[jira] [Comment Edited] (CASSANDRA-15514) Properly clean up DefaultSessionProvider#sessionCache when closing QueryReplayer
[
https://issues.apache.org/jira/browse/CASSANDRA-15514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17018520#comment-17018520
]
Yifan Cai edited comment on CASSANDRA-15514 at 1/18/20 6:34 AM:
[PR|https://github.com/apache/cassandra/pull/427],
[code|https://github.com/yifan-c/cassandra/tree/CASSANDRA-15514],
[test|https://app.circleci.com/github/yifan-c/cassandra/pipelines/538442be-e3c6-4b1d-844a-5ff48a37fa55/workflows/658295e7-331a-4f6d-a014-f8fa189043a9]
Briefly, the patch includes
* Remove the session from cache when the {{DefaultSessionProvider}} is being
closed as closing the query replayer.
* Add in-jvm dtest to simulate replaying queries end-to-end. Replay and close
twice to show the sessions are released.
was (Author: yifanc):
[PR|https://github.com/apache/cassandra/pull/427],
[code|https://github.com/yifan-c/cassandra/tree/CASSANDRA-15514],
[test|https://app.circleci.com/github/yifan-c/cassandra/pipelines/52714adb-1e2e-4d5e-9cca-fa8652364576/workflows/4d732111-1e44-4c2b-939e-f01049b923ec]
Briefly, the patch includes
* Remove the session from cache when the {{DefaultSessionProvider}} is being
closed as closing the query replayer.
* Add in-jvm dtest to simulate replaying queries end-to-end. Replay and close
twice to show the sessions are released.
> Properly clean up DefaultSessionProvider#sessionCache when closing
> QueryReplayer
>
>
> Key: CASSANDRA-15514
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15514
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/fql
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0-beta
>
>
> The {{DefaultSessionProvider}} used by the {{QueryReplayer}} caches the
> sessions by hostname. When closing, the sessions and their associated
> clusters are closed but they never get removed from the cache.
> When connecting to the same host, the closed session is returned, which is
> unexpected.
> Besides the unexpected behavior, the session references remaining in the
> cache leaks resources.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15514) Properly clean up DefaultSessionProvider#sessionCache when closing QueryReplayer
[
https://issues.apache.org/jira/browse/CASSANDRA-15514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifan Cai updated CASSANDRA-15514:
--
Test and Documentation Plan: Unit test and CI
Status: Patch Available (was: Open)
> Properly clean up DefaultSessionProvider#sessionCache when closing
> QueryReplayer
>
>
> Key: CASSANDRA-15514
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15514
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/fql
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0-beta
>
>
> The {{DefaultSessionProvider}} used by the {{QueryReplayer}} caches the
> sessions by hostname. When closing, the sessions and their associated
> clusters are closed but they never get removed from the cache.
> When connecting to the same host, the closed session is returned, which is
> unexpected.
> Besides the unexpected behavior, the session references remaining in the
> cache leaks resources.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15514) Properly clean up DefaultSessionProvider#sessionCache when closing QueryReplayer
[
https://issues.apache.org/jira/browse/CASSANDRA-15514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17018520#comment-17018520
]
Yifan Cai commented on CASSANDRA-15514:
---
[PR|https://github.com/apache/cassandra/pull/427],
[code|https://github.com/yifan-c/cassandra/tree/CASSANDRA-15514],
[test|https://app.circleci.com/github/yifan-c/cassandra/pipelines/52714adb-1e2e-4d5e-9cca-fa8652364576/workflows/4d732111-1e44-4c2b-939e-f01049b923ec]
Briefly, the patch includes
* Remove the session from cache when the {{DefaultSessionProvider}} is being
closed as closing the query replayer.
* Add in-jvm dtest to simulate replaying queries end-to-end. Replay and close
twice to show the sessions are released.
> Properly clean up DefaultSessionProvider#sessionCache when closing
> QueryReplayer
>
>
> Key: CASSANDRA-15514
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15514
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/fql
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0-beta
>
>
> The {{DefaultSessionProvider}} used by the {{QueryReplayer}} caches the
> sessions by hostname. When closing, the sessions and their associated
> clusters are closed but they never get removed from the cache.
> When connecting to the same host, the closed session is returned, which is
> unexpected.
> Besides the unexpected behavior, the session references remaining in the
> cache leaks resources.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15514) Properly clean up DefaultSessionProvider#sessionCache when closing QueryReplayer
[
https://issues.apache.org/jira/browse/CASSANDRA-15514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yifan Cai updated CASSANDRA-15514:
--
Bug Category: Parent values: Degradation(12984)Level 1 values: Resource
Management(12995)
Complexity: Low Hanging Fruit
Discovered By: Unit Test
Fix Version/s: 4.0-beta
Severity: Low
Status: Open (was: Triage Needed)
> Properly clean up DefaultSessionProvider#sessionCache when closing
> QueryReplayer
>
>
> Key: CASSANDRA-15514
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15514
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/fql
>Reporter: Yifan Cai
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.0-beta
>
>
> The {{DefaultSessionProvider}} used by the {{QueryReplayer}} caches the
> sessions by hostname. When closing, the sessions and their associated
> clusters are closed but they never get removed from the cache.
> When connecting to the same host, the closed session is returned, which is
> unexpected.
> Besides the unexpected behavior, the session references remaining in the
> cache leaks resources.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15514) Properly clean up DefaultSessionProvider#sessionCache when closing QueryReplayer
Yifan Cai created CASSANDRA-15514:
-
Summary: Properly clean up DefaultSessionProvider#sessionCache
when closing QueryReplayer
Key: CASSANDRA-15514
URL: https://issues.apache.org/jira/browse/CASSANDRA-15514
Project: Cassandra
Issue Type: Bug
Components: Tool/fql
Reporter: Yifan Cai
Assignee: Yifan Cai
The {{DefaultSessionProvider}} used by the {{QueryReplayer}} caches the
sessions by hostname. When closing, the sessions and their associated clusters
are closed but they never get removed from the cache.
When connecting to the same host, the closed session is returned, which is
unexpected.
Besides the unexpected behavior, the session references remaining in the cache
leaks resources.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15470) Potential Overflow in DatabaseDescriptor Functions That Convert Between KB/MB & Bytes
[
https://issues.apache.org/jira/browse/CASSANDRA-15470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mallika Kulkarni updated CASSANDRA-15470:
-
Status: Patch Available (was: In Progress)
> Potential Overflow in DatabaseDescriptor Functions That Convert Between KB/MB
> & Bytes
> -
>
> Key: CASSANDRA-15470
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15470
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Jordan West
>Assignee: Mallika Kulkarni
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-rc
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{DatabaseDescriptor}} has several functions that convert between user
> supplied sizes in KB/MB and bytes. These are implemented without much
> consistency and, while unlikely, several have the potential to overflow since
> validation on the input is missing. Meanwhile, some widen the number to a
> long correctly. Options include: widening in all places or simply doing
> better validation on start up — currently only the lower bound of the valid
> range is checked for many of these fields.
> List of Affected {{DatabaseDescriptor}} Methods:
> * {{getColumnIndexSize}}
> * {{getColumnIndexCacheSize}}
> * {{getBatchSizeWarnThreshold}}
> * {{getNativeTransportFrameBlockSize}}
> * {{getRepairSessionSpaceInMegabytes}}
> * {{getNativeTransportMaxFrameSize}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15470) Potential Overflow in DatabaseDescriptor Functions That Convert Between KB/MB & Bytes
[
https://issues.apache.org/jira/browse/CASSANDRA-15470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mallika Kulkarni updated CASSANDRA-15470:
-
Status: In Progress (was: Changes Suggested)
> Potential Overflow in DatabaseDescriptor Functions That Convert Between KB/MB
> & Bytes
> -
>
> Key: CASSANDRA-15470
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15470
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Jordan West
>Assignee: Mallika Kulkarni
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-rc
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{DatabaseDescriptor}} has several functions that convert between user
> supplied sizes in KB/MB and bytes. These are implemented without much
> consistency and, while unlikely, several have the potential to overflow since
> validation on the input is missing. Meanwhile, some widen the number to a
> long correctly. Options include: widening in all places or simply doing
> better validation on start up — currently only the lower bound of the valid
> range is checked for many of these fields.
> List of Affected {{DatabaseDescriptor}} Methods:
> * {{getColumnIndexSize}}
> * {{getColumnIndexCacheSize}}
> * {{getBatchSizeWarnThreshold}}
> * {{getNativeTransportFrameBlockSize}}
> * {{getRepairSessionSpaceInMegabytes}}
> * {{getNativeTransportMaxFrameSize}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15470) Potential Overflow in DatabaseDescriptor Functions That Convert Between KB/MB & Bytes
[
https://issues.apache.org/jira/browse/CASSANDRA-15470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17018491#comment-17018491
]
Mallika Kulkarni commented on CASSANDRA-15470:
--
Thank you [~jrwest] . I added a new commit to the Pull Request.
> Potential Overflow in DatabaseDescriptor Functions That Convert Between KB/MB
> & Bytes
> -
>
> Key: CASSANDRA-15470
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15470
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Config
>Reporter: Jordan West
>Assignee: Mallika Kulkarni
>Priority: Normal
> Labels: pull-request-available
> Fix For: 4.0-rc
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {{DatabaseDescriptor}} has several functions that convert between user
> supplied sizes in KB/MB and bytes. These are implemented without much
> consistency and, while unlikely, several have the potential to overflow since
> validation on the input is missing. Meanwhile, some widen the number to a
> long correctly. Options include: widening in all places or simply doing
> better validation on start up — currently only the lower bound of the valid
> range is checked for many of these fields.
> List of Affected {{DatabaseDescriptor}} Methods:
> * {{getColumnIndexSize}}
> * {{getColumnIndexCacheSize}}
> * {{getBatchSizeWarnThreshold}}
> * {{getNativeTransportFrameBlockSize}}
> * {{getRepairSessionSpaceInMegabytes}}
> * {{getNativeTransportMaxFrameSize}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15472) Read failure due to exception from metrics-core dependency
[
https://issues.apache.org/jira/browse/CASSANDRA-15472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
C. Scott Andreas updated CASSANDRA-15472:
-
Bug Category: Parent values: Availability(12983)Level 1 values: Response
Crash(12991)
Complexity: Normal
Discovered By: User Report
Severity: Normal
Status: Open (was: Triage Needed)
> Read failure due to exception from metrics-core dependency
> --
>
> Key: CASSANDRA-15472
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15472
> Project: Cassandra
> Issue Type: Bug
> Components: Dependencies
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Normal
> Fix For: 4.0, 3.0.x, 3.11.x
>
>
> Stacktrace
> {code:java}
> Uncaught exception on thread Thread[SharedPool-Worker-27,5,main]: {}
> java.util.NoSuchElementException: null
> at
> java.util.concurrent.ConcurrentSkipListMap.firstKey(ConcurrentSkipListMap.java:2053)
> ~[na:1.8.0_222]
> at
> com.yammer.metrics.stats.ExponentiallyDecayingSample.update(ExponentiallyDecayingSample.java:102)
> ~[metrics-core-2.2.0.jar:na]
> at
> com.yammer.metrics.stats.ExponentiallyDecayingSample.update(ExponentiallyDecayingSample.java:81)
> ~[metrics-core-2.2.0.jar:na]
> at com.yammer.metrics.core.Histogram.update(Histogram.java:110)
> ~[metrics-core-2.2.0.jar:na]
> at com.yammer.metrics.core.Timer.update(Timer.java:198)
> ~[metrics-core-2.2.0.jar:na]
> at com.yammer.metrics.core.Timer.update(Timer.java:76)
> ~[metrics-core-2.2.0.jar:na]
> at
> org.apache.cassandra.metrics.LatencyMetrics.addNano(LatencyMetrics.java:108)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> org.apache.cassandra.metrics.LatencyMetrics.addNano(LatencyMetrics.java:114)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1897)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:353)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:85)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:47)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:64)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_222]
> at
> org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
> [nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_222]
> {code}
> This [issue|https://github.com/dropwizard/metrics/issues/1278] has been
> [fixed|https://github.com/dropwizard/metrics/pull/1436] in
> [v4.0.6|https://github.com/dropwizard/metrics/releases/tag/v4.0.6].
> This is observed on a 2.1.19 cluster, but this would impact pretty much any
> version of C* since we depend on lower versions of metrics-core that do not
> have the fix.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15473) New feature - Virtual Tables
[ https://issues.apache.org/jira/browse/CASSANDRA-15473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] C. Scott Andreas updated CASSANDRA-15473: - Change Category: Code Clarity Complexity: Normal Status: Open (was: Triage Needed) > New feature - Virtual Tables > > > Key: CASSANDRA-15473 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15473 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > > Added a page on virtual tables, a new feature. > https://github.com/apache/cassandra/pull/402 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15473) New feature - Virtual Tables
[ https://issues.apache.org/jira/browse/CASSANDRA-15473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] C. Scott Andreas updated CASSANDRA-15473: - Test and Documentation Plan: [ Review of documentation ] Status: Patch Available (was: Open) > New feature - Virtual Tables > > > Key: CASSANDRA-15473 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15473 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > > Added a page on virtual tables, a new feature. > https://github.com/apache/cassandra/pull/402 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15474) Audit Logging - New Feature
[ https://issues.apache.org/jira/browse/CASSANDRA-15474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] C. Scott Andreas updated CASSANDRA-15474: - Test and Documentation Plan: [ Review of documentation ] Status: Patch Available (was: Open) > Audit Logging - New Feature > --- > > Key: CASSANDRA-15474 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15474 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > > Added a page on audit logging, a new feature. > https://github.com/apache/cassandra/pull/403 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15474) Audit Logging - New Feature
[ https://issues.apache.org/jira/browse/CASSANDRA-15474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] C. Scott Andreas updated CASSANDRA-15474: - Change Category: Code Clarity Complexity: Normal Status: Open (was: Triage Needed) > Audit Logging - New Feature > --- > > Key: CASSANDRA-15474 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15474 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > > Added a page on audit logging, a new feature. > https://github.com/apache/cassandra/pull/403 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15476) Transient Replication - New Feature
[ https://issues.apache.org/jira/browse/CASSANDRA-15476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] C. Scott Andreas updated CASSANDRA-15476: - Change Category: Code Clarity Complexity: Normal Status: Open (was: Triage Needed) > Transient Replication - New Feature > --- > > Key: CASSANDRA-15476 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15476 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > > Added a page on transient replication, a new feature. > https://github.com/apache/cassandra/pull/405 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15479) Backups - updated
[ https://issues.apache.org/jira/browse/CASSANDRA-15479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] C. Scott Andreas updated CASSANDRA-15479: - Change Category: Code Clarity Complexity: Normal Status: Open (was: Triage Needed) > Backups - updated > - > > Key: CASSANDRA-15479 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15479 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > > Added Backups page. > https://github.com/apache/cassandra/pull/408 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15480) Bulk Loading
[ https://issues.apache.org/jira/browse/CASSANDRA-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] C. Scott Andreas updated CASSANDRA-15480: - Test and Documentation Plan: [ Review of documentation ] Status: Patch Available (was: Open) > Bulk Loading > > > Key: CASSANDRA-15480 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15480 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > > Added bulk loading page. > https://github.com/apache/cassandra/pull/409 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15476) Transient Replication - New Feature
[ https://issues.apache.org/jira/browse/CASSANDRA-15476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] C. Scott Andreas updated CASSANDRA-15476: - Test and Documentation Plan: [ Review of documentation ] Status: Patch Available (was: Open) > Transient Replication - New Feature > --- > > Key: CASSANDRA-15476 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15476 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > > Added a page on transient replication, a new feature. > https://github.com/apache/cassandra/pull/405 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15479) Backups - updated
[ https://issues.apache.org/jira/browse/CASSANDRA-15479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] C. Scott Andreas updated CASSANDRA-15479: - Test and Documentation Plan: [ Review of documentation ] Status: Patch Available (was: Open) > Backups - updated > - > > Key: CASSANDRA-15479 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15479 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > > Added Backups page. > https://github.com/apache/cassandra/pull/408 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15480) Bulk Loading
[ https://issues.apache.org/jira/browse/CASSANDRA-15480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] C. Scott Andreas updated CASSANDRA-15480: - Change Category: Code Clarity Complexity: Normal Status: Open (was: Triage Needed) > Bulk Loading > > > Key: CASSANDRA-15480 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15480 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > > Added bulk loading page. > https://github.com/apache/cassandra/pull/409 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15485) Added Full Repair Example
[ https://issues.apache.org/jira/browse/CASSANDRA-15485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] C. Scott Andreas updated CASSANDRA-15485: - Change Category: Code Clarity Complexity: Normal Status: Open (was: Triage Needed) > Added Full Repair Example > - > > Key: CASSANDRA-15485 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15485 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > > Added full repair example. > https://github.com/apache/cassandra/pull/414 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15483) Overview
[ https://issues.apache.org/jira/browse/CASSANDRA-15483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] C. Scott Andreas updated CASSANDRA-15483: - Change Category: Code Clarity Complexity: Normal Status: Open (was: Triage Needed) > Overview > > > Key: CASSANDRA-15483 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15483 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > > Added page on Overview. > https://github.com/apache/cassandra/pull/412 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15483) Overview
[ https://issues.apache.org/jira/browse/CASSANDRA-15483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] C. Scott Andreas updated CASSANDRA-15483: - Test and Documentation Plan: [ Review of documentation ] Status: Patch Available (was: Open) > Overview > > > Key: CASSANDRA-15483 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15483 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > > Added page on Overview. > https://github.com/apache/cassandra/pull/412 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15485) Added Full Repair Example
[ https://issues.apache.org/jira/browse/CASSANDRA-15485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] C. Scott Andreas updated CASSANDRA-15485: - Test and Documentation Plan: [ Review of documentation ] Status: Patch Available (was: Open) > Added Full Repair Example > - > > Key: CASSANDRA-15485 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15485 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > > Added full repair example. > https://github.com/apache/cassandra/pull/414 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15491) Hints
[ https://issues.apache.org/jira/browse/CASSANDRA-15491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] C. Scott Andreas updated CASSANDRA-15491: - Test and Documentation Plan: [ Review of documentation ] Status: Patch Available (was: Open) > Hints > - > > Key: CASSANDRA-15491 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15491 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > > Added page for hints. > https://github.com/apache/cassandra/pull/419 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15492) DDL update
[ https://issues.apache.org/jira/browse/CASSANDRA-15492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] C. Scott Andreas updated CASSANDRA-15492: - Change Category: Code Clarity Complexity: Normal Status: Open (was: Triage Needed) > DDL update > -- > > Key: CASSANDRA-15492 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15492 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > > - Added more detail and examples to some sections such as Speculative retry > - Added table option 'cdc' > https://github.com/apache/cassandra/pull/420 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15491) Hints
[ https://issues.apache.org/jira/browse/CASSANDRA-15491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] C. Scott Andreas updated CASSANDRA-15491: - Change Category: Code Clarity Complexity: Normal Status: Open (was: Triage Needed) > Hints > - > > Key: CASSANDRA-15491 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15491 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > > Added page for hints. > https://github.com/apache/cassandra/pull/419 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15492) DDL update
[ https://issues.apache.org/jira/browse/CASSANDRA-15492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] C. Scott Andreas updated CASSANDRA-15492: - Test and Documentation Plan: [ Review of documentation ] Status: Patch Available (was: Open) > DDL update > -- > > Key: CASSANDRA-15492 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15492 > Project: Cassandra > Issue Type: Sub-task > Components: Documentation/Website >Reporter: DeepakVohra >Assignee: DeepakVohra >Priority: Normal > > - Added more detail and examples to some sections such as Speculative retry > - Added table option 'cdc' > https://github.com/apache/cassandra/pull/420 -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15495) Separate cqlsh into it's own git repo
[ https://issues.apache.org/jira/browse/CASSANDRA-15495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] C. Scott Andreas updated CASSANDRA-15495: - Resolution: Information Provided Status: Resolved (was: Triage Needed) > Separate cqlsh into it's own git repo > - > > Key: CASSANDRA-15495 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15495 > Project: Cassandra > Issue Type: Improvement > Components: Packaging >Reporter: Israel Fruchter >Priority: Normal > > While looking into the CASSANDRA-10190, and seeing the long time there was no > new version on pypi > [https://pypi.org/project/cqlsh/#history] > > I thouhgt maybe it's time to separate cqlsh into it's own git repo, and > handle more frequent release of it. > while have it installable regardless to Cassandra installation > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15500) Slow query logging emits incomplete predicates
[ https://issues.apache.org/jira/browse/CASSANDRA-15500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] C. Scott Andreas updated CASSANDRA-15500: - Bug Category: Parent values: Correctness(12982) Complexity: Normal Discovered By: User Report Severity: Normal Status: Open (was: Triage Needed) > Slow query logging emits incomplete predicates > -- > > Key: CASSANDRA-15500 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15500 > Project: Cassandra > Issue Type: Bug > Components: Observability/Logging >Reporter: onmstester >Priority: Normal > > Using Apache Cassandra 3.11.2, defined a table like this: > > _create table my_table(_ > __ _partition text,_ > __ _clustering1 int,_ > _clustering2 text,_ > _data set,_ > ** _*primary key (partition, clustering1, > clustering2))*_ > > and configured slow queries threshold to 1ms in yaml to see how queries > passed to cassandra. Query below: > > _select * from my_table where partition='a' and clustering1= 1 and > clustering2='b'_ > > would be like this in debug.log of cassandra: > > _select * from my_table where partition='a' LIMIT 100> (it means that the > two cluster key restriction did not push down to storage engine and the whole > partition been retrieved)_ > > but this query: > > _select * from my_table where partition='a' and clustering1= 1_ > > _would be_ > > _select * from my_table where partition='a' and_ _clustering1= 1_ _LIMIT 100> > (single cluster key been pushed down to storage engine)_ > > > _So it seems to me that, we could not restrict multiple clustering keys in > select because it would retrieve the whole partition ?!_ -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15500) Slow query logging emits incomplete predicates
[ https://issues.apache.org/jira/browse/CASSANDRA-15500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] C. Scott Andreas updated CASSANDRA-15500: - Summary: Slow query logging emits incomplete predicates (was: only partition key push down when multiple cluster keys restricted in where clause) > Slow query logging emits incomplete predicates > -- > > Key: CASSANDRA-15500 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15500 > Project: Cassandra > Issue Type: Bug > Components: Observability/Logging >Reporter: onmstester >Priority: Normal > > Using Apache Cassandra 3.11.2, defined a table like this: > > _create table my_table(_ > __ _partition text,_ > __ _clustering1 int,_ > _clustering2 text,_ > _data set,_ > ** _*primary key (partition, clustering1, > clustering2))*_ > > and configured slow queries threshold to 1ms in yaml to see how queries > passed to cassandra. Query below: > > _select * from my_table where partition='a' and clustering1= 1 and > clustering2='b'_ > > would be like this in debug.log of cassandra: > > _select * from my_table where partition='a' LIMIT 100> (it means that the > two cluster key restriction did not push down to storage engine and the whole > partition been retrieved)_ > > but this query: > > _select * from my_table where partition='a' and clustering1= 1_ > > _would be_ > > _select * from my_table where partition='a' and_ _clustering1= 1_ _LIMIT 100> > (single cluster key been pushed down to storage engine)_ > > > _So it seems to me that, we could not restrict multiple clustering keys in > select because it would retrieve the whole partition ?!_ -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15500) only partition key push down when multiple cluster keys restricted in where clause
[ https://issues.apache.org/jira/browse/CASSANDRA-15500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] C. Scott Andreas updated CASSANDRA-15500: - Component/s: Observability/Logging > only partition key push down when multiple cluster keys restricted in where > clause > -- > > Key: CASSANDRA-15500 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15500 > Project: Cassandra > Issue Type: Bug > Components: Observability/Logging >Reporter: onmstester >Priority: Normal > > Using Apache Cassandra 3.11.2, defined a table like this: > > _create table my_table(_ > __ _partition text,_ > __ _clustering1 int,_ > _clustering2 text,_ > _data set,_ > ** _*primary key (partition, clustering1, > clustering2))*_ > > and configured slow queries threshold to 1ms in yaml to see how queries > passed to cassandra. Query below: > > _select * from my_table where partition='a' and clustering1= 1 and > clustering2='b'_ > > would be like this in debug.log of cassandra: > > _select * from my_table where partition='a' LIMIT 100> (it means that the > two cluster key restriction did not push down to storage engine and the whole > partition been retrieved)_ > > but this query: > > _select * from my_table where partition='a' and clustering1= 1_ > > _would be_ > > _select * from my_table where partition='a' and_ _clustering1= 1_ _LIMIT 100> > (single cluster key been pushed down to storage engine)_ > > > _So it seems to me that, we could not restrict multiple clustering keys in > select because it would retrieve the whole partition ?!_ -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15503) Slow query log indicates opposite LTE when GTE operator
[
https://issues.apache.org/jira/browse/CASSANDRA-15503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
C. Scott Andreas updated CASSANDRA-15503:
-
Bug Category: Parent values: Correctness(12982)
Discovered By: User Report
Severity: Normal
Since Version: 3.10
> Slow query log indicates opposite LTE when GTE operator
> ---
>
> Key: CASSANDRA-15503
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15503
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Logging
>Reporter: Wallace Baggaley
>Priority: Normal
>
> Slow query log is indicating a '<=' when a ">=" operator was sent. This
> appears to be a logging only issue, but it threw off development for a day
> figuring this out. Please fix.
> How to reproduce. Set slow query log timeout to 1 millisecond.
> In cqlsh run
> {noformat}
> CREATE TABLE atable (
> .id text,
> timestamp timestamp,
> PRIMARY KEY ((id), timestamp)
> ) WITH CLUSTERING ORDER BY (timestamp DESC);
> insert into atable (id, timestamp) VALUES ( '1',1);
> insert into atable (id, timestamp) VALUES ( '2',2);
> insert into atable (id, timestamp) VALUES ( '3',3);
> insert into atable (id, timestamp) VALUES ( '4',4);
> insert into atable (id, timestamp) VALUES ( '5',5);
> insert into atable (id, timestamp) VALUES ( '6',6);
> insert into atable (id, timestamp) VALUES ( '7',7);
> insert into atable (id, timestamp) VALUES ( '8',8);
> insert into atable (id, timestamp) VALUES ( '9',9);
> insert into atable (id, timestamp) VALUES ( '10',10);
> insert into atable (id, timestamp) VALUES ( '11',11);
> select * from atable where timestamp >= '1970-01-01 00:00:00.006+' allow
> filtering;
> {noformat}
> In the logs it prints:
> {noformat}
> DEBUG 1 operations were slow in the last 5003 msecs:
> ,
> time 7 msec - slow timeout 1 msec
> {noformat}
> But the query works appropriately and returns
> {noformat}
> id | timestamp
> +-
> 6 | 1970-01-01 00:00:00.006000+
> 7 | 1970-01-01 00:00:00.007000+
> 9 | 1970-01-01 00:00:00.009000+
> 10 | 1970-01-01 00:00:00.01+
> 8 | 1970-01-01 00:00:00.008000+
> 11 | 1970-01-01 00:00:00.011000+
> (6 rows)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15503) Slow query log indicates opposite LTE when GTE operator
[
https://issues.apache.org/jira/browse/CASSANDRA-15503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
C. Scott Andreas updated CASSANDRA-15503:
-
Complexity: Normal
Status: Open (was: Triage Needed)
> Slow query log indicates opposite LTE when GTE operator
> ---
>
> Key: CASSANDRA-15503
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15503
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Logging
>Reporter: Wallace Baggaley
>Priority: Normal
>
> Slow query log is indicating a '<=' when a ">=" operator was sent. This
> appears to be a logging only issue, but it threw off development for a day
> figuring this out. Please fix.
> How to reproduce. Set slow query log timeout to 1 millisecond.
> In cqlsh run
> {noformat}
> CREATE TABLE atable (
> .id text,
> timestamp timestamp,
> PRIMARY KEY ((id), timestamp)
> ) WITH CLUSTERING ORDER BY (timestamp DESC);
> insert into atable (id, timestamp) VALUES ( '1',1);
> insert into atable (id, timestamp) VALUES ( '2',2);
> insert into atable (id, timestamp) VALUES ( '3',3);
> insert into atable (id, timestamp) VALUES ( '4',4);
> insert into atable (id, timestamp) VALUES ( '5',5);
> insert into atable (id, timestamp) VALUES ( '6',6);
> insert into atable (id, timestamp) VALUES ( '7',7);
> insert into atable (id, timestamp) VALUES ( '8',8);
> insert into atable (id, timestamp) VALUES ( '9',9);
> insert into atable (id, timestamp) VALUES ( '10',10);
> insert into atable (id, timestamp) VALUES ( '11',11);
> select * from atable where timestamp >= '1970-01-01 00:00:00.006+' allow
> filtering;
> {noformat}
> In the logs it prints:
> {noformat}
> DEBUG 1 operations were slow in the last 5003 msecs:
> ,
> time 7 msec - slow timeout 1 msec
> {noformat}
> But the query works appropriately and returns
> {noformat}
> id | timestamp
> +-
> 6 | 1970-01-01 00:00:00.006000+
> 7 | 1970-01-01 00:00:00.007000+
> 9 | 1970-01-01 00:00:00.009000+
> 10 | 1970-01-01 00:00:00.01+
> 8 | 1970-01-01 00:00:00.008000+
> 11 | 1970-01-01 00:00:00.011000+
> (6 rows)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15503) Slow query log indicates opposite LTE when GTE operator
[
https://issues.apache.org/jira/browse/CASSANDRA-15503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17018464#comment-17018464
]
C. Scott Andreas commented on CASSANDRA-15503:
--
Confirmed this reproduces on 3.11.5, affecting strict equalities as well.
Strict equality expressions aren't printed at all:
{{cqlsh> select * from test.atable where timestamp = '1975-01-01
00:00:00.006+' allow filtering;}}
{{=> , time 11 msec - slow timeout
1 msec/cross-node}}
And as Wallace notes, inequalities are reversed:
{{cqlsh> select * from test.atable where timestamp > '1975-01-01
00:00:00.006+' allow filtering;}}
{{=> , time 11 msec - slow timeout 1 msec/cross-node}}
> Slow query log indicates opposite LTE when GTE operator
> ---
>
> Key: CASSANDRA-15503
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15503
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Observability
>Reporter: Wallace Baggaley
>Priority: Normal
>
> Slow query log is indicating a '<=' when a ">=" operator was sent. This
> appears to be a logging only issue, but it threw off development for a day
> figuring this out. Please fix.
> How to reproduce. Set slow query log timeout to 1 millisecond.
> In cqlsh run
> {noformat}
> CREATE TABLE atable (
> .id text,
> timestamp timestamp,
> PRIMARY KEY ((id), timestamp)
> ) WITH CLUSTERING ORDER BY (timestamp DESC);
> insert into atable (id, timestamp) VALUES ( '1',1);
> insert into atable (id, timestamp) VALUES ( '2',2);
> insert into atable (id, timestamp) VALUES ( '3',3);
> insert into atable (id, timestamp) VALUES ( '4',4);
> insert into atable (id, timestamp) VALUES ( '5',5);
> insert into atable (id, timestamp) VALUES ( '6',6);
> insert into atable (id, timestamp) VALUES ( '7',7);
> insert into atable (id, timestamp) VALUES ( '8',8);
> insert into atable (id, timestamp) VALUES ( '9',9);
> insert into atable (id, timestamp) VALUES ( '10',10);
> insert into atable (id, timestamp) VALUES ( '11',11);
> select * from atable where timestamp >= '1970-01-01 00:00:00.006+' allow
> filtering;
> {noformat}
> In the logs it prints:
> {noformat}
> DEBUG 1 operations were slow in the last 5003 msecs:
> ,
> time 7 msec - slow timeout 1 msec
> {noformat}
> But the query works appropriately and returns
> {noformat}
> id | timestamp
> +-
> 6 | 1970-01-01 00:00:00.006000+
> 7 | 1970-01-01 00:00:00.007000+
> 9 | 1970-01-01 00:00:00.009000+
> 10 | 1970-01-01 00:00:00.01+
> 8 | 1970-01-01 00:00:00.008000+
> 11 | 1970-01-01 00:00:00.011000+
> (6 rows)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15503) Slow query log indicates opposite LTE when GTE operator
[
https://issues.apache.org/jira/browse/CASSANDRA-15503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
C. Scott Andreas updated CASSANDRA-15503:
-
Component/s: (was: Legacy/Observability)
Observability/Logging
> Slow query log indicates opposite LTE when GTE operator
> ---
>
> Key: CASSANDRA-15503
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15503
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/Logging
>Reporter: Wallace Baggaley
>Priority: Normal
>
> Slow query log is indicating a '<=' when a ">=" operator was sent. This
> appears to be a logging only issue, but it threw off development for a day
> figuring this out. Please fix.
> How to reproduce. Set slow query log timeout to 1 millisecond.
> In cqlsh run
> {noformat}
> CREATE TABLE atable (
> .id text,
> timestamp timestamp,
> PRIMARY KEY ((id), timestamp)
> ) WITH CLUSTERING ORDER BY (timestamp DESC);
> insert into atable (id, timestamp) VALUES ( '1',1);
> insert into atable (id, timestamp) VALUES ( '2',2);
> insert into atable (id, timestamp) VALUES ( '3',3);
> insert into atable (id, timestamp) VALUES ( '4',4);
> insert into atable (id, timestamp) VALUES ( '5',5);
> insert into atable (id, timestamp) VALUES ( '6',6);
> insert into atable (id, timestamp) VALUES ( '7',7);
> insert into atable (id, timestamp) VALUES ( '8',8);
> insert into atable (id, timestamp) VALUES ( '9',9);
> insert into atable (id, timestamp) VALUES ( '10',10);
> insert into atable (id, timestamp) VALUES ( '11',11);
> select * from atable where timestamp >= '1970-01-01 00:00:00.006+' allow
> filtering;
> {noformat}
> In the logs it prints:
> {noformat}
> DEBUG 1 operations were slow in the last 5003 msecs:
> ,
> time 7 msec - slow timeout 1 msec
> {noformat}
> But the query works appropriately and returns
> {noformat}
> id | timestamp
> +-
> 6 | 1970-01-01 00:00:00.006000+
> 7 | 1970-01-01 00:00:00.007000+
> 9 | 1970-01-01 00:00:00.009000+
> 10 | 1970-01-01 00:00:00.01+
> 8 | 1970-01-01 00:00:00.008000+
> 11 | 1970-01-01 00:00:00.011000+
> (6 rows)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15508) Failing jvm dtest: FailingRepairTest.testFailingMessage
[
https://issues.apache.org/jira/browse/CASSANDRA-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17018449#comment-17018449
]
David Capwell commented on CASSANDRA-15508:
---
FYI [~mck]; trying to change circle ci build for in-jvm dtests.
> Failing jvm dtest: FailingRepairTest.testFailingMessage
> ---
>
> Key: CASSANDRA-15508
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15508
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Marcus Eriksson
>Assignee: David Capwell
>Priority: Normal
> Labels: pull-request-available
> Attachments: Screen Shot 2020-01-17 at 4.51.17 PM.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It seems we can't run parameterized unit tests with {{ant testsome}}:
> {code}
> $ ant testsome
> -Dtest.name=org.apache.cassandra.distributed.test.FailingRepairTest
> -Dtest.methods=testFailingMessage
>
> [junit-timeout] Testcase:
> initializationError(org.junit.runner.manipulation.Filter):Caused an ERROR
> [junit-timeout] No tests found matching Method
> testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest)
> from org.junit.internal.requests.ClassRequest@4d95d2a2
> [junit-timeout] java.lang.Exception: No tests found matching Method
> testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest)
> from org.junit.internal.requests.ClassRequest@4d95d2a2
> [junit-timeout] at
> java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15508) Failing jvm dtest: FailingRepairTest.testFailingMessage
[
https://issues.apache.org/jira/browse/CASSANDRA-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-15508:
--
Attachment: Screen Shot 2020-01-17 at 4.51.17 PM.png
> Failing jvm dtest: FailingRepairTest.testFailingMessage
> ---
>
> Key: CASSANDRA-15508
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15508
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Marcus Eriksson
>Assignee: David Capwell
>Priority: Normal
> Labels: pull-request-available
> Attachments: Screen Shot 2020-01-17 at 4.51.17 PM.png
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It seems we can't run parameterized unit tests with {{ant testsome}}:
> {code}
> $ ant testsome
> -Dtest.name=org.apache.cassandra.distributed.test.FailingRepairTest
> -Dtest.methods=testFailingMessage
>
> [junit-timeout] Testcase:
> initializationError(org.junit.runner.manipulation.Filter):Caused an ERROR
> [junit-timeout] No tests found matching Method
> testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest)
> from org.junit.internal.requests.ClassRequest@4d95d2a2
> [junit-timeout] java.lang.Exception: No tests found matching Method
> testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest)
> from org.junit.internal.requests.ClassRequest@4d95d2a2
> [junit-timeout] at
> java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15508) Failing jvm dtest: FailingRepairTest.testFailingMessage
[
https://issues.apache.org/jira/browse/CASSANDRA-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17018442#comment-17018442
]
David Capwell commented on CASSANDRA-15508:
---
Pushed the change so that upgrade tests are their own build and still require
approval; I think the only real change here is that the approval button is now
top level so I can trigger the dtest builds at the same time as building trunk.
> Failing jvm dtest: FailingRepairTest.testFailingMessage
> ---
>
> Key: CASSANDRA-15508
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15508
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Marcus Eriksson
>Assignee: David Capwell
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It seems we can't run parameterized unit tests with {{ant testsome}}:
> {code}
> $ ant testsome
> -Dtest.name=org.apache.cassandra.distributed.test.FailingRepairTest
> -Dtest.methods=testFailingMessage
>
> [junit-timeout] Testcase:
> initializationError(org.junit.runner.manipulation.Filter):Caused an ERROR
> [junit-timeout] No tests found matching Method
> testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest)
> from org.junit.internal.requests.ClassRequest@4d95d2a2
> [junit-timeout] java.lang.Exception: No tests found matching Method
> testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest)
> from org.junit.internal.requests.ClassRequest@4d95d2a2
> [junit-timeout] at
> java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15504) INT is incompatible with previous type SMALLINT
[
https://issues.apache.org/jira/browse/CASSANDRA-15504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
C. Scott Andreas updated CASSANDRA-15504:
-
Component/s: Cluster/Schema
> INT is incompatible with previous type SMALLINT
> ---
>
> Key: CASSANDRA-15504
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15504
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Schema
>Reporter: Marcus Truscello
>Priority: Normal
>
> With the release of Cassandra 3.11.5 and the fixing of CASSANDRA-14948, it
> now appears that you can no longer re-add a SMALLINT column as an INT type.
> This is rather surprising as any SMALLINT value should be representable by an
> INT type.
> The following example was run on Cassandra 3.11.5 on CentOS 7 installed from
> official RedHat repo:
>
>
> {noformat}
> cqlsh> CREATE KEYSPACE demo WITH replication = {'class':'SimpleStrategy',
> 'replication_factor' : 1};
> cqlsh> CREATE TABLE demo.demo_table (
> ... user_id BIGINT,
> ... created TIMESTAMP,
> ... points SMALLINT,
> ... PRIMARY KEY (user_id, created)
> ... ) WITH CLUSTERING ORDER BY (created DESC);
> cqlsh> ALTER TABLE demo.demo_table DROP points;
> cqlsh> ALTER TABLE demo.demo_table ADD points INT;
> InvalidRequest: Error from server: code=2200 [Invalid query] message="Cannot
> re-add previously dropped column 'points' of type int, incompatible with
> previous type smallint"{noformat}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15513) Repair -full sets "repairedAt" value
[
https://issues.apache.org/jira/browse/CASSANDRA-15513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17018436#comment-17018436
]
Brandon Williams commented on CASSANDRA-15513:
--
It seems to me after CASSANDRA-5351, this should be a significant problem,
since these sstables will not compact against each other. Am I missing
something?
> Repair -full sets "repairedAt" value
>
>
> Key: CASSANDRA-15513
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15513
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Repair
>Reporter: Sean Fuller
>Priority: Normal
>
> {code:java}
> src/java/org/apache/cassandra/service/ActiveRepairService.java:{code}
> {code:java}
> * we only want to set repairedAt for incremental repairs including all
> replicas for a token range. For non-global
> * incremental repairs, forced incremental repairs, and full repairs, the
> UNREPAIRED_SSTABLE value will prevent
> * sstables from being promoted to repaired or preserve the
> repairedAt/pendingRepair values, respectively.{code}
> Verified in 3.11.5:
> {code:java}
> $ for f in *Data.db; do echo $f; sudo /usr/bin/sstablemetadata $f |grep -i
> repaired; done
> md-10-big-Data.db
> Repaired at: 0
> md-5-big-Data.db
> Repaired at: 0
> md-6-big-Data.db
> Repaired at: 0
> md-7-big-Data.db
> Repaired at: 0
> md-8-big-Data.db
> Repaired at: 0
> md-9-big-Data.db
> Repaired at: 0
> $ nodetool repair -full
> [2020-01-17 00:25:35,192] Starting repair command #1
> (e0edeee0-38bf-11ea-b3f5-891068749d18), repairing keyspace ext_alt_live with
> repair options (parallelism: parallel, primary range: false, incremental:
> false, job threads: 1, ColumnFamilies: [], dataCenters: [], hosts: [], # of
> ranges: 3, pull repair: false)
> [2020-01-17 00:27:51,874] Repair session e1930880-38bf-11ea-b3f5-891068749d18
> for range [(3074457345618258602,-9223372036854775808],
> (-9223372036854775808,-3074457345618258603],
> (-3074457345618258603,3074457345618258602]] finished (progress: 71%)
> [2020-01-17 00:27:51,920] Repair completed successfully
> [2020-01-17 00:27:51,925] Repair command #1 finished in 2 minutes 16 seconds
> [2020-01-17 00:27:51,980] Replication factor is 1. No repair is needed for
> keyspace 'system_auth'
> [2020-01-17 00:27:52,003] Starting repair command #2
> (327adb10-38c0-11ea-b3f5-891068749d18), repairing keyspace system_traces with
> repair options (parallelism: parallel, primary range: false, incremental:
> false, job threads: 1, ColumnFamilies: [], dataCenters: [], hosts: [], # of
> ranges: 2, pull repair: false)
> [2020-01-17 00:27:52,052] Repair session 327d4c10-38c0-11ea-b3f5-891068749d18
> for range [(3074457345618258602,-9223372036854775808]] finished (progress:
> 83%)
> [2020-01-17 00:27:52,081] Repair session 327dc140-38c0-11ea-b3f5-891068749d18
> for range [(-3074457345618258603,3074457345618258602]] finished (progress:
> 100%)
> [2020-01-17 00:27:52,086] Repair completed successfully
> [2020-01-17 00:27:52,090] Repair command #2 finished in 0 seconds
> $ for f in *Data.db; do echo $f; sudo /usr/bin/sstablemetadata $f |grep -i
> repaired; done
> md-11-big-Data.db
> Repaired at: 0
> md-12-big-Data.db
> Repaired at: 0
> md-13-big-Data.db
> Repaired at: 1579220735212
> md-14-big-Data.db
> Repaired at: 1579220735212
> md-15-big-Data.db
> Repaired at: 1579220735212
> md-16-big-Data.db
> Repaired at: 1579220735212{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15513) Repair -full sets "repairedAt" value
[
https://issues.apache.org/jira/browse/CASSANDRA-15513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
C. Scott Andreas updated CASSANDRA-15513:
-
Component/s: Consistency/Repair
> Repair -full sets "repairedAt" value
>
>
> Key: CASSANDRA-15513
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15513
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Repair
>Reporter: Sean Fuller
>Priority: Normal
>
> {code:java}
> src/java/org/apache/cassandra/service/ActiveRepairService.java:{code}
> {code:java}
> * we only want to set repairedAt for incremental repairs including all
> replicas for a token range. For non-global
> * incremental repairs, forced incremental repairs, and full repairs, the
> UNREPAIRED_SSTABLE value will prevent
> * sstables from being promoted to repaired or preserve the
> repairedAt/pendingRepair values, respectively.{code}
> Verified in 3.11.5:
> {code:java}
> $ for f in *Data.db; do echo $f; sudo /usr/bin/sstablemetadata $f |grep -i
> repaired; done
> md-10-big-Data.db
> Repaired at: 0
> md-5-big-Data.db
> Repaired at: 0
> md-6-big-Data.db
> Repaired at: 0
> md-7-big-Data.db
> Repaired at: 0
> md-8-big-Data.db
> Repaired at: 0
> md-9-big-Data.db
> Repaired at: 0
> $ nodetool repair -full
> [2020-01-17 00:25:35,192] Starting repair command #1
> (e0edeee0-38bf-11ea-b3f5-891068749d18), repairing keyspace ext_alt_live with
> repair options (parallelism: parallel, primary range: false, incremental:
> false, job threads: 1, ColumnFamilies: [], dataCenters: [], hosts: [], # of
> ranges: 3, pull repair: false)
> [2020-01-17 00:27:51,874] Repair session e1930880-38bf-11ea-b3f5-891068749d18
> for range [(3074457345618258602,-9223372036854775808],
> (-9223372036854775808,-3074457345618258603],
> (-3074457345618258603,3074457345618258602]] finished (progress: 71%)
> [2020-01-17 00:27:51,920] Repair completed successfully
> [2020-01-17 00:27:51,925] Repair command #1 finished in 2 minutes 16 seconds
> [2020-01-17 00:27:51,980] Replication factor is 1. No repair is needed for
> keyspace 'system_auth'
> [2020-01-17 00:27:52,003] Starting repair command #2
> (327adb10-38c0-11ea-b3f5-891068749d18), repairing keyspace system_traces with
> repair options (parallelism: parallel, primary range: false, incremental:
> false, job threads: 1, ColumnFamilies: [], dataCenters: [], hosts: [], # of
> ranges: 2, pull repair: false)
> [2020-01-17 00:27:52,052] Repair session 327d4c10-38c0-11ea-b3f5-891068749d18
> for range [(3074457345618258602,-9223372036854775808]] finished (progress:
> 83%)
> [2020-01-17 00:27:52,081] Repair session 327dc140-38c0-11ea-b3f5-891068749d18
> for range [(-3074457345618258603,3074457345618258602]] finished (progress:
> 100%)
> [2020-01-17 00:27:52,086] Repair completed successfully
> [2020-01-17 00:27:52,090] Repair command #2 finished in 0 seconds
> $ for f in *Data.db; do echo $f; sudo /usr/bin/sstablemetadata $f |grep -i
> repaired; done
> md-11-big-Data.db
> Repaired at: 0
> md-12-big-Data.db
> Repaired at: 0
> md-13-big-Data.db
> Repaired at: 1579220735212
> md-14-big-Data.db
> Repaired at: 1579220735212
> md-15-big-Data.db
> Repaired at: 1579220735212
> md-16-big-Data.db
> Repaired at: 1579220735212{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15508) Failing jvm dtest: FailingRepairTest.testFailingMessage
[
https://issues.apache.org/jira/browse/CASSANDRA-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17018433#comment-17018433
]
David Capwell commented on CASSANDRA-15508:
---
Ok, after diving through ant implementation and looking at which processes get
forked; here is a dump of what I see.
1) testclasslist creates a new JVM for each class.
2) unless you override -Dtest.runner the default is 1; testclasslist runs with
1 runner so no concurrent tests (in the same container)
3) current model creates a JVM for each method, but does not support
paramaiterized tests (same method has different names)
4) the test Marcus is working on had one test fail to gossip then the second
test couldn't start since the port was still open; this looks like a resource
leak within a single test class.
Given this, I do feel that its fair to say that in-jvm dtest tests are expected
to run within a single JVM for all the tests in a single class; this is also
how unit tests run. By also having this requirement, we simplify the build
since we don't need the in-jvm tasks, since you can just say
-Dtest.classlistprefix=distributed; this means we don't fragment the build.
Without looking at the code, this is what I changed circle ci to do
{code}
ant testclasslist -Dtest.classlistfile=<( echo
"org/apache/cassandra/distributed/test/ReadRepairTest.java" )
-Dtest.classlistprefix=distributed
{code}
this means that the split and run commands in circle ci can be shared by unit,
distributed, long, and burn.
> Failing jvm dtest: FailingRepairTest.testFailingMessage
> ---
>
> Key: CASSANDRA-15508
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15508
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Marcus Eriksson
>Assignee: David Capwell
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It seems we can't run parameterized unit tests with {{ant testsome}}:
> {code}
> $ ant testsome
> -Dtest.name=org.apache.cassandra.distributed.test.FailingRepairTest
> -Dtest.methods=testFailingMessage
>
> [junit-timeout] Testcase:
> initializationError(org.junit.runner.manipulation.Filter):Caused an ERROR
> [junit-timeout] No tests found matching Method
> testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest)
> from org.junit.internal.requests.ClassRequest@4d95d2a2
> [junit-timeout] java.lang.Exception: No tests found matching Method
> testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest)
> from org.junit.internal.requests.ClassRequest@4d95d2a2
> [junit-timeout] at
> java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15513) Repair -full sets "repairedAt" value
[
https://issues.apache.org/jira/browse/CASSANDRA-15513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-15513:
-
Since Version: 3.11.2
> Repair -full sets "repairedAt" value
>
>
> Key: CASSANDRA-15513
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15513
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sean Fuller
>Priority: Normal
>
> {code:java}
> src/java/org/apache/cassandra/service/ActiveRepairService.java:{code}
> {code:java}
> * we only want to set repairedAt for incremental repairs including all
> replicas for a token range. For non-global
> * incremental repairs, forced incremental repairs, and full repairs, the
> UNREPAIRED_SSTABLE value will prevent
> * sstables from being promoted to repaired or preserve the
> repairedAt/pendingRepair values, respectively.{code}
> Verified in 3.11.5:
> {code:java}
> $ for f in *Data.db; do echo $f; sudo /usr/bin/sstablemetadata $f |grep -i
> repaired; done
> md-10-big-Data.db
> Repaired at: 0
> md-5-big-Data.db
> Repaired at: 0
> md-6-big-Data.db
> Repaired at: 0
> md-7-big-Data.db
> Repaired at: 0
> md-8-big-Data.db
> Repaired at: 0
> md-9-big-Data.db
> Repaired at: 0
> $ nodetool repair -full
> [2020-01-17 00:25:35,192] Starting repair command #1
> (e0edeee0-38bf-11ea-b3f5-891068749d18), repairing keyspace ext_alt_live with
> repair options (parallelism: parallel, primary range: false, incremental:
> false, job threads: 1, ColumnFamilies: [], dataCenters: [], hosts: [], # of
> ranges: 3, pull repair: false)
> [2020-01-17 00:27:51,874] Repair session e1930880-38bf-11ea-b3f5-891068749d18
> for range [(3074457345618258602,-9223372036854775808],
> (-9223372036854775808,-3074457345618258603],
> (-3074457345618258603,3074457345618258602]] finished (progress: 71%)
> [2020-01-17 00:27:51,920] Repair completed successfully
> [2020-01-17 00:27:51,925] Repair command #1 finished in 2 minutes 16 seconds
> [2020-01-17 00:27:51,980] Replication factor is 1. No repair is needed for
> keyspace 'system_auth'
> [2020-01-17 00:27:52,003] Starting repair command #2
> (327adb10-38c0-11ea-b3f5-891068749d18), repairing keyspace system_traces with
> repair options (parallelism: parallel, primary range: false, incremental:
> false, job threads: 1, ColumnFamilies: [], dataCenters: [], hosts: [], # of
> ranges: 2, pull repair: false)
> [2020-01-17 00:27:52,052] Repair session 327d4c10-38c0-11ea-b3f5-891068749d18
> for range [(3074457345618258602,-9223372036854775808]] finished (progress:
> 83%)
> [2020-01-17 00:27:52,081] Repair session 327dc140-38c0-11ea-b3f5-891068749d18
> for range [(-3074457345618258603,3074457345618258602]] finished (progress:
> 100%)
> [2020-01-17 00:27:52,086] Repair completed successfully
> [2020-01-17 00:27:52,090] Repair command #2 finished in 0 seconds
> $ for f in *Data.db; do echo $f; sudo /usr/bin/sstablemetadata $f |grep -i
> repaired; done
> md-11-big-Data.db
> Repaired at: 0
> md-12-big-Data.db
> Repaired at: 0
> md-13-big-Data.db
> Repaired at: 1579220735212
> md-14-big-Data.db
> Repaired at: 1579220735212
> md-15-big-Data.db
> Repaired at: 1579220735212
> md-16-big-Data.db
> Repaired at: 1579220735212{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15496) Declarative Jenkins pipeline builds (and removing `ant test-all`)
[
https://issues.apache.org/jira/browse/CASSANDRA-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-15496:
--
Status: Changes Suggested (was: Review In Progress)
LGTM, once
doc/source/development/testing.rs
gets updated then ready to commit IMO
> Declarative Jenkins pipeline builds (and removing `ant test-all`)
> -
>
> Key: CASSANDRA-15496
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15496
> Project: Cassandra
> Issue Type: Task
> Components: Test/unit
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
> Attachments: Screenshot 2020-01-17 at 21.14.52.png, Screenshot
> 2020-01-17 at 21.16.05.png, Screenshot 2020-01-17 at 21.16.47.png, Screenshot
> 2020-01-17 at 21.18.49.png
>
>
> *Declarative Jenkins pipeline*
> Currently all the jenkins build jobs are generated by the DSL job from the
> cassandra-builds repository.
> For a given branch, there are numerous builds: artifacts, unit tests, and
> dtests.
> Using a (declarative) {{Jenkinsfile}} in-tree these builds can be put
> together into per-branch pipelines.
> Per-branch pipelines will give a nicer UI (via blue ocean) and one place to
> look at test results for any branch and commit. From there we can post one
> complete test summary on commits back to the related jira ticket.
> ref:
> https://lists.apache.org/thread.html/rc8d5a55142ea3bfb742e3347b8ea924946796bad03a1e089c8fb9ee7%40%3Cdev.cassandra.apache.org%3E
> *Removing {{`ant test-all`}}*
> Currently the test-all target does nothing but execute the test target.
> This is because the test targets finishes with unit test failures.
> As these jenkins jobs are automated here:
> https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L44
> ; it is easy enough to remove the "test-all" target, replacing it with
> separate jobs for 'stress-test', 'fqltool-test', and 'long-test'.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15367) Memtable memory allocations may deadlock
[
https://issues.apache.org/jira/browse/CASSANDRA-15367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17018429#comment-17018429
]
Benedict Elliott Smith commented on CASSANDRA-15367:
[~bdeggleston] do you want to polish your patch at least for 3.0, 3.x, and I
guess 4.0 for now? It's definitely the least invasive approach. I'll file a
separate ticket for moving towards removing the lock entirely, and we can
consider if/when we want to address it. I would prefer 4.0 had a "better"
(conceptually) solution to the problem, but it's very much up for debate (one
that can be had on another ticket)
> Memtable memory allocations may deadlock
>
>
> Key: CASSANDRA-15367
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15367
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Commit Log, Local/Memtable
>Reporter: Benedict Elliott Smith
>Assignee: Benedict Elliott Smith
>Priority: Normal
> Fix For: 4.0, 2.2.x, 3.0.x, 3.11.x
>
>
> * Under heavy contention, we guard modifications to a partition with a mutex,
> for the lifetime of the memtable.
> * Memtables block for the completion of all {{OpOrder.Group}} started before
> their flush began
> * Memtables permit operations from this cohort to fall-through to the
> following Memtable, in order to guarantee a precise commitLogUpperBound
> * Memtable memory limits may be lifted for operations in the first cohort,
> since they block flush (and hence block future memory allocation)
> With very unfortunate scheduling
> * A contended partition may rapidly escalate to a mutex
> * The system may reach memory limits that prevent allocations for the new
> Memtable’s cohort (C2)
> * An operation from C2 may hold the mutex when this occurs
> * Operations from a prior Memtable’s cohort (C1), for a contended partition,
> may fall-through to the next Memtable
> * The operations from C1 may execute after the above is encountered by those
> from C2
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15496) Declarative Jenkins pipeline builds (and removing `ant test-all`)
[
https://issues.apache.org/jira/browse/CASSANDRA-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17018426#comment-17018426
]
David Capwell commented on CASSANDRA-15496:
---
The recent changes LGTM, but a small issue
{code}
$ grep -r 'long-testsome' *
build.xml: ant long-testsome -Dtest.name=org.apache.cassandra.cql3.ViewLongTest
-Dtest.methods=testConflictResolution
build.xml:
[davidcapwell ~/src/github/apache/cassandra] (trunk) $ grep -r 'long-test' *
build.xml: ant long-testsome -Dtest.name=org.apache.cassandra.cql3.ViewLongTest
-Dtest.methods=testConflictResolution
build.xml:
build.xml:
build.xml:
depends="eclipse-warnings,test,long-test,test-compression,stress-test,fqltool-test"
doc/source/development/testing.rst:Test that consume a significant amount of
time during execution can be found in the ``test/long`` directory and executed
as a regular JUnit test or standalone program. Except for the execution time,
there’s nothing really special about them. However, ant will execute tests
under ``test/long`` only when using the ``ant long-test`` target.
doc/source/development/testing.rst:Cassandra ships with a default `CircleCI
`_ configuration, to enable running tests on your
branches, you need to go the CircleCI website, click "Login" and log in with
your github account. Then you need to give CircleCI permission to watch your
repositories. Once you have done that, you can optionally configure CircleCI to
run tests in parallel - click "Projects", then your github account and then
click the settings for the project. If you leave the parallelism at 1 for
Cassandra, only ``ant eclipse-warnings`` and ``ant test`` will be run. If you
up the parallelism to 4, it also runs ``ant long-test``, ``ant
test-compression`` and ``ant stress-test``
ide/idea/workspace.xml:
{code}
since you are renaming that task, can you update those two lines in the docs?
Once that is done then +1 from me.
I have a question about this and java 8/11; currently this looks like it would
only be one but do you have plans to run both as one pipeline, or do you see
that as a second pipeline? What I see is that the test results will conflict
when we do that, and not sure if the copy will be overridden (wasn't able to
run since I only had dumb builds).
> Declarative Jenkins pipeline builds (and removing `ant test-all`)
> -
>
> Key: CASSANDRA-15496
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15496
> Project: Cassandra
> Issue Type: Task
> Components: Test/unit
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
> Attachments: Screenshot 2020-01-17 at 21.14.52.png, Screenshot
> 2020-01-17 at 21.16.05.png, Screenshot 2020-01-17 at 21.16.47.png, Screenshot
> 2020-01-17 at 21.18.49.png
>
>
> *Declarative Jenkins pipeline*
> Currently all the jenkins build jobs are generated by the DSL job from the
> cassandra-builds repository.
> For a given branch, there are numerous builds: artifacts, unit tests, and
> dtests.
> Using a (declarative) {{Jenkinsfile}} in-tree these builds can be put
> together into per-branch pipelines.
> Per-branch pipelines will give a nicer UI (via blue ocean) and one place to
> look at test results for any branch and commit. From there we can post one
> complete test summary on commits back to the related jira ticket.
> ref:
> https://lists.apache.org/thread.html/rc8d5a55142ea3bfb742e3347b8ea924946796bad03a1e089c8fb9ee7%40%3Cdev.cassandra.apache.org%3E
> *Removing {{`ant test-all`}}*
> Currently the test-all target does nothing but execute the test target.
> This is because the test targets finishes with unit test failures.
> As these jenkins jobs are automated here:
> https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L44
> ; it is easy enough to remove the "test-all" target, replacing it with
> separate jobs for 'stress-test', 'fqltool-test', and 'long-test'.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-15513) Repair -full sets "repairedAt" value
Sean Fuller created CASSANDRA-15513:
---
Summary: Repair -full sets "repairedAt" value
Key: CASSANDRA-15513
URL: https://issues.apache.org/jira/browse/CASSANDRA-15513
Project: Cassandra
Issue Type: Bug
Reporter: Sean Fuller
{code:java}
src/java/org/apache/cassandra/service/ActiveRepairService.java:{code}
{code:java}
* we only want to set repairedAt for incremental repairs including all replicas
for a token range. For non-global
* incremental repairs, forced incremental repairs, and full repairs, the
UNREPAIRED_SSTABLE value will prevent
* sstables from being promoted to repaired or preserve the
repairedAt/pendingRepair values, respectively.{code}
Verified in 3.11.5:
{code:java}
$ for f in *Data.db; do echo $f; sudo /usr/bin/sstablemetadata $f |grep -i
repaired; done
md-10-big-Data.db
Repaired at: 0
md-5-big-Data.db
Repaired at: 0
md-6-big-Data.db
Repaired at: 0
md-7-big-Data.db
Repaired at: 0
md-8-big-Data.db
Repaired at: 0
md-9-big-Data.db
Repaired at: 0
$ nodetool repair -full
[2020-01-17 00:25:35,192] Starting repair command #1
(e0edeee0-38bf-11ea-b3f5-891068749d18), repairing keyspace ext_alt_live with
repair options (parallelism: parallel, primary range: false, incremental:
false, job threads: 1, ColumnFamilies: [], dataCenters: [], hosts: [], # of
ranges: 3, pull repair: false)
[2020-01-17 00:27:51,874] Repair session e1930880-38bf-11ea-b3f5-891068749d18
for range [(3074457345618258602,-9223372036854775808],
(-9223372036854775808,-3074457345618258603],
(-3074457345618258603,3074457345618258602]] finished (progress: 71%)
[2020-01-17 00:27:51,920] Repair completed successfully
[2020-01-17 00:27:51,925] Repair command #1 finished in 2 minutes 16 seconds
[2020-01-17 00:27:51,980] Replication factor is 1. No repair is needed for
keyspace 'system_auth'
[2020-01-17 00:27:52,003] Starting repair command #2
(327adb10-38c0-11ea-b3f5-891068749d18), repairing keyspace system_traces with
repair options (parallelism: parallel, primary range: false, incremental:
false, job threads: 1, ColumnFamilies: [], dataCenters: [], hosts: [], # of
ranges: 2, pull repair: false)
[2020-01-17 00:27:52,052] Repair session 327d4c10-38c0-11ea-b3f5-891068749d18
for range [(3074457345618258602,-9223372036854775808]] finished (progress: 83%)
[2020-01-17 00:27:52,081] Repair session 327dc140-38c0-11ea-b3f5-891068749d18
for range [(-3074457345618258603,3074457345618258602]] finished (progress: 100%)
[2020-01-17 00:27:52,086] Repair completed successfully
[2020-01-17 00:27:52,090] Repair command #2 finished in 0 seconds
$ for f in *Data.db; do echo $f; sudo /usr/bin/sstablemetadata $f |grep -i
repaired; done
md-11-big-Data.db
Repaired at: 0
md-12-big-Data.db
Repaired at: 0
md-13-big-Data.db
Repaired at: 1579220735212
md-14-big-Data.db
Repaired at: 1579220735212
md-15-big-Data.db
Repaired at: 1579220735212
md-16-big-Data.db
Repaired at: 1579220735212{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14970) New releases must supply SHA-256 and/or SHA-512 checksums
[ https://issues.apache.org/jira/browse/CASSANDRA-14970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17018398#comment-17018398 ] Michael Semb Wever edited comment on CASSANDRA-14970 at 1/17/20 11:23 PM: -- Patches updated at [cassandra|https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/trunk_14970] and [cassandra-builds|https://github.com/apache/cassandra-builds/compare/master...thelastpickle:mck/14970_sha512-checksums]. Still to do… * test rpm docker stuff inside script * test prepare_release.sh * make the patches for 2.2, 3.0, 3.11 was (Author: michaelsembwever): Still to do… * test rpm docker stuff inside script * test prepare_release.sh * make the patches for 2.2, 3.0, 3.11 > New releases must supply SHA-256 and/or SHA-512 checksums > - > > Key: CASSANDRA-14970 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14970 > Project: Cassandra > Issue Type: Bug > Components: Packaging >Reporter: Michael Shuler >Assignee: Michael Semb Wever >Priority: Urgent > Fix For: 2.2.16, 3.0.20, 3.11.6, 4.0 > > Attachments: > 0001-Update-downloads-for-sha256-sha512-checksum-files.patch, > 0001-Update-release-checksum-algorithms-to-SHA-256-SHA-512.patch, > ant-publish-checksum-fail.jpg, build_cassandra-2.1.png, build_trunk.png > > > Release policy was updated around 9/2018 to state: > "For new releases, PMCs MUST supply SHA-256 and/or SHA-512; and SHOULD NOT > supply MD5 or SHA-1. Existing releases do not need to be changed." > build.xml needs to be updated from MD5 & SHA-1 to, at least, SHA-256 or both. > cassandra-builds/cassandra-release scripts need to be updated to work with > the new checksum files. > http://www.apache.org/dev/release-distribution#sigs-and-sums -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14970) New releases must supply SHA-256 and/or SHA-512 checksums
[ https://issues.apache.org/jira/browse/CASSANDRA-14970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17018398#comment-17018398 ] Michael Semb Wever commented on CASSANDRA-14970: Still to do… * test rpm docker stuff inside script * test prepare_release.sh * make the patches for 2.2, 3.0, 3.11 > New releases must supply SHA-256 and/or SHA-512 checksums > - > > Key: CASSANDRA-14970 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14970 > Project: Cassandra > Issue Type: Bug > Components: Packaging >Reporter: Michael Shuler >Assignee: Michael Semb Wever >Priority: Urgent > Fix For: 2.2.16, 3.0.20, 3.11.6, 4.0 > > Attachments: > 0001-Update-downloads-for-sha256-sha512-checksum-files.patch, > 0001-Update-release-checksum-algorithms-to-SHA-256-SHA-512.patch, > ant-publish-checksum-fail.jpg, build_cassandra-2.1.png, build_trunk.png > > > Release policy was updated around 9/2018 to state: > "For new releases, PMCs MUST supply SHA-256 and/or SHA-512; and SHOULD NOT > supply MD5 or SHA-1. Existing releases do not need to be changed." > build.xml needs to be updated from MD5 & SHA-1 to, at least, SHA-256 or both. > cassandra-builds/cassandra-release scripts need to be updated to work with > the new checksum files. > http://www.apache.org/dev/release-distribution#sigs-and-sums -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15496) Declarative Jenkins pipeline builds (and removing `ant test-all`)
[
https://issues.apache.org/jira/browse/CASSANDRA-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-15496:
--
Reviewers: David Capwell, David Capwell (was: David Capwell)
David Capwell, David Capwell
Status: Review In Progress (was: Patch Available)
> Declarative Jenkins pipeline builds (and removing `ant test-all`)
> -
>
> Key: CASSANDRA-15496
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15496
> Project: Cassandra
> Issue Type: Task
> Components: Test/unit
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
> Attachments: Screenshot 2020-01-17 at 21.14.52.png, Screenshot
> 2020-01-17 at 21.16.05.png, Screenshot 2020-01-17 at 21.16.47.png, Screenshot
> 2020-01-17 at 21.18.49.png
>
>
> *Declarative Jenkins pipeline*
> Currently all the jenkins build jobs are generated by the DSL job from the
> cassandra-builds repository.
> For a given branch, there are numerous builds: artifacts, unit tests, and
> dtests.
> Using a (declarative) {{Jenkinsfile}} in-tree these builds can be put
> together into per-branch pipelines.
> Per-branch pipelines will give a nicer UI (via blue ocean) and one place to
> look at test results for any branch and commit. From there we can post one
> complete test summary on commits back to the related jira ticket.
> ref:
> https://lists.apache.org/thread.html/rc8d5a55142ea3bfb742e3347b8ea924946796bad03a1e089c8fb9ee7%40%3Cdev.cassandra.apache.org%3E
> *Removing {{`ant test-all`}}*
> Currently the test-all target does nothing but execute the test target.
> This is because the test targets finishes with unit test failures.
> As these jenkins jobs are automated here:
> https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L44
> ; it is easy enough to remove the "test-all" target, replacing it with
> separate jobs for 'stress-test', 'fqltool-test', and 'long-test'.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15496) Declarative Jenkins pipeline builds (and removing `ant test-all`)
[
https://issues.apache.org/jira/browse/CASSANDRA-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17018344#comment-17018344
]
David Capwell commented on CASSANDRA-15496:
---
Ok, took a stab at reuse using the function and got something working. Turns
out the pipeline DSL is very restrictive and doesn't allow arbitrary groovy.
The Node syntax is pure groovy so able to
{code}
def targets = ['stress-test', 'fqltoolt']
node {
stage('Init') {
cleanWs()
}
stage('Test') {
parallel targets.collectEntries {
["Build ${it}" : runBuild(it)]
}
}
}
def runBuild(target) {
return {
stage(target) {
warnError('Tests unstable') {
build job: "${env.JOB_NAME}-$target"
}
warnError('missing test xml files') {
script {
step ([$class: 'CopyArtifact',
projectName: "${env.JOB_NAME}-$target",
optional: true,
fingerprintArtifacts: true,
selector: [$class: 'StatusBuildSelector', stable:
false],
target: target])
}
unstable {
warnError('missing test xml files') {
script {
step ([$class: 'CopyArtifact',
projectName: "${env.JOB_NAME}-$target",
optional: true,
fingerprintArtifacts: true,
selector: [$class: 'StatusBuildSelector', stable:
false],
target: $target])
}
}
}
}
}
}
}
{code}
> Declarative Jenkins pipeline builds (and removing `ant test-all`)
> -
>
> Key: CASSANDRA-15496
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15496
> Project: Cassandra
> Issue Type: Task
> Components: Test/unit
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
> Attachments: Screenshot 2020-01-17 at 21.14.52.png, Screenshot
> 2020-01-17 at 21.16.05.png, Screenshot 2020-01-17 at 21.16.47.png, Screenshot
> 2020-01-17 at 21.18.49.png
>
>
> *Declarative Jenkins pipeline*
> Currently all the jenkins build jobs are generated by the DSL job from the
> cassandra-builds repository.
> For a given branch, there are numerous builds: artifacts, unit tests, and
> dtests.
> Using a (declarative) {{Jenkinsfile}} in-tree these builds can be put
> together into per-branch pipelines.
> Per-branch pipelines will give a nicer UI (via blue ocean) and one place to
> look at test results for any branch and commit. From there we can post one
> complete test summary on commits back to the related jira ticket.
> ref:
> https://lists.apache.org/thread.html/rc8d5a55142ea3bfb742e3347b8ea924946796bad03a1e089c8fb9ee7%40%3Cdev.cassandra.apache.org%3E
> *Removing {{`ant test-all`}}*
> Currently the test-all target does nothing but execute the test target.
> This is because the test targets finishes with unit test failures.
> As these jenkins jobs are automated here:
> https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L44
> ; it is easy enough to remove the "test-all" target, replacing it with
> separate jobs for 'stress-test', 'fqltool-test', and 'long-test'.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15496) Declarative Jenkins pipeline builds (and removing `ant test-all`)
[
https://issues.apache.org/jira/browse/CASSANDRA-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17018281#comment-17018281
]
Michael Semb Wever edited comment on CASSANDRA-15496 at 1/17/20 8:24 PM:
-
On my own jenkins, putting this together I get…
Cassandra-2.2
!Screenshot 2020-01-17 at 21.14.52.png|width=250!
Cassandra-3.0
!Screenshot 2020-01-17 at 21.16.05.png|width=250!
Cassandra-3.11
!Screenshot 2020-01-17 at 21.16.47.png|width=250!
Cassandra-trunk
!Screenshot 2020-01-17 at 21.18.49.png|width=250!
(the build steps were intentionally cancelled)
was (Author: michaelsembwever):
On my own jenkins, putting this together I get…
Cassandra-2.2
!Screenshot 2020-01-17 at 21.14.52.png|width=250!
Cassandra-3.0
!Screenshot 2020-01-17 at 21.16.05.png|width=250!
Cassandra-3.11
!Screenshot 2020-01-17 at 21.16.47.png|width=250!
Cassandra-trunk
!Screenshot 2020-01-17 at 21.18.49.png|width=250!
> Declarative Jenkins pipeline builds (and removing `ant test-all`)
> -
>
> Key: CASSANDRA-15496
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15496
> Project: Cassandra
> Issue Type: Task
> Components: Test/unit
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
> Attachments: Screenshot 2020-01-17 at 21.14.52.png, Screenshot
> 2020-01-17 at 21.16.05.png, Screenshot 2020-01-17 at 21.16.47.png, Screenshot
> 2020-01-17 at 21.18.49.png
>
>
> *Declarative Jenkins pipeline*
> Currently all the jenkins build jobs are generated by the DSL job from the
> cassandra-builds repository.
> For a given branch, there are numerous builds: artifacts, unit tests, and
> dtests.
> Using a (declarative) {{Jenkinsfile}} in-tree these builds can be put
> together into per-branch pipelines.
> Per-branch pipelines will give a nicer UI (via blue ocean) and one place to
> look at test results for any branch and commit. From there we can post one
> complete test summary on commits back to the related jira ticket.
> ref:
> https://lists.apache.org/thread.html/rc8d5a55142ea3bfb742e3347b8ea924946796bad03a1e089c8fb9ee7%40%3Cdev.cassandra.apache.org%3E
> *Removing {{`ant test-all`}}*
> Currently the test-all target does nothing but execute the test target.
> This is because the test targets finishes with unit test failures.
> As these jenkins jobs are automated here:
> https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L44
> ; it is easy enough to remove the "test-all" target, replacing it with
> separate jobs for 'stress-test', 'fqltool-test', and 'long-test'.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15496) Declarative Jenkins pipeline builds (and removing `ant test-all`)
[
https://issues.apache.org/jira/browse/CASSANDRA-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17018281#comment-17018281
]
Michael Semb Wever edited comment on CASSANDRA-15496 at 1/17/20 8:23 PM:
-
On my own jenkins, putting this together I get…
Cassandra-2.2
!Screenshot 2020-01-17 at 21.14.52.png|width=250!
Cassandra-3.0
!Screenshot 2020-01-17 at 21.16.05.png|width=250!
Cassandra-3.11
!Screenshot 2020-01-17 at 21.16.47.png|width=250!
Cassandra-trunk
!Screenshot 2020-01-17 at 21.18.49.png|width=250!
was (Author: michaelsembwever):
On my own jenkins, putting this together I get…
Cassandra-2.2
!Screenshot 2020-01-17 at 21.14.52.png|thumbnail!
Cassandra-3.0
!Screenshot 2020-01-17 at 21.16.05.png|thumbnail!
Cassandra-3.11
!Screenshot 2020-01-17 at 21.16.47.png|thumbnail!
Cassandra-trunk
!Screenshot 2020-01-17 at 21.18.49.png|thumbnail!
> Declarative Jenkins pipeline builds (and removing `ant test-all`)
> -
>
> Key: CASSANDRA-15496
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15496
> Project: Cassandra
> Issue Type: Task
> Components: Test/unit
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
> Attachments: Screenshot 2020-01-17 at 21.14.52.png, Screenshot
> 2020-01-17 at 21.16.05.png, Screenshot 2020-01-17 at 21.16.47.png, Screenshot
> 2020-01-17 at 21.18.49.png
>
>
> *Declarative Jenkins pipeline*
> Currently all the jenkins build jobs are generated by the DSL job from the
> cassandra-builds repository.
> For a given branch, there are numerous builds: artifacts, unit tests, and
> dtests.
> Using a (declarative) {{Jenkinsfile}} in-tree these builds can be put
> together into per-branch pipelines.
> Per-branch pipelines will give a nicer UI (via blue ocean) and one place to
> look at test results for any branch and commit. From there we can post one
> complete test summary on commits back to the related jira ticket.
> ref:
> https://lists.apache.org/thread.html/rc8d5a55142ea3bfb742e3347b8ea924946796bad03a1e089c8fb9ee7%40%3Cdev.cassandra.apache.org%3E
> *Removing {{`ant test-all`}}*
> Currently the test-all target does nothing but execute the test target.
> This is because the test targets finishes with unit test failures.
> As these jenkins jobs are automated here:
> https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L44
> ; it is easy enough to remove the "test-all" target, replacing it with
> separate jobs for 'stress-test', 'fqltool-test', and 'long-test'.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15496) Declarative Jenkins pipeline builds (and removing `ant test-all`)
[
https://issues.apache.org/jira/browse/CASSANDRA-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17018281#comment-17018281
]
Michael Semb Wever edited comment on CASSANDRA-15496 at 1/17/20 8:21 PM:
-
On my own jenkins, putting this together I get…
Cassandra-2.2
!Screenshot 2020-01-17 at 21.14.52.png|thumbnail!
Cassandra-3.0
!Screenshot 2020-01-17 at 21.16.05.png|thumbnail!
Cassandra-3.11
!Screenshot 2020-01-17 at 21.16.47.png|thumbnail!
Cassandra-trunk
!Screenshot 2020-01-17 at 21.18.49.png|thumbnail!
was (Author: michaelsembwever):
On my own jenkins, putting this together I get…
Cassandra-2.2
!Screenshot 2020-01-17 at 21.14.52.png!
Cassandra-3.0
> Declarative Jenkins pipeline builds (and removing `ant test-all`)
> -
>
> Key: CASSANDRA-15496
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15496
> Project: Cassandra
> Issue Type: Task
> Components: Test/unit
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
> Attachments: Screenshot 2020-01-17 at 21.14.52.png, Screenshot
> 2020-01-17 at 21.16.05.png, Screenshot 2020-01-17 at 21.16.47.png, Screenshot
> 2020-01-17 at 21.18.49.png
>
>
> *Declarative Jenkins pipeline*
> Currently all the jenkins build jobs are generated by the DSL job from the
> cassandra-builds repository.
> For a given branch, there are numerous builds: artifacts, unit tests, and
> dtests.
> Using a (declarative) {{Jenkinsfile}} in-tree these builds can be put
> together into per-branch pipelines.
> Per-branch pipelines will give a nicer UI (via blue ocean) and one place to
> look at test results for any branch and commit. From there we can post one
> complete test summary on commits back to the related jira ticket.
> ref:
> https://lists.apache.org/thread.html/rc8d5a55142ea3bfb742e3347b8ea924946796bad03a1e089c8fb9ee7%40%3Cdev.cassandra.apache.org%3E
> *Removing {{`ant test-all`}}*
> Currently the test-all target does nothing but execute the test target.
> This is because the test targets finishes with unit test failures.
> As these jenkins jobs are automated here:
> https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L44
> ; it is easy enough to remove the "test-all" target, replacing it with
> separate jobs for 'stress-test', 'fqltool-test', and 'long-test'.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15496) Declarative Jenkins pipeline builds (and removing `ant test-all`)
[
https://issues.apache.org/jira/browse/CASSANDRA-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15496:
---
Attachment: Screenshot 2020-01-17 at 21.18.49.png
> Declarative Jenkins pipeline builds (and removing `ant test-all`)
> -
>
> Key: CASSANDRA-15496
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15496
> Project: Cassandra
> Issue Type: Task
> Components: Test/unit
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
> Attachments: Screenshot 2020-01-17 at 21.14.52.png, Screenshot
> 2020-01-17 at 21.16.05.png, Screenshot 2020-01-17 at 21.16.47.png, Screenshot
> 2020-01-17 at 21.18.49.png
>
>
> *Declarative Jenkins pipeline*
> Currently all the jenkins build jobs are generated by the DSL job from the
> cassandra-builds repository.
> For a given branch, there are numerous builds: artifacts, unit tests, and
> dtests.
> Using a (declarative) {{Jenkinsfile}} in-tree these builds can be put
> together into per-branch pipelines.
> Per-branch pipelines will give a nicer UI (via blue ocean) and one place to
> look at test results for any branch and commit. From there we can post one
> complete test summary on commits back to the related jira ticket.
> ref:
> https://lists.apache.org/thread.html/rc8d5a55142ea3bfb742e3347b8ea924946796bad03a1e089c8fb9ee7%40%3Cdev.cassandra.apache.org%3E
> *Removing {{`ant test-all`}}*
> Currently the test-all target does nothing but execute the test target.
> This is because the test targets finishes with unit test failures.
> As these jenkins jobs are automated here:
> https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L44
> ; it is easy enough to remove the "test-all" target, replacing it with
> separate jobs for 'stress-test', 'fqltool-test', and 'long-test'.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15496) Declarative Jenkins pipeline builds (and removing `ant test-all`)
[
https://issues.apache.org/jira/browse/CASSANDRA-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15496:
---
Attachment: Screenshot 2020-01-17 at 21.16.47.png
Screenshot 2020-01-17 at 21.16.05.png
> Declarative Jenkins pipeline builds (and removing `ant test-all`)
> -
>
> Key: CASSANDRA-15496
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15496
> Project: Cassandra
> Issue Type: Task
> Components: Test/unit
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
> Attachments: Screenshot 2020-01-17 at 21.14.52.png, Screenshot
> 2020-01-17 at 21.16.05.png, Screenshot 2020-01-17 at 21.16.47.png
>
>
> *Declarative Jenkins pipeline*
> Currently all the jenkins build jobs are generated by the DSL job from the
> cassandra-builds repository.
> For a given branch, there are numerous builds: artifacts, unit tests, and
> dtests.
> Using a (declarative) {{Jenkinsfile}} in-tree these builds can be put
> together into per-branch pipelines.
> Per-branch pipelines will give a nicer UI (via blue ocean) and one place to
> look at test results for any branch and commit. From there we can post one
> complete test summary on commits back to the related jira ticket.
> ref:
> https://lists.apache.org/thread.html/rc8d5a55142ea3bfb742e3347b8ea924946796bad03a1e089c8fb9ee7%40%3Cdev.cassandra.apache.org%3E
> *Removing {{`ant test-all`}}*
> Currently the test-all target does nothing but execute the test target.
> This is because the test targets finishes with unit test failures.
> As these jenkins jobs are automated here:
> https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L44
> ; it is easy enough to remove the "test-all" target, replacing it with
> separate jobs for 'stress-test', 'fqltool-test', and 'long-test'.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15496) Declarative Jenkins pipeline builds (and removing `ant test-all`)
[
https://issues.apache.org/jira/browse/CASSANDRA-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17018281#comment-17018281
]
Michael Semb Wever commented on CASSANDRA-15496:
On my own jenkins, putting this together I get…
Cassandra-2.2
!Screenshot 2020-01-17 at 21.14.52.png!
Cassandra-3.0
> Declarative Jenkins pipeline builds (and removing `ant test-all`)
> -
>
> Key: CASSANDRA-15496
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15496
> Project: Cassandra
> Issue Type: Task
> Components: Test/unit
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
> Attachments: Screenshot 2020-01-17 at 21.14.52.png
>
>
> *Declarative Jenkins pipeline*
> Currently all the jenkins build jobs are generated by the DSL job from the
> cassandra-builds repository.
> For a given branch, there are numerous builds: artifacts, unit tests, and
> dtests.
> Using a (declarative) {{Jenkinsfile}} in-tree these builds can be put
> together into per-branch pipelines.
> Per-branch pipelines will give a nicer UI (via blue ocean) and one place to
> look at test results for any branch and commit. From there we can post one
> complete test summary on commits back to the related jira ticket.
> ref:
> https://lists.apache.org/thread.html/rc8d5a55142ea3bfb742e3347b8ea924946796bad03a1e089c8fb9ee7%40%3Cdev.cassandra.apache.org%3E
> *Removing {{`ant test-all`}}*
> Currently the test-all target does nothing but execute the test target.
> This is because the test targets finishes with unit test failures.
> As these jenkins jobs are automated here:
> https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L44
> ; it is easy enough to remove the "test-all" target, replacing it with
> separate jobs for 'stress-test', 'fqltool-test', and 'long-test'.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15496) Declarative Jenkins pipeline builds (and removing `ant test-all`)
[
https://issues.apache.org/jira/browse/CASSANDRA-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15496:
---
Attachment: Screenshot 2020-01-17 at 21.14.52.png
> Declarative Jenkins pipeline builds (and removing `ant test-all`)
> -
>
> Key: CASSANDRA-15496
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15496
> Project: Cassandra
> Issue Type: Task
> Components: Test/unit
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
> Attachments: Screenshot 2020-01-17 at 21.14.52.png
>
>
> *Declarative Jenkins pipeline*
> Currently all the jenkins build jobs are generated by the DSL job from the
> cassandra-builds repository.
> For a given branch, there are numerous builds: artifacts, unit tests, and
> dtests.
> Using a (declarative) {{Jenkinsfile}} in-tree these builds can be put
> together into per-branch pipelines.
> Per-branch pipelines will give a nicer UI (via blue ocean) and one place to
> look at test results for any branch and commit. From there we can post one
> complete test summary on commits back to the related jira ticket.
> ref:
> https://lists.apache.org/thread.html/rc8d5a55142ea3bfb742e3347b8ea924946796bad03a1e089c8fb9ee7%40%3Cdev.cassandra.apache.org%3E
> *Removing {{`ant test-all`}}*
> Currently the test-all target does nothing but execute the test target.
> This is because the test targets finishes with unit test failures.
> As these jenkins jobs are automated here:
> https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L44
> ; it is easy enough to remove the "test-all" target, replacing it with
> separate jobs for 'stress-test', 'fqltool-test', and 'long-test'.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15496) Declarative Jenkins pipeline builds (and removing `ant test-all`)
[
https://issues.apache.org/jira/browse/CASSANDRA-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17018244#comment-17018244
]
David Capwell edited comment on CASSANDRA-15496 at 1/17/20 7:11 PM:
I am new to Jenkinsfile but the docs look like its groovy. What I see in the
test section is each test is copy/paste with a change in the build target name,
I am wondering if this could be extracted into a function which then lets you
define the target to generate for?
With that, I think the body of test could be
{code}
stage('Test') {
parallel {
runBuild("stress-test")
runBuild("fqltoo-testl")
runBuild("test-jvm-dtest-forking")
...
}
}
{code}
was (Author: dcapwell):
I am new to Jenkinsfile but the docs look like its groovy. What I see in the
test section is each test is copy/paste with a change in the build target name,
I am wondering if this could be extracted into a function which then lets you
define the target to generate for?
> Declarative Jenkins pipeline builds (and removing `ant test-all`)
> -
>
> Key: CASSANDRA-15496
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15496
> Project: Cassandra
> Issue Type: Task
> Components: Test/unit
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
>
> *Declarative Jenkins pipeline*
> Currently all the jenkins build jobs are generated by the DSL job from the
> cassandra-builds repository.
> For a given branch, there are numerous builds: artifacts, unit tests, and
> dtests.
> Using a (declarative) {{Jenkinsfile}} in-tree these builds can be put
> together into per-branch pipelines.
> Per-branch pipelines will give a nicer UI (via blue ocean) and one place to
> look at test results for any branch and commit. From there we can post one
> complete test summary on commits back to the related jira ticket.
> ref:
> https://lists.apache.org/thread.html/rc8d5a55142ea3bfb742e3347b8ea924946796bad03a1e089c8fb9ee7%40%3Cdev.cassandra.apache.org%3E
> *Removing {{`ant test-all`}}*
> Currently the test-all target does nothing but execute the test target.
> This is because the test targets finishes with unit test failures.
> As these jenkins jobs are automated here:
> https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L44
> ; it is easy enough to remove the "test-all" target, replacing it with
> separate jobs for 'stress-test', 'fqltool-test', and 'long-test'.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15496) Declarative Jenkins pipeline builds (and removing `ant test-all`)
[
https://issues.apache.org/jira/browse/CASSANDRA-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17018244#comment-17018244
]
David Capwell commented on CASSANDRA-15496:
---
I am new to Jenkinsfile but the docs look like its groovy. What I see in the
test section is each test is copy/paste with a change in the build target name,
I am wondering if this could be extracted into a function which then lets you
define the target to generate for?
> Declarative Jenkins pipeline builds (and removing `ant test-all`)
> -
>
> Key: CASSANDRA-15496
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15496
> Project: Cassandra
> Issue Type: Task
> Components: Test/unit
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
>
> *Declarative Jenkins pipeline*
> Currently all the jenkins build jobs are generated by the DSL job from the
> cassandra-builds repository.
> For a given branch, there are numerous builds: artifacts, unit tests, and
> dtests.
> Using a (declarative) {{Jenkinsfile}} in-tree these builds can be put
> together into per-branch pipelines.
> Per-branch pipelines will give a nicer UI (via blue ocean) and one place to
> look at test results for any branch and commit. From there we can post one
> complete test summary on commits back to the related jira ticket.
> ref:
> https://lists.apache.org/thread.html/rc8d5a55142ea3bfb742e3347b8ea924946796bad03a1e089c8fb9ee7%40%3Cdev.cassandra.apache.org%3E
> *Removing {{`ant test-all`}}*
> Currently the test-all target does nothing but execute the test target.
> This is because the test targets finishes with unit test failures.
> As these jenkins jobs are automated here:
> https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L44
> ; it is easy enough to remove the "test-all" target, replacing it with
> separate jobs for 'stress-test', 'fqltool-test', and 'long-test'.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15496) Declarative Jenkins pipeline builds (and removing `ant test-all`)
[
https://issues.apache.org/jira/browse/CASSANDRA-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15496:
---
Status: Patch Available (was: In Progress)
> Declarative Jenkins pipeline builds (and removing `ant test-all`)
> -
>
> Key: CASSANDRA-15496
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15496
> Project: Cassandra
> Issue Type: Task
> Components: Test/unit
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
>
> *Declarative Jenkins pipeline*
> Currently all the jenkins build jobs are generated by the DSL job from the
> cassandra-builds repository.
> For a given branch, there are numerous builds: artifacts, unit tests, and
> dtests.
> Using a (declarative) {{Jenkinsfile}} in-tree these builds can be put
> together into per-branch pipelines.
> Per-branch pipelines will give a nicer UI (via blue ocean) and one place to
> look at test results for any branch and commit. From there we can post one
> complete test summary on commits back to the related jira ticket.
> ref:
> https://lists.apache.org/thread.html/rc8d5a55142ea3bfb742e3347b8ea924946796bad03a1e089c8fb9ee7%40%3Cdev.cassandra.apache.org%3E
> *Removing {{`ant test-all`}}*
> Currently the test-all target does nothing but execute the test target.
> This is because the test targets finishes with unit test failures.
> As these jenkins jobs are automated here:
> https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L44
> ; it is easy enough to remove the "test-all" target, replacing it with
> separate jobs for 'stress-test', 'fqltool-test', and 'long-test'.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15508) Failing jvm dtest: FailingRepairTest.testFailingMessage
[
https://issues.apache.org/jira/browse/CASSANDRA-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17018223#comment-17018223
]
David Capwell commented on CASSANDRA-15508:
---
spoke with [~marcuse] and one issue with this patch is that his patch he is
working on (upgrade tests) fails. Ill try to rework this so it works with his
patch; ill also keep upgrade as separate and move to its own runner so it can
use different resources than normal jvm dtest.
> Failing jvm dtest: FailingRepairTest.testFailingMessage
> ---
>
> Key: CASSANDRA-15508
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15508
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Marcus Eriksson
>Assignee: David Capwell
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It seems we can't run parameterized unit tests with {{ant testsome}}:
> {code}
> $ ant testsome
> -Dtest.name=org.apache.cassandra.distributed.test.FailingRepairTest
> -Dtest.methods=testFailingMessage
>
> [junit-timeout] Testcase:
> initializationError(org.junit.runner.manipulation.Filter):Caused an ERROR
> [junit-timeout] No tests found matching Method
> testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest)
> from org.junit.internal.requests.ClassRequest@4d95d2a2
> [junit-timeout] java.lang.Exception: No tests found matching Method
> testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest)
> from org.junit.internal.requests.ClassRequest@4d95d2a2
> [junit-timeout] at
> java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15496) Declarative Jenkins pipeline builds (and removing `ant test-all`)
[
https://issues.apache.org/jira/browse/CASSANDRA-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17018208#comment-17018208
]
Michael Semb Wever edited comment on CASSANDRA-15496 at 1/17/20 5:59 PM:
-
||branch||circleci||jenkins pipeline||jenkins dtests||
|[cassandra-2.2_15496|https://github.com/apache/cassandra/compare/cassandra-2.2...thelastpickle:mck/cassandra-2.2_15496]|[circleci|https://circleci.com/gh/thelastpickle/workflows/cassandra/tree/mck%2Fcassandra-2.2_15496]|[!https://builds.apache.org/job/Cassandra-devbranch/5/badge/icon!|https://builds.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/5]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/726/badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/726]|
|[cassandra-3.0_15496|https://github.com/apache/cassandra/compare/cassandra-3.0...thelastpickle:mck/cassandra-3.0_15496]|[circleci|https://circleci.com/gh/thelastpickle/workflows/cassandra/tree/mck%2Fcassandra-3.0_15496]|[!https://builds.apache.org/job/Cassandra-devbranch/2/badge/icon!|https://builds.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/2]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/723/badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/723]|
|[cassandra-3.11_15496|https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:mck/cassandra-3.11_15496]|[circleci|https://circleci.com/gh/thelastpickle/workflows/cassandra/tree/mck%2Fcassandra-3.11_15496]|[!https://builds.apache.org/job/Cassandra-devbranch-pipeline/3/badge/icon!|https://builds.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/3]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/724/badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/724]|
|[trunk_15496|https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/trunk_15496]|[circleci|https://circleci.com/gh/thelastpickle/workflows/cassandra/tree/mck%2Ftrunk_15496]|[!https://builds.apache.org/job/Cassandra-devbranch-pipeline/4/badge/icon!|https://builds.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/4]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/725/badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/725]|
The additional {{cassandra-builds}} is in
https://github.com/apache/cassandra-builds/compare/master...thelastpickle:mck/jenkins-dsl-pipeline
, but can not be pushed until the in-tree {{Jenkinsfile}} files are in place.
was (Author: michaelsembwever):
||branch||circleci||jenkins pipeline||jenkins dtests||
|[cassandra-2.2_15496|https://github.com/apache/cassandra/compare/cassandra-2.2...thelastpickle:mck/cassandra-2.2_15496]|[circleci|https://circleci.com/gh/thelastpickle/workflows/cassandra/tree/mck%2Fcassandra-2.2_15496]|[!https://builds.apache.org/job/Cassandra-devbranch/5/badge/icon!|https://builds.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/5]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/726/badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/726]|
|[cassandra-3.0_15496|https://github.com/apache/cassandra/compare/cassandra-3.0...thelastpickle:mck/cassandra-3.0_15496]|[circleci|https://circleci.com/gh/thelastpickle/workflows/cassandra/tree/mck%2Fcassandra-3.0_15496]|[!https://builds.apache.org/job/Cassandra-devbranch/2/badge/icon!|https://builds.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/2]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/723/badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/723]|
|[cassandra-3.11_15496|https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:mck/cassandra-3.11_15496]|[circleci|https://circleci.com/gh/thelastpickle/workflows/cassandra/tree/mck%2Fcassandra-3.11_15496]|[!https://builds.apache.org/job/Cassandra-devbranch-pipeline/3/badge/icon!|https://builds.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/3]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/724/badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/724]|
|[trunk_15496|https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/trunk_15496]|[circleci|https://circleci.com/gh/thelastpickle/workflows/cassandra/tree/mck%2Ftrunk_15496]|[!https://builds.apache.org/job/Cassandra-devbranch-pipeline/4/badge/icon!|https://builds.apac
[jira] [Assigned] (CASSANDRA-15307) Fix flakey test_remote_query - cql_test.TestCQLSlowQuery test
[
https://issues.apache.org/jira/browse/CASSANDRA-15307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova reassigned CASSANDRA-15307:
---
Assignee: Ekaterina Dimitrova
> Fix flakey test_remote_query - cql_test.TestCQLSlowQuery test
> --
>
> Key: CASSANDRA-15307
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15307
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: Joey Lynch
>Assignee: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.0-alpha
>
>
> Example failure:
> [https://circleci.com/gh/jolynch/cassandra/554#tests/containers/61]
>
> {noformat}
> Your job ran 959 tests with 1 failure
> - test_remote_query cql_test.TestCQLSlowQuerycql_test.py
> ccmlib.node.TimeoutError: 05 Sep 2019 23:05:07 [node2] Missing: ['operations
> were slow', 'SELECT \\* FROM ks.test2 WHERE id = 1']: DEBUG [BatchlogTasks:1]
> 2019-09-05 23:04:24,437 Ba. See debug.log for remainder
> self =
> def test_remote_query(self):
> """
> Check that a query running on a node other than the coordinator
> is reported as slow:
>
> - populate the cluster with 2 nodes
> - start one node without having it join the ring
> - start the other one node with slow_query_log_timeout_in_ms set
> to a small value
> and the read request timeouts set to a large value (to ensure
> the query is not aborted) and
> read_iteration_delay set to a value big enough for the query to
> exceed slow_query_log_timeout_in_ms
> (this will cause read queries to take longer than the slow
> query timeout)
> - CREATE a table
> - INSERT 5000 rows on a session on the node that is not a member
> of the ring
> - run SELECT statements and check that the slow query messages
> are present in the debug logs
> (we cannot check the logs at info level because the no spam
> logger has unpredictable results)
>
> @jira_ticket CASSANDRA-12403
> """
> cluster = self.cluster
>
> cluster.set_configuration_options(values={'slow_query_log_timeout_in_ms': 10,
> 'request_timeout_in_ms':
> 12,
>
> 'read_request_timeout_in_ms': 12,
>
> 'range_request_timeout_in_ms': 12})
>
> cluster.populate(2)
> node1, node2 = cluster.nodelist()
>
> node1.start(wait_for_binary_proto=True, join_ring=False) # ensure
> other node executes queries
> node2.start(wait_for_binary_proto=True,
> jvm_args=["-Dcassandra.monitoring_report_interval_ms=10",
> "-Dcassandra.test.read_iteration_delay_ms=1"])
> # see above for explanation
>
> session = self.patient_exclusive_cql_connection(node1)
>
> create_ks(session, 'ks', 1)
> session.execute("""
> CREATE TABLE test2 (
> id int,
> col int,
> val text,
> PRIMARY KEY(id, col)
> );
> """)
>
> for i, j in itertools.product(list(range(100)), list(range(10))):
> session.execute("INSERT INTO test2 (id, col, val) VALUES ({}, {},
> 'foo')".format(i, j))
>
> # only check debug logs because at INFO level the no-spam logger has
> unpredictable results
> mark = node2.mark_log(filename='debug.log')
> session.execute(SimpleStatement("SELECT * from test2",
>
> consistency_level=ConsistencyLevel.ONE,
>
> retry_policy=FallthroughRetryPolicy()))
> node2.watch_log_for(["operations were slow", "SELECT \* FROM
> ks.test2"],
> from_mark=mark, filename='debug.log', timeout=60)
>
>
> mark = node2.mark_log(filename='debug.log')
> session.execute(SimpleStatement("SELECT * from test2 where id = 1",
>
> consistency_level=ConsistencyLevel.ONE,
>
> retry_policy=FallthroughRetryPolicy()))
> node2.watch_log_for(["operations were slow", "SELECT \* FROM ks.test2
> WHERE id = 1"],
> > from_mark=mark, filename='debug.log', timeout=60)
> cql_test.py:1150:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> self =
> exprs = ['operations were slow', 'SELECT \\* FROM ks.test2 WHERE id = 1']
> from_mark = 166214, timeout = 60, process = None, verbos
[jira] [Commented] (CASSANDRA-15496) Declarative Jenkins pipeline builds (and removing `ant test-all`)
[
https://issues.apache.org/jira/browse/CASSANDRA-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17018208#comment-17018208
]
Michael Semb Wever commented on CASSANDRA-15496:
||branch||circleci||jenkins pipeline||jenkins dtests||
|[cassandra-2.2_15496|https://github.com/apache/cassandra/compare/cassandra-2.2...thelastpickle:mck/cassandra-2.2_15496]|[circleci|https://circleci.com/gh/thelastpickle/workflows/cassandra/tree/mck%2Fcassandra-2.2_15496]|[!https://builds.apache.org/job/Cassandra-devbranch/5/badge/icon!|https://builds.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/5]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/726/badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/726]|
|[cassandra-3.0_15496|https://github.com/apache/cassandra/compare/cassandra-3.0...thelastpickle:mck/cassandra-3.0_15496]|[circleci|https://circleci.com/gh/thelastpickle/workflows/cassandra/tree/mck%2Fcassandra-3.0_15496]|[!https://builds.apache.org/job/Cassandra-devbranch/2/badge/icon!|https://builds.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/2]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/723/badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/723]|
|[cassandra-3.11_15496|https://github.com/apache/cassandra/compare/cassandra-3.11...thelastpickle:mck/cassandra-3.11_15496]|[circleci|https://circleci.com/gh/thelastpickle/workflows/cassandra/tree/mck%2Fcassandra-3.11_15496]|[!https://builds.apache.org/job/Cassandra-devbranch-pipeline/3/badge/icon!|https://builds.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/3]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/724/badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/724]|
|[trunk_15496|https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/trunk_15496]|[circleci|https://circleci.com/gh/thelastpickle/workflows/cassandra/tree/mck%2Ftrunk_15496]|[!https://builds.apache.org/job/Cassandra-devbranch-pipeline/4/badge/icon!|https://builds.apache.org/blue/organizations/jenkins/Cassandra-devbranch/detail/Cassandra-devbranch/4]|[!https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/725/badge/icon!|https://builds.apache.org/view/A-D/view/Cassandra/job/Cassandra-devbranch-dtest/725]|
> Declarative Jenkins pipeline builds (and removing `ant test-all`)
> -
>
> Key: CASSANDRA-15496
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15496
> Project: Cassandra
> Issue Type: Task
> Components: Test/unit
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
>
> *Declarative Jenkins pipeline*
> Currently all the jenkins build jobs are generated by the DSL job from the
> cassandra-builds repository.
> For a given branch, there are numerous builds: artifacts, unit tests, and
> dtests.
> Using a (declarative) {{Jenkinsfile}} in-tree these builds can be put
> together into per-branch pipelines.
> Per-branch pipelines will give a nicer UI (via blue ocean) and one place to
> look at test results for any branch and commit. From there we can post one
> complete test summary on commits back to the related jira ticket.
> ref:
> https://lists.apache.org/thread.html/rc8d5a55142ea3bfb742e3347b8ea924946796bad03a1e089c8fb9ee7%40%3Cdev.cassandra.apache.org%3E
> *Removing {{`ant test-all`}}*
> Currently the test-all target does nothing but execute the test target.
> This is because the test targets finishes with unit test failures.
> As these jenkins jobs are automated here:
> https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L44
> ; it is easy enough to remove the "test-all" target, replacing it with
> separate jobs for 'stress-test', 'fqltool-test', and 'long-test'.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15456) Backport CASSANDRA-15332 dtest changes to 3.x
[ https://issues.apache.org/jira/browse/CASSANDRA-15456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17018196#comment-17018196 ] David Capwell commented on CASSANDRA-15456: --- Should include CASSANDRA-15450 > Backport CASSANDRA-15332 dtest changes to 3.x > - > > Key: CASSANDRA-15456 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15456 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > > When CASSANDRA-15332 was merged there was no work to back port the dtest > changes to 3.x. As of this moment, we want to make sure 3.x and 4.x dtest > share the same API, so since CASSANDRA-15332 changed the API we need to make > sure those changes are reflected in 3.x -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15505) Add message interceptors to in-jvm dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-15505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17018158#comment-17018158 ] David Capwell commented on CASSANDRA-15505: --- Looks like I was remembering when I was testing all messages but only validation got merged https://github.com/apache/cassandra/blob/trunk/test/distributed/org/apache/cassandra/distributed/test/FailingRepairTest.java When I saw prepare and the other message (don’t remember name...) I changed the implementation to fail the request. Why I call it out, the messaging service filters are very useful so if I was going to bring back the tests I didn’t merge I would need to ask myself “messaging service or this”. If you have a use case to collect/consume, I would prefer that isn’t a primitive and was syntactic sugar on a lower level feature; maybe something like messaging service filters? With that verb filtering could be the same. > Add message interceptors to in-jvm dtests > - > > Key: CASSANDRA-15505 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15505 > Project: Cassandra > Issue Type: New Feature > Components: Test/dtest >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > > Currently we only have means to filter messages in in-jvm tests. We need a > facility to intercept and modify the messages between nodes for testing > purposes. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15496) Declarative Jenkins pipeline builds (and removing `ant test-all`)
[
https://issues.apache.org/jira/browse/CASSANDRA-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15496:
---
Status: In Progress (was: Patch Available)
> Declarative Jenkins pipeline builds (and removing `ant test-all`)
> -
>
> Key: CASSANDRA-15496
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15496
> Project: Cassandra
> Issue Type: Task
> Components: Test/unit
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
>
> *Declarative Jenkins pipeline*
> Currently all the jenkins build jobs are generated by the DSL job from the
> cassandra-builds repository.
> For a given branch, there are numerous builds: artifacts, unit tests, and
> dtests.
> Using a (declarative) {{Jenkinsfile}} in-tree these builds can be put
> together into per-branch pipelines.
> Per-branch pipelines will give a nicer UI (via blue ocean) and one place to
> look at test results for any branch and commit. From there we can post one
> complete test summary on commits back to the related jira ticket.
> ref:
> https://lists.apache.org/thread.html/rc8d5a55142ea3bfb742e3347b8ea924946796bad03a1e089c8fb9ee7%40%3Cdev.cassandra.apache.org%3E
> *Removing {{`ant test-all`}}*
> Currently the test-all target does nothing but execute the test target.
> This is because the test targets finishes with unit test failures.
> As these jenkins jobs are automated here:
> https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L44
> ; it is easy enough to remove the "test-all" target, replacing it with
> separate jobs for 'stress-test', 'fqltool-test', and 'long-test'.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14781) Log message when mutation passed to CommitLog#add(Mutation) is too large is not descriptive enough
[ https://issues.apache.org/jira/browse/CASSANDRA-14781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17018144#comment-17018144 ] Jordan West commented on CASSANDRA-14781: - Thanks [~n.v.harikrishna]. I think the duplicate code is probably simpler than making things more nuanced and thanks for testing the sorting performance. The code looks ok me and I am ok w/ the separate ticket approach so that there is some improvements for operators at a minimum but I would like to get [~aleksey]'s input as well. > Log message when mutation passed to CommitLog#add(Mutation) is too large is > not descriptive enough > -- > > Key: CASSANDRA-14781 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14781 > Project: Cassandra > Issue Type: Bug > Components: Consistency/Hints, Local/Commit Log, Messaging/Client >Reporter: Jordan West >Assignee: Tom Petracca >Priority: Normal > Labels: protocolv5 > Fix For: 4.0-beta > > Attachments: CASSANDRA-14781.patch, CASSANDRA-14781_3.0.patch, > CASSANDRA-14781_3.11.patch > > > When hitting > [https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/commitlog/CommitLog.java#L256-L257], > the log message produced does not help the operator track down what data is > being written. At a minimum the keyspace and cfIds involved would be useful > (and are available) – more detail might not be reasonable to include. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15508) Failing jvm dtest: FailingRepairTest.testFailingMessage
[
https://issues.apache.org/jira/browse/CASSANDRA-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17018117#comment-17018117
]
David Capwell commented on CASSANDRA-15508:
---
That’s fair; my Main thoughts were that I hope it’s short term (there is a Jira
to publish the jars), and it simplifies the build. If it’s desired I can keep
it the same way (two builds)
No idea what has changed, but testclasslist is how I run jvm dtests so know it
works for 3.0 and trunk.
> Failing jvm dtest: FailingRepairTest.testFailingMessage
> ---
>
> Key: CASSANDRA-15508
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15508
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Marcus Eriksson
>Assignee: David Capwell
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It seems we can't run parameterized unit tests with {{ant testsome}}:
> {code}
> $ ant testsome
> -Dtest.name=org.apache.cassandra.distributed.test.FailingRepairTest
> -Dtest.methods=testFailingMessage
>
> [junit-timeout] Testcase:
> initializationError(org.junit.runner.manipulation.Filter):Caused an ERROR
> [junit-timeout] No tests found matching Method
> testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest)
> from org.junit.internal.requests.ClassRequest@4d95d2a2
> [junit-timeout] java.lang.Exception: No tests found matching Method
> testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest)
> from org.junit.internal.requests.ClassRequest@4d95d2a2
> [junit-timeout] at
> java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15496) Declarative Jenkins pipeline builds (and removing `ant test-all`)
[
https://issues.apache.org/jira/browse/CASSANDRA-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15496:
---
Description:
*Declarative Jenkins pipeline*
Currently all the jenkins build jobs are generated by the DSL job from the
cassandra-builds repository.
For a given branch, there are numerous builds: artifacts, unit tests, and
dtests.
Using a (declarative) {{Jenkinsfile}} in-tree these builds can be put together
into per-branch pipelines.
Per-branch pipelines will give a nicer UI (via blue ocean) and one place to
look at test results for any branch and commit. From there we can post one
complete test summary on commits back to the related jira ticket.
ref:
https://lists.apache.org/thread.html/rc8d5a55142ea3bfb742e3347b8ea924946796bad03a1e089c8fb9ee7%40%3Cdev.cassandra.apache.org%3E
*Removing {{`ant test-all`}}*
Currently the test-all target does nothing but execute the test target.
This is because the test targets finishes with unit test failures.
As these jenkins jobs are automated here:
https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L44
; it is easy enough to remove the "test-all" target, replacing it with
separate jobs for 'stress-test', 'fqltool-test', and 'long-test'.
was:
*Declarative Jenkins pipeline*
Currently all the jenkins build jobs are generated by the DSL job from the
cassandra-builds repository.
For a given branch, there are numerous builds: artifacts, unit tests, and
dtests.
Using a (declarative) {{Jenkinsfile}} in-tree these builds can be put together
into per-branch pipelines.
Per-branch pipelines will give a nicer UI (via blue ocean) and one place to
look at test results for any branch and commit.
*Removing {{`ant test-all`}}*
Currently the test-all target does nothing but execute the test target.
This is because the test targets finishes with unit test failures.
As these jenkins jobs are automated here:
https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L44
; it is easy enough to remove the "test-all" target, replacing it with
separate jobs for 'stress-test', 'fqltool-test', and 'long-test'.
> Declarative Jenkins pipeline builds (and removing `ant test-all`)
> -
>
> Key: CASSANDRA-15496
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15496
> Project: Cassandra
> Issue Type: Task
> Components: Test/unit
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 4.x
>
>
> *Declarative Jenkins pipeline*
> Currently all the jenkins build jobs are generated by the DSL job from the
> cassandra-builds repository.
> For a given branch, there are numerous builds: artifacts, unit tests, and
> dtests.
> Using a (declarative) {{Jenkinsfile}} in-tree these builds can be put
> together into per-branch pipelines.
> Per-branch pipelines will give a nicer UI (via blue ocean) and one place to
> look at test results for any branch and commit. From there we can post one
> complete test summary on commits back to the related jira ticket.
> ref:
> https://lists.apache.org/thread.html/rc8d5a55142ea3bfb742e3347b8ea924946796bad03a1e089c8fb9ee7%40%3Cdev.cassandra.apache.org%3E
> *Removing {{`ant test-all`}}*
> Currently the test-all target does nothing but execute the test target.
> This is because the test targets finishes with unit test failures.
> As these jenkins jobs are automated here:
> https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L44
> ; it is easy enough to remove the "test-all" target, replacing it with
> separate jobs for 'stress-test', 'fqltool-test', and 'long-test'.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15496) Declarative Jenkins pipeline builds (and removing `ant test-all`)
[
https://issues.apache.org/jira/browse/CASSANDRA-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15496:
---
Description:
*Declarative Jenkins pipeline*
Currently all the jenkins build jobs are generated by the DSL job from the
cassandra-builds repository.
For a given branch, there are numerous builds: artifacts, unit tests, and
dtests.
Using a (declarative) {{Jenkinsfile}} in-tree these builds can be put together
into per-branch pipelines.
Per-branch pipelines will give a nicer UI (via blue ocean) and one place to
look at test results for any branch and commit.
*Removing {{`ant test-all`}}*
Currently the test-all target does nothing but execute the test target.
This is because the test targets finishes with unit test failures.
As these jenkins jobs are automated here:
https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L44
; it is easy enough to remove the "test-all" target, replacing it with
separate jobs for 'stress-test', 'fqltool-test', and 'long-test'.
was:
Currently the test-all target does nothing but execute the test target.
This is because the test targets finishes with unit test failures.
As these jenkins jobs are automated here:
https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L44
; it is easy enough to remove the "test-all" target, replacing it with
separate jobs for 'stress-test', 'fqltool-test', and 'long-test'.
> Declarative Jenkins pipeline builds (and removing `ant test-all`)
> -
>
> Key: CASSANDRA-15496
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15496
> Project: Cassandra
> Issue Type: Task
> Components: Test/unit
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
>
> *Declarative Jenkins pipeline*
> Currently all the jenkins build jobs are generated by the DSL job from the
> cassandra-builds repository.
> For a given branch, there are numerous builds: artifacts, unit tests, and
> dtests.
> Using a (declarative) {{Jenkinsfile}} in-tree these builds can be put
> together into per-branch pipelines.
> Per-branch pipelines will give a nicer UI (via blue ocean) and one place to
> look at test results for any branch and commit.
> *Removing {{`ant test-all`}}*
> Currently the test-all target does nothing but execute the test target.
> This is because the test targets finishes with unit test failures.
> As these jenkins jobs are automated here:
> https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L44
> ; it is easy enough to remove the "test-all" target, replacing it with
> separate jobs for 'stress-test', 'fqltool-test', and 'long-test'.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15496) Declarative Jenkins pipeline builds (and removing `ant test-all`)
[
https://issues.apache.org/jira/browse/CASSANDRA-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15496:
---
Fix Version/s: 3.11.x
3.0.x
2.2.x
> Declarative Jenkins pipeline builds (and removing `ant test-all`)
> -
>
> Key: CASSANDRA-15496
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15496
> Project: Cassandra
> Issue Type: Task
> Components: Test/unit
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.x
>
>
> *Declarative Jenkins pipeline*
> Currently all the jenkins build jobs are generated by the DSL job from the
> cassandra-builds repository.
> For a given branch, there are numerous builds: artifacts, unit tests, and
> dtests.
> Using a (declarative) {{Jenkinsfile}} in-tree these builds can be put
> together into per-branch pipelines.
> Per-branch pipelines will give a nicer UI (via blue ocean) and one place to
> look at test results for any branch and commit. From there we can post one
> complete test summary on commits back to the related jira ticket.
> ref:
> https://lists.apache.org/thread.html/rc8d5a55142ea3bfb742e3347b8ea924946796bad03a1e089c8fb9ee7%40%3Cdev.cassandra.apache.org%3E
> *Removing {{`ant test-all`}}*
> Currently the test-all target does nothing but execute the test target.
> This is because the test targets finishes with unit test failures.
> As these jenkins jobs are automated here:
> https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L44
> ; it is easy enough to remove the "test-all" target, replacing it with
> separate jobs for 'stress-test', 'fqltool-test', and 'long-test'.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15496) Declarative Jenkins pipeline builds (and removing `ant test-all`)
[
https://issues.apache.org/jira/browse/CASSANDRA-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Semb Wever updated CASSANDRA-15496:
---
Fix Version/s: 4.x
> Declarative Jenkins pipeline builds (and removing `ant test-all`)
> -
>
> Key: CASSANDRA-15496
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15496
> Project: Cassandra
> Issue Type: Task
> Components: Test/unit
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 4.x
>
>
> *Declarative Jenkins pipeline*
> Currently all the jenkins build jobs are generated by the DSL job from the
> cassandra-builds repository.
> For a given branch, there are numerous builds: artifacts, unit tests, and
> dtests.
> Using a (declarative) {{Jenkinsfile}} in-tree these builds can be put
> together into per-branch pipelines.
> Per-branch pipelines will give a nicer UI (via blue ocean) and one place to
> look at test results for any branch and commit.
> *Removing {{`ant test-all`}}*
> Currently the test-all target does nothing but execute the test target.
> This is because the test targets finishes with unit test failures.
> As these jenkins jobs are automated here:
> https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L44
> ; it is easy enough to remove the "test-all" target, replacing it with
> separate jobs for 'stress-test', 'fqltool-test', and 'long-test'.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15496) Declarative Jenkins pipeline builds (and removing `ant test-all`)
[ https://issues.apache.org/jira/browse/CASSANDRA-15496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-15496: --- Summary: Declarative Jenkins pipeline builds (and removing `ant test-all`) (was: Split out Jenkins test-all builds to individual builds for each of the test targets) > Declarative Jenkins pipeline builds (and removing `ant test-all`) > - > > Key: CASSANDRA-15496 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15496 > Project: Cassandra > Issue Type: Task > Components: Test/unit >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > > Currently the test-all target does nothing but execute the test target. > This is because the test targets finishes with unit test failures. > As these jenkins jobs are automated here: > https://github.com/apache/cassandra-builds/blob/master/jenkins-dsl/cassandra_job_dsl_seed.groovy#L44 > ; it is easy enough to remove the "test-all" target, replacing it with > separate jobs for 'stress-test', 'fqltool-test', and 'long-test'. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15507) Test org.apache.cassandra.distributed.test.DistributedReadWritePathTest#failingReadRepairTest does not test a failing read repair and should be updated to actually t
[ https://issues.apache.org/jira/browse/CASSANDRA-15507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Petrov updated CASSANDRA-15507: Fix Version/s: 4.0 Since Version: 4.0 Source Control Link: https://github.com/apache/cassandra/commit/815c397f4876aae9ed2ae5a9578c5ec7087643ab Resolution: Fixed Status: Resolved (was: Ready to Commit) > Test > org.apache.cassandra.distributed.test.DistributedReadWritePathTest#failingReadRepairTest > does not test a failing read repair and should be updated to actually > trigger a failed read repair > > > Key: CASSANDRA-15507 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15507 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Fix For: 4.0 > > Time Spent: 20m > Remaining Estimate: 0h > > The test > org.apache.cassandra.distributed.test.DistributedReadWritePathTest#failingReadRepairTest > makes a few assumptions which are not valid at the moment. > 1) the write to node 1 and 2 have the same digest (they don’t, this is caused > by the timestamp being different) > 2) node 3 will participate with the read; it won’t give the fact that > org.apache.cassandra.locator.ReplicaPlans#contactForRead will speculate the > first 2 nodes always, so node 3 won’t get involved with the repair > 3) node 3 will attempt to get repaired (it won’t because its never looked at) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15507) Test org.apache.cassandra.distributed.test.DistributedReadWritePathTest#failingReadRepairTest does not test a failing read repair and should be updated to actually t
[ https://issues.apache.org/jira/browse/CASSANDRA-15507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Petrov updated CASSANDRA-15507: Reviewers: Alex Petrov, Alex Petrov (was: Alex Petrov) Alex Petrov, Alex Petrov Status: Review In Progress (was: Patch Available) > Test > org.apache.cassandra.distributed.test.DistributedReadWritePathTest#failingReadRepairTest > does not test a failing read repair and should be updated to actually > trigger a failed read repair > > > Key: CASSANDRA-15507 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15507 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > The test > org.apache.cassandra.distributed.test.DistributedReadWritePathTest#failingReadRepairTest > makes a few assumptions which are not valid at the moment. > 1) the write to node 1 and 2 have the same digest (they don’t, this is caused > by the timestamp being different) > 2) node 3 will participate with the read; it won’t give the fact that > org.apache.cassandra.locator.ReplicaPlans#contactForRead will speculate the > first 2 nodes always, so node 3 won’t get involved with the repair > 3) node 3 will attempt to get repaired (it won’t because its never looked at) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15507) Test org.apache.cassandra.distributed.test.DistributedReadWritePathTest#failingReadRepairTest does not test a failing read repair and should be updated to actually t
[ https://issues.apache.org/jira/browse/CASSANDRA-15507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Petrov updated CASSANDRA-15507: Status: Ready to Commit (was: Review In Progress) > Test > org.apache.cassandra.distributed.test.DistributedReadWritePathTest#failingReadRepairTest > does not test a failing read repair and should be updated to actually > trigger a failed read repair > > > Key: CASSANDRA-15507 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15507 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > The test > org.apache.cassandra.distributed.test.DistributedReadWritePathTest#failingReadRepairTest > makes a few assumptions which are not valid at the moment. > 1) the write to node 1 and 2 have the same digest (they don’t, this is caused > by the timestamp being different) > 2) node 3 will participate with the read; it won’t give the fact that > org.apache.cassandra.locator.ReplicaPlans#contactForRead will speculate the > first 2 nodes always, so node 3 won’t get involved with the repair > 3) node 3 will attempt to get repaired (it won’t because its never looked at) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15507) Test org.apache.cassandra.distributed.test.DistributedReadWritePathTest#failingReadRepairTest does not test a failing read repair and should be updated to actually
[ https://issues.apache.org/jira/browse/CASSANDRA-15507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17018041#comment-17018041 ] Alex Petrov commented on CASSANDRA-15507: - You're right; the way it is implemented now is not quite right. Patch looks good to me, +1. Committed to trunk with [815c397f4876aae9ed2ae5a9578c5ec7087643ab|https://github.com/apache/cassandra/commit/815c397f4876aae9ed2ae5a9578c5ec7087643ab] > Test > org.apache.cassandra.distributed.test.DistributedReadWritePathTest#failingReadRepairTest > does not test a failing read repair and should be updated to actually > trigger a failed read repair > > > Key: CASSANDRA-15507 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15507 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > The test > org.apache.cassandra.distributed.test.DistributedReadWritePathTest#failingReadRepairTest > makes a few assumptions which are not valid at the moment. > 1) the write to node 1 and 2 have the same digest (they don’t, this is caused > by the timestamp being different) > 2) node 3 will participate with the read; it won’t give the fact that > org.apache.cassandra.locator.ReplicaPlans#contactForRead will speculate the > first 2 nodes always, so node 3 won’t get involved with the repair > 3) node 3 will attempt to get repaired (it won’t because its never looked at) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: failingReadRepairTest was not triggering a repair on the third node as expected, so fixed the test to make the expected semantics
This is an automated email from the ASF dual-hosted git repository.
ifesdjeen pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new 815c397 failingReadRepairTest was not triggering a repair on the
third node as expected, so fixed the test to make the expected semantics
815c397 is described below
commit 815c397f4876aae9ed2ae5a9578c5ec7087643ab
Author: David Capwell
AuthorDate: Wed Jan 15 15:03:40 2020 -0800
failingReadRepairTest was not triggering a repair on the third node as
expected, so fixed the test to make the expected semantics
Patch by David Capwell, reviewed by Alex Petrov for CASSANDRA-15507.
---
.../test/DistributedReadWritePathTest.java | 49 ++
.../distributed/test/DistributedTestBase.java | 2 +-
2 files changed, 41 insertions(+), 10 deletions(-)
diff --git
a/test/distributed/org/apache/cassandra/distributed/test/DistributedReadWritePathTest.java
b/test/distributed/org/apache/cassandra/distributed/test/DistributedReadWritePathTest.java
index 0870ab3..679a90f 100644
---
a/test/distributed/org/apache/cassandra/distributed/test/DistributedReadWritePathTest.java
+++
b/test/distributed/org/apache/cassandra/distributed/test/DistributedReadWritePathTest.java
@@ -29,6 +29,7 @@ import org.apache.cassandra.db.ConsistencyLevel;
import org.apache.cassandra.db.Keyspace;
import org.apache.cassandra.distributed.Cluster;
import org.apache.cassandra.distributed.impl.IInvokableInstance;
+import org.apache.cassandra.metrics.ReadRepairMetrics;
import static org.apache.cassandra.distributed.api.Feature.NETWORK;
import static org.apache.cassandra.net.Verb.READ_REPAIR_RSP;
@@ -166,22 +167,52 @@ public class DistributedReadWritePathTest extends
DistributedTestBase
@Test
public void failingReadRepairTest() throws Throwable
{
+// This test makes a explicit assumption about which nodes are read
from; that 1 and 2 will be "contacts", and that 3 will be ignored.
+// This is a implementation detail of
org.apache.cassandra.locator.ReplicaPlans#contactForRead and
+//
org.apache.cassandra.locator.AbstractReplicationStrategy.getNaturalReplicasForToken
that may change
+// in a future release; when that happens this test could start to
fail but should only fail with the explicit
+// check that detects this behavior has changed.
+// see CASSANDRA-15507
try (Cluster cluster = init(Cluster.create(3)))
{
cluster.schemaChange("CREATE TABLE " + KEYSPACE + ".tbl (pk int,
ck int, v int, PRIMARY KEY (pk, ck)) WITH read_repair='blocking'");
-cluster.get(1).executeInternal("INSERT INTO " + KEYSPACE + ".tbl
(pk, ck, v) VALUES (1, 1, 1)");
-cluster.get(2).executeInternal("INSERT INTO " + KEYSPACE + ".tbl
(pk, ck, v) VALUES (1, 1, 1)");
+// nodes 1 and 3 are identical
+cluster.get(1).executeInternal("INSERT INTO " + KEYSPACE + ".tbl
(pk, ck, v) VALUES (1, 1, 1) USING TIMESTAMP 43");
+cluster.get(3).executeInternal("INSERT INTO " + KEYSPACE + ".tbl
(pk, ck, v) VALUES (1, 1, 1) USING TIMESTAMP 43");
-assertRows(cluster.get(3).executeInternal("SELECT * FROM " +
KEYSPACE + ".tbl WHERE pk = 1"));
+// node 2 is different because of the timestamp; a repair is needed
+cluster.get(2).executeInternal("INSERT INTO " + KEYSPACE + ".tbl
(pk, ck, v) VALUES (1, 1, 1) USING TIMESTAMP 42");
-cluster.verbs(READ_REPAIR_REQ).to(3).drop();
-assertRows(cluster.coordinator(1).execute("SELECT * FROM " +
KEYSPACE + ".tbl WHERE pk = 1",
- ConsistencyLevel.QUORUM),
- row(1, 1, 1));
+// 2 is out of date so needs to be repaired. This will make sure
that the repair does not happen on the node
+// which will trigger the coordinator to write to node 3
+cluster.verbs(READ_REPAIR_REQ).to(2).drop();
-// Data was not repaired
-assertRows(cluster.get(3).executeInternal("SELECT * FROM " +
KEYSPACE + ".tbl WHERE pk = 1"));
+// save the state of the counters so its known if the contacts
list changed
+long readRepairRequestsBefore = cluster.get(1).callOnInstance(()
->
Keyspace.open(KEYSPACE).getColumnFamilyStore("tbl").metric.readRepairRequests.getCount());
+long speculatedWriteBefore = cluster.get(1).callOnInstance(() ->
ReadRepairMetrics.speculatedWrite.getCount());
+
+Object[][] rows = cluster.coordinator(1)
+ .execute("SELECT pk, ck, v, WRITETIME(v) FROM " +
KEYSPACE + ".tbl WHERE pk = 1", ConsistencyLevel.QUORUM);
+
+// make sure to check counters first, so can detect if read repair
executed as expected
+long readRepairRequests
[jira] [Updated] (CASSANDRA-15450) in-jvm dtest cluster uncaughtExceptions propagation of exception goes to the wrong instance, it uses cluster generation when it should be using the instance id
[ https://issues.apache.org/jira/browse/CASSANDRA-15450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Petrov updated CASSANDRA-15450: Fix Version/s: 4.0 Since Version: 4.0 Source Control Link: https://github.com/apache/cassandra/commit/8f355ca2c25836784085f55eb464ad12ffaa1716 Resolution: Fixed Status: Resolved (was: Ready to Commit) > in-jvm dtest cluster uncaughtExceptions propagation of exception goes to the > wrong instance, it uses cluster generation when it should be using the > instance id > --- > > Key: CASSANDRA-15450 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15450 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Fix For: 4.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > In AbstractCluster.uncaughtExceptions, we attempt to get the instance from > the class loader and used the “generation”. This value is actually the > cluster id, so causes tests to fail when multiple tests share the same JVM; > it should be using the “id” field which represents the instance id relative > to the cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15450) in-jvm dtest cluster uncaughtExceptions propagation of exception goes to the wrong instance, it uses cluster generation when it should be using the instance id
[ https://issues.apache.org/jira/browse/CASSANDRA-15450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Petrov updated CASSANDRA-15450: Status: Ready to Commit (was: Review In Progress) > in-jvm dtest cluster uncaughtExceptions propagation of exception goes to the > wrong instance, it uses cluster generation when it should be using the > instance id > --- > > Key: CASSANDRA-15450 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15450 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > In AbstractCluster.uncaughtExceptions, we attempt to get the instance from > the class loader and used the “generation”. This value is actually the > cluster id, so causes tests to fail when multiple tests share the same JVM; > it should be using the “id” field which represents the instance id relative > to the cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15450) in-jvm dtest cluster uncaughtExceptions propagation of exception goes to the wrong instance, it uses cluster generation when it should be using the instance id
[ https://issues.apache.org/jira/browse/CASSANDRA-15450?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Petrov updated CASSANDRA-15450: Reviewers: Alex Petrov, Alex Petrov (was: Alex Petrov) Alex Petrov, Alex Petrov Status: Review In Progress (was: Patch Available) > in-jvm dtest cluster uncaughtExceptions propagation of exception goes to the > wrong instance, it uses cluster generation when it should be using the > instance id > --- > > Key: CASSANDRA-15450 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15450 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > In AbstractCluster.uncaughtExceptions, we attempt to get the instance from > the class loader and used the “generation”. This value is actually the > cluster id, so causes tests to fail when multiple tests share the same JVM; > it should be using the “id” field which represents the instance id relative > to the cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15450) in-jvm dtest cluster uncaughtExceptions propagation of exception goes to the wrong instance, it uses cluster generation when it should be using the instance id
[ https://issues.apache.org/jira/browse/CASSANDRA-15450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17018028#comment-17018028 ] Alex Petrov commented on CASSANDRA-15450: - Committed only to trunk, under assumption that other branches are going to be fixed with a correct back ported version: [8f355ca2c25836784085f55eb464ad12ffaa1716|https://github.com/apache/cassandra/commit/8f355ca2c25836784085f55eb464ad12ffaa1716] > in-jvm dtest cluster uncaughtExceptions propagation of exception goes to the > wrong instance, it uses cluster generation when it should be using the > instance id > --- > > Key: CASSANDRA-15450 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15450 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > In AbstractCluster.uncaughtExceptions, we attempt to get the instance from > the class loader and used the “generation”. This value is actually the > cluster id, so causes tests to fail when multiple tests share the same JVM; > it should be using the “id” field which represents the instance id relative > to the cluster. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: In-JVM dtest cluster uncaughtExceptions propagation of exception goes to the wrong instance, it uses cluster generation when it should be using the instance id
This is an automated email from the ASF dual-hosted git repository.
ifesdjeen pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new 8f355ca In-JVM dtest cluster uncaughtExceptions propagation of
exception goes to the wrong instance, it uses cluster generation when it should
be using the instance id
8f355ca is described below
commit 8f355ca2c25836784085f55eb464ad12ffaa1716
Author: David Capwell
AuthorDate: Thu Dec 12 10:14:23 2019 -0800
In-JVM dtest cluster uncaughtExceptions propagation of exception goes to
the wrong instance, it uses cluster generation when it should be using the
instance id
Patch by David Capwell; reviewed by Alex Petrov for CASSANDRA-15450
---
.../org/apache/cassandra/distributed/impl/AbstractCluster.java | 3 +--
.../org/apache/cassandra/distributed/impl/InstanceClassLoader.java | 7 ++-
2 files changed, 7 insertions(+), 3 deletions(-)
diff --git
a/test/distributed/org/apache/cassandra/distributed/impl/AbstractCluster.java
b/test/distributed/org/apache/cassandra/distributed/impl/AbstractCluster.java
index dd01896..47f76fd 100644
---
a/test/distributed/org/apache/cassandra/distributed/impl/AbstractCluster.java
+++
b/test/distributed/org/apache/cassandra/distributed/impl/AbstractCluster.java
@@ -43,7 +43,6 @@ import com.google.common.collect.Sets;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
-import org.apache.cassandra.config.DatabaseDescriptor;
import org.apache.cassandra.db.ColumnFamilyStore;
import org.apache.cassandra.db.ConsistencyLevel;
import org.apache.cassandra.db.Keyspace;
@@ -497,7 +496,7 @@ public abstract class AbstractCluster
implements ICluster,
return;
}
InstanceClassLoader cl = (InstanceClassLoader)
thread.getContextClassLoader();
-get(cl.getGeneration()).uncaughtException(thread, error);
+get(cl.getInstanceId()).uncaughtException(thread, error);
}
protected interface Factory>
diff --git
a/test/distributed/org/apache/cassandra/distributed/impl/InstanceClassLoader.java
b/test/distributed/org/apache/cassandra/distributed/impl/InstanceClassLoader.java
index 5ada263..75a8912 100644
---
a/test/distributed/org/apache/cassandra/distributed/impl/InstanceClassLoader.java
+++
b/test/distributed/org/apache/cassandra/distributed/impl/InstanceClassLoader.java
@@ -81,11 +81,16 @@ public class InstanceClassLoader extends URLClassLoader
this.id = id;
}
-public int getGeneration()
+public int getClusterGeneration()
{
return generation;
}
+public int getInstanceId()
+{
+return id;
+}
+
@Override
public Class loadClass(String name) throws ClassNotFoundException
{
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15430) Cassandra 3.0.18: BatchMessage.execute - 10x more on-heap allocations compared to 2.1.18
[
https://issues.apache.org/jira/browse/CASSANDRA-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17017992#comment-17017992
]
Benedict Elliott Smith commented on CASSANDRA-15430:
If you want to file a separate ticket for that, please do. Attach similar
information, including schema and any profile information / JFR recordings you
have.
There's a limited amount of my available time I can dedicate to this kind of
work, to warn you. I cannot promise swift progress, but will steal what time I
can to see how we have regressed since 2.1, and if it's in a manner we can
easily rectify.
> Cassandra 3.0.18: BatchMessage.execute - 10x more on-heap allocations
> compared to 2.1.18
>
>
> Key: CASSANDRA-15430
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15430
> Project: Cassandra
> Issue Type: Bug
>Reporter: Thomas Steinmaurer
>Priority: Normal
> Attachments: dashboard.png, jfr_allocations.png, mutation_stage.png,
> screenshot-1.png, screenshot-2.png, screenshot-3.png, screenshot-4.png
>
>
> In a 6 node loadtest cluster, we have been running with 2.1.18 a certain
> production-like workload constantly and sufficiently. After upgrading one
> node to 3.0.18 (remaining 5 still on 2.1.18 after we have seen that sort of
> regression described below), 3.0.18 is showing increased CPU usage, increase
> GC, high mutation stage pending tasks, dropped mutation messages ...
> Some spec. All 6 nodes equally sized:
> * Bare metal, 32 physical cores, 512G RAM
> * Xmx31G, G1, max pause millis = 2000ms
> * cassandra.yaml basically unchanged, thus same settings in regard to number
> of threads, compaction throttling etc.
> Following dashboard shows highlighted areas (CPU, suspension) with metrics
> for all 6 nodes and the one outlier being the node upgraded to Cassandra
> 3.0.18.
> !dashboard.png|width=1280!
> Additionally we see a large increase on pending tasks in the mutation stage
> after the upgrade:
> !mutation_stage.png!
> And dropped mutation messages, also confirmed in the Cassandra log:
> {noformat}
> INFO [ScheduledTasks:1] 2019-11-15 08:24:24,780 MessagingService.java:1022 -
> MUTATION messages were dropped in last 5000 ms: 41552 for internal timeout
> and 0 for cross node timeout
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,157 StatusLogger.java:52 - Pool
> NameActive Pending Completed Blocked All Time
> Blocked
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 -
> MutationStage 256 81824 3360532756 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 -
> ViewMutationStage 0 0 0 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 -
> ReadStage 0 0 62862266 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 -
> RequestResponseStage 0 0 2176659856 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 -
> ReadRepairStage 0 0 0 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 -
> CounterMutationStage 0 0 0 0
> 0
> ...
> {noformat}
> Judging from a 15min JFR session for both, 3.0.18 and 2.1.18 on a different
> node, high-level, it looks like the code path underneath
> {{BatchMessage.execute}} is producing ~ 10x more on-heap allocations in
> 3.0.18 compared to 2.1.18.
> !jfr_allocations.png!
> Left => 3.0.18
> Right => 2.1.18
> JFRs zipped are exceeding the 60MB limit to directly attach to the ticket. I
> can upload them, if there is another destination available.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15430) Cassandra 3.0.18: BatchMessage.execute - 10x more on-heap allocations compared to 2.1.18
[
https://issues.apache.org/jira/browse/CASSANDRA-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17017990#comment-17017990
]
Thomas Steinmaurer edited comment on CASSANDRA-15430 at 1/17/20 1:09 PM:
-
[~benedict], thanks!
Just another internal high-level overview on our 3.0 experience compared to 2.1
when it comes to the write path.
!screenshot-4.png!
"Timeseries (TS)" written is simply our "payload" (data point) we are ingesting
in a 6 node load test environment. While 2.1.18 was able to handle ~ 2 mio TS
payloads / min / Cassandra JVM at 3-4% GC suspension and 25% CPU usage on a 64
vCore box (32 physical cores) without dropping mutation messages, 3.0.18 looks
much worse. All 3.0.18 based tests in the table have been done without being in
a mixed Cassandra binary version scenario.
So, any low-hanging fruit would be much appreciated. :-)
was (Author: tsteinmaurer):
[~benedict], thanks!
Just another internal high-level overview on our 3.0 experience compared to 2.1
when it comes to the write patch.
!screenshot-4.png!
"Timeseries (TS)" written is simply our "payload" (data point) we are ingesting
in a 6 node load test environment. While 2.1.18 was able to handle ~ 2 mio TS
payloads / min / Cassandra JVM at 3-4% GC suspension and 25% CPU usage on a 64
vCore box (32 physical cores) without dropping mutation messages, 3.0.18 looks
much worse. All 3.0.18 based tests in the table have been done without being in
a mixed Cassandra binary version scenario.
So, any low-hanging fruit would be much appreciated. :-)
> Cassandra 3.0.18: BatchMessage.execute - 10x more on-heap allocations
> compared to 2.1.18
>
>
> Key: CASSANDRA-15430
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15430
> Project: Cassandra
> Issue Type: Bug
>Reporter: Thomas Steinmaurer
>Priority: Normal
> Attachments: dashboard.png, jfr_allocations.png, mutation_stage.png,
> screenshot-1.png, screenshot-2.png, screenshot-3.png, screenshot-4.png
>
>
> In a 6 node loadtest cluster, we have been running with 2.1.18 a certain
> production-like workload constantly and sufficiently. After upgrading one
> node to 3.0.18 (remaining 5 still on 2.1.18 after we have seen that sort of
> regression described below), 3.0.18 is showing increased CPU usage, increase
> GC, high mutation stage pending tasks, dropped mutation messages ...
> Some spec. All 6 nodes equally sized:
> * Bare metal, 32 physical cores, 512G RAM
> * Xmx31G, G1, max pause millis = 2000ms
> * cassandra.yaml basically unchanged, thus same settings in regard to number
> of threads, compaction throttling etc.
> Following dashboard shows highlighted areas (CPU, suspension) with metrics
> for all 6 nodes and the one outlier being the node upgraded to Cassandra
> 3.0.18.
> !dashboard.png|width=1280!
> Additionally we see a large increase on pending tasks in the mutation stage
> after the upgrade:
> !mutation_stage.png!
> And dropped mutation messages, also confirmed in the Cassandra log:
> {noformat}
> INFO [ScheduledTasks:1] 2019-11-15 08:24:24,780 MessagingService.java:1022 -
> MUTATION messages were dropped in last 5000 ms: 41552 for internal timeout
> and 0 for cross node timeout
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,157 StatusLogger.java:52 - Pool
> NameActive Pending Completed Blocked All Time
> Blocked
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 -
> MutationStage 256 81824 3360532756 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 -
> ViewMutationStage 0 0 0 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 -
> ReadStage 0 0 62862266 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 -
> RequestResponseStage 0 0 2176659856 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 -
> ReadRepairStage 0 0 0 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 -
> CounterMutationStage 0 0 0 0
> 0
> ...
> {noformat}
> Judging from a 15min JFR session for both, 3.0.18 and 2.1.18 on a different
> node, high-level, it looks like the code path underneath
> {{BatchMessage.execute}} is producing ~ 10x more on-heap allocations in
> 3.0.18 compared to 2.1.18.
> !jfr_allocations.png!
> Left => 3.0.18
> Right => 2.1
[jira] [Commented] (CASSANDRA-15430) Cassandra 3.0.18: BatchMessage.execute - 10x more on-heap allocations compared to 2.1.18
[
https://issues.apache.org/jira/browse/CASSANDRA-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17017990#comment-17017990
]
Thomas Steinmaurer commented on CASSANDRA-15430:
[~benedict], thanks!
Just another internal high-level overview on our 3.0 experience compared to 2.1
when it comes to the write patch.
!screenshot-4.png!
"Timeseries (TS)" written is simply our "payload" (data point) we are ingesting
in a 6 node load test environment. While 2.1.18 was able to handle ~ 2 mio TS
payloads / min / Cassandra JVM at 3-4% GC suspension and 25% CPU usage on a 64
vCore box (32 physical cores) without dropping mutation messages, 3.0.18 looks
much worse. All 3.0.18 based tests in the table have been done without being in
a mixed Cassandra binary version scenario.
So, any low-hanging fruit would be much appreciated. :-)
> Cassandra 3.0.18: BatchMessage.execute - 10x more on-heap allocations
> compared to 2.1.18
>
>
> Key: CASSANDRA-15430
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15430
> Project: Cassandra
> Issue Type: Bug
>Reporter: Thomas Steinmaurer
>Priority: Normal
> Attachments: dashboard.png, jfr_allocations.png, mutation_stage.png,
> screenshot-1.png, screenshot-2.png, screenshot-3.png, screenshot-4.png
>
>
> In a 6 node loadtest cluster, we have been running with 2.1.18 a certain
> production-like workload constantly and sufficiently. After upgrading one
> node to 3.0.18 (remaining 5 still on 2.1.18 after we have seen that sort of
> regression described below), 3.0.18 is showing increased CPU usage, increase
> GC, high mutation stage pending tasks, dropped mutation messages ...
> Some spec. All 6 nodes equally sized:
> * Bare metal, 32 physical cores, 512G RAM
> * Xmx31G, G1, max pause millis = 2000ms
> * cassandra.yaml basically unchanged, thus same settings in regard to number
> of threads, compaction throttling etc.
> Following dashboard shows highlighted areas (CPU, suspension) with metrics
> for all 6 nodes and the one outlier being the node upgraded to Cassandra
> 3.0.18.
> !dashboard.png|width=1280!
> Additionally we see a large increase on pending tasks in the mutation stage
> after the upgrade:
> !mutation_stage.png!
> And dropped mutation messages, also confirmed in the Cassandra log:
> {noformat}
> INFO [ScheduledTasks:1] 2019-11-15 08:24:24,780 MessagingService.java:1022 -
> MUTATION messages were dropped in last 5000 ms: 41552 for internal timeout
> and 0 for cross node timeout
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,157 StatusLogger.java:52 - Pool
> NameActive Pending Completed Blocked All Time
> Blocked
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 -
> MutationStage 256 81824 3360532756 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 -
> ViewMutationStage 0 0 0 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 -
> ReadStage 0 0 62862266 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 -
> RequestResponseStage 0 0 2176659856 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 -
> ReadRepairStage 0 0 0 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 -
> CounterMutationStage 0 0 0 0
> 0
> ...
> {noformat}
> Judging from a 15min JFR session for both, 3.0.18 and 2.1.18 on a different
> node, high-level, it looks like the code path underneath
> {{BatchMessage.execute}} is producing ~ 10x more on-heap allocations in
> 3.0.18 compared to 2.1.18.
> !jfr_allocations.png!
> Left => 3.0.18
> Right => 2.1.18
> JFRs zipped are exceeding the 60MB limit to directly attach to the ticket. I
> can upload them, if there is another destination available.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15430) Cassandra 3.0.18: BatchMessage.execute - 10x more on-heap allocations compared to 2.1.18
[
https://issues.apache.org/jira/browse/CASSANDRA-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Steinmaurer updated CASSANDRA-15430:
---
Attachment: screenshot-4.png
> Cassandra 3.0.18: BatchMessage.execute - 10x more on-heap allocations
> compared to 2.1.18
>
>
> Key: CASSANDRA-15430
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15430
> Project: Cassandra
> Issue Type: Bug
>Reporter: Thomas Steinmaurer
>Priority: Normal
> Attachments: dashboard.png, jfr_allocations.png, mutation_stage.png,
> screenshot-1.png, screenshot-2.png, screenshot-3.png, screenshot-4.png
>
>
> In a 6 node loadtest cluster, we have been running with 2.1.18 a certain
> production-like workload constantly and sufficiently. After upgrading one
> node to 3.0.18 (remaining 5 still on 2.1.18 after we have seen that sort of
> regression described below), 3.0.18 is showing increased CPU usage, increase
> GC, high mutation stage pending tasks, dropped mutation messages ...
> Some spec. All 6 nodes equally sized:
> * Bare metal, 32 physical cores, 512G RAM
> * Xmx31G, G1, max pause millis = 2000ms
> * cassandra.yaml basically unchanged, thus same settings in regard to number
> of threads, compaction throttling etc.
> Following dashboard shows highlighted areas (CPU, suspension) with metrics
> for all 6 nodes and the one outlier being the node upgraded to Cassandra
> 3.0.18.
> !dashboard.png|width=1280!
> Additionally we see a large increase on pending tasks in the mutation stage
> after the upgrade:
> !mutation_stage.png!
> And dropped mutation messages, also confirmed in the Cassandra log:
> {noformat}
> INFO [ScheduledTasks:1] 2019-11-15 08:24:24,780 MessagingService.java:1022 -
> MUTATION messages were dropped in last 5000 ms: 41552 for internal timeout
> and 0 for cross node timeout
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,157 StatusLogger.java:52 - Pool
> NameActive Pending Completed Blocked All Time
> Blocked
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 -
> MutationStage 256 81824 3360532756 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 -
> ViewMutationStage 0 0 0 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 -
> ReadStage 0 0 62862266 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 -
> RequestResponseStage 0 0 2176659856 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 -
> ReadRepairStage 0 0 0 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 -
> CounterMutationStage 0 0 0 0
> 0
> ...
> {noformat}
> Judging from a 15min JFR session for both, 3.0.18 and 2.1.18 on a different
> node, high-level, it looks like the code path underneath
> {{BatchMessage.execute}} is producing ~ 10x more on-heap allocations in
> 3.0.18 compared to 2.1.18.
> !jfr_allocations.png!
> Left => 3.0.18
> Right => 2.1.18
> JFRs zipped are exceeding the 60MB limit to directly attach to the ticket. I
> can upload them, if there is another destination available.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15430) Cassandra 3.0.18: BatchMessage.execute - 10x more on-heap allocations compared to 2.1.18
[
https://issues.apache.org/jira/browse/CASSANDRA-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17017985#comment-17017985
]
Benedict Elliott Smith commented on CASSANDRA-15430:
Yep, I'll try to find time next week to take a look at the JFR recordings.
In the meantime, I note that 140GiB of the heap pressure, or around 1/3rd of
the pressure attributed to this work, is down to mixed-mode operation of the
cluster. All of the work under {{LegacyLayout}} is incurred to talk to the 2.1
nodes. So a fully upgraded cluster should "only" be seeing ~5x increase in
heap pressure :)
I'll see if I can see some obvious causes that can be mitigated for the
remainder.
> Cassandra 3.0.18: BatchMessage.execute - 10x more on-heap allocations
> compared to 2.1.18
>
>
> Key: CASSANDRA-15430
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15430
> Project: Cassandra
> Issue Type: Bug
>Reporter: Thomas Steinmaurer
>Priority: Normal
> Attachments: dashboard.png, jfr_allocations.png, mutation_stage.png,
> screenshot-1.png, screenshot-2.png, screenshot-3.png
>
>
> In a 6 node loadtest cluster, we have been running with 2.1.18 a certain
> production-like workload constantly and sufficiently. After upgrading one
> node to 3.0.18 (remaining 5 still on 2.1.18 after we have seen that sort of
> regression described below), 3.0.18 is showing increased CPU usage, increase
> GC, high mutation stage pending tasks, dropped mutation messages ...
> Some spec. All 6 nodes equally sized:
> * Bare metal, 32 physical cores, 512G RAM
> * Xmx31G, G1, max pause millis = 2000ms
> * cassandra.yaml basically unchanged, thus same settings in regard to number
> of threads, compaction throttling etc.
> Following dashboard shows highlighted areas (CPU, suspension) with metrics
> for all 6 nodes and the one outlier being the node upgraded to Cassandra
> 3.0.18.
> !dashboard.png|width=1280!
> Additionally we see a large increase on pending tasks in the mutation stage
> after the upgrade:
> !mutation_stage.png!
> And dropped mutation messages, also confirmed in the Cassandra log:
> {noformat}
> INFO [ScheduledTasks:1] 2019-11-15 08:24:24,780 MessagingService.java:1022 -
> MUTATION messages were dropped in last 5000 ms: 41552 for internal timeout
> and 0 for cross node timeout
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,157 StatusLogger.java:52 - Pool
> NameActive Pending Completed Blocked All Time
> Blocked
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 -
> MutationStage 256 81824 3360532756 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 -
> ViewMutationStage 0 0 0 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 -
> ReadStage 0 0 62862266 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 -
> RequestResponseStage 0 0 2176659856 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 -
> ReadRepairStage 0 0 0 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 -
> CounterMutationStage 0 0 0 0
> 0
> ...
> {noformat}
> Judging from a 15min JFR session for both, 3.0.18 and 2.1.18 on a different
> node, high-level, it looks like the code path underneath
> {{BatchMessage.execute}} is producing ~ 10x more on-heap allocations in
> 3.0.18 compared to 2.1.18.
> !jfr_allocations.png!
> Left => 3.0.18
> Right => 2.1.18
> JFRs zipped are exceeding the 60MB limit to directly attach to the ticket. I
> can upload them, if there is another destination available.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-13917) COMPACT STORAGE queries on dense static tables accept hidden column1 and value columns
[
https://issues.apache.org/jira/browse/CASSANDRA-13917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17017897#comment-17017897
]
Alex Petrov edited comment on CASSANDRA-13917 at 1/17/20 12:51 PM:
---
[~slebresne] you are right. In fact, I've found several more places where this
could cause a problem. What we should've done initially is just add a separate
method for CQL layer, and let the rest of the calls (e.g., all internal stuff)
to use the old one. Committed version of the patch also caused problems in
mixed version environments which was caught by one of the upgrade in-jvm
dtests. Also, things like thrift and compact storage compatibility layer was
impacted in a way, too.
I'm still thinking if {{RowUpdateBuilder}} should be using CQL columns or all
columns. I'd argue that CQL only (e.g., no hidden), since this is also how we
query them for the most part. We don't really those hidden column even on
internal paths.
I've made a patch and have added a couple more tests that validates the
situations that weren't covered previously. Thanks to [CASSANDRA-15506], we can
also make sure that upgrade tests are going to be running.
|[patch|https://github.com/apache/cassandra/compare/trunk...ifesdjeen:CASSANDRA-13917-followup]|[CI|https://circleci.com/gh/ifesdjeen/cassandra/tree/CASSANDRA-13917-followup]|
|[patch|https://github.com/apache/cassandra/compare/trunk...ifesdjeen:CASSANDRA-13917-followup-3.11]|[CI|https://circleci.com/gh/ifesdjeen/cassandra/tree/CASSANDRA-13917-followup-3.11]|
was (Author: ifesdjeen):
[~slebresne] you are right. In fact, I've found several more places where this
could cause a problem. What we should've done initially is just add a separate
method for CQL layer, and let the rest of the calls (e.g., all internal stuff)
to use the old one. Committed version of the patch also caused problems in
mixed version environments which was caught by one of the upgrade in-jvm
dtests. Also, things like thrift and compact storage compatibility layer was
impacted in a way, too.
I'm still thinking if {{RowUpdateBuilder}} should be using CQL columns or all
columns. I'd argue that CQL only (e.g., no hidden), since this is also how we
query them for the most part. We don't really those hidden column even on
internal paths.
I've made a patch and have added a couple more tests that validates the
situations that weren't covered previously. Thanks to [CASSANDRA-15506], we can
also make sure that upgrade tests are going to be running.
|[patch|https://github.com/apache/cassandra/compare/trunk...ifesdjeen:CASSANDRA-13917-followup]|[CI|https://circleci.com/gh/ifesdjeen/cassandra/tree/CASSANDRA-13917-followup]|
> COMPACT STORAGE queries on dense static tables accept hidden column1 and
> value columns
> --
>
> Key: CASSANDRA-13917
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13917
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core
>Reporter: Alex Petrov
>Assignee: Aleksandr Sorokoumov
>Priority: Low
> Labels: lhf
> Fix For: 3.0.x, 3.11.x
>
> Attachments: 13917-3.0-testall-13.12.2019,
> 13917-3.0-testall-16.01.2020, 13917-3.0-testall-2.png,
> 13917-3.0-testall-20.11.2019.png, 13917-3.0-upgrade-16.01.2020,
> 13917-3.0.png, 13917-3.11-testall-13.12.2019,
> 13917-3.11-testall-16.01.2020.png, 13917-3.11-testall-2.png,
> 13917-3.11-testall-20.11.2019.png, 13917-3.11-upgrade-16.01.2020.png,
> 13917-3.11.png
>
>
> Test for the issue:
> {code}
> @Test
> public void testCompactStorage() throws Throwable
> {
> createTable("CREATE TABLE %s (a int PRIMARY KEY, b int, c int) WITH
> COMPACT STORAGE");
> assertInvalid("INSERT INTO %s (a, b, c, column1) VALUES (?, ?, ?,
> ?)", 1, 1, 1, ByteBufferUtil.bytes('a'));
> // This one fails with Some clustering keys are missing: column1,
> which is still wrong
> assertInvalid("INSERT INTO %s (a, b, c, value) VALUES (?, ?, ?, ?)",
> 1, 1, 1, ByteBufferUtil.bytes('a'));
> assertInvalid("INSERT INTO %s (a, b, c, column1, value) VALUES (?, ?,
> ?, ?, ?)", 1, 1, 1, ByteBufferUtil.bytes('a'), ByteBufferUtil.bytes('b'));
> assertEmpty(execute("SELECT * FROM %s"));
> }
> {code}
> Gladly, these writes are no-op, even though they succeed.
> {{value}} and {{column1}} should be completely hidden. Fixing this one should
> be as easy as just adding validations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15430) Cassandra 3.0.18: BatchMessage.execute - 10x more on-heap allocations compared to 2.1.18
[
https://issues.apache.org/jira/browse/CASSANDRA-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17017955#comment-17017955
]
Thomas Steinmaurer commented on CASSANDRA-15430:
[~benedict], as discussed, a few more JFR-based screens to better show the
differences, until you find time to open JFR files yourself. Again, it is all
about the write path (only) executing batch messages. Both had the same *JFR
duration*, namely *15min*.
Cassandra *2.1.18*:
* BatchMessage.execute - In total: 42,76 GByte with next level top contributors:
** BatchStatement.getMutations => 19,46 GByte
** BatchStatement.executeWithoutConditions => 16,97 GByte
!screenshot-1.png!
Cassandra *3.0.18*:
* BatchMessage.execute - In total: 451,86 GByte (factor 10 more) with next
level top contributors:
** BatchStatement.executeWithoutConditions => 214,23 GByte
** BatchStatement.getMutations => 205,52 GByte
For *3.0.18*, more in-depth drill-down for
BatchMessage.executeWithoutConditions (214,23 GByte)
!screenshot-2.png!
resp.: BatchMessage.getMutations (205,52 GByte)
!screenshot-3.png!
A bit hard to give sufficient details with screen shots, so likely it would be
simply the best option, to work directly with the provided JFR files.
Thanks a lot!
> Cassandra 3.0.18: BatchMessage.execute - 10x more on-heap allocations
> compared to 2.1.18
>
>
> Key: CASSANDRA-15430
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15430
> Project: Cassandra
> Issue Type: Bug
>Reporter: Thomas Steinmaurer
>Priority: Normal
> Attachments: dashboard.png, jfr_allocations.png, mutation_stage.png,
> screenshot-1.png, screenshot-2.png, screenshot-3.png
>
>
> In a 6 node loadtest cluster, we have been running with 2.1.18 a certain
> production-like workload constantly and sufficiently. After upgrading one
> node to 3.0.18 (remaining 5 still on 2.1.18 after we have seen that sort of
> regression described below), 3.0.18 is showing increased CPU usage, increase
> GC, high mutation stage pending tasks, dropped mutation messages ...
> Some spec. All 6 nodes equally sized:
> * Bare metal, 32 physical cores, 512G RAM
> * Xmx31G, G1, max pause millis = 2000ms
> * cassandra.yaml basically unchanged, thus same settings in regard to number
> of threads, compaction throttling etc.
> Following dashboard shows highlighted areas (CPU, suspension) with metrics
> for all 6 nodes and the one outlier being the node upgraded to Cassandra
> 3.0.18.
> !dashboard.png|width=1280!
> Additionally we see a large increase on pending tasks in the mutation stage
> after the upgrade:
> !mutation_stage.png!
> And dropped mutation messages, also confirmed in the Cassandra log:
> {noformat}
> INFO [ScheduledTasks:1] 2019-11-15 08:24:24,780 MessagingService.java:1022 -
> MUTATION messages were dropped in last 5000 ms: 41552 for internal timeout
> and 0 for cross node timeout
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,157 StatusLogger.java:52 - Pool
> NameActive Pending Completed Blocked All Time
> Blocked
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 -
> MutationStage 256 81824 3360532756 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 -
> ViewMutationStage 0 0 0 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 -
> ReadStage 0 0 62862266 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 -
> RequestResponseStage 0 0 2176659856 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 -
> ReadRepairStage 0 0 0 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 -
> CounterMutationStage 0 0 0 0
> 0
> ...
> {noformat}
> Judging from a 15min JFR session for both, 3.0.18 and 2.1.18 on a different
> node, high-level, it looks like the code path underneath
> {{BatchMessage.execute}} is producing ~ 10x more on-heap allocations in
> 3.0.18 compared to 2.1.18.
> !jfr_allocations.png!
> Left => 3.0.18
> Right => 2.1.18
> JFRs zipped are exceeding the 60MB limit to directly attach to the ticket. I
> can upload them, if there is another destination available.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For a
[jira] [Updated] (CASSANDRA-15430) Cassandra 3.0.18: BatchMessage.execute - 10x more on-heap allocations compared to 2.1.18
[
https://issues.apache.org/jira/browse/CASSANDRA-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Steinmaurer updated CASSANDRA-15430:
---
Attachment: screenshot-3.png
> Cassandra 3.0.18: BatchMessage.execute - 10x more on-heap allocations
> compared to 2.1.18
>
>
> Key: CASSANDRA-15430
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15430
> Project: Cassandra
> Issue Type: Bug
>Reporter: Thomas Steinmaurer
>Priority: Normal
> Attachments: dashboard.png, jfr_allocations.png, mutation_stage.png,
> screenshot-1.png, screenshot-2.png, screenshot-3.png
>
>
> In a 6 node loadtest cluster, we have been running with 2.1.18 a certain
> production-like workload constantly and sufficiently. After upgrading one
> node to 3.0.18 (remaining 5 still on 2.1.18 after we have seen that sort of
> regression described below), 3.0.18 is showing increased CPU usage, increase
> GC, high mutation stage pending tasks, dropped mutation messages ...
> Some spec. All 6 nodes equally sized:
> * Bare metal, 32 physical cores, 512G RAM
> * Xmx31G, G1, max pause millis = 2000ms
> * cassandra.yaml basically unchanged, thus same settings in regard to number
> of threads, compaction throttling etc.
> Following dashboard shows highlighted areas (CPU, suspension) with metrics
> for all 6 nodes and the one outlier being the node upgraded to Cassandra
> 3.0.18.
> !dashboard.png|width=1280!
> Additionally we see a large increase on pending tasks in the mutation stage
> after the upgrade:
> !mutation_stage.png!
> And dropped mutation messages, also confirmed in the Cassandra log:
> {noformat}
> INFO [ScheduledTasks:1] 2019-11-15 08:24:24,780 MessagingService.java:1022 -
> MUTATION messages were dropped in last 5000 ms: 41552 for internal timeout
> and 0 for cross node timeout
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,157 StatusLogger.java:52 - Pool
> NameActive Pending Completed Blocked All Time
> Blocked
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 -
> MutationStage 256 81824 3360532756 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 -
> ViewMutationStage 0 0 0 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 -
> ReadStage 0 0 62862266 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 -
> RequestResponseStage 0 0 2176659856 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 -
> ReadRepairStage 0 0 0 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 -
> CounterMutationStage 0 0 0 0
> 0
> ...
> {noformat}
> Judging from a 15min JFR session for both, 3.0.18 and 2.1.18 on a different
> node, high-level, it looks like the code path underneath
> {{BatchMessage.execute}} is producing ~ 10x more on-heap allocations in
> 3.0.18 compared to 2.1.18.
> !jfr_allocations.png!
> Left => 3.0.18
> Right => 2.1.18
> JFRs zipped are exceeding the 60MB limit to directly attach to the ticket. I
> can upload them, if there is another destination available.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15430) Cassandra 3.0.18: BatchMessage.execute - 10x more on-heap allocations compared to 2.1.18
[
https://issues.apache.org/jira/browse/CASSANDRA-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Steinmaurer updated CASSANDRA-15430:
---
Attachment: screenshot-2.png
> Cassandra 3.0.18: BatchMessage.execute - 10x more on-heap allocations
> compared to 2.1.18
>
>
> Key: CASSANDRA-15430
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15430
> Project: Cassandra
> Issue Type: Bug
>Reporter: Thomas Steinmaurer
>Priority: Normal
> Attachments: dashboard.png, jfr_allocations.png, mutation_stage.png,
> screenshot-1.png, screenshot-2.png
>
>
> In a 6 node loadtest cluster, we have been running with 2.1.18 a certain
> production-like workload constantly and sufficiently. After upgrading one
> node to 3.0.18 (remaining 5 still on 2.1.18 after we have seen that sort of
> regression described below), 3.0.18 is showing increased CPU usage, increase
> GC, high mutation stage pending tasks, dropped mutation messages ...
> Some spec. All 6 nodes equally sized:
> * Bare metal, 32 physical cores, 512G RAM
> * Xmx31G, G1, max pause millis = 2000ms
> * cassandra.yaml basically unchanged, thus same settings in regard to number
> of threads, compaction throttling etc.
> Following dashboard shows highlighted areas (CPU, suspension) with metrics
> for all 6 nodes and the one outlier being the node upgraded to Cassandra
> 3.0.18.
> !dashboard.png|width=1280!
> Additionally we see a large increase on pending tasks in the mutation stage
> after the upgrade:
> !mutation_stage.png!
> And dropped mutation messages, also confirmed in the Cassandra log:
> {noformat}
> INFO [ScheduledTasks:1] 2019-11-15 08:24:24,780 MessagingService.java:1022 -
> MUTATION messages were dropped in last 5000 ms: 41552 for internal timeout
> and 0 for cross node timeout
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,157 StatusLogger.java:52 - Pool
> NameActive Pending Completed Blocked All Time
> Blocked
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 -
> MutationStage 256 81824 3360532756 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 -
> ViewMutationStage 0 0 0 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 -
> ReadStage 0 0 62862266 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 -
> RequestResponseStage 0 0 2176659856 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 -
> ReadRepairStage 0 0 0 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 -
> CounterMutationStage 0 0 0 0
> 0
> ...
> {noformat}
> Judging from a 15min JFR session for both, 3.0.18 and 2.1.18 on a different
> node, high-level, it looks like the code path underneath
> {{BatchMessage.execute}} is producing ~ 10x more on-heap allocations in
> 3.0.18 compared to 2.1.18.
> !jfr_allocations.png!
> Left => 3.0.18
> Right => 2.1.18
> JFRs zipped are exceeding the 60MB limit to directly attach to the ticket. I
> can upload them, if there is another destination available.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15430) Cassandra 3.0.18: BatchMessage.execute - 10x more on-heap allocations compared to 2.1.18
[
https://issues.apache.org/jira/browse/CASSANDRA-15430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Steinmaurer updated CASSANDRA-15430:
---
Attachment: screenshot-1.png
> Cassandra 3.0.18: BatchMessage.execute - 10x more on-heap allocations
> compared to 2.1.18
>
>
> Key: CASSANDRA-15430
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15430
> Project: Cassandra
> Issue Type: Bug
>Reporter: Thomas Steinmaurer
>Priority: Normal
> Attachments: dashboard.png, jfr_allocations.png, mutation_stage.png,
> screenshot-1.png
>
>
> In a 6 node loadtest cluster, we have been running with 2.1.18 a certain
> production-like workload constantly and sufficiently. After upgrading one
> node to 3.0.18 (remaining 5 still on 2.1.18 after we have seen that sort of
> regression described below), 3.0.18 is showing increased CPU usage, increase
> GC, high mutation stage pending tasks, dropped mutation messages ...
> Some spec. All 6 nodes equally sized:
> * Bare metal, 32 physical cores, 512G RAM
> * Xmx31G, G1, max pause millis = 2000ms
> * cassandra.yaml basically unchanged, thus same settings in regard to number
> of threads, compaction throttling etc.
> Following dashboard shows highlighted areas (CPU, suspension) with metrics
> for all 6 nodes and the one outlier being the node upgraded to Cassandra
> 3.0.18.
> !dashboard.png|width=1280!
> Additionally we see a large increase on pending tasks in the mutation stage
> after the upgrade:
> !mutation_stage.png!
> And dropped mutation messages, also confirmed in the Cassandra log:
> {noformat}
> INFO [ScheduledTasks:1] 2019-11-15 08:24:24,780 MessagingService.java:1022 -
> MUTATION messages were dropped in last 5000 ms: 41552 for internal timeout
> and 0 for cross node timeout
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,157 StatusLogger.java:52 - Pool
> NameActive Pending Completed Blocked All Time
> Blocked
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 -
> MutationStage 256 81824 3360532756 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 -
> ViewMutationStage 0 0 0 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,168 StatusLogger.java:56 -
> ReadStage 0 0 62862266 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 -
> RequestResponseStage 0 0 2176659856 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 -
> ReadRepairStage 0 0 0 0
> 0
> INFO [ScheduledTasks:1] 2019-11-15 08:24:25,169 StatusLogger.java:56 -
> CounterMutationStage 0 0 0 0
> 0
> ...
> {noformat}
> Judging from a 15min JFR session for both, 3.0.18 and 2.1.18 on a different
> node, high-level, it looks like the code path underneath
> {{BatchMessage.execute}} is producing ~ 10x more on-heap allocations in
> 3.0.18 compared to 2.1.18.
> !jfr_allocations.png!
> Left => 3.0.18
> Right => 2.1.18
> JFRs zipped are exceeding the 60MB limit to directly attach to the ticket. I
> can upload them, if there is another destination available.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13917) COMPACT STORAGE queries on dense static tables accept hidden column1 and value columns
[
https://issues.apache.org/jira/browse/CASSANDRA-13917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17017897#comment-17017897
]
Alex Petrov commented on CASSANDRA-13917:
-
[~slebresne] you are right. In fact, I've found several more places where this
could cause a problem. What we should've done initially is just add a separate
method for CQL layer, and let the rest of the calls (e.g., all internal stuff)
to use the old one. Committed version of the patch also caused problems in
mixed version environments which was caught by one of the upgrade in-jvm
dtests. Also, things like thrift and compact storage compatibility layer was
impacted in a way, too.
I'm still thinking if {{RowUpdateBuilder}} should be using CQL columns or all
columns. I'd argue that CQL only (e.g., no hidden), since this is also how we
query them for the most part. We don't really those hidden column even on
internal paths.
I've made a patch and have added a couple more tests that validates the
situations that weren't covered previously. Thanks to [CASSANDRA-15506], we can
also make sure that upgrade tests are going to be running.
|[patch|https://github.com/apache/cassandra/compare/trunk...ifesdjeen:CASSANDRA-13917-followup]|[CI|https://circleci.com/gh/ifesdjeen/cassandra/tree/CASSANDRA-13917-followup]|
> COMPACT STORAGE queries on dense static tables accept hidden column1 and
> value columns
> --
>
> Key: CASSANDRA-13917
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13917
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core
>Reporter: Alex Petrov
>Assignee: Aleksandr Sorokoumov
>Priority: Low
> Labels: lhf
> Fix For: 3.0.x, 3.11.x
>
> Attachments: 13917-3.0-testall-13.12.2019,
> 13917-3.0-testall-16.01.2020, 13917-3.0-testall-2.png,
> 13917-3.0-testall-20.11.2019.png, 13917-3.0-upgrade-16.01.2020,
> 13917-3.0.png, 13917-3.11-testall-13.12.2019,
> 13917-3.11-testall-16.01.2020.png, 13917-3.11-testall-2.png,
> 13917-3.11-testall-20.11.2019.png, 13917-3.11-upgrade-16.01.2020.png,
> 13917-3.11.png
>
>
> Test for the issue:
> {code}
> @Test
> public void testCompactStorage() throws Throwable
> {
> createTable("CREATE TABLE %s (a int PRIMARY KEY, b int, c int) WITH
> COMPACT STORAGE");
> assertInvalid("INSERT INTO %s (a, b, c, column1) VALUES (?, ?, ?,
> ?)", 1, 1, 1, ByteBufferUtil.bytes('a'));
> // This one fails with Some clustering keys are missing: column1,
> which is still wrong
> assertInvalid("INSERT INTO %s (a, b, c, value) VALUES (?, ?, ?, ?)",
> 1, 1, 1, ByteBufferUtil.bytes('a'));
> assertInvalid("INSERT INTO %s (a, b, c, column1, value) VALUES (?, ?,
> ?, ?, ?)", 1, 1, 1, ByteBufferUtil.bytes('a'), ByteBufferUtil.bytes('b'));
> assertEmpty(execute("SELECT * FROM %s"));
> }
> {code}
> Gladly, these writes are no-op, even though they succeed.
> {{value}} and {{column1}} should be completely hidden. Fixing this one should
> be as easy as just adding validations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15505) Add message interceptors to in-jvm dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-15505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17017884#comment-17017884 ] Alex Petrov edited comment on CASSANDRA-15505 at 1/17/20 10:27 AM: --- [~dcapwell] main motivation to make API the way it is right now is: 1. Internal Cassandra messages are version-dependent and even if their classes matched (which is not the case after the recent refactoring), we should not expose Cassandra internals through in-jvm dtest framework. This means that we can only pass already serialised messages. 2. Serialization mechanisms (Cassandra and Java) here unfortunately serve different purposes. Cassandra serialiser just encodes message to a byte buffer and wraps it into a class that performs actual routing. Java serialisation makes sure that the message can pass the boundary of the instance. 3. Messages are not updated and the intent here is not allowing updating, but inspecting them. Inspecting messages is possible only in when you have Cassandra on class path, and often requires running instance state for decoding (like tables, keyspaces, etc), so I decided to make an example in tests that performs inspection directly on the node. As regards the tests this could be useful for, since I initially thought we could count received repair messages using interceptors in the failing repair test. I'm also frequently adding netty handler for logging for tracing messages between the nodes, and I can imagine a few tests where we could rely on message visibility for validation. I have tried to find an example of "calling run like in failing repair test", and I assume you mean that you mean we can either read metrics, or we can even subscribe to the message sink without having this possibility built in in the framework. I still see some value to have it directly here though. was (Author: ifesdjeen): [~dcapwell] main motivation to make API the way it is right now is: 1. Internal Cassandra messages are version-dependent and even if their classes matched (which is not the case after the recent refactoring), we should not expose Cassandra internals through in-jvm dtest framework. This means that we can only pass already serialised messages. 2. Serialization mechanisms (Cassandra and Java here unfortunately serve different purposes). Cassandra serialiser just encodes message to a byte buffer and wraps it into a class that performs actual routing. Java serialisation makes sure that the message can pass the boundary of the instance. 3. Messages are not updated and the intent here is not allowing updating, but inspecting them. But since inspecting messages is possible only in when you have Cassandra on class path, and often requires running instance state for decoding (like tables, keyspaces, etc), I decided to make an example in tests that performs inspection directly on the node. As regards the tests this could be useful for, since I initially thought we could count received repair messages using interceptors in the failing repair test. I'm also frequently adding netty handler for logging for tracing messages between the nodes, and I can imagine a few tests where we could rely on message visibility for validation. I have tried to find an example of "calling run like in failing repair test", and I assume you mean that you mean we can either read metrics, or we can even subscribe to the message sink without having this possibility built in in the framework. I still see some value to have it directly here though. > Add message interceptors to in-jvm dtests > - > > Key: CASSANDRA-15505 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15505 > Project: Cassandra > Issue Type: New Feature > Components: Test/dtest >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > > Currently we only have means to filter messages in in-jvm tests. We need a > facility to intercept and modify the messages between nodes for testing > purposes. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15505) Add message interceptors to in-jvm dtests
[ https://issues.apache.org/jira/browse/CASSANDRA-15505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17017884#comment-17017884 ] Alex Petrov commented on CASSANDRA-15505: - [~dcapwell] main motivation to make API the way it is right now is: 1. Internal Cassandra messages are version-dependent and even if their classes matched (which is not the case after the recent refactoring), we should not expose Cassandra internals through in-jvm dtest framework. This means that we can only pass already serialised messages. 2. Serialization mechanisms (Cassandra and Java here unfortunately serve different purposes). Cassandra serialiser just encodes message to a byte buffer and wraps it into a class that performs actual routing. Java serialisation makes sure that the message can pass the boundary of the instance. 3. Messages are not updated and the intent here is not allowing updating, but inspecting them. But since inspecting messages is possible only in when you have Cassandra on class path, and often requires running instance state for decoding (like tables, keyspaces, etc), I decided to make an example in tests that performs inspection directly on the node. As regards the tests this could be useful for, since I initially thought we could count received repair messages using interceptors in the failing repair test. I'm also frequently adding netty handler for logging for tracing messages between the nodes, and I can imagine a few tests where we could rely on message visibility for validation. I have tried to find an example of "calling run like in failing repair test", and I assume you mean that you mean we can either read metrics, or we can even subscribe to the message sink without having this possibility built in in the framework. I still see some value to have it directly here though. > Add message interceptors to in-jvm dtests > - > > Key: CASSANDRA-15505 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15505 > Project: Cassandra > Issue Type: New Feature > Components: Test/dtest >Reporter: Alex Petrov >Assignee: Alex Petrov >Priority: Normal > > Currently we only have means to filter messages in in-jvm tests. We need a > facility to intercept and modify the messages between nodes for testing > purposes. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15508) Failing jvm dtest: FailingRepairTest.testFailingMessage
[
https://issues.apache.org/jira/browse/CASSANDRA-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17017851#comment-17017851
]
Marcus Eriksson edited comment on CASSANDRA-15508 at 1/17/20 9:47 AM:
--
the reason I separated them and had the dtest jar build as "type: approval" was
to avoid rebuilding cassandra 5 extra times on every commit
what has changed since CASSANDRA-15014 to make {{ant testclasslist}} work now?
was (Author: krummas):
the reason I separated them and had the dtest jar build as "type: approval" was
to avoid rebuilding cassandra 5 extra times on every commit
> Failing jvm dtest: FailingRepairTest.testFailingMessage
> ---
>
> Key: CASSANDRA-15508
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15508
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Marcus Eriksson
>Assignee: David Capwell
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It seems we can't run parameterized unit tests with {{ant testsome}}:
> {code}
> $ ant testsome
> -Dtest.name=org.apache.cassandra.distributed.test.FailingRepairTest
> -Dtest.methods=testFailingMessage
>
> [junit-timeout] Testcase:
> initializationError(org.junit.runner.manipulation.Filter):Caused an ERROR
> [junit-timeout] No tests found matching Method
> testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest)
> from org.junit.internal.requests.ClassRequest@4d95d2a2
> [junit-timeout] java.lang.Exception: No tests found matching Method
> testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest)
> from org.junit.internal.requests.ClassRequest@4d95d2a2
> [junit-timeout] at
> java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15508) Failing jvm dtest: FailingRepairTest.testFailingMessage
[
https://issues.apache.org/jira/browse/CASSANDRA-15508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-15508:
Reviewers: Marcus Eriksson
> Failing jvm dtest: FailingRepairTest.testFailingMessage
> ---
>
> Key: CASSANDRA-15508
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15508
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Marcus Eriksson
>Assignee: David Capwell
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It seems we can't run parameterized unit tests with {{ant testsome}}:
> {code}
> $ ant testsome
> -Dtest.name=org.apache.cassandra.distributed.test.FailingRepairTest
> -Dtest.methods=testFailingMessage
>
> [junit-timeout] Testcase:
> initializationError(org.junit.runner.manipulation.Filter):Caused an ERROR
> [junit-timeout] No tests found matching Method
> testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest)
> from org.junit.internal.requests.ClassRequest@4d95d2a2
> [junit-timeout] java.lang.Exception: No tests found matching Method
> testFailingMessage(org.apache.cassandra.distributed.test.FailingRepairTest)
> from org.junit.internal.requests.ClassRequest@4d95d2a2
> [junit-timeout] at
> java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
