[cassandra-builds] branch master updated: In Jenkins, propagate FAILURE from stages to pipeline
This is an automated email from the ASF dual-hosted git repository.
mck pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git
The following commit(s) were added to refs/heads/master by this push:
new bc7baa8 In Jenkins, propagate FAILURE from stages to pipeline
bc7baa8 is described below
commit bc7baa86bfdb79c58eefaae96e7b2e96ed9d93a0
Author: Mick Semb Wever
AuthorDate: Thu Aug 27 07:39:54 2020 +0200
In Jenkins, propagate FAILURE from stages to pipeline
ref: https://the-asf.slack.com/archives/C0162JU2CKY/p1598292846001000
---
jenkins-dsl/cassandra_pipeline.groovy | 11 +++
1 file changed, 11 insertions(+)
diff --git a/jenkins-dsl/cassandra_pipeline.groovy
b/jenkins-dsl/cassandra_pipeline.groovy
index e8eb114..12a7428 100644
--- a/jenkins-dsl/cassandra_pipeline.groovy
+++ b/jenkins-dsl/cassandra_pipeline.groovy
@@ -31,6 +31,7 @@ pipeline {
script {
stress = build job: "${env.JOB_NAME}-stress-test",
parameters: [string(name: 'REPO', value: params.REPO), string(name: 'BRANCH',
value: params.BRANCH)], propagate: false
if (stress.result != 'SUCCESS') unstable('stress test
failures')
+ if (stress.result == 'FAILURE')
currentBuild.result='FAILURE'
}
}
post {
@@ -48,6 +49,7 @@ pipeline {
script {
fqltool = build job: "${env.JOB_NAME}-fqltool-test",
parameters: [string(name: 'REPO', value: params.REPO), string(name: 'BRANCH',
value: params.BRANCH)], propagate: false
if (fqltool.result != 'SUCCESS') unstable('fqltool test
failures')
+ if (fqltool.result == 'FAILURE')
currentBuild.result='FAILURE'
}
}
post {
@@ -65,6 +67,7 @@ pipeline {
script {
jvm_dtest = build job: "${env.JOB_NAME}-jvm-dtest",
parameters: [string(name: 'REPO', value: params.REPO), string(name: 'BRANCH',
value: params.BRANCH)], propagate: false
if (jvm_dtest.result != 'SUCCESS') unstable('jvm-dtest
failures')
+ if (jvm_dtest.result == 'FAILURE')
currentBuild.result='FAILURE'
}
}
post {
@@ -82,6 +85,7 @@ pipeline {
script {
jvm_dtest_upgrade = build job:
"${env.JOB_NAME}-jvm-dtest-upgrade", parameters: [string(name: 'REPO', value:
params.REPO), string(name: 'BRANCH', value: params.BRANCH)], propagate: false
if (jvm_dtest_upgrade.result != 'SUCCESS')
unstable('jvm-dtest-upgrade failures')
+ if (jvm_dtest_upgrade.result == 'FAILURE')
currentBuild.result='FAILURE'
}
}
post {
@@ -99,6 +103,7 @@ pipeline {
script {
test = build job: "${env.JOB_NAME}-test", parameters:
[string(name: 'REPO', value: params.REPO), string(name: 'BRANCH', value:
params.BRANCH)], propagate: false
if (test.result != 'SUCCESS') unstable('unit test failures')
+ if (test.result == 'FAILURE') currentBuild.result='FAILURE'
}
}
post {
@@ -116,6 +121,7 @@ pipeline {
script {
long_test = build job: "${env.JOB_NAME}-long-test",
parameters: [string(name: 'REPO', value: params.REPO), string(name: 'BRANCH',
value: params.BRANCH)], propagate: false
if (long_test.result != 'SUCCESS') unstable('long unit test
failures')
+ if (long_test.result == 'FAILURE')
currentBuild.result='FAILURE'
}
}
post {
@@ -133,6 +139,7 @@ pipeline {
script {
burn = build job: "${env.JOB_NAME}-test-burn", parameters:
[string(name: 'REPO', value: params.REPO), string(name: 'BRANCH', value:
params.BRANCH)], propagate: false
if (burn.result != 'SUCCESS') unstable('burn test failures')
+ if (burn.result == 'FAILURE') currentBuild.result='FAILURE'
}
}
post {
@@ -150,6 +157,7 @@ pipeline {
script {
cdc = build job: "${env.JOB_NAME}-test-cdc", parameters:
[string(name: 'REPO', value: params.REPO), string(name: 'BRANCH', value:
params.BRANCH)], propagate: false
if (cdc.result != 'SUCCESS') unstable('cdc failures')
+ if (cdc.result == 'FAILURE') currentBuild.result='FAILURE'
}
}
post {
@@ -167,6 +175,7 @@ pipeline {
script {
compression = build job: "${env.JOB_NAME}-test-compression",
parameters: [string(name: 'REPO', value: params.REPO), string(name: 'BRANCH',
value: params.BRANCH)], propagate: false
if
[jira] [Comment Edited] (CASSANDRA-16073) cql_tracing_test.py failing repeatedly on jenkins
[ https://issues.apache.org/jira/browse/CASSANDRA-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185603#comment-17185603 ] Berenguer Blasi edited comment on CASSANDRA-16073 at 8/27/20, 5:37 AM: --- Seems latest jenkins run spike to 50-ish failures. This is strange as the dtest runs pre-merge where clean. I have pushed a new [PR|https://github.com/apache/cassandra-dtest/pull/92] to revert the generic timeout change in favor of a specific test fix. -I will need sbdy to fire a dtest ci for me...- Edit: Waiting on CI was (Author: bereng): Seems latest jenkins run spike to 50-ish failures. This is strange as the dtest runs pre-merge where clean. I have pushed a new [PR|https://github.com/apache/cassandra-dtest/pull/92] to revert the generic timeout change in favor of a specific test fix. I will need sbdy to fire a dtest ci for me... > cql_tracing_test.py failing repeatedly on jenkins > - > > Key: CASSANDRA-16073 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16073 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 50m > Remaining Estimate: 0h > > Jenkins has been failing tracing tests repeatedly -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16073) cql_tracing_test.py failing repeatedly on jenkins
[ https://issues.apache.org/jira/browse/CASSANDRA-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-16073: Status: Patch Available (was: Open) > cql_tracing_test.py failing repeatedly on jenkins > - > > Key: CASSANDRA-16073 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16073 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 50m > Remaining Estimate: 0h > > Jenkins has been failing tracing tests repeatedly -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16073) cql_tracing_test.py failing repeatedly on jenkins
[ https://issues.apache.org/jira/browse/CASSANDRA-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-16073: Status: Open (was: Resolved) Seems latest jenkins run spike to 50-ish failures. This is strange as the dtest runs pre-merge where clean. I have pushed a new [PR|https://github.com/apache/cassandra-dtest/pull/92] to revert the generic timeout change in favor of a specific test fix. I will need sbdy to fire a dtest ci for me... > cql_tracing_test.py failing repeatedly on jenkins > - > > Key: CASSANDRA-16073 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16073 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 50m > Remaining Estimate: 0h > > Jenkins has been failing tracing tests repeatedly -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16029) Create better Apache Cassandra 4.0 docs
[
https://issues.apache.org/jira/browse/CASSANDRA-16029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185507#comment-17185507
]
Lorina Poland edited comment on CASSANDRA-16029 at 8/26/20, 10:40 PM:
--
Note to self: Need to make sure the Prev/Next works in the UI.
You have to enable the {{page-pagination}} attribute:
[https://docs.antora.org/antora/2.3/whats-new/#smarter-pagination]
was (Author: polandll):
Note to self: Need to make sure the Prev/Next works in the UI.
> Create better Apache Cassandra 4.0 docs
> ---
>
> Key: CASSANDRA-16029
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16029
> Project: Cassandra
> Issue Type: Improvement
> Components: Documentation/Website
>Reporter: Lorina Poland
>Assignee: Lorina Poland
>Priority: Normal
> Fix For: 4.0
>
> Attachments: CurrentDocs.png, NewImprovedDocs.png
>
>
> The proof of concept to create new Documentation showed that a combination of
> the static site generator (SSG) Antora + Asciidoc files would provide the
> most flexibility in redesigning the Apache C* OSS pages.
> Current proof of concept: [https://polandll.github.io/site/Cassandra/4.0/]
> To update the docs, the tools to write needed an update, too, to provide a
> better UX and modern look and feel. The old tools (reStructured Text, Sphinx)
> will be replaced with the new tools ([AsciiDoc|http://asciidoc.org/],
> [Antora|http://antora.org/]).
> See the attachments for screenshots of the current docs and the POC docs.
> The advantages of asciidoc+antora:
> * Rich markdown language that is directly editable.
> * Good organizational structure for docs files
> * Flexible build/publishing capabilities, including content from multiple
> repositories and branches
> * Flexible web UI, built separately from the doc files and static site.
> * JS extensions that enable additional functionality.
> To complete the conversion, the following steps are required:
> * Pandoc used for conversion of rSt files to adoc files
> ** Followed by significant manual editing to fix the errors left over after
> conversion
> * Creation of new antora files (site.yml, antora.yml, CSS and layout)
> ** Integration with plug-ins (lunr for search, tabs for additional codeblock
> capabilities)
> ** Finish the UI to match current Apache C* UI, or engage designer to do a
> new design
> *** Put the ASF pulldown in the main header banner, like Apache Spark does
> ([https://spark.apache.org/downloads.html])
> * Modification of build scripts to work with Antora
> ** Modification of python scripts that generation YAML and nodetool docs
> *** Make sure that these scripts are run for each version
> ** Modification of cassandra/doc Makefile
> ** Dockerfile and docker-entrypoint.sh files to use Docker for documentation
> build
> * Modification of cassandra-website to work with Antora
> ** Dockerfile and docker-entrypoint.sh files to use Docker for documentation
> build
> * Figure out how to handle older versions that are not converted to asciidoc
> now
> ** Put in TBD? Copy 4.0 branch with note to rewrite later?
> * Figure out why tabs-block antora extension is working locally but not in
> GH pages (may or may not be important)
> Other tasks:
> * Complete conversion, plus editing the current docs before Apache C* 4.0 GA
> * Further improvements for docs organization
> **
> [https://docs.google.com/document/d/1aeNtgyPAsKcNa0GSKvl2ywlFEj30714ry4Sb5turWeE/edit?usp=sharing]
> ** Get some sections out of Docs, like Developing Code info -> Community
> * Solve the issue of CQL highlighting/lexer/parser issue
> ** Need CQL to be submitted to pygments
> * Addition of more useful documentation - tutorials, how-to guides, separate
> reference guide
> Current work:
> [https://github.com/polandll/cassandra/blob/doc_redo_asciidoc/|https://github.com/polandll/cassandra/blob/doc_redo_asciidoc/doc/Dockerfile]
> Relevant Links:
> Original source for antora-ui: [https://gitlab.com/antora/antora-ui-default]
> (Mozilla public license 2.0)
> Current working repo for antora-ui:
> [https://github.com/ianjevans/antora-ui-datastax] in oss branch
> Lunr search implemented with: [https://github.com/Mogztter/antora-lunr] and
> [https://github.com/Mogztter/antora-site-generator-lunr]
> (MIT license)
> Codeblock tabs implemented with: Dan Allen's tabs-block.js code (finding
> source).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15393) Add byte array backed cells
[ https://issues.apache.org/jira/browse/CASSANDRA-15393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185510#comment-17185510 ] Caleb Rackliffe commented on CASSANDRA-15393: - bq. The idea of combining the byte array util and the accessor has also been floating around, and I'd prefer we didn't for both the reason above, and because people are pretty used to having ByteBufferUtil handy. Having ByteArrayUtil handy would make the appearance of byte[] everywhere a little less jarring (hopefully). bq. NativeCell prevents moving most of that into AbstractCell. Given the simplicity of the parts that can be factored, I'd lean towards keeping the class hierarchy simple over minimizing the size of the array and buffer cell implementations. Agreed on both points. > Add byte array backed cells > --- > > Key: CASSANDRA-15393 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15393 > Project: Cassandra > Issue Type: Sub-task > Components: Local/Compaction >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 20m > Remaining Estimate: 0h > > We currently materialize all values as on heap byte buffers. Byte buffers > have a fairly high overhead given how frequently they’re used, and on the > compaction and local read path we don’t do anything that needs them. Use of > byte buffer methods only happens on the coordinator. Using cells that are > backed by byte arrays instead in these situations reduces compaction and read > garbage up to 22% in many cases. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15958) org.apache.cassandra.net.ConnectionTest testMessagePurging
[ https://issues.apache.org/jira/browse/CASSANDRA-15958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17183526#comment-17183526 ] Adam Holmberg edited comment on CASSANDRA-15958 at 8/26/20, 10:39 PM: -- The timeout occurs [waiting for this close future|https://github.com/aholmberg/cassandra/blob/CASSANDRA-15958/test/unit/org/apache/cassandra/net/ConnectionTest.java#L268], failing intermittently due to a race condition. The test [closes|https://github.com/aholmberg/cassandra/blob/CASSANDRA-15958/test/unit/org/apache/cassandra/net/ConnectionTest.java#L723] the inbound connection [twice|https://github.com/aholmberg/cassandra/blob/CASSANDRA-15958/test/unit/org/apache/cassandra/net/ConnectionTest.java#L268]. If the first execution finishes and [shuts down the executor|https://github.com/aholmberg/cassandra/blob/CASSANDRA-15958/src/java/org/apache/cassandra/net/InboundSockets.java#L128] before the second, the [FutureCombiner|https://github.com/aholmberg/cassandra/blob/CASSANDRA-15958/src/java/org/apache/cassandra/net/InboundSockets.java#L125-L126] fails to [addListener|https://github.com/aholmberg/cassandra/blob/CASSANDRA-15958/src/java/org/apache/cassandra/net/FutureCombiner.java#L83] to the channel, and the [done future|https://github.com/aholmberg/cassandra/blob/CASSANDRA-15958/src/java/org/apache/cassandra/net/InboundSockets.java#L131] will never complete. was (Author: aholmber): [patch|https://github.com/apache/cassandra/compare/trunk...aholmberg:CASSANDRA-15958?expand=1] The timeout occurs [waiting for this close future|https://github.com/aholmberg/cassandra/blob/CASSANDRA-15958/test/unit/org/apache/cassandra/net/ConnectionTest.java#L268], failing intermittently due to a race condition. The test [closes|https://github.com/aholmberg/cassandra/blob/CASSANDRA-15958/test/unit/org/apache/cassandra/net/ConnectionTest.java#L723] the inbound connection [twice|https://github.com/aholmberg/cassandra/blob/CASSANDRA-15958/test/unit/org/apache/cassandra/net/ConnectionTest.java#L268]. If the first execution finishes and [shuts down the executor|https://github.com/aholmberg/cassandra/blob/CASSANDRA-15958/src/java/org/apache/cassandra/net/InboundSockets.java#L128] before the second, the [FutureCombiner|https://github.com/aholmberg/cassandra/blob/CASSANDRA-15958/src/java/org/apache/cassandra/net/InboundSockets.java#L125-L126] fails to [addListener|https://github.com/aholmberg/cassandra/blob/CASSANDRA-15958/src/java/org/apache/cassandra/net/FutureCombiner.java#L83] to the channel, and the [done future|https://github.com/aholmberg/cassandra/blob/CASSANDRA-15958/src/java/org/apache/cassandra/net/InboundSockets.java#L131] will never complete. > org.apache.cassandra.net.ConnectionTest testMessagePurging > -- > > Key: CASSANDRA-15958 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15958 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: David Capwell >Assignee: Adam Holmberg >Priority: Normal > Fix For: 4.0-beta > > > Build: > https://ci-cassandra.apache.org/job/Cassandra-trunk-test/196/testReport/junit/org.apache.cassandra.net/ConnectionTest/testMessagePurging/ > Build: > https://ci-cassandra.apache.org/job/Cassandra-trunk-test/194/testReport/junit/org.apache.cassandra.net/ConnectionTest/testMessagePurging/ > java.util.concurrent.TimeoutException > at org.apache.cassandra.net.AsyncPromise.get(AsyncPromise.java:258) > at org.apache.cassandra.net.FutureDelegate.get(FutureDelegate.java:143) > at > org.apache.cassandra.net.ConnectionTest.doTestManual(ConnectionTest.java:268) > at > org.apache.cassandra.net.ConnectionTest.testManual(ConnectionTest.java:236) > at > org.apache.cassandra.net.ConnectionTest.testMessagePurging(ConnectionTest.java:679) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16029) Create better Apache Cassandra 4.0 docs
[ https://issues.apache.org/jira/browse/CASSANDRA-16029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185507#comment-17185507 ] Lorina Poland commented on CASSANDRA-16029: --- Note to self: Need to make sure the Prev/Next works in the UI. > Create better Apache Cassandra 4.0 docs > --- > > Key: CASSANDRA-16029 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16029 > Project: Cassandra > Issue Type: Improvement > Components: Documentation/Website >Reporter: Lorina Poland >Assignee: Lorina Poland >Priority: Normal > Fix For: 4.0 > > Attachments: CurrentDocs.png, NewImprovedDocs.png > > > The proof of concept to create new Documentation showed that a combination of > the static site generator (SSG) Antora + Asciidoc files would provide the > most flexibility in redesigning the Apache C* OSS pages. > Current proof of concept: [https://polandll.github.io/site/Cassandra/4.0/] > To update the docs, the tools to write needed an update, too, to provide a > better UX and modern look and feel. The old tools (reStructured Text, Sphinx) > will be replaced with the new tools ([AsciiDoc|http://asciidoc.org/], > [Antora|http://antora.org/]). > See the attachments for screenshots of the current docs and the POC docs. > The advantages of asciidoc+antora: > * Rich markdown language that is directly editable. > * Good organizational structure for docs files > * Flexible build/publishing capabilities, including content from multiple > repositories and branches > * Flexible web UI, built separately from the doc files and static site. > * JS extensions that enable additional functionality. > To complete the conversion, the following steps are required: > * Pandoc used for conversion of rSt files to adoc files > ** Followed by significant manual editing to fix the errors left over after > conversion > * Creation of new antora files (site.yml, antora.yml, CSS and layout) > ** Integration with plug-ins (lunr for search, tabs for additional codeblock > capabilities) > ** Finish the UI to match current Apache C* UI, or engage designer to do a > new design > *** Put the ASF pulldown in the main header banner, like Apache Spark does > ([https://spark.apache.org/downloads.html]) > * Modification of build scripts to work with Antora > ** Modification of python scripts that generation YAML and nodetool docs > *** Make sure that these scripts are run for each version > ** Modification of cassandra/doc Makefile > ** Dockerfile and docker-entrypoint.sh files to use Docker for documentation > build > * Modification of cassandra-website to work with Antora > ** Dockerfile and docker-entrypoint.sh files to use Docker for documentation > build > * Figure out how to handle older versions that are not converted to asciidoc > now > ** Put in TBD? Copy 4.0 branch with note to rewrite later? > * Figure out why tabs-block antora extension is working locally but not in > GH pages (may or may not be important) > Other tasks: > * Complete conversion, plus editing the current docs before Apache C* 4.0 GA > * Further improvements for docs organization > ** > [https://docs.google.com/document/d/1aeNtgyPAsKcNa0GSKvl2ywlFEj30714ry4Sb5turWeE/edit?usp=sharing] > ** Get some sections out of Docs, like Developing Code info -> Community > * Solve the issue of CQL highlighting/lexer/parser issue > ** Need CQL to be submitted to pygments > * Addition of more useful documentation - tutorials, how-to guides, separate > reference guide > Current work: > [https://github.com/polandll/cassandra/blob/doc_redo_asciidoc/|https://github.com/polandll/cassandra/blob/doc_redo_asciidoc/doc/Dockerfile] > Relevant Links: > Original source for antora-ui: [https://gitlab.com/antora/antora-ui-default] > (Mozilla public license 2.0) > Current working repo for antora-ui: > [https://github.com/ianjevans/antora-ui-datastax] in oss branch > Lunr search implemented with: [https://github.com/Mogztter/antora-lunr] and > [https://github.com/Mogztter/antora-site-generator-lunr] > (MIT license) > Codeblock tabs implemented with: Dan Allen's tabs-block.js code (finding > source). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15393) Add byte array backed cells
[
https://issues.apache.org/jira/browse/CASSANDRA-15393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185506#comment-17185506
]
Caleb Rackliffe edited comment on CASSANDRA-15393 at 8/26/20, 10:35 PM:
bq. I'd done something sort of along these lines (albeit in the opposite
direction) by adding the `ValueAware` class, so that AbstractType et al can
operate directly on Cell objects without having to know what a cell is.
Although I seem to default to keeping type information on the serializer / type
side of things, there are benefits to each which we should discuss. And
possibly a better name for ValueAware.
Unless we have a concrete plan for {{ValueAware}}, why don't we just remove it
and use {{Cell}} directly? There might not be a real benefit to hiding the
concept of a cell from the types and serializers. My comments here (ex.
{{getLong()}}) are driving toward trying to eliminate as many places where we
actually have to explicitly pair value and accessor as possible. All {{Cell}}
instances know what they are backed by, and hiding as much of that from callers
as possible seems valuable.
was (Author: maedhroz):
bq. I'd done something sort of along these lines (albeit in the opposite
direction) by adding the `ValueAware` class, so that AbstractType et al can
operate directly on Cell objects without having to know what a cell is.
Although I seem to default to keeping type information on the serializer / type
side of things, there are benefits to each which we should discuss. And
possibly a better name for ValueAware.
Unless we have a concrete plan for {{ValueAware}}, why don't we just remove it
and use {{Cell}} directly? There might not be a real benefit to hiding the
concept of a cell from the types and serializers. My comments here (ex.
{{getLong()}}) are driving toward trying to eliminate as many places where we
actually have to explicitly pair value and accessor as possible.
> Add byte array backed cells
> ---
>
> Key: CASSANDRA-15393
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15393
> Project: Cassandra
> Issue Type: Sub-task
> Components: Local/Compaction
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> We currently materialize all values as on heap byte buffers. Byte buffers
> have a fairly high overhead given how frequently they’re used, and on the
> compaction and local read path we don’t do anything that needs them. Use of
> byte buffer methods only happens on the coordinator. Using cells that are
> backed by byte arrays instead in these situations reduces compaction and read
> garbage up to 22% in many cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15393) Add byte array backed cells
[
https://issues.apache.org/jira/browse/CASSANDRA-15393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185506#comment-17185506
]
Caleb Rackliffe commented on CASSANDRA-15393:
-
bq. I'd done something sort of along these lines (albeit in the opposite
direction) by adding the `ValueAware` class, so that AbstractType et al can
operate directly on Cell objects without having to know what a cell is.
Although I seem to default to keeping type information on the serializer / type
side of things, there are benefits to each which we should discuss. And
possibly a better name for ValueAware.
Unless we have a concrete plan for {{ValueAware}}, why don't we just remove it
and use {{Cell}} directly? There might not be a real benefit to hiding the
concept of a cell from the types and serializers. My comments here (ex.
{{getLong()}}) are driving toward trying to eliminate as many places where we
actually have to explicitly pair value and accessor as possible.
> Add byte array backed cells
> ---
>
> Key: CASSANDRA-15393
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15393
> Project: Cassandra
> Issue Type: Sub-task
> Components: Local/Compaction
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> We currently materialize all values as on heap byte buffers. Byte buffers
> have a fairly high overhead given how frequently they’re used, and on the
> compaction and local read path we don’t do anything that needs them. Use of
> byte buffer methods only happens on the coordinator. Using cells that are
> backed by byte arrays instead in these situations reduces compaction and read
> garbage up to 22% in many cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16064) Add test which validates that Message serializedSize(version) == serialize(out, version).length
[ https://issues.apache.org/jira/browse/CASSANDRA-16064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-16064: -- Fix Version/s: (was: 4.0-beta) 4.0-beta2 Since Version: 4.0-beta1 Source Control Link: https://github.com/apache/cassandra/commit/0697489f7455fa078cb454776b98236f18f82dd4 Resolution: Fixed Status: Resolved (was: Ready to Commit) CI results: https://app.circleci.com/pipelines/github/dcapwell/cassandra/455/workflows/a60aef20-a3e0-4f3a-b055-93e9b040d82a there were flaky tests and I forgot to merge the python dtest changes in CASSANDRA-7544 so those tests also failed (so pushed the tests). > Add test which validates that Message serializedSize(version) == > serialize(out, version).length > --- > > Key: CASSANDRA-16064 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16064 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Internode >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta2 > > > In 4.0 we require serializedSize(version) == serialize(out, version).length > for correctness in post40 message format as we write it into the message > header. Given that this is a strong requirement for correct deserialization > of the message, we should have tests which help enforce this property. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Add test which validates that Message serializedSize(version) == serialize(out, version).length
This is an automated email from the ASF dual-hosted git repository.
dcapwell pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new 0697489 Add test which validates that Message serializedSize(version)
== serialize(out, version).length
0697489 is described below
commit 0697489f7455fa078cb454776b98236f18f82dd4
Author: David Capwell
AuthorDate: Wed Aug 26 14:34:32 2020 -0700
Add test which validates that Message serializedSize(version) ==
serialize(out, version).length
patch by David Capwell; reviewed by Caleb Rackliffe, Jon Meredith for
CASSANDRA-16064
---
build.xml | 2 +-
lib/commons-lang3-3.1.jar | Bin 315805 -> 0 bytes
lib/commons-lang3-3.11.jar | Bin 0 -> 577742 bytes
src/java/org/apache/cassandra/db/ReadCommand.java | 20 +-
.../org/apache/cassandra/db/SchemaCQLHelper.java | 22 ++
.../apache/cassandra/db/marshal/AbstractType.java | 2 +-
.../org/apache/cassandra/db/marshal/EmptyType.java | 8 +
.../org/apache/cassandra/db/marshal/UserType.java | 45 ++-
src/java/org/apache/cassandra/net/Message.java | 2 +-
.../org/apache/cassandra/net/MessagingService.java | 17 +-
src/java/org/apache/cassandra/net/PingRequest.java | 27 +-
src/java/org/apache/cassandra/schema/Schema.java | 19 +-
.../org/apache/cassandra/schema/TableMetadata.java | 2 +-
.../cassandra/schema/TableMetadataProvider.java| 24 ++
.../cassandra/serializers/TimestampSerializer.java | 10 +-
.../cassandra/serializers/UserTypeSerializer.java | 13 +-
.../org/apache/cassandra/utils/ByteArrayUtil.java | 209 +++
.../apache/cassandra/utils/FailingConsumer.java| 42 +++
.../cql3/validation/entities/TupleTypeTest.java| 157 +
.../cassandra/db/marshal/TimestampTypeTest.java| 45 +++
.../cassandra/db/marshal/TypeValidationTest.java | 132 ++-
.../net/MessageSerializationPropertyTest.java | 144
.../cassandra/utils/AbstractTypeGenerators.java| 381 +
.../apache/cassandra/utils/ByteArrayUtilTest.java | 211
.../cassandra/utils/CassandraGenerators.java | 286
.../cassandra/utils/FixedMonotonicClock.java | 68
.../org/apache/cassandra/utils/Generators.java | 354 ++-
.../org/apache/cassandra/utils/GeneratorsTest.java | 30 ++
28 files changed, 2203 insertions(+), 69 deletions(-)
diff --git a/build.xml b/build.xml
index 7b67314..66a0d2e 100644
--- a/build.xml
+++ b/build.xml
@@ -563,7 +563,7 @@
-
+
diff --git a/lib/commons-lang3-3.1.jar b/lib/commons-lang3-3.1.jar
deleted file mode 100644
index a85e539..000
Binary files a/lib/commons-lang3-3.1.jar and /dev/null differ
diff --git a/lib/commons-lang3-3.11.jar b/lib/commons-lang3-3.11.jar
new file mode 100644
index 000..bbaa8a6
Binary files /dev/null and b/lib/commons-lang3-3.11.jar differ
diff --git a/src/java/org/apache/cassandra/db/ReadCommand.java
b/src/java/org/apache/cassandra/db/ReadCommand.java
index ffdfc7c..60ddaa9 100644
--- a/src/java/org/apache/cassandra/db/ReadCommand.java
+++ b/src/java/org/apache/cassandra/db/ReadCommand.java
@@ -27,6 +27,7 @@ import java.util.function.Function;
import javax.annotation.Nullable;
+import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.Preconditions;
import com.google.common.collect.Iterables;
import com.google.common.collect.Sets;
@@ -60,6 +61,7 @@ import org.apache.cassandra.schema.Schema;
import org.apache.cassandra.schema.SchemaConstants;
import org.apache.cassandra.schema.TableId;
import org.apache.cassandra.schema.TableMetadata;
+import org.apache.cassandra.schema.TableMetadataProvider;
import org.apache.cassandra.service.ActiveRepairService;
import org.apache.cassandra.service.ClientWarn;
import org.apache.cassandra.tracing.Tracing;
@@ -904,8 +906,22 @@ public abstract class ReadCommand extends AbstractReadQuery
}
}
-private static class Serializer implements
IVersionedSerializer
+@VisibleForTesting
+public static class Serializer implements IVersionedSerializer
{
+private final TableMetadataProvider schema;
+
+public Serializer()
+{
+this(Schema.instance);
+}
+
+@VisibleForTesting
+public Serializer(TableMetadataProvider schema)
+{
+this.schema = Objects.requireNonNull(schema, "schema");
+}
+
private static int digestFlag(boolean isDigest)
{
return isDigest ? 0x01 : 0;
@@ -983,7 +999,7 @@ public abstract class ReadCommand extends AbstractReadQuery
boolean hasIndex = hasIndex(flags);
int digestVersion = isDigest ?
[jira] [Commented] (CASSANDRA-15393) Add byte array backed cells
[ https://issues.apache.org/jira/browse/CASSANDRA-15393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185499#comment-17185499 ] Caleb Rackliffe commented on CASSANDRA-15393: - bq. I'm inclined to go the other way, and make underlying type information opaque everywhere except the serializers themselves. There are very few places where we need to see it, and making it explicit anywhere it doesn't need to be just makes it more difficult to make changes later on. I suppose my starting disposition is wondering why we wouldn't _leave_ it explicit (i.e. as it is now) if the change hurts readability/clarity for the benefit of future changes we might not actually make (or if we do make them, might not look exactly the way we imagine they will here). > Add byte array backed cells > --- > > Key: CASSANDRA-15393 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15393 > Project: Cassandra > Issue Type: Sub-task > Components: Local/Compaction >Reporter: Blake Eggleston >Assignee: Blake Eggleston >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 20m > Remaining Estimate: 0h > > We currently materialize all values as on heap byte buffers. Byte buffers > have a fairly high overhead given how frequently they’re used, and on the > compaction and local read path we don’t do anything that needs them. Use of > byte buffer methods only happens on the coordinator. Using cells that are > backed by byte arrays instead in these situations reduces compaction and read > garbage up to 22% in many cases. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-dtest] branch master updated: Resolve JMX output inconsistencies from CASSANDRA-7544 storage-port-configurable-per-node
This is an automated email from the ASF dual-hosted git repository.
dcapwell pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git
The following commit(s) were added to refs/heads/master by this push:
new 36e2514 Resolve JMX output inconsistencies from CASSANDRA-7544
storage-port-configurable-per-node
36e2514 is described below
commit 36e25140a013843faf91be938c3301495957ed1d
Author: Jon Meredith
AuthorDate: Wed Aug 26 12:57:07 2020 -0700
Resolve JMX output inconsistencies from CASSANDRA-7544
storage-port-configurable-per-node
patch by Jon Meredith; reviewed by David Capwell for CASSANDRA-15937
---
replace_address_test.py | 23 +++
secondary_indexes_test.py | 2 +-
snitch_test.py| 24 +++-
3 files changed, 27 insertions(+), 22 deletions(-)
diff --git a/replace_address_test.py b/replace_address_test.py
index 0a34ac8..18b01a3 100644
--- a/replace_address_test.py
+++ b/replace_address_test.py
@@ -158,15 +158,14 @@ class BaseReplaceAddressTest(Tester):
def _verify_replacement(self, node, same_address):
if not same_address:
if self.cluster.cassandra_version() >= '2.2.7':
-address_prefix = '' if self.cluster.cassandra_version() >=
'4.0' else '/'
-node.watch_log_for("Node {}{} is replacing {}{}"
- .format(address_prefix,
self.replacement_node.address_for_current_version(),
- address_prefix,
self.replaced_node.address_for_current_version()),
+node.watch_log_for("Node {} is replacing {}"
+
.format(self.replacement_node.address_for_current_version_slashy(),
+
self.replaced_node.address_for_current_version_slashy()),
timeout=60, filename='debug.log')
-node.watch_log_for("Node {}{} will complete replacement of
{}{} for tokens"
- .format(address_prefix,
self.replacement_node.address_for_current_version(),
- address_prefix,
self.replaced_node.address_for_current_version()), timeout=10)
-node.watch_log_for("removing endpoint
{}{}".format(address_prefix, self.replaced_node.address_for_current_version()),
+node.watch_log_for("Node {} will complete replacement of {}
for tokens"
+
.format(self.replacement_node.address_for_current_version_slashy(),
+
self.replaced_node.address_for_current_version_slashy()), timeout=10)
+node.watch_log_for("removing endpoint
{}".format(self.replaced_node.address_for_current_version_slashy()),
timeout=60, filename='debug.log')
else:
node.watch_log_for("between /{} and /{}; /{} is the new owner"
@@ -183,8 +182,8 @@ class BaseReplaceAddressTest(Tester):
num_tokens =
int(self.replacement_node.get_conf_option('num_tokens'))
logger.debug("Verifying {} tokens migrated
successfully".format(num_tokens))
-replmnt_address = ("/" + self.replacement_node.address()) if
self.cluster.version() < '4.0' else self.replacement_node.address_and_port()
-repled_address = ("/" + self.replaced_node.address()) if
self.cluster.version() < '4.0' else self.replaced_node.address_and_port()
+replmnt_address =
self.replacement_node.address_for_current_version_slashy()
+repled_address =
self.replaced_node.address_for_current_version_slashy()
token_ownership_log = r"Token (.*?) changing ownership from {} to
{}".format(repled_address,
replmnt_address)
logs = self.replacement_node.grep_log(token_ownership_log)
@@ -324,7 +323,7 @@ class TestReplaceAddress(BaseReplaceAddressTest):
self._do_replace(replace_address='127.0.0.5',
wait_for_binary_proto=False)
logger.debug("Waiting for replace to fail")
-node_log_str = "/127.0.0.5" if self.cluster.version() < '4.0' else
"127.0.0.5:7000"
+node_log_str = "/127.0.0.5" if self.cluster.version() < '4.0' else
"/127.0.0.5:7000"
self.replacement_node.watch_log_for("java.lang.RuntimeException:
Cannot replace_address "
+ node_log_str + " because it
doesn't exist in gossip")
assert_not_running(self.replacement_node)
@@ -509,7 +508,7 @@ class TestReplaceAddress(BaseReplaceAddressTest):
self.replacement_node.stop()
logger.debug("Waiting other nodes to detect node stopped")
-node_log_str = ("/" + self.replacement_node.address()) if
self.cluster.version() < '4.0' else self.replacement_node.address_and_port()
+
[jira] [Commented] (CASSANDRA-15968) nodetool drain fails in jvm dtest
[
https://issues.apache.org/jira/browse/CASSANDRA-15968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185497#comment-17185497
]
David Capwell commented on CASSANDRA-15968:
---
my test ended up not needing this and lost my time so can't push this through.
The code was working and had a few comments to improve them, so should be low
hanging if someone wanted to pick up.
> nodetool drain fails in jvm dtest
> -
>
> Key: CASSANDRA-15968
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15968
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: David Capwell
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-beta
>
>
> Drain fails since MessagingService doesn’t expect to be shutdown twice.
> {code}
> Caused by: java.lang.AssertionError
> at
> org.apache.cassandra.net.MessagingService.shutdown(MessagingService.java:766)
> at
> org.apache.cassandra.distributed.impl.Instance.lambda$parallelRun$28(Instance.java:805)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ... 3 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15968) nodetool drain fails in jvm dtest
[
https://issues.apache.org/jira/browse/CASSANDRA-15968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell reassigned CASSANDRA-15968:
-
Assignee: (was: David Capwell)
> nodetool drain fails in jvm dtest
> -
>
> Key: CASSANDRA-15968
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15968
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: David Capwell
>Priority: Normal
> Fix For: 2.2.x, 3.0.x, 3.11.x, 4.0-beta
>
>
> Drain fails since MessagingService doesn’t expect to be shutdown twice.
> {code}
> Caused by: java.lang.AssertionError
> at
> org.apache.cassandra.net.MessagingService.shutdown(MessagingService.java:766)
> at
> org.apache.cassandra.distributed.impl.Instance.lambda$parallelRun$28(Instance.java:805)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> ... 3 more
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15393) Add byte array backed cells
[
https://issues.apache.org/jira/browse/CASSANDRA-15393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185489#comment-17185489
]
Caleb Rackliffe commented on CASSANDRA-15393:
-
bq. I'd prefer that we keep the type/serializers consistent wrt type. In other
words, I'd prefer some methods "unneccesarily" switch to using an accessor than
having some parts of a class use accessors and other parts use byte buffers
directly
I'm not sure. There is a clear penalty in terms of readability in places like
{{CompositeType#decompose()}}, and the main/public version of
{{AbstractType#decompose()}} already explicitly _does_ return a {{ByteBuffer}}.
This is also a case where remaining explicit reduces the sheer size of the
patch.
> Add byte array backed cells
> ---
>
> Key: CASSANDRA-15393
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15393
> Project: Cassandra
> Issue Type: Sub-task
> Components: Local/Compaction
>Reporter: Blake Eggleston
>Assignee: Blake Eggleston
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> We currently materialize all values as on heap byte buffers. Byte buffers
> have a fairly high overhead given how frequently they’re used, and on the
> compaction and local read path we don’t do anything that needs them. Use of
> byte buffer methods only happens on the coordinator. Using cells that are
> backed by byte arrays instead in these situations reduces compaction and read
> garbage up to 22% in many cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16064) Add test which validates that Message serializedSize(version) == serialize(out, version).length
[ https://issues.apache.org/jira/browse/CASSANDRA-16064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-16064: -- Status: Ready to Commit (was: Review In Progress) Caleb and Jon both +1 in PR > Add test which validates that Message serializedSize(version) == > serialize(out, version).length > --- > > Key: CASSANDRA-16064 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16064 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Internode >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > In 4.0 we require serializedSize(version) == serialize(out, version).length > for correctness in post40 message format as we write it into the message > header. Given that this is a strong requirement for correct deserialization > of the message, we should have tests which help enforce this property. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15937) JMX output inconsistencies from CASSANDRA-7544 storage-port-configurable-per-node
[
https://issues.apache.org/jira/browse/CASSANDRA-15937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-15937:
--
Fix Version/s: (was: 4.0-beta)
4.0-beta2
Since Version: 4.0-beta1
Source Control Link:
https://github.com/apache/cassandra/commit/a062ff5f9f0d3221def8fccbcd21e933e7cd
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Can with ci multiple times. There were multiple failures but they were flaky
tests or issues with recent changes outside of this change.
https://app.circleci.com/pipelines/github/dcapwell/cassandra/450/workflows/bdad1231-c6f5-4ffc-ab7f-1c1a9895e3e3
https://app.circleci.com/pipelines/github/dcapwell/cassandra/451/workflows/e4a1c41d-9a3a-4bc1-8b5d-2e5cf9584852
> JMX output inconsistencies from CASSANDRA-7544
> storage-port-configurable-per-node
> -
>
> Key: CASSANDRA-15937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15937
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/JMX
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0-beta2
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> CASSANDRA-7544 introduced changes to allow the storage port number to be
> configured per-node. As part of that work it introduces new MBeans for
> MessagingService, FailureDetector providing new 'WithPort' versions that
> include the new port information, however there are some mistakes and
> inconsistencies.
> {code:java}
> 3.11.6 trunk trunk
> w/Port Notes
>
> AllEndpointStates /127.0.0.1\n... /127.0.0.3\n...
> 127.0.0.3:7000\n (trunk /w port different)
> SimpleStates /127.0.0.2=UP /127.0.0.2=UP
> 127.0.0.3:7000=UP (trunk /w port different)
> LargeMessagePendingTasks /127.0.0.1=0 /127.0.0.1=0
> 127.0.0.3:7000=0 (trunk /w port different)
> TimeoutsPerHost 127.0.0.1=0 /127.0.0.1=0
> 127.0.0.3:7000=0 3.0/3.11.6 & trunk differ.
> BackPressurePerHost 127.0.0.1=Infinity /127.0.0.2=Infinity
> /127.0.0.2=Infinity 3.11 & trunk differ, missing port number for
> BackPressurePerHostWithPort
> SchemaVersions {...=[127.0.0.1,...]} {...=[127.0.0.1,...]}
> {...=[127.0.0.1:7000,...]
>
> TokenToEndpointMap {-92...8=127.0.0.1, -92...8=127.0.0.1
> -92..8=127.0.0.1:7000
> HostIdMap 127.0.0.1=1ee..6f0af 127.0.0.1=e06...7e MISSING
> Deprecated for EndpointToHostId
> EndpointToHostId 127.0.0.1=1ee..6f0a 127.0.0.1=e06...7e
> 127.0.0.1:7000=e0..7e
> HostIdToEndpoint 1ee..6f0a=127.0.0.1 e06..7e=127.0.0.1
> e06..7e=127.0.0.1:7000
> LoadMap 127.0.0.1=185.01 KiB 127.0.0.1:7000=106.08 KiB
> 127.0.0.1=106.08 Ki LoadMap and LoadMapWithPort are flipped.
> LiveNodes 127.0.0.1 127.0.0.1
> 127.0.0.1:7000
> Ownership /127.0.0.1=0.33 /127.0.0.1=0.33
> 127.0.0.1:7000=0.33
> Scores /127.0.0.1=0.0 /127.0.0.1=0.0
> 127.0.0.1:7000=0.0
> {code}
>
> Proposed changes
>
> 1) AllEndpointStats, SimpleStates, Connection message tracking,
> TimeoutsPerHost - include the host/ip:port in the WithPort version
> 2) Add port number to BackPressurePerHostWithPort
> 3) Correct LoadMap to omit port / LoadMapWithPort to include port
> 4) Ownership - update with port to host/ip:port version
> 5) Scores - update with port to host/ip:port version
>
>
> Additionally while dumping out all of the JMX info with `sjk mxdump`
>
> 6) DynamicEndpointSnitch.getScoresWithPort now returns an InetAddressAndPort
> which should just be a String
> 7) ClientMetrics.clientsByProtocolVersion returns a Guava Immutable map
> 8) StorageService.getIdealConsistencyLevel fails if none set (as we try and
> call ConsistencyLevel.toString on a null pointer).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Resolve JMX output inconsistencies from CASSANDRA-7544 storage-port-configurable-per-node
This is an automated email from the ASF dual-hosted git repository.
dcapwell pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new a062ff5 Resolve JMX output inconsistencies from CASSANDRA-7544
storage-port-configurable-per-node
a062ff5 is described below
commit a062ff5f9f0d3221def8fccbcd21e933e7cd
Author: Jon Meredith
AuthorDate: Wed Aug 26 12:58:10 2020 -0700
Resolve JMX output inconsistencies from CASSANDRA-7544
storage-port-configurable-per-node
patch by Jon Meredith; reviewed by David Capwell for CASSANDRA-15937
---
CHANGES.txt| 1 +
src/java/org/apache/cassandra/gms/Gossiper.java| 2 +-
.../org/apache/cassandra/gms/GossiperEvent.java| 2 +-
.../org/apache/cassandra/gms/VersionedValue.java | 12 +-
src/java/org/apache/cassandra/hints/HintEvent.java | 2 +-
.../cassandra/locator/DynamicEndpointSnitch.java | 4 +-
.../locator/DynamicEndpointSnitchMBean.java| 2 +-
.../cassandra/locator/Ec2MultiRegionSnitch.java| 2 +-
.../locator/GossipingPropertyFileSnitch.java | 2 +-
.../cassandra/locator/InetAddressAndPort.java | 58 +-
.../apache/cassandra/metrics/ClientMetrics.java| 2 +-
.../cassandra/metrics/InternodeInboundMetrics.java | 2 +-
.../metrics/InternodeOutboundMetrics.java | 2 +-
.../repair/SystemDistributedKeyspace.java | 2 +-
.../repair/consistent/LocalSessionInfo.java| 2 +-
.../cassandra/repair/consistent/LocalSessions.java | 2 +-
.../cassandra/schema/SchemaMigrationEvent.java | 2 +-
.../org/apache/cassandra/service/StorageProxy.java | 2 +-
.../apache/cassandra/service/StorageService.java | 6 +-
.../apache/cassandra/streaming/StreamSession.java | 10 +-
.../distributed/impl/DistributedTestSnitch.java| 2 +-
.../distributed/test/GossipSettlesTest.java| 128 -
.../test/RepairCoordinatorNeighbourDown.java | 8 +-
.../distributed/test/SimpleReadWriteTest.java | 8 +-
.../distributed/test/TopologyChangeTest.java | 7 +-
.../apache/cassandra/gms/EndpointStateTest.java| 9 +-
.../locator/GossipingPropertyFileSnitchTest.java | 2 +-
.../cassandra/locator/InetAddressAndPortTest.java | 47 +++-
.../cassandra/locator/PropertyFileSnitchTest.java | 50
.../cassandra/service/ActiveRepairServiceTest.java | 2 +-
30 files changed, 298 insertions(+), 84 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 123efdf..3c33a6b 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -18,6 +18,7 @@
* Fix version parsing logic when upgrading from 3.0 (CASSANDRA-15973)
* Optimize NoSpamLogger use in hot paths (CASSANDRA-15766)
* Verify sstable components on startup (CASSANDRA-15945)
+ * Resolve JMX output inconsistencies from CASSANDRA-7544
storage-port-configurable-per-node (CASSANDRA-15937)
Merged from 3.11:
* Correctly interpret SASI's `max_compaction_flush_memory_in_mb` setting in
megabytes not bytes (CASSANDRA-16071)
* Fix short read protection for GROUP BY queries (CASSANDRA-15459)
diff --git a/src/java/org/apache/cassandra/gms/Gossiper.java
b/src/java/org/apache/cassandra/gms/Gossiper.java
index 37ad6bb..9e1082a 100644
--- a/src/java/org/apache/cassandra/gms/Gossiper.java
+++ b/src/java/org/apache/cassandra/gms/Gossiper.java
@@ -1949,7 +1949,7 @@ public class Gossiper implements
IFailureDetectionEventListener, GossiperMBean
hosts = new ArrayList<>();
results.put(stringVersion, hosts);
}
-hosts.add(host.getHostAddress(true));
+hosts.add(host.getHostAddressAndPort());
}
return results;
diff --git a/src/java/org/apache/cassandra/gms/GossiperEvent.java
b/src/java/org/apache/cassandra/gms/GossiperEvent.java
index ef7bd8d..4ec0cf4 100644
--- a/src/java/org/apache/cassandra/gms/GossiperEvent.java
+++ b/src/java/org/apache/cassandra/gms/GossiperEvent.java
@@ -95,7 +95,7 @@ public final class GossiperEvent extends DiagnosticEvent
{
// be extra defensive against nulls and bugs
HashMap ret = new HashMap<>();
-if (endpoint != null) ret.put("endpoint",
endpoint.getHostAddress(true));
+if (endpoint != null) ret.put("endpoint",
endpoint.getHostAddressAndPort());
ret.put("quarantineExpiration", quarantineExpiration);
ret.put("localState", String.valueOf(localState));
ret.put("endpointStateMap", String.valueOf(endpointStateMap));
diff --git a/src/java/org/apache/cassandra/gms/VersionedValue.java
b/src/java/org/apache/cassandra/gms/VersionedValue.java
index 94b8cb8..caf6ce9 100644
--- a/src/java/org/apache/cassandra/gms/VersionedValue.java
+++ b/src/java/org/apache/cassandra/gms/VersionedValue.java
@@ -143,7 +143,7 @@ public class VersionedValue implements
Comparable
[cassandra-dtest] branch master updated: CASSANDRA-15899 - fix repair_admin tests
This is an automated email from the ASF dual-hosted git repository.
bdeggleston pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git
The following commit(s) were added to refs/heads/master by this push:
new e8b48f5 CASSANDRA-15899 - fix repair_admin tests
e8b48f5 is described below
commit e8b48f5fe8c097ccb0037c14f5332659c58bdeb3
Author: Blake Eggleston
AuthorDate: Wed Aug 26 13:35:50 2020 -0700
CASSANDRA-15899 - fix repair_admin tests
---
repair_tests/incremental_repair_test.py | 10 +-
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/repair_tests/incremental_repair_test.py
b/repair_tests/incremental_repair_test.py
index e379355..c8e7024 100644
--- a/repair_tests/incremental_repair_test.py
+++ b/repair_tests/incremental_repair_test.py
@@ -231,10 +231,10 @@ class TestIncRepair(Tester):
assert re.match(str(session_id), line)
assert "REPAIRING" in line
-node1.nodetool("repair_admin --cancel {}".format(session_id))
+node1.nodetool("repair_admin cancel --session {}".format(session_id))
for node in self.cluster.nodelist():
-out = node.nodetool('repair_admin --all')
+out = node.nodetool('repair_admin list --all')
lines = out.stdout.split('\n')
assert len(lines) > 1
line = lines[1]
@@ -272,7 +272,7 @@ class TestIncRepair(Tester):
assert "REPAIRING" in line
try:
-node2.nodetool("repair_admin --cancel {}".format(session_id))
+node2.nodetool("repair_admin --cancel --session
{}".format(session_id))
self.fail("cancel from a non coordinator should fail")
except ToolError:
pass # expected
@@ -315,10 +315,10 @@ class TestIncRepair(Tester):
assert re.match(str(session_id), line)
assert "REPAIRING" in line
-node2.nodetool("repair_admin --cancel {} --force".format(session_id))
+node2.nodetool("repair_admin cancel --session {}
--force".format(session_id))
for node in self.cluster.nodelist():
-out = node.nodetool('repair_admin --all')
+out = node.nodetool('repair_admin list --all')
lines = out.stdout.split('\n')
assert len(lines) > 1
line = lines[1]
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16061) transient_replication_ring_test.py::TestTransientReplicationRing::test_move_forwards_and_cleanup
[ https://issues.apache.org/jira/browse/CASSANDRA-16061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-16061: Fix Version/s: (was: 4.0) 4.0-beta > transient_replication_ring_test.py::TestTransientReplicationRing::test_move_forwards_and_cleanup > > > Key: CASSANDRA-16061 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16061 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0-beta > > > Failing here, also locally: > [https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra/312/workflows/da4ce69c-e778-467e-b9f3-27ab166a8321/jobs/1945] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: ninja: fix PendingStats composite type
This is an automated email from the ASF dual-hosted git repository. bdeggleston pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new 236a3bb ninja: fix PendingStats composite type 236a3bb is described below commit 236a3bb7f9c6beee8a334eef8a39bc55a5fee5f8 Author: Blake Eggleston AuthorDate: Wed Aug 26 12:14:19 2020 -0700 ninja: fix PendingStats composite type --- src/java/org/apache/cassandra/repair/consistent/admin/PendingStats.java | 1 + 1 file changed, 1 insertion(+) diff --git a/src/java/org/apache/cassandra/repair/consistent/admin/PendingStats.java b/src/java/org/apache/cassandra/repair/consistent/admin/PendingStats.java index 9dbe0df..f1b515e 100644 --- a/src/java/org/apache/cassandra/repair/consistent/admin/PendingStats.java +++ b/src/java/org/apache/cassandra/repair/consistent/admin/PendingStats.java @@ -45,6 +45,7 @@ public class PendingStats SimpleType.STRING, PendingStat.COMPOSITE_TYPE, PendingStat.COMPOSITE_TYPE, + PendingStat.COMPOSITE_TYPE, PendingStat.COMPOSITE_TYPE}; COMPOSITE_TYPE = new CompositeType(RepairStats.Section.class.getName(), "PendingStats", COMPOSITE_NAMES, COMPOSITE_NAMES, COMPOSITE_TYPES); } - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15937) JMX output inconsistencies from CASSANDRA-7544 storage-port-configurable-per-node
[
https://issues.apache.org/jira/browse/CASSANDRA-15937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185420#comment-17185420
]
David Capwell commented on CASSANDRA-15937:
---
ccm was merged, going to rebase and start the commits now.
> JMX output inconsistencies from CASSANDRA-7544
> storage-port-configurable-per-node
> -
>
> Key: CASSANDRA-15937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15937
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/JMX
>Reporter: Jon Meredith
>Assignee: Jon Meredith
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> CASSANDRA-7544 introduced changes to allow the storage port number to be
> configured per-node. As part of that work it introduces new MBeans for
> MessagingService, FailureDetector providing new 'WithPort' versions that
> include the new port information, however there are some mistakes and
> inconsistencies.
> {code:java}
> 3.11.6 trunk trunk
> w/Port Notes
>
> AllEndpointStates /127.0.0.1\n... /127.0.0.3\n...
> 127.0.0.3:7000\n (trunk /w port different)
> SimpleStates /127.0.0.2=UP /127.0.0.2=UP
> 127.0.0.3:7000=UP (trunk /w port different)
> LargeMessagePendingTasks /127.0.0.1=0 /127.0.0.1=0
> 127.0.0.3:7000=0 (trunk /w port different)
> TimeoutsPerHost 127.0.0.1=0 /127.0.0.1=0
> 127.0.0.3:7000=0 3.0/3.11.6 & trunk differ.
> BackPressurePerHost 127.0.0.1=Infinity /127.0.0.2=Infinity
> /127.0.0.2=Infinity 3.11 & trunk differ, missing port number for
> BackPressurePerHostWithPort
> SchemaVersions {...=[127.0.0.1,...]} {...=[127.0.0.1,...]}
> {...=[127.0.0.1:7000,...]
>
> TokenToEndpointMap {-92...8=127.0.0.1, -92...8=127.0.0.1
> -92..8=127.0.0.1:7000
> HostIdMap 127.0.0.1=1ee..6f0af 127.0.0.1=e06...7e MISSING
> Deprecated for EndpointToHostId
> EndpointToHostId 127.0.0.1=1ee..6f0a 127.0.0.1=e06...7e
> 127.0.0.1:7000=e0..7e
> HostIdToEndpoint 1ee..6f0a=127.0.0.1 e06..7e=127.0.0.1
> e06..7e=127.0.0.1:7000
> LoadMap 127.0.0.1=185.01 KiB 127.0.0.1:7000=106.08 KiB
> 127.0.0.1=106.08 Ki LoadMap and LoadMapWithPort are flipped.
> LiveNodes 127.0.0.1 127.0.0.1
> 127.0.0.1:7000
> Ownership /127.0.0.1=0.33 /127.0.0.1=0.33
> 127.0.0.1:7000=0.33
> Scores /127.0.0.1=0.0 /127.0.0.1=0.0
> 127.0.0.1:7000=0.0
> {code}
>
> Proposed changes
>
> 1) AllEndpointStats, SimpleStates, Connection message tracking,
> TimeoutsPerHost - include the host/ip:port in the WithPort version
> 2) Add port number to BackPressurePerHostWithPort
> 3) Correct LoadMap to omit port / LoadMapWithPort to include port
> 4) Ownership - update with port to host/ip:port version
> 5) Scores - update with port to host/ip:port version
>
>
> Additionally while dumping out all of the JMX info with `sjk mxdump`
>
> 6) DynamicEndpointSnitch.getScoresWithPort now returns an InetAddressAndPort
> which should just be a String
> 7) ClientMetrics.clientsByProtocolVersion returns a Guava Immutable map
> 8) StorageService.getIdealConsistencyLevel fails if none set (as we try and
> call ConsistencyLevel.toString on a null pointer).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-15992) Fix flaky python dtest test_13595 - consistency_test.TestConsistency
[
https://issues.apache.org/jira/browse/CASSANDRA-15992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Holmberg reassigned CASSANDRA-15992:
-
Assignee: Adam Holmberg
> Fix flaky python dtest test_13595 - consistency_test.TestConsistency
>
>
> Key: CASSANDRA-15992
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15992
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest
>Reporter: David Capwell
>Assignee: Adam Holmberg
>Priority: Normal
> Fix For: 4.0-beta
>
>
> https://app.circleci.com/pipelines/github/dcapwell/cassandra/355/workflows/7b8df61d-706f-4094-a206-7cdc6b4e0451/jobs/1818
> {code}
> > assert 9 == jmx.read_attribute(srp, 'Count')
> E AssertionError: assert 9 == 5
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15958) org.apache.cassandra.net.ConnectionTest testMessagePurging
[ https://issues.apache.org/jira/browse/CASSANDRA-15958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185385#comment-17185385 ] Adam Holmberg commented on CASSANDRA-15958: --- A third alternative would be to skip synchronization and computation and just make it periodic. I'm not sure what the genesis of the more precise calculation was. > org.apache.cassandra.net.ConnectionTest testMessagePurging > -- > > Key: CASSANDRA-15958 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15958 > Project: Cassandra > Issue Type: Bug > Components: Test/unit >Reporter: David Capwell >Assignee: Adam Holmberg >Priority: Normal > Fix For: 4.0-beta > > > Build: > https://ci-cassandra.apache.org/job/Cassandra-trunk-test/196/testReport/junit/org.apache.cassandra.net/ConnectionTest/testMessagePurging/ > Build: > https://ci-cassandra.apache.org/job/Cassandra-trunk-test/194/testReport/junit/org.apache.cassandra.net/ConnectionTest/testMessagePurging/ > java.util.concurrent.TimeoutException > at org.apache.cassandra.net.AsyncPromise.get(AsyncPromise.java:258) > at org.apache.cassandra.net.FutureDelegate.get(FutureDelegate.java:143) > at > org.apache.cassandra.net.ConnectionTest.doTestManual(ConnectionTest.java:268) > at > org.apache.cassandra.net.ConnectionTest.testManual(ConnectionTest.java:236) > at > org.apache.cassandra.net.ConnectionTest.testMessagePurging(ConnectionTest.java:679) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16064) Add test which validates that Message serializedSize(version) == serialize(out, version).length
[ https://issues.apache.org/jira/browse/CASSANDRA-16064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185372#comment-17185372 ] David Capwell commented on CASSANDRA-16064: --- Thanks for all the reviews so far. I have wrapped up everything and running through CI right now. java8: https://app.circleci.com/pipelines/github/dcapwell/cassandra/448/workflows/3982d772-6f37-4ff2-bc16-af1c1613462b only failing test so far looks to be CASSANDRA-15861, so so far so good. > Add test which validates that Message serializedSize(version) == > serialize(out, version).length > --- > > Key: CASSANDRA-16064 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16064 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Internode >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 4.0-beta > > > In 4.0 we require serializedSize(version) == serialize(out, version).length > for correctness in post40 message format as we write it into the message > header. Given that this is a strong requirement for correct deserialization > of the message, we should have tests which help enforce this property. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16073) cql_tracing_test.py failing repeatedly on jenkins
[ https://issues.apache.org/jira/browse/CASSANDRA-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-16073: --- Since Version: 4.0-alpha4 Source Control Link: https://github.com/apache/cassandra-dtest/commit/faf202c6c377a875a2c52ee971436c14962d1132 Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed into DTest master at faf202c6c377a875a2c52ee971436c14962d1132 > cql_tracing_test.py failing repeatedly on jenkins > - > > Key: CASSANDRA-16073 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16073 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 40m > Remaining Estimate: 0h > > Jenkins has been failing tracing tests repeatedly -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16073) cql_tracing_test.py failing repeatedly on jenkins
[ https://issues.apache.org/jira/browse/CASSANDRA-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-16073: --- Status: Ready to Commit (was: Review In Progress) > cql_tracing_test.py failing repeatedly on jenkins > - > > Key: CASSANDRA-16073 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16073 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 40m > Remaining Estimate: 0h > > Jenkins has been failing tracing tests repeatedly -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16073) cql_tracing_test.py failing repeatedly on jenkins
[ https://issues.apache.org/jira/browse/CASSANDRA-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-16073: --- Reviewers: Benjamin Lerer, Benjamin Lerer (was: Benjamin Lerer) Benjamin Lerer, Benjamin Lerer (was: Benjamin Lerer) Status: Review In Progress (was: Patch Available) > cql_tracing_test.py failing repeatedly on jenkins > - > > Key: CASSANDRA-16073 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16073 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 40m > Remaining Estimate: 0h > > Jenkins has been failing tracing tests repeatedly -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16073) cql_tracing_test.py failing repeatedly on jenkins
[ https://issues.apache.org/jira/browse/CASSANDRA-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185272#comment-17185272 ] Benjamin Lerer commented on CASSANDRA-16073: Your test run was looking fine to me :-) > cql_tracing_test.py failing repeatedly on jenkins > - > > Key: CASSANDRA-16073 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16073 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 40m > Remaining Estimate: 0h > > Jenkins has been failing tracing tests repeatedly -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-dtest] branch master updated: Increase several timeouts on test failures
This is an automated email from the ASF dual-hosted git repository.
blerer pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git
The following commit(s) were added to refs/heads/master by this push:
new faf202c Increase several timeouts on test failures
faf202c is described below
commit faf202c6c377a875a2c52ee971436c14962d1132
Author: Bereng
AuthorDate: Mon Aug 24 17:19:45 2020 +0200
Increase several timeouts on test failures
patch by Berenguer Blasi; reviewed by Benjamin Lerer for CASSANDRA-16073
---
cql_tracing_test.py | 6 --
dtest_setup.py | 2 +-
2 files changed, 5 insertions(+), 3 deletions(-)
diff --git a/cql_tracing_test.py b/cql_tracing_test.py
index fa6d4b8..cb3805a 100644
--- a/cql_tracing_test.py
+++ b/cql_tracing_test.py
@@ -78,7 +78,8 @@ class TestCqlTracing(Tester):
out, err, _ = node1.run_cqlsh(
"CONSISTENCY ALL; TRACING ON; "
"INSERT INTO ks.users (userid, firstname, lastname, age) "
-"VALUES (550e8400-e29b-41d4-a716-44665544, 'Frodo', 'Baggins',
32)")
+"VALUES (550e8400-e29b-41d4-a716-44665544, 'Frodo', 'Baggins',
32)",
+cqlsh_options=['--request-timeout', '30'])
logger.debug(out)
assert 'Tracing session: ' in out
@@ -92,7 +93,8 @@ class TestCqlTracing(Tester):
# Queries
out, err, _ = node1.run_cqlsh('CONSISTENCY ALL; TRACING ON; '
'SELECT firstname, lastname '
- 'FROM ks.users WHERE userid =
550e8400-e29b-41d4-a716-44665544')
+ 'FROM ks.users WHERE userid =
550e8400-e29b-41d4-a716-44665544',
+ cqlsh_options=['--request-timeout',
'30'])
logger.debug(out)
assert 'Tracing session: ' in out
diff --git a/dtest_setup.py b/dtest_setup.py
index 5688604..e9bb74a 100644
--- a/dtest_setup.py
+++ b/dtest_setup.py
@@ -399,7 +399,7 @@ class DTestSetup(object):
else:
repaired_data_tracking_values = {}
-timeout = 15000
+timeout = 3
if self.cluster_options is not None and len(self.cluster_options) > 0:
values = merge_dicts(self.cluster_options, phi_values,
repaired_data_tracking_values)
else:
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15406) Show the progress of data streaming and index build
[ https://issues.apache.org/jira/browse/CASSANDRA-15406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185261#comment-17185261 ] Benjamin Lerer commented on CASSANDRA-15406: Sorry, [~stefan.miklosovic] I got dragged into other things. I will review it tomorrow. > Show the progress of data streaming and index build > > > Key: CASSANDRA-15406 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15406 > Project: Cassandra > Issue Type: Improvement > Components: Consistency/Streaming, Legacy/Streaming and Messaging, > Tool/nodetool >Reporter: maxwellguo >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 4.0, 4.x > > Time Spent: 3h 10m > Remaining Estimate: 0h > > I found that we should supply a command to show the progress of streaming > when we do the operation of bootstrap/move/decommission/removenode. For when > do data streaming , noboday knows which steps there program are in , so I > think a command to show the joing/leaving node's is needed . > > PR [https://github.com/apache/cassandra/pull/558] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15897) Dropping compact storage with 2.1-sstables on disk make them unreadable
[ https://issues.apache.org/jira/browse/CASSANDRA-15897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185250#comment-17185250 ] Ekaterina Dimitrova edited comment on CASSANDRA-15897 at 8/26/20, 3:04 PM: --- I just added as part of CASSANDRA-16063 the following [commit|https://github.com/ekaterinadimitrova2/cassandra/commit/9ff9130808c751c9253bdecaa27c453bb5e7a71c] You might want to check it. I will soon publish the full patch for that ticket (it covers a couple of patches in a couple of repos), I have to run CI and confirm some of the upgrade tests but this was one of the things that was brought as part of that ticket. was (Author: e.dimitrova): I just added as part of CASSANDRA-16063 the following [commit|https://github.com/ekaterinadimitrova2/cassandra/commit/9ff9130808c751c9253bdecaa27c453bb5e7a71c] You might want to check it. I will soon publish the full patch, I have to run CI and confirm some of the upgrade tests but this was one of the things that was brought as part of that ticket. > Dropping compact storage with 2.1-sstables on disk make them unreadable > --- > > Key: CASSANDRA-15897 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15897 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Marcus Eriksson >Assignee: Sylvain Lebresne >Priority: Normal > > Test reproducing: > https://github.com/krummas/cassandra/commits/marcuse/dropcompactstorage -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15958) org.apache.cassandra.net.ConnectionTest testMessagePurging
[
https://issues.apache.org/jira/browse/CASSANDRA-15958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185251#comment-17185251
]
Adam Holmberg edited comment on CASSANDRA-15958 at 8/26/20, 3:03 PM:
-
Looked at the {{OutboundMessageQueue}} issue a bit more. Put more simply,
there's a race [adding to this queue and updating the expiration
deadline|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L89-L92]
while another thread is draining (
[1|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L256]
[2|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L148]
[3|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L484]
) and also updating (
[1|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L288]
[2|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L488]
). The race is there, but I'm not certain it would be a problem in an
operating server, since nothing is spinning on an inactive queue waiting for
messages to be evacuated, like this test is. In other words, new incoming
messages and ongoing delivery would break this loose naturally.
Two ways to proceed:
1.) We can agree that it's not a problem, and I can make this test not
susceptible to the timeout.
2.) We try to fix by adding synchronization around both the external queue and
expiry update. I would need to expand the analysis quite a bit to understand
what performance implications that might have (since the apparent point of the
two queue design is efficiency).
[~yifanc] I'm interested in your take.
also /cc [~benedict] [~aleksey] for ideas since this is part of your newish
messaging rewrite.
was (Author: aholmber):
Looked at the {{OutboundMessageQueue}} issue a bit more. Put more simply,
there's a race [adding to this queue and updating the expiration
deadline|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L89-L92]
while another thread is draining (
[1|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L256]
[2|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L148]
[3|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L484]
) and also updating
([1|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L288][2|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L488]).
The race is there, but I'm not certain it would be a problem in an operating
server, since nothing is spinning on an inactive queue waiting for messages to
be evacuated, like this test is. In other words, new incoming messages and
ongoing delivery would break this loose naturally.
Two ways to proceed:
1.) We can agree that it's not a problem, and I can make this test not
susceptible to the timeout.
2.) We try to fix by adding synchronization around both the external queue and
expiry update. I would need to expand the analysis quite a bit to understand
what performance implications that might have (since the apparent point of the
two queue design is efficiency).
[~yifanc] I'm interested in your take.
also /cc [~benedict] [~aleksey] for ideas since this is part of your newish
messaging rewrite.
> org.apache.cassandra.net.ConnectionTest testMessagePurging
> --
>
> Key: CASSANDRA-15958
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15958
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Adam Holmberg
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Build:
> https://ci-cassandra.apache.org/job/Cassandra-trunk-test/196/testReport/junit/org.apache.cassandra.net/ConnectionTest/testMessagePurging/
> Build:
> https://ci-cassandra.apache.org/job/Cassandra-trunk-test/194/testReport/junit/org.apache.cassandra.net/ConnectionTest/testMessagePurging/
>
[jira] [Comment Edited] (CASSANDRA-15958) org.apache.cassandra.net.ConnectionTest testMessagePurging
[
https://issues.apache.org/jira/browse/CASSANDRA-15958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185251#comment-17185251
]
Adam Holmberg edited comment on CASSANDRA-15958 at 8/26/20, 3:03 PM:
-
Looked at the {{OutboundMessageQueue}} issue a bit more. Put more simply,
there's a race [adding to this queue and updating the expiration
deadline|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L89-L92]
while another thread is draining (
[1|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L256]
[2|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L148]
[3|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L484]
) and also updating
([1|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L288][2|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L488]).
The race is there, but I'm not certain it would be a problem in an operating
server, since nothing is spinning on an inactive queue waiting for messages to
be evacuated, like this test is. In other words, new incoming messages and
ongoing delivery would break this loose naturally.
Two ways to proceed:
1.) We can agree that it's not a problem, and I can make this test not
susceptible to the timeout.
2.) We try to fix by adding synchronization around both the external queue and
expiry update. I would need to expand the analysis quite a bit to understand
what performance implications that might have (since the apparent point of the
two queue design is efficiency).
[~yifanc] I'm interested in your take.
also /cc [~benedict] [~aleksey] for ideas since this is part of your newish
messaging rewrite.
was (Author: aholmber):
Looked at the {{OutboundMessageQueue}} issue a bit more. Put more simply,
there's a race [adding to this queue and updating the expiration
deadline|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L89-L92]
while another thread is draining (
[1|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L256][2|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L148][3|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L484])
and also updating
([1|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L288][2|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L488]).
The race is there, but I'm not certain it would be a problem in an operating
server, since nothing is spinning on an inactive queue waiting for messages to
be evacuated, like this test is. In other words, new incoming messages and
ongoing delivery would break this loose naturally.
Two ways to proceed:
1.) We can agree that it's not a problem, and I can make this test not
susceptible to the timeout.
2.) We try to fix by adding synchronization around both the external queue and
expiry update. I would need to expand the analysis quite a bit to understand
what performance implications that might have (since the apparent point of the
two queue design is efficiency).
[~yifanc] I'm interested in your take.
also /cc [~benedict] [~aleksey] for ideas since this is part of your newish
messaging rewrite.
> org.apache.cassandra.net.ConnectionTest testMessagePurging
> --
>
> Key: CASSANDRA-15958
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15958
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Adam Holmberg
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Build:
> https://ci-cassandra.apache.org/job/Cassandra-trunk-test/196/testReport/junit/org.apache.cassandra.net/ConnectionTest/testMessagePurging/
> Build:
> https://ci-cassandra.apache.org/job/Cassandra-trunk-test/194/testReport/junit/org.apache.cassandra.net/ConnectionTest/testMessagePurging/
>
[jira] [Comment Edited] (CASSANDRA-15958) org.apache.cassandra.net.ConnectionTest testMessagePurging
[
https://issues.apache.org/jira/browse/CASSANDRA-15958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185251#comment-17185251
]
Adam Holmberg edited comment on CASSANDRA-15958 at 8/26/20, 3:02 PM:
-
Looked at the {{OutboundMessageQueue}} issue a bit more. Put more simply,
there's a race [adding to this queue and updating the expiration
deadline|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L89-L92]
while another thread is draining (
[1|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L256][2|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L148][3|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L484])
and also updating
([1|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L288][2|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L488]).
The race is there, but I'm not certain it would be a problem in an operating
server, since nothing is spinning on an inactive queue waiting for messages to
be evacuated, like this test is. In other words, new incoming messages and
ongoing delivery would break this loose naturally.
Two ways to proceed:
1.) We can agree that it's not a problem, and I can make this test not
susceptible to the timeout.
2.) We try to fix by adding synchronization around both the external queue and
expiry update. I would need to expand the analysis quite a bit to understand
what performance implications that might have (since the apparent point of the
two queue design is efficiency).
[~yifanc] I'm interested in your take.
also /cc [~benedict] [~aleksey] for ideas since this is part of your newish
messaging rewrite.
was (Author: aholmber):
Looked at the {{OutboundMessageQueue}} issue a bit more. Put more simply,
there's a race [adding to this queue and updating the expiration
deadline|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L89-L92]
while another thread is draining
([1|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L256][2|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L148][3|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L484])
and also updating
([1|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L288][2|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L488]).
The race is there, but I'm not certain it would be a problem in an operating
server, since nothing is spinning on an inactive queue waiting for messages to
be evacuated, like this test is. In other words, new incoming messages and
ongoing delivery would break this loose naturally.
Two ways to proceed:
1.) We can agree that it's not a problem, and I can make this test not
susceptible to the timeout.
2.) We try to fix by adding synchronization around both the external queue and
expiry update. I would need to expand the analysis quite a bit to understand
what performance implications that might have (since the apparent point of the
two queue design is efficiency).
[~yifanc] I'm interested in your take.
also /cc [~benedict] [~aleksey] for ideas since this is part of your newish
messaging rewrite.
> org.apache.cassandra.net.ConnectionTest testMessagePurging
> --
>
> Key: CASSANDRA-15958
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15958
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Adam Holmberg
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Build:
> https://ci-cassandra.apache.org/job/Cassandra-trunk-test/196/testReport/junit/org.apache.cassandra.net/ConnectionTest/testMessagePurging/
> Build:
> https://ci-cassandra.apache.org/job/Cassandra-trunk-test/194/testReport/junit/org.apache.cassandra.net/ConnectionTest/testMessagePurging/
>
[jira] [Commented] (CASSANDRA-15958) org.apache.cassandra.net.ConnectionTest testMessagePurging
[
https://issues.apache.org/jira/browse/CASSANDRA-15958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185251#comment-17185251
]
Adam Holmberg commented on CASSANDRA-15958:
---
Looked at the {{OutboundMessageQueue}} issue a bit more. Put more simply,
there's a race [adding to this queue and updating the expiration
deadline|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L89-L92]
while another thread is draining
([1|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L256][2|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L148][3|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L484])
and also updating
([1|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L288][2|https://github.com/apache/cassandra/blob/405e2dd8b5610208596ab4cb0bb6b9be7a159f5e/src/java/org/apache/cassandra/net/OutboundMessageQueue.java#L488]).
The race is there, but I'm not certain it would be a problem in an operating
server, since nothing is spinning on an inactive queue waiting for messages to
be evacuated, like this test is. In other words, new incoming messages and
ongoing delivery would break this loose naturally.
Two ways to proceed:
1.) We can agree that it's not a problem, and I can make this test not
susceptible to the timeout.
2.) We try to fix by adding synchronization around both the external queue and
expiry update. I would need to expand the analysis quite a bit to understand
what performance implications that might have (since the apparent point of the
two queue design is efficiency).
[~yifanc] I'm interested in your take.
also /cc [~benedict] [~aleksey] for ideas since this is part of your newish
messaging rewrite.
> org.apache.cassandra.net.ConnectionTest testMessagePurging
> --
>
> Key: CASSANDRA-15958
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15958
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: David Capwell
>Assignee: Adam Holmberg
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Build:
> https://ci-cassandra.apache.org/job/Cassandra-trunk-test/196/testReport/junit/org.apache.cassandra.net/ConnectionTest/testMessagePurging/
> Build:
> https://ci-cassandra.apache.org/job/Cassandra-trunk-test/194/testReport/junit/org.apache.cassandra.net/ConnectionTest/testMessagePurging/
> java.util.concurrent.TimeoutException
> at org.apache.cassandra.net.AsyncPromise.get(AsyncPromise.java:258)
> at org.apache.cassandra.net.FutureDelegate.get(FutureDelegate.java:143)
> at
> org.apache.cassandra.net.ConnectionTest.doTestManual(ConnectionTest.java:268)
> at
> org.apache.cassandra.net.ConnectionTest.testManual(ConnectionTest.java:236)
> at
> org.apache.cassandra.net.ConnectionTest.testMessagePurging(ConnectionTest.java:679)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15897) Dropping compact storage with 2.1-sstables on disk make them unreadable
[ https://issues.apache.org/jira/browse/CASSANDRA-15897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185250#comment-17185250 ] Ekaterina Dimitrova edited comment on CASSANDRA-15897 at 8/26/20, 2:59 PM: --- I just added as part of CASSANDRA-16063 the following [commit|https://github.com/ekaterinadimitrova2/cassandra/commit/9ff9130808c751c9253bdecaa27c453bb5e7a71c] You might want to check it. I will soon publish the full patch, I have to run CI and confirm some of the upgrade tests but this was one of the things that was brought as part of that ticket. was (Author: e.dimitrova): I just added as part of CASSANDRA-16063 the following [commit | https://github.com/ekaterinadimitrova2/cassandra/commit/9ff9130808c751c9253bdecaa27c453bb5e7a71c] You might want to check it. I will soon publish the full patch, I have to run CI and confirm some of the upgrade tests but this was one of the things that was brought as part of that ticket. > Dropping compact storage with 2.1-sstables on disk make them unreadable > --- > > Key: CASSANDRA-15897 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15897 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Marcus Eriksson >Assignee: Sylvain Lebresne >Priority: Normal > > Test reproducing: > https://github.com/krummas/cassandra/commits/marcuse/dropcompactstorage -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15897) Dropping compact storage with 2.1-sstables on disk make them unreadable
[ https://issues.apache.org/jira/browse/CASSANDRA-15897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185250#comment-17185250 ] Ekaterina Dimitrova commented on CASSANDRA-15897: - I just added as part of CASSANDRA-16063 the following [commit | https://github.com/ekaterinadimitrova2/cassandra/commit/9ff9130808c751c9253bdecaa27c453bb5e7a71c] You might want to check it. I will soon publish the full patch, I have to run CI and confirm some of the upgrade tests but this was one of the things that was brought as part of that ticket. > Dropping compact storage with 2.1-sstables on disk make them unreadable > --- > > Key: CASSANDRA-15897 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15897 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Local Write-Read Paths >Reporter: Marcus Eriksson >Assignee: Sylvain Lebresne >Priority: Normal > > Test reproducing: > https://github.com/krummas/cassandra/commits/marcuse/dropcompactstorage -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16075) Backport CASSANDRA-15653 to 2.1
[ https://issues.apache.org/jira/browse/CASSANDRA-16075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16075: Fix Version/s: (was: 2.1.x) 2.1.22 > Backport CASSANDRA-15653 to 2.1 > --- > > Key: CASSANDRA-16075 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16075 > Project: Cassandra > Issue Type: Task > Components: Local/Other, Observability/JMX >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: High > Fix For: 2.1.22 > > > When committing CASSANDRA-15653, the 2.1 branch was omitted, so we should > backport the fix. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16075) Backport CASSANDRA-15653 to 2.1
[
https://issues.apache.org/jira/browse/CASSANDRA-16075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-16075:
Source Control Link:
https://github.com/apache/cassandra/commit/d28f769fc6dae2ddbc150723a4069c765fc80c08
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Thanks, committed to 2.1 in {{d28f769fc6dae2ddbc150723a4069c765fc80c08}} and
merged up with {{-s ours}}.
> Backport CASSANDRA-15653 to 2.1
> ---
>
> Key: CASSANDRA-16075
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16075
> Project: Cassandra
> Issue Type: Task
> Components: Local/Other, Observability/JMX
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: High
> Fix For: 2.1.x
>
>
> When committing CASSANDRA-15653, the 2.1 branch was omitted, so we should
> backport the fix.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-2.2 updated (bb2562b -> 3f74a9c)
This is an automated email from the ASF dual-hosted git repository. samt pushed a change to branch cassandra-2.2 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from bb2562b Jenkinsfile fixes for wrong test reports being copied, and reports getting clobbered new d28f769 Disable JMX rebinding new 3f74a9c Merge branch 'cassandra-2.1' into cassandra-2.2 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated (035dc65 -> 2fc090c)
This is an automated email from the ASF dual-hosted git repository. samt pushed a change to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 035dc65 add default wipe action to circle pip3 new d28f769 Disable JMX rebinding new 3f74a9c Merge branch 'cassandra-2.1' into cassandra-2.2 new 2fc090c Merge branch 'cassandra-2.2' into cassandra-3.0 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-2.1 updated: Disable JMX rebinding
This is an automated email from the ASF dual-hosted git repository.
samt pushed a commit to branch cassandra-2.1
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-2.1 by this push:
new d28f769 Disable JMX rebinding
d28f769 is described below
commit d28f769fc6dae2ddbc150723a4069c765fc80c08
Author: Colm O hEigeartaigh
AuthorDate: Mon Mar 23 14:05:03 2020 +
Disable JMX rebinding
Patch by Colm O hEigeartaigh; reviewed by Sam Tunnicliffe for
CASSANDRA-15653/CASSANDRA-16075
---
CHANGES.txt| 3 +
.../apache/cassandra/service/CassandraDaemon.java | 80 +-
2 files changed, 66 insertions(+), 17 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 0a6268c..367504b 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,3 +1,6 @@
+2.1.22
+ * Disable JMX rebinding (CASSANDRA-15653, CASSANDRA-16075)
+
2.1.21
* Fix writing of snapshot manifest when the table has table-backed secondary
indexes (CASSANDRA-10968)
* Fix parse error in cqlsh COPY FROM and formatting for map of blobs
(CASSANDRA-15679)
diff --git a/src/java/org/apache/cassandra/service/CassandraDaemon.java
b/src/java/org/apache/cassandra/service/CassandraDaemon.java
index dc22834..afd376e 100644
--- a/src/java/org/apache/cassandra/service/CassandraDaemon.java
+++ b/src/java/org/apache/cassandra/service/CassandraDaemon.java
@@ -23,7 +23,13 @@ import java.lang.management.ManagementFactory;
import java.lang.management.MemoryPoolMXBean;
import java.net.InetAddress;
import java.net.UnknownHostException;
-import java.rmi.registry.LocateRegistry;
+import java.rmi.AccessException;
+import java.rmi.AlreadyBoundException;
+import java.rmi.NotBoundException;
+import java.rmi.Remote;
+import java.rmi.RemoteException;
+import java.rmi.registry.Registry;
+import java.rmi.server.RMIClientSocketFactory;
import java.rmi.server.RMIServerSocketFactory;
import java.util.List;
import java.util.*;
@@ -35,6 +41,7 @@ import javax.management.StandardMBean;
import javax.management.remote.JMXConnectorServer;
import javax.management.remote.JMXServiceURL;
import javax.management.remote.rmi.RMIConnectorServer;
+import javax.management.remote.rmi.RMIJRMPServerImpl;
import com.google.common.annotations.VisibleForTesting;
import com.google.common.collect.Iterables;
@@ -97,23 +104,18 @@ public class CassandraDaemon
try
{
RMIServerSocketFactory serverFactory = new
RMIServerSocketFactoryImpl();
-LocateRegistry.createRegistry(Integer.valueOf(jmxPort),
null, serverFactory);
-
-StringBuffer url = new StringBuffer();
-url.append("service:jmx:");
-url.append("rmi://localhost/jndi/");
-
url.append("rmi://localhost:").append(jmxPort).append("/jmxrmi");
-
-Map env = new HashMap();
-
env.put(RMIConnectorServer.RMI_SERVER_SOCKET_FACTORY_ATTRIBUTE, serverFactory);
-
-jmxServer = new RMIConnectorServer(
-new JMXServiceURL(url.toString()),
-env,
-ManagementFactory.getPlatformMBeanServer()
-);
-
+Map env =
Collections.singletonMap(RMIConnectorServer.RMI_SERVER_SOCKET_FACTORY_ATTRIBUTE,
serverFactory);
+
+Registry registry = new
JmxRegistry(Integer.valueOf(jmxPort), null, serverFactory, "jmxrmi");
+JMXServiceURL url = new
JMXServiceURL(String.format("service:jmx:rmi://localhost/jndi/rmi://localhost:%s/jmxrmi",
jmxPort));
+@SuppressWarnings("resource")
+RMIJRMPServerImpl server = new
RMIJRMPServerImpl(Integer.valueOf(jmxPort),
+ null,
+
(RMIServerSocketFactory)
env.get(RMIConnectorServer.RMI_SERVER_SOCKET_FACTORY_ATTRIBUTE),
+ env);
+jmxServer = new RMIConnectorServer(url, env, server,
ManagementFactory.getPlatformMBeanServer());
jmxServer.start();
+
((JmxRegistry)registry).setRemoteServerStub(server.toStub());
}
catch (IOException e)
{
@@ -689,4 +691,48 @@ public class CassandraDaemon
*/
public boolean isRunning();
}
+
+
+@SuppressWarnings("restriction")
+private static class JmxRegistry extends sun.rmi.registry.RegistryImpl {
+private final String lookupName;
+private Remote remoteServerStub;
+
+JmxRegistry(final int port,
+final RMIClientSocketFactory csf,
+RMIServerSocketFactory ssf,
+
[cassandra] branch cassandra-3.11 updated (97cb2ba -> 318bd1a)
This is an automated email from the ASF dual-hosted git repository. samt pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 97cb2ba Merge branch 'cassandra-3.0' into cassandra-3.11 new d28f769 Disable JMX rebinding new 3f74a9c Merge branch 'cassandra-2.1' into cassandra-2.2 new 2fc090c Merge branch 'cassandra-2.2' into cassandra-3.0 new 318bd1a Merge branch 'cassandra-3.0' into cassandra-3.11 The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.0' into cassandra-3.11
This is an automated email from the ASF dual-hosted git repository. samt pushed a commit to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 318bd1ae9993d91ff25e3376f438cc2a9c40f11f Merge: 97cb2ba 2fc090c Author: Sam Tunnicliffe AuthorDate: Wed Aug 26 13:42:37 2020 +0100 Merge branch 'cassandra-3.0' into cassandra-3.11 - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-2.1' into cassandra-2.2
This is an automated email from the ASF dual-hosted git repository. samt pushed a commit to branch cassandra-2.2 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 3f74a9c9d308638a5d54cd3c56195f5f5cca2478 Merge: bb2562b d28f769 Author: Sam Tunnicliffe AuthorDate: Wed Aug 26 13:25:03 2020 +0100 Merge branch 'cassandra-2.1' into cassandra-2.2 - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-2.2' into cassandra-3.0
This is an automated email from the ASF dual-hosted git repository. samt pushed a commit to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 2fc090c180140c8114238ee565dbcd13c198e882 Merge: 035dc65 3f74a9c Author: Sam Tunnicliffe AuthorDate: Wed Aug 26 13:41:10 2020 +0100 Merge branch 'cassandra-2.2' into cassandra-3.0 - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (3375e14 -> 405e2dd)
This is an automated email from the ASF dual-hosted git repository. samt pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git. from 3375e14 Merge branch 'cassandra-3.11' into trunk new d28f769 Disable JMX rebinding new 3f74a9c Merge branch 'cassandra-2.1' into cassandra-2.2 new 2fc090c Merge branch 'cassandra-2.2' into cassandra-3.0 new 318bd1a Merge branch 'cassandra-3.0' into cassandra-3.11 new 405e2dd Merge branch 'cassandra-3.11' into trunk The 5 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-3.11' into trunk
This is an automated email from the ASF dual-hosted git repository. samt pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 405e2dd8b5610208596ab4cb0bb6b9be7a159f5e Merge: 3375e14 318bd1a Author: Sam Tunnicliffe AuthorDate: Wed Aug 26 13:44:12 2020 +0100 Merge branch 'cassandra-3.11' into trunk - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15977) 4.0 quality testing: Read Repair
[ https://issues.apache.org/jira/browse/CASSANDRA-15977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185148#comment-17185148 ] Andres de la Peña commented on CASSANDRA-15977: --- CI results: [ci-cassandra utest|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-test/245/] [ci-cassandra jvm-dtest|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-jvm-dtest/189/] [ci-cassandra jvm-dtest-upgrade|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-jvm-dtest-upgrade/13/] [ci-cassandra dtest|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/44/] [CircleCI j8 |https://app.circleci.com/pipelines/github/adelapena/cassandra/95/workflows/1b89e3a8-89bf-4f34-98b2-b4546526da1e] [CircleCI j11|https://app.circleci.com/pipelines/github/adelapena/cassandra/95/workflows/ca050684-6b42-42c2-8555-9c2c7c718820] > 4.0 quality testing: Read Repair > > > Key: CASSANDRA-15977 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15977 > Project: Cassandra > Issue Type: Task > Components: Test/dtest, Test/unit >Reporter: Andres de la Peña >Assignee: Andres de la Peña >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 40m > Remaining Estimate: 0h > > This is a subtask of CASSANDRA-15579 focusing on read repair. > [This > document|https://docs.google.com/document/d/1-gldHcdLSMRbDhhI8ahs_tPeAZsjurjXr38xABVjWHE/edit?usp=sharing] > lists and describes the existing functional tests for read repair, so we can > have a broad view of what is currently covered. We can comment on this > document and add ideas for new cases/tests, so it can gradually evolve to a > more or less detailed test plan. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16075) Backport CASSANDRA-15653 to 2.1
[ https://issues.apache.org/jira/browse/CASSANDRA-16075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185135#comment-17185135 ] Marcus Eriksson commented on CASSANDRA-16075: - (y) > Backport CASSANDRA-15653 to 2.1 > --- > > Key: CASSANDRA-16075 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16075 > Project: Cassandra > Issue Type: Task > Components: Local/Other, Observability/JMX >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: High > Fix For: 2.1.x > > > When committing CASSANDRA-15653, the 2.1 branch was omitted, so we should > backport the fix. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16075) Backport CASSANDRA-15653 to 2.1
[
https://issues.apache.org/jira/browse/CASSANDRA-16075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185131#comment-17185131
]
Sam Tunnicliffe commented on CASSANDRA-16075:
-
CI run is a bit of a problem unfortunately, as both Jenkins and Circle now
depends on changes to {{build.xml}} which are 2.2+.
So I've run both the jmx dtests locally as well as a handful of other dtests
which start up the service with local JMX enabled.
These all look good and the manual testing with
encrypted/unencrypted/authenticated/unauthenticated non-local connections seems
fine.
wdyt [~marcuse]?
> Backport CASSANDRA-15653 to 2.1
> ---
>
> Key: CASSANDRA-16075
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16075
> Project: Cassandra
> Issue Type: Task
> Components: Local/Other, Observability/JMX
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: High
> Fix For: 2.1.x
>
>
> When committing CASSANDRA-15653, the 2.1 branch was omitted, so we should
> backport the fix.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15977) 4.0 quality testing: Read Repair
[
https://issues.apache.org/jira/browse/CASSANDRA-15977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185123#comment-17185123
]
Andres de la Peña commented on CASSANDRA-15977:
---
The changes proposed in the PR are:
* Remove duplicated read repair tests from {{SimpleReadWriteTest}}
({{readRepairTest}} and {{failingReadRepairTest}}), and move
{{readRepairTimeoutTest}} to {{ReadRepairTest}}.
* Port Python dtests in {{read_repair_test.py:TestReadRepairGuarantees}} to
in-JVM {{ReadRepairGuaranteesTest}}, using Byte Buddy to drop internal messages.
* Port Python dtest
{{read_repair_test.py:TestReadRepair.test_alter_rf_and_run_read_repair}} to
in-JVM {{ReadRepairTest#alterRFAndRunReadRepair}}.
* Add {{ReadRepairQueryTypesTest}} in-JVM test class, which contains tests
similar to {{test_alter_rf_and_run_read_repair}} for different query types and
schemas. The tests verify that each type of query propagates writes, checking
how only the selected columns are propagated. The tests are parameterized by
read repair strategy, usage of paging, whether the outdated node is the
coordinator, and whether the data is in-memory or on-disk. There are 34 tests
and 16 parameter configurations, so it produces 544 test runs, which take
around 7 minutes locally.
* Port Python dtest
{{read_repair_test.py:TestReadRepair.test_range_slice_query_with_tombstones}}
to in-JVM {{ReadRepairTest#testRangeSliceQueryWithTombstones}}, extended with
some additional types of tombstone and parameterized for in-memory/on-disk data.
* Port Python dtest
{{read_repair_test.py:TestReadRepair.test_gcable_tombstone_resurrection_on_range_slice_query}}
to in-JVM {{ReadRepairTest#testGCableTombstoneResurrectionOnRangeSliceQuery}},
extended with some additional types of tombstone.
* Port Python dtest {{consistency_test.py:TestConsistency.test_readrepair}} to
in-JVM {{ReadRepairTest#readRepairLongTest}}. This test is very similar to
{{ReadRepairTest#readRepairTest}}, so I'm not sure we need it.
* Extend the coverage of {{ReadRepairTest#emptyRangeTombstones}} and
{{ReadRepairTest#emptyRangeTombstonesFromPaging}} with a new test class named
{{ReadRepairEmptyRangeTombstonesTest}}. The new class tests different overlaps
of range queries and tombstone ranges and partition deletions, and it is
parameterized by read repair strategy, usage of paging, whether the outdated
node is the coordinator, whether the rows are in the same node as the
tombstones, and whether the clustering order is reversed. There are 14 tests
and 16 parameter configurations, so it produces 224 test runs, which take
around 1.5 minutes locally.
* Extend {{MixedModeReadRepairTest}} with new basic tests to verify the
propagation of insertions, updates and deletions between the upgraded and the
not-upgraded node, in both directions.
> 4.0 quality testing: Read Repair
>
>
> Key: CASSANDRA-15977
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15977
> Project: Cassandra
> Issue Type: Task
> Components: Test/dtest, Test/unit
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 4.0-beta
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> This is a subtask of CASSANDRA-15579 focusing on read repair.
> [This
> document|https://docs.google.com/document/d/1-gldHcdLSMRbDhhI8ahs_tPeAZsjurjXr38xABVjWHE/edit?usp=sharing]
> lists and describes the existing functional tests for read repair, so we can
> have a broad view of what is currently covered. We can comment on this
> document and add ideas for new cases/tests, so it can gradually evolve to a
> more or less detailed test plan.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16075) Backport CASSANDRA-15653 to 2.1
[ https://issues.apache.org/jira/browse/CASSANDRA-16075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-16075: Reviewers: Marcus Eriksson, Marcus Eriksson (was: Marcus Eriksson) Marcus Eriksson, Marcus Eriksson (was: Marcus Eriksson) Status: Review In Progress (was: Patch Available) > Backport CASSANDRA-15653 to 2.1 > --- > > Key: CASSANDRA-16075 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16075 > Project: Cassandra > Issue Type: Task > Components: Local/Other, Observability/JMX >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: High > Fix For: 2.1.x > > > When committing CASSANDRA-15653, the 2.1 branch was omitted, so we should > backport the fix. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16075) Backport CASSANDRA-15653 to 2.1
[ https://issues.apache.org/jira/browse/CASSANDRA-16075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185121#comment-17185121 ] Marcus Eriksson edited comment on CASSANDRA-16075 at 8/26/20, 11:37 AM: +1 (assuming CI run clean) was (Author: krummas): +1 > Backport CASSANDRA-15653 to 2.1 > --- > > Key: CASSANDRA-16075 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16075 > Project: Cassandra > Issue Type: Task > Components: Local/Other, Observability/JMX >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: High > Fix For: 2.1.x > > > When committing CASSANDRA-15653, the 2.1 branch was omitted, so we should > backport the fix. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16075) Backport CASSANDRA-15653 to 2.1
[ https://issues.apache.org/jira/browse/CASSANDRA-16075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-16075: Status: Ready to Commit (was: Review In Progress) +1 > Backport CASSANDRA-15653 to 2.1 > --- > > Key: CASSANDRA-16075 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16075 > Project: Cassandra > Issue Type: Task > Components: Local/Other, Observability/JMX >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: High > Fix For: 2.1.x > > > When committing CASSANDRA-15653, the 2.1 branch was omitted, so we should > backport the fix. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16075) Backport CASSANDRA-15653 to 2.1
[
https://issues.apache.org/jira/browse/CASSANDRA-16075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sam Tunnicliffe updated CASSANDRA-16075:
Test and Documentation Plan:
Run {{jmx_test.py}} and {{jmx_auth_test.py}} locally and manually verified
authenticated/unauthenticated/encrypted connections.
CI run WIP.
Status: Patch Available (was: Open)
https://github.com/beobal/cassandra/tree/16075-2.1
> Backport CASSANDRA-15653 to 2.1
> ---
>
> Key: CASSANDRA-16075
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16075
> Project: Cassandra
> Issue Type: Task
> Components: Local/Other, Observability/JMX
>Reporter: Sam Tunnicliffe
>Assignee: Sam Tunnicliffe
>Priority: High
> Fix For: 2.1.x
>
>
> When committing CASSANDRA-15653, the 2.1 branch was omitted, so we should
> backport the fix.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16075) Backport CASSANDRA-15653 to 2.1
[ https://issues.apache.org/jira/browse/CASSANDRA-16075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Tunnicliffe updated CASSANDRA-16075: Change Category: Operability Complexity: Normal Fix Version/s: 2.1.x Reviewers: Marcus Eriksson Priority: High (was: Normal) Status: Open (was: Triage Needed) > Backport CASSANDRA-15653 to 2.1 > --- > > Key: CASSANDRA-16075 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16075 > Project: Cassandra > Issue Type: Task > Components: Local/Other, Observability/JMX >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: High > Fix For: 2.1.x > > > When committing CASSANDRA-15653, the 2.1 branch was omitted, so we should > backport the fix. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15988) Add nodetool getfullquerylog
[ https://issues.apache.org/jira/browse/CASSANDRA-15988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185085#comment-17185085 ] Stefan Miklosovic commented on CASSANDRA-15988: --- hi [~marcuse], would you mind to be a reviewer? Thank you. > Add nodetool getfullquerylog > - > > Key: CASSANDRA-15988 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15988 > Project: Cassandra > Issue Type: Improvement > Components: Tool/fql >Reporter: Ekaterina Dimitrova >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 4.0-rc > > > This ticket is raised based on CASSANDRA-15791 and valuable feedback provided > by [~jshook]. > There are two outstanding questions: > * forming the exact shape of such a command and how it can benefit the > users; to be discussed in detail in this ticket > * Is this a thing we as a project can add to 4.0 beta or it should be > considered in 4.1 for example > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16075) Backport CASSANDRA-15653 to 2.1
Sam Tunnicliffe created CASSANDRA-16075: --- Summary: Backport CASSANDRA-15653 to 2.1 Key: CASSANDRA-16075 URL: https://issues.apache.org/jira/browse/CASSANDRA-16075 Project: Cassandra Issue Type: Task Components: Local/Other, Observability/JMX Reporter: Sam Tunnicliffe Assignee: Sam Tunnicliffe When committing CASSANDRA-15653, the 2.1 branch was omitted, so we should backport the fix. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16072) Reduce thread contention in CommitLogSegment and HintsBuffer by rewriting CAS loops to atomic adds
[
https://issues.apache.org/jira/browse/CASSANDRA-16072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185073#comment-17185073
]
Michael Semb Wever commented on CASSANDRA-16072:
Patches updated, with cast, and an assert added in HintsBuffer.create(..) to
make explicit the context's constraints.
> Reduce thread contention in CommitLogSegment and HintsBuffer by rewriting CAS
> loops to atomic adds
> --
>
> Key: CASSANDRA-16072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16072
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Hints, Local/Commit Log
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 3.11.x, 4.0-beta
>
>
> Follow up to CASSANDRA-15922
> Both CommitLogSegment and HintsBuffer use AtomicIntegers for the current
> offset when allocating. Like in CASSANDRA\-15922 the loops on
> {{.compareAndSet(..)}} can be replaced with atomic adds using the {{.
> getAndAdd(..)}} method.
> In highly contended environments the CAS failures can be high, starving
> writes in a running Cassandra node. On the same cluster CASSANDRA\-15922 was
> found, after CASSANDRA\-15922's fix was deployed, there was still problems
> around commit log flushing and hints. No flamegraph was collected that
> demonstrated the thread contention as clearly as was found in
> CASSANDRA\-15922, but the performance fix proposed here hopefully is obvious
> enough.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16072) Reduce thread contention in CommitLogSegment and HintsBuffer by rewriting CAS loops to atomic adds
[
https://issues.apache.org/jira/browse/CASSANDRA-16072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185058#comment-17185058
]
Berenguer Blasi commented on CASSANDRA-16072:
-
Ok after speaking to [~mck] yes it will return _some_ negative number but not
-1 as stated in the comment. All makes sense now and comment + code will be
synched :-)
> Reduce thread contention in CommitLogSegment and HintsBuffer by rewriting CAS
> loops to atomic adds
> --
>
> Key: CASSANDRA-16072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16072
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Hints, Local/Commit Log
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 3.11.x, 4.0-beta
>
>
> Follow up to CASSANDRA-15922
> Both CommitLogSegment and HintsBuffer use AtomicIntegers for the current
> offset when allocating. Like in CASSANDRA\-15922 the loops on
> {{.compareAndSet(..)}} can be replaced with atomic adds using the {{.
> getAndAdd(..)}} method.
> In highly contended environments the CAS failures can be high, starving
> writes in a running Cassandra node. On the same cluster CASSANDRA\-15922 was
> found, after CASSANDRA\-15922's fix was deployed, there was still problems
> around commit log flushing and hints. No flamegraph was collected that
> demonstrated the thread contention as clearly as was found in
> CASSANDRA\-15922, but the performance fix proposed here hopefully is obvious
> enough.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16072) Reduce thread contention in CommitLogSegment and HintsBuffer by rewriting CAS loops to atomic adds
[
https://issues.apache.org/jira/browse/CASSANDRA-16072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185047#comment-17185047
]
Michael Semb Wever edited comment on CASSANDRA-16072 at 8/26/20, 9:08 AM:
--
[~Bereng], ways {{(prev + size)}} could wrap around…
- prev itself has wrapped around, the method still then returns negative,
- {{`prev > endOfBuffer`}} and {{`prev + size > Integer.MAX_VALUE`}}.
In the later, {{`endOfBuffer + size> Integer.MAX_VALUE`}} must also be true, so
it would be when {{commitlog_segment_size_in_mb}} is set to some value close
(relative to the size of mutations) to 2GB.
To prevent this would it be enough to
{code}if (((long)prev + size) >= endOfBuffer){code}
was (Author: michaelsembwever):
[~Bereng], ways {{(prev + size)}} could wrap around…
- prev itself has wrapped around, the method still then returns negative,
- {{prev > endOfBuffer}} and {{prev + size > Integer.MAX_VALUE}}.
In the later, {{endOfBuffer + size> Integer.MAX_VALUE}} must also be true, so
it would be when commitlog_segment_size_in_mb is set to some value close
(relative to the size of mutations) to 2GB.
To prevent this would it be enough to
{code}if (((long)prev + size) >= endOfBuffer){code}
> Reduce thread contention in CommitLogSegment and HintsBuffer by rewriting CAS
> loops to atomic adds
> --
>
> Key: CASSANDRA-16072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16072
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Hints, Local/Commit Log
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 3.11.x, 4.0-beta
>
>
> Follow up to CASSANDRA-15922
> Both CommitLogSegment and HintsBuffer use AtomicIntegers for the current
> offset when allocating. Like in CASSANDRA\-15922 the loops on
> {{.compareAndSet(..)}} can be replaced with atomic adds using the {{.
> getAndAdd(..)}} method.
> In highly contended environments the CAS failures can be high, starving
> writes in a running Cassandra node. On the same cluster CASSANDRA\-15922 was
> found, after CASSANDRA\-15922's fix was deployed, there was still problems
> around commit log flushing and hints. No flamegraph was collected that
> demonstrated the thread contention as clearly as was found in
> CASSANDRA\-15922, but the performance fix proposed here hopefully is obvious
> enough.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16072) Reduce thread contention in CommitLogSegment and HintsBuffer by rewriting CAS loops to atomic adds
[
https://issues.apache.org/jira/browse/CASSANDRA-16072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185047#comment-17185047
]
Michael Semb Wever commented on CASSANDRA-16072:
[~Bereng], ways {{(prev + size)}} could wrap around…
- prev itself has wrapped around, the method still then returns negative,
- {{prev > endOfBuffer}} and {{prev + size > Integer.MAX_VALUE}}.
In the later, {{endOfBuffer + size> Integer.MAX_VALUE}} must also be true, so
it would be when commitlog_segment_size_in_mb is set to some value close
(relative to the size of mutations) to 2GB.
To prevent this would it be enough to
{code}if (((long)prev + size) >= endOfBuffer){code}
> Reduce thread contention in CommitLogSegment and HintsBuffer by rewriting CAS
> loops to atomic adds
> --
>
> Key: CASSANDRA-16072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16072
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Hints, Local/Commit Log
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 3.11.x, 4.0-beta
>
>
> Follow up to CASSANDRA-15922
> Both CommitLogSegment and HintsBuffer use AtomicIntegers for the current
> offset when allocating. Like in CASSANDRA\-15922 the loops on
> {{.compareAndSet(..)}} can be replaced with atomic adds using the {{.
> getAndAdd(..)}} method.
> In highly contended environments the CAS failures can be high, starving
> writes in a running Cassandra node. On the same cluster CASSANDRA\-15922 was
> found, after CASSANDRA\-15922's fix was deployed, there was still problems
> around commit log flushing and hints. No flamegraph was collected that
> demonstrated the thread contention as clearly as was found in
> CASSANDRA\-15922, but the performance fix proposed here hopefully is obvious
> enough.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16072) Reduce thread contention in CommitLogSegment and HintsBuffer by rewriting CAS loops to atomic adds
[
https://issues.apache.org/jira/browse/CASSANDRA-16072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Berenguer Blasi updated CASSANDRA-16072:
Reviewers: Benjamin Lerer, Berenguer Blasi (was: Benjamin Lerer)
> Reduce thread contention in CommitLogSegment and HintsBuffer by rewriting CAS
> loops to atomic adds
> --
>
> Key: CASSANDRA-16072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16072
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Hints, Local/Commit Log
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 3.11.x, 4.0-beta
>
>
> Follow up to CASSANDRA-15922
> Both CommitLogSegment and HintsBuffer use AtomicIntegers for the current
> offset when allocating. Like in CASSANDRA\-15922 the loops on
> {{.compareAndSet(..)}} can be replaced with atomic adds using the {{.
> getAndAdd(..)}} method.
> In highly contended environments the CAS failures can be high, starving
> writes in a running Cassandra node. On the same cluster CASSANDRA\-15922 was
> found, after CASSANDRA\-15922's fix was deployed, there was still problems
> around commit log flushing and hints. No flamegraph was collected that
> demonstrated the thread contention as clearly as was found in
> CASSANDRA\-15922, but the performance fix proposed here hopefully is obvious
> enough.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16072) Reduce thread contention in CommitLogSegment and HintsBuffer by rewriting CAS loops to atomic adds
[
https://issues.apache.org/jira/browse/CASSANDRA-16072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185035#comment-17185035
]
Berenguer Blasi commented on CASSANDRA-16072:
-
[~mck] if
[here|https://github.com/apache/cassandra/compare/trunk...thelastpickle:mck/trunk_cas_improvements#diff-7720d4b5123a354876e0b3139222f34eR255]
{{(prev + size)}} wrap around they can produce sthg smaller than
{{endOfBuffer}} and _not_ return -1. Or am I holding the wrong end of the stick?
> Reduce thread contention in CommitLogSegment and HintsBuffer by rewriting CAS
> loops to atomic adds
> --
>
> Key: CASSANDRA-16072
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16072
> Project: Cassandra
> Issue Type: Improvement
> Components: Consistency/Hints, Local/Commit Log
>Reporter: Michael Semb Wever
>Assignee: Michael Semb Wever
>Priority: Normal
> Fix For: 3.11.x, 4.0-beta
>
>
> Follow up to CASSANDRA-15922
> Both CommitLogSegment and HintsBuffer use AtomicIntegers for the current
> offset when allocating. Like in CASSANDRA\-15922 the loops on
> {{.compareAndSet(..)}} can be replaced with atomic adds using the {{.
> getAndAdd(..)}} method.
> In highly contended environments the CAS failures can be high, starving
> writes in a running Cassandra node. On the same cluster CASSANDRA\-15922 was
> found, after CASSANDRA\-15922's fix was deployed, there was still problems
> around commit log flushing and hints. No flamegraph was collected that
> demonstrated the thread contention as clearly as was found in
> CASSANDRA\-15922, but the performance fix proposed here hopefully is obvious
> enough.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15861) Mutating sstable component may race with entire-sstable-streaming(ZCS) causing checksum validation failure
[
https://issues.apache.org/jira/browse/CASSANDRA-15861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17185031#comment-17185031
]
ZhaoYang commented on CASSANDRA-15861:
--
[~bdeggleston] I have restored previous commits, sorry for the trouble
> Mutating sstable component may race with entire-sstable-streaming(ZCS)
> causing checksum validation failure
> --
>
> Key: CASSANDRA-15861
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15861
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Repair, Consistency/Streaming,
> Local/Compaction
>Reporter: ZhaoYang
>Assignee: ZhaoYang
>Priority: Normal
> Fix For: 4.0-beta
>
>
> Flaky dtest: [test_dead_sync_initiator -
> repair_tests.repair_test.TestRepair|https://ci-cassandra.apache.org/view/all/job/Cassandra-devbranch-dtest/143/testReport/junit/dtest.repair_tests.repair_test/TestRepair/test_dead_sync_initiator/]
> {code:java|title=stacktrace}
> Unexpected error found in node logs (see stdout for full details). Errors:
> [ERROR [Stream-Deserializer-127.0.0.1:7000-570871f3] 2020-06-03 04:05:19,081
> CassandraEntireSSTableStreamReader.java:145 - [Stream
> 6f1c3360-a54f-11ea-a808-2f23710fdc90] Error while reading sstable from stream
> for table = keyspace1.standard1
> org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted:
> /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-te4ty0r9/test/node3/data0/keyspace1/standard1-5f5ab140a54f11eaa8082f23710fdc90/na-2-big-Statistics.db
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.maybeValidateChecksum(MetadataSerializer.java:219)
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:198)
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:129)
> at
> org.apache.cassandra.io.sstable.metadata.MetadataSerializer.mutate(MetadataSerializer.java:226)
> at
> org.apache.cassandra.db.streaming.CassandraEntireSSTableStreamReader.read(CassandraEntireSSTableStreamReader.java:140)
> at
> org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:78)
> at
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:49)
> at
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:36)
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:49)
> at
> org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:181)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.io.IOException: Checksums do not match for
> /home/cassandra/cassandra/cassandra-dtest/tmp/dtest-te4ty0r9/test/node3/data0/keyspace1/standard1-5f5ab140a54f11eaa8082f23710fdc90/na-2-big-Statistics.db
> {code}
>
> In the above test, it executes "nodetool repair" on node1 and kills node2
> during repair. At the end, node3 reports checksum validation failure on
> sstable transferred from node1.
> {code:java|title=what happened}
> 1. When repair started on node1, it performs anti-compaction which modifies
> sstable's repairAt to 0 and pending repair id to session-id.
> 2. Then node1 creates {{ComponentManifest}} which contains file lengths to be
> transferred to node3.
> 3. Before node1 actually sends the files to node3, node2 is killed and node1
> starts to broadcast repair-failure-message to all participants in
> {{CoordinatorSession#fail}}
> 4. Node1 receives its own repair-failure-message and fails its local repair
> sessions at {{LocalSessions#failSession}} which triggers async background
> compaction.
> 5. Node1's background compaction will mutate sstable's repairAt to 0 and
> pending repair id to null via
> {{PendingRepairManager#getNextRepairFinishedTask}}, as there is no more
> in-progress repair.
> 6. Node1 actually sends the sstable to node3 where the sstable's STATS
> component size is different from the original size recorded in the manifest.
> 7. At the end, node3 reports checksum validation failure when it tries to
> mutate sstable level and "isTransient" attribute in
> {{CassandraEntireSSTableStreamReader#read}}.
> {code}
> Currently, entire-sstable-streaming requires sstable components to be
> immutable, because \{{ComponentManifest}}
> with component sizes are sent before sending actual files. This isn't a
> problem in legacy streaming as STATS file length didn't matter.
>
> Ideally it will be great to make sstable
[jira] [Updated] (CASSANDRA-16051) sstablemetadata line 44: : command not found
[ https://issues.apache.org/jira/browse/CASSANDRA-16051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi updated CASSANDRA-16051: Resolution: Cannot Reproduce Status: Resolved (was: Open) > sstablemetadata line 44: : command not found > > > Key: CASSANDRA-16051 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16051 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: Arvinder Singh >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta2 > > > sstablemetadata shipped with > [4.0-beta|https://issues.apache.org/jira/issues/?jql=project+%3D+CASSANDRA+AND+fixVersion+%3D+4.0-beta] > is missing the code "#Use JAVA_HOME if set, otherwise look for java in PATH" > and report error: > sstablemetadata: line 44: : command not found > > The fix is to merge below code from 3.x: > ^# Use JAVA_HOME if set, otherwise look for java in PATH^ > ^if [ -x "$JAVA_HOME/bin/java" ]; then^ > ^JAVA="$JAVA_HOME/bin/java"^ > ^else^ > ^JAVA="`which java`"^ > ^fi^ > ^if [ "x$JAVA" = "x" ]; then^ > ^echo "Java executable not found (hint: set JAVA_HOME)" >&2^ > ^exit 1^ > ^fi^ -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16051) sstablemetadata line 44: : command not found
[
https://issues.apache.org/jira/browse/CASSANDRA-16051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17184999#comment-17184999
]
Berenguer Blasi commented on CASSANDRA-16051:
-
sstablemetadata works correctly as far as I can see on latest trunk:
{noformat}
tools/bin/sstablemetadata
test/data/legacy-sstables/ma/legacy_tables/legacy_ma_simple/ma-1-big-Data.db
SSTable:
/tmp/test/test/data/legacy-sstables/ma/legacy_tables/legacy_ma_simple/ma-1-big
Partitioner: org.apache.cassandra.dht.ByteOrderedPartitioner
Bloom Filter FP chance: 0.01
Minimum timestamp: 1445008632854000 (10/16/2015 17:17:12)
Maximum timestamp: 1445008633275000 (10/16/2015 17:17:13)
SSTable min local deletion time: 2147483647 (no tombstones)
SSTable max local deletion time: 2147483647 (no tombstones)
Compressor: org.apache.cassandra.io.compress.LZ4Compressor
Compression ratio: -1.0
TTL min: 0
TTL max: 0
First token: 30 (0)
Last token: 34 (4)
minClusteringValues: []
maxClusteringValues: []
Estimated droppable tombstones: 0.0
SSTable Level: 0
Repaired at: 0
Pending repair: --
Replay positions covered: {CommitLogPosition(segmentId=-1,
position=0)=CommitLogPosition(segmentId=1445008629850, position=682736)}
totalColumnsSet: 5
totalRows: 5
Estimated tombstone drop times:
Drop Time| Count (%) Histogram
2147483646 (01/19/2038 04:14:06) | 5 (100) OO
Percentiles
50th 2395318855 (11/26/2045 15:20:55)
75th 2395318855 (11/26/2045 15:20:55)
95th 2395318855 (11/26/2045 15:20:55)
98th 2395318855 (11/26/2045 15:20:55)
99th 2395318855 (11/26/2045 15:20:55)
Min 1996099047 (04/03/2033 01:57:27)
Max 2395318855 (11/26/2045 15:20:55)
Partition Size:
Size (bytes) | Count (%) Histogram
35 (35 B)| 5 (100) OO
Percentiles
50th 35 (35 B)
75th 35 (35 B)
95th 35 (35 B)
98th 35 (35 B)
99th 35 (35 B)
Min 30 (30 B)
Max 35 (35 B)
Column Count:
Columns | Count (%) Histogram
1 | 5 (100) OO
Percentiles
50th 1
75th 1
95th 1
98th 1
99th 1
Min 0
Max 1
Estimated cardinality: 5
EncodingStats minTTL: 0
EncodingStats minLocalDeletionTime: 144288 (09/22/2015 02:00:00)
EncodingStats minTimestamp: 1445008632854000 (10/16/2015 17:17:12)
KeyType: org.apache.cassandra.db.marshal.UTF8Type
ClusteringTypes: []
StaticColumns:
RegularColumns: val:org.apache.cassandra.db.marshal.UTF8Type
IsTransient: false
{noformat}
Also I have been running it lately often as part of CASSANDRA-16016 without
problems. I will close as I can't seem to repro but feel free to correct me and
reopen if it's still a problem. Some indications on how to repro would help.
Thx in advance.
> sstablemetadata line 44: : command not found
>
>
> Key: CASSANDRA-16051
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16051
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/sstable
>Reporter: Arvinder Singh
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.0-beta2
>
>
> sstablemetadata shipped with
> [4.0-beta|https://issues.apache.org/jira/issues/?jql=project+%3D+CASSANDRA+AND+fixVersion+%3D+4.0-beta]
> is missing the code "#Use JAVA_HOME if set, otherwise look for java in PATH"
> and report error:
> sstablemetadata: line 44: : command not found
>
> The fix is to merge below code from 3.x:
> ^# Use JAVA_HOME if set, otherwise look for java in PATH^
> ^if [ -x "$JAVA_HOME/bin/java" ]; then^
> ^JAVA="$JAVA_HOME/bin/java"^
> ^else^
> ^JAVA="`which java`"^
> ^fi^
> ^if [ "x$JAVA" = "x" ]; then^
> ^echo "Java executable not found (hint: set JAVA_HOME)" >&2^
> ^exit 1^
> ^fi^
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16051) sstablemetadata line 44: : command not found
[ https://issues.apache.org/jira/browse/CASSANDRA-16051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Berenguer Blasi reassigned CASSANDRA-16051: --- Assignee: Berenguer Blasi > sstablemetadata line 44: : command not found > > > Key: CASSANDRA-16051 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16051 > Project: Cassandra > Issue Type: Bug > Components: Tool/sstable >Reporter: Arvinder Singh >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta2 > > > sstablemetadata shipped with > [4.0-beta|https://issues.apache.org/jira/issues/?jql=project+%3D+CASSANDRA+AND+fixVersion+%3D+4.0-beta] > is missing the code "#Use JAVA_HOME if set, otherwise look for java in PATH" > and report error: > sstablemetadata: line 44: : command not found > > The fix is to merge below code from 3.x: > ^# Use JAVA_HOME if set, otherwise look for java in PATH^ > ^if [ -x "$JAVA_HOME/bin/java" ]; then^ > ^JAVA="$JAVA_HOME/bin/java"^ > ^else^ > ^JAVA="`which java`"^ > ^fi^ > ^if [ "x$JAVA" = "x" ]; then^ > ^echo "Java executable not found (hint: set JAVA_HOME)" >&2^ > ^exit 1^ > ^fi^ -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16073) cql_tracing_test.py failing repeatedly on jenkins
[ https://issues.apache.org/jira/browse/CASSANDRA-16073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17184961#comment-17184961 ] Berenguer Blasi commented on CASSANDRA-16073: - [~blerer] did you fire another CI run in the end? I am curious to see the results... > cql_tracing_test.py failing repeatedly on jenkins > - > > Key: CASSANDRA-16073 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16073 > Project: Cassandra > Issue Type: Bug > Components: Test/dtest >Reporter: Berenguer Blasi >Assignee: Berenguer Blasi >Priority: Normal > Fix For: 4.0-beta > > Time Spent: 20m > Remaining Estimate: 0h > > Jenkins has been failing tracing tests repeatedly -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15988) Add nodetool getfullquerylog
[ https://issues.apache.org/jira/browse/CASSANDRA-15988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-15988: -- Test and Documentation Plan: The code is tested via standard unit tests. Status: Patch Available (was: Open) https://github.com/apache/cassandra/pull/721 > Add nodetool getfullquerylog > - > > Key: CASSANDRA-15988 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15988 > Project: Cassandra > Issue Type: Improvement > Components: Tool/fql >Reporter: Ekaterina Dimitrova >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 4.0-rc > > > This ticket is raised based on CASSANDRA-15791 and valuable feedback provided > by [~jshook]. > There are two outstanding questions: > * forming the exact shape of such a command and how it can benefit the > users; to be discussed in detail in this ticket > * Is this a thing we as a project can add to 4.0 beta or it should be > considered in 4.1 for example > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
