[jira] [Comment Edited] (CASSANDRA-17698) sstabledump errors when dumping data from index
[
https://issues.apache.org/jira/browse/CASSANDRA-17698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575062#comment-17575062
]
maxwellguo edited comment on CASSANDRA-17698 at 8/4/22 6:49 AM:
Just have this problem for me now , vesion is : trunk and 3.11.4, Assign this
issue to me now .
was (Author: maxwellguo):
Just meet this problem, for trunk and 3.11.4, Assign this issue to me now .
> sstabledump errors when dumping data from index
> ---
>
> Key: CASSANDRA-17698
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17698
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/sstable
>Reporter: Stefan Miklosovic
>Priority: Normal
>
> {code:java}
> cqlsh> CREATE KEYSPACE ks1 WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> cqlsh> CREATE TABLE ks1.tb1 ( id text, name text, primary key (id));
> cqlsh> CREATE INDEX IF NOT EXISTS ON ks1.tb1(name);
> cqlsh> INSERT INTO ks1.tb1 (id, name ) VALUES ( '1', 'Joe');
> cqlsh> exit
> ./bin/nodetool flush
> ./tools/bin/sstabledump
> data/data/ks1/tb1-1c3c5f10ee4711ecab82eda2f44200b3/.tb1_name_idx/nb-1-big-Data.db
>
> [
> {
> "partition" : {
> "key" : [ "Joe" ],
> "position" : 0
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 17,
> "clustering" : [ ] } ] } ]Exception in thread "main"
> java.lang.UnsupportedOperationException
> at
> org.apache.cassandra.db.marshal.PartitionerDefinedOrder.toJSONString(PartitionerDefinedOrder.java:87)
> at
> org.apache.cassandra.db.marshal.AbstractType.toJSONString(AbstractType.java:187)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeClustering(JsonTransformer.java:372)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeRow(JsonTransformer.java:269)
> at
> org.apache.cassandra.tools.JsonTransformer.serializePartition(JsonTransformer.java:235)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
> at java.util.Iterator.forEachRemaining(Iterator.java:116)
> at
> java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
> at
> java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at
> java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> at
> org.apache.cassandra.tools.JsonTransformer.toJson(JsonTransformer.java:113)
> at
> org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:214) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-17698) sstabledump errors when dumping data from index
[
https://issues.apache.org/jira/browse/CASSANDRA-17698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
maxwellguo reassigned CASSANDRA-17698:
--
Assignee: maxwellguo
> sstabledump errors when dumping data from index
> ---
>
> Key: CASSANDRA-17698
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17698
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/sstable
>Reporter: Stefan Miklosovic
>Assignee: maxwellguo
>Priority: Normal
>
> {code:java}
> cqlsh> CREATE KEYSPACE ks1 WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> cqlsh> CREATE TABLE ks1.tb1 ( id text, name text, primary key (id));
> cqlsh> CREATE INDEX IF NOT EXISTS ON ks1.tb1(name);
> cqlsh> INSERT INTO ks1.tb1 (id, name ) VALUES ( '1', 'Joe');
> cqlsh> exit
> ./bin/nodetool flush
> ./tools/bin/sstabledump
> data/data/ks1/tb1-1c3c5f10ee4711ecab82eda2f44200b3/.tb1_name_idx/nb-1-big-Data.db
>
> [
> {
> "partition" : {
> "key" : [ "Joe" ],
> "position" : 0
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 17,
> "clustering" : [ ] } ] } ]Exception in thread "main"
> java.lang.UnsupportedOperationException
> at
> org.apache.cassandra.db.marshal.PartitionerDefinedOrder.toJSONString(PartitionerDefinedOrder.java:87)
> at
> org.apache.cassandra.db.marshal.AbstractType.toJSONString(AbstractType.java:187)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeClustering(JsonTransformer.java:372)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeRow(JsonTransformer.java:269)
> at
> org.apache.cassandra.tools.JsonTransformer.serializePartition(JsonTransformer.java:235)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
> at java.util.Iterator.forEachRemaining(Iterator.java:116)
> at
> java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
> at
> java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at
> java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> at
> org.apache.cassandra.tools.JsonTransformer.toJson(JsonTransformer.java:113)
> at
> org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:214) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17698) sstabledump errors when dumping data from index
[
https://issues.apache.org/jira/browse/CASSANDRA-17698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575062#comment-17575062
]
maxwellguo commented on CASSANDRA-17698:
Just meet this problem, for trunk and 3.11.4, Assign this issue to me now .
> sstabledump errors when dumping data from index
> ---
>
> Key: CASSANDRA-17698
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17698
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/sstable
>Reporter: Stefan Miklosovic
>Priority: Normal
>
> {code:java}
> cqlsh> CREATE KEYSPACE ks1 WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};
> cqlsh> CREATE TABLE ks1.tb1 ( id text, name text, primary key (id));
> cqlsh> CREATE INDEX IF NOT EXISTS ON ks1.tb1(name);
> cqlsh> INSERT INTO ks1.tb1 (id, name ) VALUES ( '1', 'Joe');
> cqlsh> exit
> ./bin/nodetool flush
> ./tools/bin/sstabledump
> data/data/ks1/tb1-1c3c5f10ee4711ecab82eda2f44200b3/.tb1_name_idx/nb-1-big-Data.db
>
> [
> {
> "partition" : {
> "key" : [ "Joe" ],
> "position" : 0
> },
> "rows" : [
> {
> "type" : "row",
> "position" : 17,
> "clustering" : [ ] } ] } ]Exception in thread "main"
> java.lang.UnsupportedOperationException
> at
> org.apache.cassandra.db.marshal.PartitionerDefinedOrder.toJSONString(PartitionerDefinedOrder.java:87)
> at
> org.apache.cassandra.db.marshal.AbstractType.toJSONString(AbstractType.java:187)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeClustering(JsonTransformer.java:372)
> at
> org.apache.cassandra.tools.JsonTransformer.serializeRow(JsonTransformer.java:269)
> at
> org.apache.cassandra.tools.JsonTransformer.serializePartition(JsonTransformer.java:235)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at
> java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
> at java.util.Iterator.forEachRemaining(Iterator.java:116)
> at
> java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
> at
> java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at
> java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> at
> org.apache.cassandra.tools.JsonTransformer.toJson(JsonTransformer.java:113)
> at
> org.apache.cassandra.tools.SSTableExport.main(SSTableExport.java:214) {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14940) Cassandra update operation failed silently
[
https://issues.apache.org/jira/browse/CASSANDRA-14940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17575051#comment-17575051
]
Claude Warren commented on CASSANDRA-14940:
---
[~wqg6848] this issue has been open for some time, is it still and issue for
you? Do you have a work around?
> Cassandra update operation failed silently
> --
>
> Key: CASSANDRA-14940
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14940
> Project: Cassandra
> Issue Type: Bug
> Environment: ENV：
> {color:#FF}Single Cassandra node{color}
> Ubuntu 16.04 LTS
> Memory: 7.8 GiB
> Processor: Intel® Core™ i7-7700K CPU @ 4.20GHz × 8
> go: v1.9.2
> github.com/gocql/gocql : 70385f88b28b43805bd83d212169ab2d38810b15
> github.com/scylladb/gocqlx: dcec9d031fb594a4bc470473d1c98d33656f93e6
> docker image: cassandra:lastest(8ea89760ce2b)
>Reporter: wings
>Priority: Urgent
> Attachments: Screenshot from 2018-12-18 19-03-19.png, Screenshot from
> 2018-12-18 19-04-49.png, demo.tar.gz, event.txt, event.txt, session.txt,
> session.txt, test.txt, test.txt
>
>
> *Inserting a piece of data into Cassandra and then updating this data may
> fail silently.*
>
>
>
> Operation record： #
> sudo docker run -d -v /home/xxx/Desktop/demo/:/demo cassandra
> #
> sudo docker ps
> {color:#FF}CONTAINER ID{color} IMAGE
> df35191c6957 cassandra
>
> #
> sudo docker exec -it df35191c6957 bash
> ---
> {color:#FF}root@df35191c6957{color}:/# cd /demo/cassandra/
> {color:#FF}root@df35191c6957{color}:/demo/cassandra# ./cassandra -order
> init
> Create KeySpace: wqg
> Init Cassandra Env Sucess
> {color:#FF}root@df35191c6957{color}:/demo/cassandra# ./cassandra -l 50 -s > 0
> Clean Done
> Insert Success: 0
> Update Success: 0
> .
> Insert Success: 23
> Update Success: 23
> .
> Insert Success: 49
> Update Success: 49
> {color:#FF}*[Abnormal]: 23*{color}
>
> {color:#FF}root@df35191c6957{color}:/demo/cassandra# cqlsh
> Connected to Test Cluster at 127.0.0.1:9042.
> [cqlsh 5.0.1 | Cassandra 3.11.3 | CQL spec 3.4.4 | Native protocol v4]
> Use HELP for help.
> cqlsh> select * from wqg.info where id=23;
> id | deleted | name | time
> +-+-+-
> 23 | {color:#FF}False{color} | wqg-test-23 | 2018-12-18
> 09:58:11.777000+
> (1 rows)
>
> Fake code:
> {code:java}
> for i := 0; i < 50; i ++ {
> // sql: INSERT INTO info (id,name,time,deleted) VALUES (?,?,?,?)
> // NOTICE: The delete field will be set to "false"
> insert()
> // sql: UPDATE %s SET deleted = ? WHERE id = ? IF deleted=False
> // NOTICE: The delete field will be set to "true"
> update()
> }
>
> {code}
>
> If you can't reproduce, increase the number of loops, for example
> "./cassandra -l 100 -s 0"
>
> Thanks
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17799) WEBSITE - August 2022 blog "Watch the Cassandra World Party"
[ https://issues.apache.org/jira/browse/CASSANDRA-17799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Diogenese Topper updated CASSANDRA-17799: - Status: Open (was: Triage Needed) Text reviewed, translated to .adoc format - need someone for final check in staging and commit. > WEBSITE - August 2022 blog "Watch the Cassandra World Party" > > > Key: CASSANDRA-17799 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17799 > Project: Cassandra > Issue Type: Task > Components: Documentation/Blog >Reporter: Diogenese Topper >Priority: Normal > > This ticket is to capture the work associated with publishing the August 2022 > blog "Watch the Cassandra World Party" > If this blog cannot be published by the *August 4, 2022 publish date*, please > contact me, suggest changes, or correct the date when possible in the pull > request for the appropriate time that the blog will go live (on both the > blog.adoc and the blog post's file). -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17799) WEBSITE - August 2022 blog "Watch the Cassandra World Party"
[ https://issues.apache.org/jira/browse/CASSANDRA-17799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Diogenese Topper updated CASSANDRA-17799: - Status: Patch Available (was: Open) https://github.com/apache/cassandra-website/pull/156 > WEBSITE - August 2022 blog "Watch the Cassandra World Party" > > > Key: CASSANDRA-17799 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17799 > Project: Cassandra > Issue Type: Task > Components: Documentation/Blog >Reporter: Diogenese Topper >Priority: Normal > Labels: pull-request-available > > This ticket is to capture the work associated with publishing the August 2022 > blog "Watch the Cassandra World Party" > If this blog cannot be published by the *August 4, 2022 publish date*, please > contact me, suggest changes, or correct the date when possible in the pull > request for the appropriate time that the blog will go live (on both the > blog.adoc and the blog post's file). -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17799) WEBSITE - August 2022 blog "Watch the Cassandra World Party"
[ https://issues.apache.org/jira/browse/CASSANDRA-17799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated CASSANDRA-17799: --- Labels: pull-request-available (was: ) > WEBSITE - August 2022 blog "Watch the Cassandra World Party" > > > Key: CASSANDRA-17799 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17799 > Project: Cassandra > Issue Type: Task > Components: Documentation/Blog >Reporter: Diogenese Topper >Priority: Normal > Labels: pull-request-available > > This ticket is to capture the work associated with publishing the August 2022 > blog "Watch the Cassandra World Party" > If this blog cannot be published by the *August 4, 2022 publish date*, please > contact me, suggest changes, or correct the date when possible in the pull > request for the appropriate time that the blog will go live (on both the > blog.adoc and the blog post's file). -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17799) WEBSITE - August 2022 blog "Watch the Cassandra World Party"
[ https://issues.apache.org/jira/browse/CASSANDRA-17799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Diogenese Topper updated CASSANDRA-17799: - Authors: Chris Thornett, Diogenese Topper Change Category: Semantic Complexity: Normal Component/s: Documentation/Blog Impacts: Docs (was: None) Test and Documentation Plan: Add blog post titled "Watch the Cassandra World Party" Modify blog index page Add image for blog: "CWP-stream1.png" > WEBSITE - August 2022 blog "Watch the Cassandra World Party" > > > Key: CASSANDRA-17799 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17799 > Project: Cassandra > Issue Type: Task > Components: Documentation/Blog >Reporter: Diogenese Topper >Priority: Normal > > This ticket is to capture the work associated with publishing the August 2022 > blog "Watch the Cassandra World Party" > If this blog cannot be published by the *August 4, 2022 publish date*, please > contact me, suggest changes, or correct the date when possible in the pull > request for the appropriate time that the blog will go live (on both the > blog.adoc and the blog post's file). -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17799) WEBSITE - August 2022 blog "Watch the Cassandra World Party"
Diogenese Topper created CASSANDRA-17799: Summary: WEBSITE - August 2022 blog "Watch the Cassandra World Party" Key: CASSANDRA-17799 URL: https://issues.apache.org/jira/browse/CASSANDRA-17799 Project: Cassandra Issue Type: Task Reporter: Diogenese Topper This ticket is to capture the work associated with publishing the August 2022 blog "Watch the Cassandra World Party" If this blog cannot be published by the *August 4, 2022 publish date*, please contact me, suggest changes, or correct the date when possible in the pull request for the appropriate time that the blog will go live (on both the blog.adoc and the blog post's file). -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch asf-staging updated (49b37b95d -> 6f4f570a8)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git discard 49b37b95d generate docs for 93aef61a new 6f4f570a8 generate docs for 93aef61a This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (49b37b95d) \ N -- N -- N refs/heads/asf-staging (6f4f570a8) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: site-ui/build/ui-bundle.zip | Bin 4740078 -> 4740078 bytes 1 file changed, 0 insertions(+), 0 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17679) Make resumable bootstrap feature optional
[
https://issues.apache.org/jira/browse/CASSANDRA-17679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17574965#comment-17574965
]
Jeremiah Jordan edited comment on CASSANDRA-17679 at 8/3/22 11:15 PM:
--
{quote}All {{-Dcassandra.reset_bootstrap_progress}} seems to do is truncate the
system keyspace of ranges, but none of the already-bootstrapped-data?
{quote}
correct. But I don’t see a proposal here to add truncating? I would definitely
be in favor of adding the ability to remove all existing data before starting
the bootstrap again. All the patch here seems to do is skip looking at the
already streamed ranges, which has the exact same effect as deleting the ranges
from the system table.
I’m not against changing the default behavior. I just don’t like adding new
flags that do the same thing as existing ones. If we want to change the default
behavior and add a new yaml flag to complement the existing -D that seems
reasonable to me.
Also if the main worry here is data resurrection then I would suggest adding
that truncation ability in. Because streaming from scratch won’t help your
resurrection problem if someone waits days between retries.
was (Author: jjordan):
{quote}All {{-Dcassandra.reset_bootstrap_progress}} seems to do is truncate the
system keyspace of ranges, but none of the already-bootstrapped-data?
{quote}
correct. But I don’t see a proposal here to add truncating? I would definitely
be in favor of adding the ability to remove all existing data before starting
the bootstrap again. All the patch here seems to do is skip looking at the
already streamed ranges, which has the exact same effect as deleting the ranges
from the system table.
I’m not against changing the default behavior. I just don’t like adding new
flags that do the same thing as existing ones. If we want to change the default
behavior and add a new yaml flag to complement the existing -D that seems
reasonable to me.
Also if the main worry here is data redirections then I would suggest adding
that truncation ability in. Because streaming from scratch won’t help your
resurrection problem if someone waits days between retries.
> Make resumable bootstrap feature optional
> -
>
> Key: CASSANDRA-17679
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17679
> Project: Cassandra
> Issue Type: New Feature
> Components: Consistency/Streaming
>Reporter: Josh McKenzie
>Assignee: Josh McKenzie
>Priority: Normal
> Fix For: 4.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> From the patch I'm working on:
> {code}
> # In certain environments, operators may want to disable resumable bootstrap
> in order to avoid potential correctness
> # violations or data loss scenarios. Largely this centers around nodes going
> down during bootstrap, tombstones being
> # written, and potential races with repair. By default we leave this on as
> it's been enabled for quite some time,
> # however the option to disable it is more palatable now that we have zero
> copy streaming as that greatly accelerates
> # bootstraps. This defaults to true.
> # resumable_bootstrap_enabled: true
> {code}
> Not really a great fit for guardrails as it's less a "feature to be toggled
> on and off" and more a subset of a specific feature that in certain
> circumstances can lead to issues.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17679) Make resumable bootstrap feature optional
[
https://issues.apache.org/jira/browse/CASSANDRA-17679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17574965#comment-17574965
]
Jeremiah Jordan commented on CASSANDRA-17679:
-
{quote}All {{-Dcassandra.reset_bootstrap_progress}} seems to do is truncate the
system keyspace of ranges, but none of the already-bootstrapped-data?
{quote}
correct. But I don’t see a proposal here to add truncating? I would definitely
be in favor of adding the ability to remove all existing data before starting
the bootstrap again. All the patch here seems to do is skip looking at the
already streamed ranges, which has the exact same effect as deleting the ranges
from the system table.
I’m not against changing the default behavior. I just don’t like adding new
flags that do the same thing as existing ones. If we want to change the default
behavior and add a new yaml flag to complement the existing -D that seems
reasonable to me.
Also if the main worry here is data redirections then I would suggest adding
that truncation ability in. Because streaming from scratch won’t help your
resurrection problem if someone waits days between retries.
> Make resumable bootstrap feature optional
> -
>
> Key: CASSANDRA-17679
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17679
> Project: Cassandra
> Issue Type: New Feature
> Components: Consistency/Streaming
>Reporter: Josh McKenzie
>Assignee: Josh McKenzie
>Priority: Normal
> Fix For: 4.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> From the patch I'm working on:
> {code}
> # In certain environments, operators may want to disable resumable bootstrap
> in order to avoid potential correctness
> # violations or data loss scenarios. Largely this centers around nodes going
> down during bootstrap, tombstones being
> # written, and potential races with repair. By default we leave this on as
> it's been enabled for quite some time,
> # however the option to disable it is more palatable now that we have zero
> copy streaming as that greatly accelerates
> # bootstraps. This defaults to true.
> # resumable_bootstrap_enabled: true
> {code}
> Not really a great fit for guardrails as it's less a "feature to be toggled
> on and off" and more a subset of a specific feature that in certain
> circumstances can lead to issues.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17679) Make resumable bootstrap feature optional
[
https://issues.apache.org/jira/browse/CASSANDRA-17679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17574964#comment-17574964
]
Josh McKenzie commented on CASSANDRA-17679:
---
{quote}Also I would not mention ZCS as a reason to disable this. ZCS applies
pretty narrowly right now since it only works for whole files. So for the most
part only works if you are using LCS, which is not the default compaction
strategy.
{quote}
Streaming in general is faster in 4.0 even without taking ZCS into account so
again, somewhat mitigated. Also, data resurrection is Bad; until someone steps
forward to fix the feature I don't think we should enable it by default even if
we haven't tackled the "bootstrapping large nodes is a pain" problem. Unless
you and others are advocating for the counter position that resurrection in
some scenarios is acceptable? And/or the scenarios enumerated on the original
ticket aren't actually an issue?
{quote}We already have the "-Dcassandra.reset_bootstrap_progress" flag which
disables resuming
{quote}
While we could better document the flag, having our default behavior of the
database be "may resurrect deleted data" is pretty unfortunate.
> Make resumable bootstrap feature optional
> -
>
> Key: CASSANDRA-17679
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17679
> Project: Cassandra
> Issue Type: New Feature
> Components: Consistency/Streaming
>Reporter: Josh McKenzie
>Assignee: Josh McKenzie
>Priority: Normal
> Fix For: 4.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> From the patch I'm working on:
> {code}
> # In certain environments, operators may want to disable resumable bootstrap
> in order to avoid potential correctness
> # violations or data loss scenarios. Largely this centers around nodes going
> down during bootstrap, tombstones being
> # written, and potential races with repair. By default we leave this on as
> it's been enabled for quite some time,
> # however the option to disable it is more palatable now that we have zero
> copy streaming as that greatly accelerates
> # bootstraps. This defaults to true.
> # resumable_bootstrap_enabled: true
> {code}
> Not really a great fit for guardrails as it's less a "feature to be toggled
> on and off" and more a subset of a specific feature that in certain
> circumstances can lead to issues.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17679) Make resumable bootstrap feature optional
[
https://issues.apache.org/jira/browse/CASSANDRA-17679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17574949#comment-17574949
]
Jeff Jirsa commented on CASSANDRA-17679:
All {{-Dcassandra.reset_bootstrap_progress}} seems to do is truncate the system
keyspace of ranges, but none of the already-bootstrapped-data?
https://github.com/apache/cassandra/blob/afaeb9302008331d8520687594afcb9f9ee664dc/src/java/org/apache/cassandra/db/SystemKeyspace.java#L1388
> Make resumable bootstrap feature optional
> -
>
> Key: CASSANDRA-17679
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17679
> Project: Cassandra
> Issue Type: New Feature
> Components: Consistency/Streaming
>Reporter: Josh McKenzie
>Assignee: Josh McKenzie
>Priority: Normal
> Fix For: 4.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> From the patch I'm working on:
> {code}
> # In certain environments, operators may want to disable resumable bootstrap
> in order to avoid potential correctness
> # violations or data loss scenarios. Largely this centers around nodes going
> down during bootstrap, tombstones being
> # written, and potential races with repair. By default we leave this on as
> it's been enabled for quite some time,
> # however the option to disable it is more palatable now that we have zero
> copy streaming as that greatly accelerates
> # bootstraps. This defaults to true.
> # resumable_bootstrap_enabled: true
> {code}
> Not really a great fit for guardrails as it's less a "feature to be toggled
> on and off" and more a subset of a specific feature that in certain
> circumstances can lead to issues.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17798) Flaky org.apache.cassandra.tools TopPartitionsTest testServiceTopPartitionsSingleTable
[
https://issues.apache.org/jira/browse/CASSANDRA-17798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-17798:
Test and Documentation Plan: (was: See PR)
> Flaky org.apache.cassandra.tools TopPartitionsTest
> testServiceTopPartitionsSingleTable
> --
>
> Key: CASSANDRA-17798
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17798
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.1-beta, 4.1.x, 4.x
>
>
> h3.
> {code:java}
> Error Message
> If this failed you probably have to raise the beginLocalSampling duration
> expected:<1> but was:<0>
> Stacktrace
> junit.framework.AssertionFailedError: If this failed you probably have to
> raise the beginLocalSampling duration expected:<1> but was:<0> at
> org.apache.cassandra.tools.TopPartitionsTest.testServiceTopPartitionsSingleTable(TopPartitionsTest.java:83)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Standard Output
> INFO [main] 2022-08-02 01:49:49,333 YamlConfigurationLoader.java:104 -
> Configuration location:
> file:home/cassandra/cassandra/build/test/cassandra.cdc.yaml DEBUG [main]
> 2022-08-02 01:49:49,339 YamlConfigurationLoader.java:124 - Loading settings
> from file:home/cassandra/cassandra/build/test/cassandra.cdc.yaml INFO
> [main] 2022-08-02 01:49:49,642 Config.java:1167 - Node
> configuration:[allocate_tokens_for_keyspace=null;
> allocate_tokens_for_local_replication_factor=null; allow_extra_insecure
> ...[truncated 50809 chars]... lizing counter cache with capacity of 2 MiBs
> INFO [MemtableFlushWriter:1] 2022-08-02 01:49:53,519 CacheService.java:163 -
> Scheduling counter cache save to every 7200 seconds (going to save all keys).
> DEBUG [MemtableFlushWriter:1] 2022-08-02 01:49:53,575
> ColumnFamilyStore.java:1330 - Flushed to
> [BigTableReader(path='/home/cassandra/cassandra/build/test/cassandra/data/system/IndexInfo-9f5c6374d48532299a0a5094af9ad1e3/nb-1-big-Data.db')]
> (1 sstables, 4.915KiB), biggest 4.915KiB, smallest 4.915KiB
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-17798) Flaky org.apache.cassandra.tools TopPartitionsTest testServiceTopPartitionsSingleTable
[
https://issues.apache.org/jira/browse/CASSANDRA-17798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova reassigned CASSANDRA-17798:
---
Assignee: (was: Berenguer Blasi)
> Flaky org.apache.cassandra.tools TopPartitionsTest
> testServiceTopPartitionsSingleTable
> --
>
> Key: CASSANDRA-17798
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17798
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 4.1-beta, 4.1.x, 4.x
>
>
> h3.
> {code:java}
> Error Message
> If this failed you probably have to raise the beginLocalSampling duration
> expected:<1> but was:<0>
> Stacktrace
> junit.framework.AssertionFailedError: If this failed you probably have to
> raise the beginLocalSampling duration expected:<1> but was:<0> at
> org.apache.cassandra.tools.TopPartitionsTest.testServiceTopPartitionsSingleTable(TopPartitionsTest.java:83)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Standard Output
> INFO [main] 2022-08-02 01:49:49,333 YamlConfigurationLoader.java:104 -
> Configuration location:
> file:home/cassandra/cassandra/build/test/cassandra.cdc.yaml DEBUG [main]
> 2022-08-02 01:49:49,339 YamlConfigurationLoader.java:124 - Loading settings
> from file:home/cassandra/cassandra/build/test/cassandra.cdc.yaml INFO
> [main] 2022-08-02 01:49:49,642 Config.java:1167 - Node
> configuration:[allocate_tokens_for_keyspace=null;
> allocate_tokens_for_local_replication_factor=null; allow_extra_insecure
> ...[truncated 50809 chars]... lizing counter cache with capacity of 2 MiBs
> INFO [MemtableFlushWriter:1] 2022-08-02 01:49:53,519 CacheService.java:163 -
> Scheduling counter cache save to every 7200 seconds (going to save all keys).
> DEBUG [MemtableFlushWriter:1] 2022-08-02 01:49:53,575
> ColumnFamilyStore.java:1330 - Flushed to
> [BigTableReader(path='/home/cassandra/cassandra/build/test/cassandra/data/system/IndexInfo-9f5c6374d48532299a0a5094af9ad1e3/nb-1-big-Data.db')]
> (1 sstables, 4.915KiB), biggest 4.915KiB, smallest 4.915KiB
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17798) Flaky org.apache.cassandra.tools TopPartitionsTest testServiceTopPartitionsSingleTable
[
https://issues.apache.org/jira/browse/CASSANDRA-17798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-17798:
Reviewers: (was: Brandon Williams)
> Flaky org.apache.cassandra.tools TopPartitionsTest
> testServiceTopPartitionsSingleTable
> --
>
> Key: CASSANDRA-17798
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17798
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.1-beta, 4.1.x, 4.x
>
>
> h3.
> {code:java}
> Error Message
> If this failed you probably have to raise the beginLocalSampling duration
> expected:<1> but was:<0>
> Stacktrace
> junit.framework.AssertionFailedError: If this failed you probably have to
> raise the beginLocalSampling duration expected:<1> but was:<0> at
> org.apache.cassandra.tools.TopPartitionsTest.testServiceTopPartitionsSingleTable(TopPartitionsTest.java:83)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Standard Output
> INFO [main] 2022-08-02 01:49:49,333 YamlConfigurationLoader.java:104 -
> Configuration location:
> file:home/cassandra/cassandra/build/test/cassandra.cdc.yaml DEBUG [main]
> 2022-08-02 01:49:49,339 YamlConfigurationLoader.java:124 - Loading settings
> from file:home/cassandra/cassandra/build/test/cassandra.cdc.yaml INFO
> [main] 2022-08-02 01:49:49,642 Config.java:1167 - Node
> configuration:[allocate_tokens_for_keyspace=null;
> allocate_tokens_for_local_replication_factor=null; allow_extra_insecure
> ...[truncated 50809 chars]... lizing counter cache with capacity of 2 MiBs
> INFO [MemtableFlushWriter:1] 2022-08-02 01:49:53,519 CacheService.java:163 -
> Scheduling counter cache save to every 7200 seconds (going to save all keys).
> DEBUG [MemtableFlushWriter:1] 2022-08-02 01:49:53,575
> ColumnFamilyStore.java:1330 - Flushed to
> [BigTableReader(path='/home/cassandra/cassandra/build/test/cassandra/data/system/IndexInfo-9f5c6374d48532299a0a5094af9ad1e3/nb-1-big-Data.db')]
> (1 sstables, 4.915KiB), biggest 4.915KiB, smallest 4.915KiB
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17798) Flaky org.apache.cassandra.tools TopPartitionsTest testServiceTopPartitionsSingleTable
[
https://issues.apache.org/jira/browse/CASSANDRA-17798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17574936#comment-17574936
]
Ekaterina Dimitrova edited comment on CASSANDRA-17798 at 8/3/22 9:45 PM:
-
It seems CASSANDRA-17455 fixed only 4.0 and we need to extend the timing
mentioned for 4.1 and trunk as the tests still occasionally fail there
was (Author: e.dimitrova):
It seems CASSANDRA-17455 fixed only 4.0 and we need to extend the timing
mentioned for 4.1 and trunk as the tests still occasionally fails there
> Flaky org.apache.cassandra.tools TopPartitionsTest
> testServiceTopPartitionsSingleTable
> --
>
> Key: CASSANDRA-17798
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17798
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.1-beta, 4.1.x, 4.x
>
>
> h3.
> {code:java}
> Error Message
> If this failed you probably have to raise the beginLocalSampling duration
> expected:<1> but was:<0>
> Stacktrace
> junit.framework.AssertionFailedError: If this failed you probably have to
> raise the beginLocalSampling duration expected:<1> but was:<0> at
> org.apache.cassandra.tools.TopPartitionsTest.testServiceTopPartitionsSingleTable(TopPartitionsTest.java:83)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Standard Output
> INFO [main] 2022-08-02 01:49:49,333 YamlConfigurationLoader.java:104 -
> Configuration location:
> file:home/cassandra/cassandra/build/test/cassandra.cdc.yaml DEBUG [main]
> 2022-08-02 01:49:49,339 YamlConfigurationLoader.java:124 - Loading settings
> from file:home/cassandra/cassandra/build/test/cassandra.cdc.yaml INFO
> [main] 2022-08-02 01:49:49,642 Config.java:1167 - Node
> configuration:[allocate_tokens_for_keyspace=null;
> allocate_tokens_for_local_replication_factor=null; allow_extra_insecure
> ...[truncated 50809 chars]... lizing counter cache with capacity of 2 MiBs
> INFO [MemtableFlushWriter:1] 2022-08-02 01:49:53,519 CacheService.java:163 -
> Scheduling counter cache save to every 7200 seconds (going to save all keys).
> DEBUG [MemtableFlushWriter:1] 2022-08-02 01:49:53,575
> ColumnFamilyStore.java:1330 - Flushed to
> [BigTableReader(path='/home/cassandra/cassandra/build/test/cassandra/data/system/IndexInfo-9f5c6374d48532299a0a5094af9ad1e3/nb-1-big-Data.db')]
> (1 sstables, 4.915KiB), biggest 4.915KiB, smallest 4.915KiB
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17798) Flaky org.apache.cassandra.tools TopPartitionsTest testServiceTopPartitionsSingleTable
[
https://issues.apache.org/jira/browse/CASSANDRA-17798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-17798:
Summary: Flaky org.apache.cassandra.tools TopPartitionsTest
testServiceTopPartitionsSingleTable (was: CLONE - Flaky
org.apache.cassandra.tools TopPartitionsTest
testServiceTopPartitionsSingleTable)
> Flaky org.apache.cassandra.tools TopPartitionsTest
> testServiceTopPartitionsSingleTable
> --
>
> Key: CASSANDRA-17798
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17798
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.1-beta, 4.1.x, 4.x
>
>
> h3.
> {code:java}
> Error Message
> If this failed you probably have to raise the beginLocalSampling duration
> expected:<1> but was:<0>
> Stacktrace
> junit.framework.AssertionFailedError: If this failed you probably have to
> raise the beginLocalSampling duration expected:<1> but was:<0> at
> org.apache.cassandra.tools.TopPartitionsTest.testServiceTopPartitionsSingleTable(TopPartitionsTest.java:83)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Standard Output
> INFO [main] 2022-08-02 01:49:49,333 YamlConfigurationLoader.java:104 -
> Configuration location:
> file:home/cassandra/cassandra/build/test/cassandra.cdc.yaml DEBUG [main]
> 2022-08-02 01:49:49,339 YamlConfigurationLoader.java:124 - Loading settings
> from file:home/cassandra/cassandra/build/test/cassandra.cdc.yaml INFO
> [main] 2022-08-02 01:49:49,642 Config.java:1167 - Node
> configuration:[allocate_tokens_for_keyspace=null;
> allocate_tokens_for_local_replication_factor=null; allow_extra_insecure
> ...[truncated 50809 chars]... lizing counter cache with capacity of 2 MiBs
> INFO [MemtableFlushWriter:1] 2022-08-02 01:49:53,519 CacheService.java:163 -
> Scheduling counter cache save to every 7200 seconds (going to save all keys).
> DEBUG [MemtableFlushWriter:1] 2022-08-02 01:49:53,575
> ColumnFamilyStore.java:1330 - Flushed to
> [BigTableReader(path='/home/cassandra/cassandra/build/test/cassandra/data/system/IndexInfo-9f5c6374d48532299a0a5094af9ad1e3/nb-1-big-Data.db')]
> (1 sstables, 4.915KiB), biggest 4.915KiB, smallest 4.915KiB
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17798) Flaky org.apache.cassandra.tools TopPartitionsTest testServiceTopPartitionsSingleTable
[
https://issues.apache.org/jira/browse/CASSANDRA-17798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-17798:
Source Control Link: (was:
https://github.com/apache/cassandra/commit/af24242af913b63b29691251025d2aa54e3890bd)
> Flaky org.apache.cassandra.tools TopPartitionsTest
> testServiceTopPartitionsSingleTable
> --
>
> Key: CASSANDRA-17798
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17798
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.1-beta, 4.1.x, 4.x
>
>
> h3.
> {code:java}
> Error Message
> If this failed you probably have to raise the beginLocalSampling duration
> expected:<1> but was:<0>
> Stacktrace
> junit.framework.AssertionFailedError: If this failed you probably have to
> raise the beginLocalSampling duration expected:<1> but was:<0> at
> org.apache.cassandra.tools.TopPartitionsTest.testServiceTopPartitionsSingleTable(TopPartitionsTest.java:83)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Standard Output
> INFO [main] 2022-08-02 01:49:49,333 YamlConfigurationLoader.java:104 -
> Configuration location:
> file:home/cassandra/cassandra/build/test/cassandra.cdc.yaml DEBUG [main]
> 2022-08-02 01:49:49,339 YamlConfigurationLoader.java:124 - Loading settings
> from file:home/cassandra/cassandra/build/test/cassandra.cdc.yaml INFO
> [main] 2022-08-02 01:49:49,642 Config.java:1167 - Node
> configuration:[allocate_tokens_for_keyspace=null;
> allocate_tokens_for_local_replication_factor=null; allow_extra_insecure
> ...[truncated 50809 chars]... lizing counter cache with capacity of 2 MiBs
> INFO [MemtableFlushWriter:1] 2022-08-02 01:49:53,519 CacheService.java:163 -
> Scheduling counter cache save to every 7200 seconds (going to save all keys).
> DEBUG [MemtableFlushWriter:1] 2022-08-02 01:49:53,575
> ColumnFamilyStore.java:1330 - Flushed to
> [BigTableReader(path='/home/cassandra/cassandra/build/test/cassandra/data/system/IndexInfo-9f5c6374d48532299a0a5094af9ad1e3/nb-1-big-Data.db')]
> (1 sstables, 4.915KiB), biggest 4.915KiB, smallest 4.915KiB
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17798) CLONE - Flaky org.apache.cassandra.tools TopPartitionsTest testServiceTopPartitionsSingleTable
[
https://issues.apache.org/jira/browse/CASSANDRA-17798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17574936#comment-17574936
]
Ekaterina Dimitrova commented on CASSANDRA-17798:
-
It seems CASSANDRA-17455 fixed only 4.0 and we need to extend the timing
mentioned for 4.1 and trunk as the tests still occasionally fails there
> CLONE - Flaky org.apache.cassandra.tools TopPartitionsTest
> testServiceTopPartitionsSingleTable
> --
>
> Key: CASSANDRA-17798
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17798
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.1-beta, 4.1.x, 4.x
>
>
> h3.
> {code:java}
> Error Message
> If this failed you probably have to raise the beginLocalSampling duration
> expected:<1> but was:<0>
> Stacktrace
> junit.framework.AssertionFailedError: If this failed you probably have to
> raise the beginLocalSampling duration expected:<1> but was:<0> at
> org.apache.cassandra.tools.TopPartitionsTest.testServiceTopPartitionsSingleTable(TopPartitionsTest.java:83)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Standard Output
> INFO [main] 2022-08-02 01:49:49,333 YamlConfigurationLoader.java:104 -
> Configuration location:
> file:home/cassandra/cassandra/build/test/cassandra.cdc.yaml DEBUG [main]
> 2022-08-02 01:49:49,339 YamlConfigurationLoader.java:124 - Loading settings
> from file:home/cassandra/cassandra/build/test/cassandra.cdc.yaml INFO
> [main] 2022-08-02 01:49:49,642 Config.java:1167 - Node
> configuration:[allocate_tokens_for_keyspace=null;
> allocate_tokens_for_local_replication_factor=null; allow_extra_insecure
> ...[truncated 50809 chars]... lizing counter cache with capacity of 2 MiBs
> INFO [MemtableFlushWriter:1] 2022-08-02 01:49:53,519 CacheService.java:163 -
> Scheduling counter cache save to every 7200 seconds (going to save all keys).
> DEBUG [MemtableFlushWriter:1] 2022-08-02 01:49:53,575
> ColumnFamilyStore.java:1330 - Flushed to
> [BigTableReader(path='/home/cassandra/cassandra/build/test/cassandra/data/system/IndexInfo-9f5c6374d48532299a0a5094af9ad1e3/nb-1-big-Data.db')]
> (1 sstables, 4.915KiB), biggest 4.915KiB, smallest 4.915KiB
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17798) CLONE - Flaky org.apache.cassandra.tools TopPartitionsTest testServiceTopPartitionsSingleTable
[
https://issues.apache.org/jira/browse/CASSANDRA-17798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-17798:
Complexity: Low Hanging Fruit (was: Normal)
> CLONE - Flaky org.apache.cassandra.tools TopPartitionsTest
> testServiceTopPartitionsSingleTable
> --
>
> Key: CASSANDRA-17798
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17798
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.1-beta, 4.1.x, 4.x
>
>
> h3.
> {code:java}
> Error Message
> If this failed you probably have to raise the beginLocalSampling duration
> expected:<1> but was:<0>
> Stacktrace
> junit.framework.AssertionFailedError: If this failed you probably have to
> raise the beginLocalSampling duration expected:<1> but was:<0> at
> org.apache.cassandra.tools.TopPartitionsTest.testServiceTopPartitionsSingleTable(TopPartitionsTest.java:83)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Standard Output
> INFO [main] 2022-08-02 01:49:49,333 YamlConfigurationLoader.java:104 -
> Configuration location:
> file:home/cassandra/cassandra/build/test/cassandra.cdc.yaml DEBUG [main]
> 2022-08-02 01:49:49,339 YamlConfigurationLoader.java:124 - Loading settings
> from file:home/cassandra/cassandra/build/test/cassandra.cdc.yaml INFO
> [main] 2022-08-02 01:49:49,642 Config.java:1167 - Node
> configuration:[allocate_tokens_for_keyspace=null;
> allocate_tokens_for_local_replication_factor=null; allow_extra_insecure
> ...[truncated 50809 chars]... lizing counter cache with capacity of 2 MiBs
> INFO [MemtableFlushWriter:1] 2022-08-02 01:49:53,519 CacheService.java:163 -
> Scheduling counter cache save to every 7200 seconds (going to save all keys).
> DEBUG [MemtableFlushWriter:1] 2022-08-02 01:49:53,575
> ColumnFamilyStore.java:1330 - Flushed to
> [BigTableReader(path='/home/cassandra/cassandra/build/test/cassandra/data/system/IndexInfo-9f5c6374d48532299a0a5094af9ad1e3/nb-1-big-Data.db')]
> (1 sstables, 4.915KiB), biggest 4.915KiB, smallest 4.915KiB
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17798) CLONE - Flaky org.apache.cassandra.tools TopPartitionsTest testServiceTopPartitionsSingleTable
[
https://issues.apache.org/jira/browse/CASSANDRA-17798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-17798:
Fix Version/s: 4.1.x
4.x
> CLONE - Flaky org.apache.cassandra.tools TopPartitionsTest
> testServiceTopPartitionsSingleTable
> --
>
> Key: CASSANDRA-17798
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17798
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.1-beta, 4.1.x, 4.x
>
>
> h3.
> {code:java}
> Error Message
> If this failed you probably have to raise the beginLocalSampling duration
> expected:<1> but was:<0>
> Stacktrace
> junit.framework.AssertionFailedError: If this failed you probably have to
> raise the beginLocalSampling duration expected:<1> but was:<0> at
> org.apache.cassandra.tools.TopPartitionsTest.testServiceTopPartitionsSingleTable(TopPartitionsTest.java:83)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Standard Output
> INFO [main] 2022-08-02 01:49:49,333 YamlConfigurationLoader.java:104 -
> Configuration location:
> file:home/cassandra/cassandra/build/test/cassandra.cdc.yaml DEBUG [main]
> 2022-08-02 01:49:49,339 YamlConfigurationLoader.java:124 - Loading settings
> from file:home/cassandra/cassandra/build/test/cassandra.cdc.yaml INFO
> [main] 2022-08-02 01:49:49,642 Config.java:1167 - Node
> configuration:[allocate_tokens_for_keyspace=null;
> allocate_tokens_for_local_replication_factor=null; allow_extra_insecure
> ...[truncated 50809 chars]... lizing counter cache with capacity of 2 MiBs
> INFO [MemtableFlushWriter:1] 2022-08-02 01:49:53,519 CacheService.java:163 -
> Scheduling counter cache save to every 7200 seconds (going to save all keys).
> DEBUG [MemtableFlushWriter:1] 2022-08-02 01:49:53,575
> ColumnFamilyStore.java:1330 - Flushed to
> [BigTableReader(path='/home/cassandra/cassandra/build/test/cassandra/data/system/IndexInfo-9f5c6374d48532299a0a5094af9ad1e3/nb-1-big-Data.db')]
> (1 sstables, 4.915KiB), biggest 4.915KiB, smallest 4.915KiB
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17798) CLONE - Flaky org.apache.cassandra.tools TopPartitionsTest testServiceTopPartitionsSingleTable
[
https://issues.apache.org/jira/browse/CASSANDRA-17798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-17798:
Description:
h3.
{code:java}
Error Message
If this failed you probably have to raise the beginLocalSampling duration
expected:<1> but was:<0>
Stacktrace
junit.framework.AssertionFailedError: If this failed you probably have to raise
the beginLocalSampling duration expected:<1> but was:<0> at
org.apache.cassandra.tools.TopPartitionsTest.testServiceTopPartitionsSingleTable(TopPartitionsTest.java:83)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method) at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
Standard Output
INFO [main] 2022-08-02 01:49:49,333 YamlConfigurationLoader.java:104 -
Configuration location:
file:home/cassandra/cassandra/build/test/cassandra.cdc.yaml DEBUG [main]
2022-08-02 01:49:49,339 YamlConfigurationLoader.java:124 - Loading settings
from file:home/cassandra/cassandra/build/test/cassandra.cdc.yaml INFO
[main] 2022-08-02 01:49:49,642 Config.java:1167 - Node
configuration:[allocate_tokens_for_keyspace=null;
allocate_tokens_for_local_replication_factor=null; allow_extra_insecure
...[truncated 50809 chars]... lizing counter cache with capacity of 2 MiBs INFO
[MemtableFlushWriter:1] 2022-08-02 01:49:53,519 CacheService.java:163 -
Scheduling counter cache save to every 7200 seconds (going to save all keys).
DEBUG [MemtableFlushWriter:1] 2022-08-02 01:49:53,575
ColumnFamilyStore.java:1330 - Flushed to
[BigTableReader(path='/home/cassandra/cassandra/build/test/cassandra/data/system/IndexInfo-9f5c6374d48532299a0a5094af9ad1e3/nb-1-big-Data.db')]
(1 sstables, 4.915KiB), biggest 4.915KiB, smallest 4.915KiB
{code}
was:
{code:java}
Error Messageexpected:<1> but
was:<0>Stacktracejunit.framework.AssertionFailedError: expected:<1> but was:<0>
at
org.apache.cassandra.tools.TopPartitionsTest.testServiceTopPartitionsSingleTable(TopPartitionsTest.java:82)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
Standard OutputINFO [main] 2022-03-17 23:37:01,750
YamlConfigurationLoader.java:103 - Configuration location:
file:/home/cassandra/cassandra/test/conf/cassandra.yaml
DEBUG [main] 2022-03-17 23:37:01,754 YamlConfigurationLoader.java:124 - Loading
settings from file:/home/cassandra/cassandra/test/conf/cassandra.yaml
INFO [main] 2022-03-17 23:37:01,878 Config.java:1119 - Node
configuration:[allocate_tokens_for_keyspace=null;
allocate_tokens_for_local_replication_factor=null;
allow_extra_insecure_udfs=false; all
...[truncated 49100 chars]...
lizing counter cache with capacity of 2 MiBs
INFO [MemtableFlushWriter:1] 2022-03-17 23:37:03,857 CacheService.java:164 -
Scheduling counter cache save to every 7200 seconds (going to save all keys).
DEBUG [MemtableFlushWriter:1] 2022-03-17 23:37:03,883
ColumnFamilyStore.java:1249 - Flushed to
[BigTableReader(path='/home/cassandra/cassandra/build/test/cassandra/data/system/IndexInfo-9f5c6374d48532299a0a5094af9ad1e3/nb-1-big-Data.db')]
(1 sstables, 4.915KiB), biggest 4.915KiB, smallest 4.915KiB {code}
> CLONE - Flaky org.apache.cassandra.tools TopPartitionsTest
> testServiceTopPartitionsSingleTable
> --
>
> Key: CASSANDRA-17798
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17798
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.1-beta
>
>
> h3.
> {code:java}
> Error Message
> If this failed you probably have to raise the beginLocalSampling duration
> expected:<1> but was:<0>
> Stacktrace
> junit.framework.AssertionFailedError: If this failed you probably have to
> raise the beginLocalSampling duration expected:<1> but was:<0> at
> org.apache.cassandra.tools.TopPartitionsTest.testServiceTopPartitionsSingleTable(TopPartitionsTest.java:83)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Standard Output
> INFO [main] 2022-08-02 01:49:49,333 YamlConfigurationLoader.java:104 -
> Configur
[jira] [Updated] (CASSANDRA-17798) CLONE - Flaky org.apache.cassandra.tools TopPartitionsTest testServiceTopPartitionsSingleTable
[
https://issues.apache.org/jira/browse/CASSANDRA-17798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-17798:
Status: Open (was: Triage Needed)
> CLONE - Flaky org.apache.cassandra.tools TopPartitionsTest
> testServiceTopPartitionsSingleTable
> --
>
> Key: CASSANDRA-17798
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17798
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Berenguer Blasi
>Priority: Normal
> Fix For: 4.1-beta
>
>
> h3.
> {code:java}
> Error Message
> If this failed you probably have to raise the beginLocalSampling duration
> expected:<1> but was:<0>
> Stacktrace
> junit.framework.AssertionFailedError: If this failed you probably have to
> raise the beginLocalSampling duration expected:<1> but was:<0> at
> org.apache.cassandra.tools.TopPartitionsTest.testServiceTopPartitionsSingleTable(TopPartitionsTest.java:83)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Standard Output
> INFO [main] 2022-08-02 01:49:49,333 YamlConfigurationLoader.java:104 -
> Configuration location:
> file:home/cassandra/cassandra/build/test/cassandra.cdc.yaml DEBUG [main]
> 2022-08-02 01:49:49,339 YamlConfigurationLoader.java:124 - Loading settings
> from file:home/cassandra/cassandra/build/test/cassandra.cdc.yaml INFO
> [main] 2022-08-02 01:49:49,642 Config.java:1167 - Node
> configuration:[allocate_tokens_for_keyspace=null;
> allocate_tokens_for_local_replication_factor=null; allow_extra_insecure
> ...[truncated 50809 chars]... lizing counter cache with capacity of 2 MiBs
> INFO [MemtableFlushWriter:1] 2022-08-02 01:49:53,519 CacheService.java:163 -
> Scheduling counter cache save to every 7200 seconds (going to save all keys).
> DEBUG [MemtableFlushWriter:1] 2022-08-02 01:49:53,575
> ColumnFamilyStore.java:1330 - Flushed to
> [BigTableReader(path='/home/cassandra/cassandra/build/test/cassandra/data/system/IndexInfo-9f5c6374d48532299a0a5094af9ad1e3/nb-1-big-Data.db')]
> (1 sstables, 4.915KiB), biggest 4.915KiB, smallest 4.915KiB
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17798) CLONE - Flaky org.apache.cassandra.tools TopPartitionsTest testServiceTopPartitionsSingleTable
Ekaterina Dimitrova created CASSANDRA-17798:
---
Summary: CLONE - Flaky org.apache.cassandra.tools
TopPartitionsTest testServiceTopPartitionsSingleTable
Key: CASSANDRA-17798
URL: https://issues.apache.org/jira/browse/CASSANDRA-17798
Project: Cassandra
Issue Type: Bug
Components: CI
Reporter: Ekaterina Dimitrova
Assignee: Berenguer Blasi
Fix For: 4.1-beta
{code:java}
Error Messageexpected:<1> but
was:<0>Stacktracejunit.framework.AssertionFailedError: expected:<1> but was:<0>
at
org.apache.cassandra.tools.TopPartitionsTest.testServiceTopPartitionsSingleTable(TopPartitionsTest.java:82)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
Standard OutputINFO [main] 2022-03-17 23:37:01,750
YamlConfigurationLoader.java:103 - Configuration location:
file:/home/cassandra/cassandra/test/conf/cassandra.yaml
DEBUG [main] 2022-03-17 23:37:01,754 YamlConfigurationLoader.java:124 - Loading
settings from file:/home/cassandra/cassandra/test/conf/cassandra.yaml
INFO [main] 2022-03-17 23:37:01,878 Config.java:1119 - Node
configuration:[allocate_tokens_for_keyspace=null;
allocate_tokens_for_local_replication_factor=null;
allow_extra_insecure_udfs=false; all
...[truncated 49100 chars]...
lizing counter cache with capacity of 2 MiBs
INFO [MemtableFlushWriter:1] 2022-03-17 23:37:03,857 CacheService.java:164 -
Scheduling counter cache save to every 7200 seconds (going to save all keys).
DEBUG [MemtableFlushWriter:1] 2022-03-17 23:37:03,883
ColumnFamilyStore.java:1249 - Flushed to
[BigTableReader(path='/home/cassandra/cassandra/build/test/cassandra/data/system/IndexInfo-9f5c6374d48532299a0a5094af9ad1e3/nb-1-big-Data.db')]
(1 sstables, 4.915KiB), biggest 4.915KiB, smallest 4.915KiB {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17103) CEP-15 (C*): Messaging and storage engine integration
[ https://issues.apache.org/jira/browse/CASSANDRA-17103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17574921#comment-17574921 ] David Capwell commented on CASSANDRA-17103: --- one more PR to fix up a few issues found https://github.com/bdeggleston/cassandra/pull/4 > CEP-15 (C*): Messaging and storage engine integration > - > > Key: CASSANDRA-17103 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17103 > Project: Cassandra > Issue Type: New Feature > Components: Accord >Reporter: Benedict Elliott Smith >Priority: Normal > > This work encompasses implementing Accord’s storage and networking interfaces > within Cassandra, so that messages may be sent around the cluster and > exectuted -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17797) All system properties and environment variables should be accessed via the new CassandraRelevantProperties and CassandraRelevantEnv classes
[ https://issues.apache.org/jira/browse/CASSANDRA-17797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-17797: Change Category: Operability Complexity: Low Hanging Fruit Component/s: Local/Config Fix Version/s: 4.x Priority: Low (was: Normal) Status: Open (was: Triage Needed) > All system properties and environment variables should be accessed via the > new CassandraRelevantProperties and CassandraRelevantEnv classes > --- > > Key: CASSANDRA-17797 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17797 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Ekaterina Dimitrova >Priority: Low > Fix For: 4.x > > > Follow up ticket for CASSANDRA-15876 - > "Always access system properties and environment variables via the new > CassandraRelevantProperties and CassandraRelevantEnv classes" > As part of that ticket we moved to those classes only properties/variables > that were currently listed in System Properties Virtual Table. > We have to move to those classes the rest of the properties around the code > and start using those classes to access all of them. > +Additional information for newcomers:+ > You might want to start by getting acquainted with > CassandraRelevantProperties and CassandraRelevantEnv classes. Also, you might > want to check what changes were done and how the first batch was transferred > to this new framework as part of > [CASSANDRA-15876|https://github.com/apache/cassandra/commit/7694c1d191531ac152db55e83bc0db6864a5441e] > We are interested into the properties accessed currently through > getProperties around the code. > As part of CASSANDRA-15876 relevant unit tests were added > (CassandraRelevantPropertiesTest). To verify the new patch we need to ensure > that all tests Cassandra pass and also to think about potential update of the > mentioned test class. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17797) All system properties and environment variables should be accessed via the new CassandraRelevantProperties and CassandraRelevantEnv classes
[ https://issues.apache.org/jira/browse/CASSANDRA-17797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-17797: Change Category: Code Clarity (was: Operability) > All system properties and environment variables should be accessed via the > new CassandraRelevantProperties and CassandraRelevantEnv classes > --- > > Key: CASSANDRA-17797 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17797 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Ekaterina Dimitrova >Priority: Low > Labels: ghc-lhf > Fix For: 4.x > > > Follow up ticket for CASSANDRA-15876 - > "Always access system properties and environment variables via the new > CassandraRelevantProperties and CassandraRelevantEnv classes" > As part of that ticket we moved to the two new classes only > properties/variables that were currently listed in System Properties Virtual > Table. > We have to move to those classes the rest of the properties around the code > and start using those classes to access all of them. > +Additional information for newcomers:+ > You might want to start by getting acquainted with > CassandraRelevantProperties and CassandraRelevantEnv classes. Also, you might > want to check what changes were done and how the first batch was transferred > to this new framework as part of > [CASSANDRA-15876|https://github.com/apache/cassandra/commit/7694c1d191531ac152db55e83bc0db6864a5441e] > We are interested into the properties accessed currently through > getProperties around the code. > As part of CASSANDRA-15876 relevant unit tests were added > (CassandraRelevantPropertiesTest). To verify the new patch we need to ensure > that all tests Cassandra pass and also to think about potential update of the > mentioned test class. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17797) All system properties and environment variables should be accessed via the new CassandraRelevantProperties and CassandraRelevantEnv classes
[ https://issues.apache.org/jira/browse/CASSANDRA-17797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-17797: Description: Follow up ticket for CASSANDRA-15876 - "Always access system properties and environment variables via the new CassandraRelevantProperties and CassandraRelevantEnv classes" As part of that ticket we moved to the two new classes only properties/variables that were currently listed in System Properties Virtual Table. We have to move to those classes the rest of the properties around the code and start using those classes to access all of them. +Additional information for newcomers:+ You might want to start by getting acquainted with CassandraRelevantProperties and CassandraRelevantEnv classes. Also, you might want to check what changes were done and how the first batch was transferred to this new framework as part of [CASSANDRA-15876|https://github.com/apache/cassandra/commit/7694c1d191531ac152db55e83bc0db6864a5441e] We are interested into the properties accessed currently through getProperties around the code. As part of CASSANDRA-15876 relevant unit tests were added (CassandraRelevantPropertiesTest). To verify the new patch we need to ensure that all tests Cassandra pass and also to think about potential update of the mentioned test class. was: Follow up ticket for CASSANDRA-15876 - "Always access system properties and environment variables via the new CassandraRelevantProperties and CassandraRelevantEnv classes" As part of that ticket we moved to those classes only properties/variables that were currently listed in System Properties Virtual Table. We have to move to those classes the rest of the properties around the code and start using those classes to access all of them. +Additional information for newcomers:+ You might want to start by getting acquainted with CassandraRelevantProperties and CassandraRelevantEnv classes. Also, you might want to check what changes were done and how the first batch was transferred to this new framework as part of [CASSANDRA-15876|https://github.com/apache/cassandra/commit/7694c1d191531ac152db55e83bc0db6864a5441e] We are interested into the properties accessed currently through getProperties around the code. As part of CASSANDRA-15876 relevant unit tests were added (CassandraRelevantPropertiesTest). To verify the new patch we need to ensure that all tests Cassandra pass and also to think about potential update of the mentioned test class. > All system properties and environment variables should be accessed via the > new CassandraRelevantProperties and CassandraRelevantEnv classes > --- > > Key: CASSANDRA-17797 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17797 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Ekaterina Dimitrova >Priority: Low > Labels: ghc-lhf > Fix For: 4.x > > > Follow up ticket for CASSANDRA-15876 - > "Always access system properties and environment variables via the new > CassandraRelevantProperties and CassandraRelevantEnv classes" > As part of that ticket we moved to the two new classes only > properties/variables that were currently listed in System Properties Virtual > Table. > We have to move to those classes the rest of the properties around the code > and start using those classes to access all of them. > +Additional information for newcomers:+ > You might want to start by getting acquainted with > CassandraRelevantProperties and CassandraRelevantEnv classes. Also, you might > want to check what changes were done and how the first batch was transferred > to this new framework as part of > [CASSANDRA-15876|https://github.com/apache/cassandra/commit/7694c1d191531ac152db55e83bc0db6864a5441e] > We are interested into the properties accessed currently through > getProperties around the code. > As part of CASSANDRA-15876 relevant unit tests were added > (CassandraRelevantPropertiesTest). To verify the new patch we need to ensure > that all tests Cassandra pass and also to think about potential update of the > mentioned test class. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17797) All system properties and environment variables should be accessed via the new CassandraRelevantProperties and CassandraRelevantEnv classes
[ https://issues.apache.org/jira/browse/CASSANDRA-17797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-17797: Labels: ghc-lhf (was: ) > All system properties and environment variables should be accessed via the > new CassandraRelevantProperties and CassandraRelevantEnv classes > --- > > Key: CASSANDRA-17797 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17797 > Project: Cassandra > Issue Type: Improvement > Components: Local/Config >Reporter: Ekaterina Dimitrova >Priority: Low > Labels: ghc-lhf > Fix For: 4.x > > > Follow up ticket for CASSANDRA-15876 - > "Always access system properties and environment variables via the new > CassandraRelevantProperties and CassandraRelevantEnv classes" > As part of that ticket we moved to those classes only properties/variables > that were currently listed in System Properties Virtual Table. > We have to move to those classes the rest of the properties around the code > and start using those classes to access all of them. > +Additional information for newcomers:+ > You might want to start by getting acquainted with > CassandraRelevantProperties and CassandraRelevantEnv classes. Also, you might > want to check what changes were done and how the first batch was transferred > to this new framework as part of > [CASSANDRA-15876|https://github.com/apache/cassandra/commit/7694c1d191531ac152db55e83bc0db6864a5441e] > We are interested into the properties accessed currently through > getProperties around the code. > As part of CASSANDRA-15876 relevant unit tests were added > (CassandraRelevantPropertiesTest). To verify the new patch we need to ensure > that all tests Cassandra pass and also to think about potential update of the > mentioned test class. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17797) All system properties and environment variables should be accessed via the new CassandraRelevantProperties and CassandraRelevantEnv classes
Ekaterina Dimitrova created CASSANDRA-17797: --- Summary: All system properties and environment variables should be accessed via the new CassandraRelevantProperties and CassandraRelevantEnv classes Key: CASSANDRA-17797 URL: https://issues.apache.org/jira/browse/CASSANDRA-17797 Project: Cassandra Issue Type: Improvement Reporter: Ekaterina Dimitrova Follow up ticket for CASSANDRA-15876 - "Always access system properties and environment variables via the new CassandraRelevantProperties and CassandraRelevantEnv classes" As part of that ticket we moved to those classes only properties/variables that were currently listed in System Properties Virtual Table. We have to move to those classes the rest of the properties around the code and start using those classes to access all of them. +Additional information for newcomers:+ You might want to start by getting acquainted with CassandraRelevantProperties and CassandraRelevantEnv classes. Also, you might want to check what changes were done and how the first batch was transferred to this new framework as part of [CASSANDRA-15876|https://github.com/apache/cassandra/commit/7694c1d191531ac152db55e83bc0db6864a5441e] We are interested into the properties accessed currently through getProperties around the code. As part of CASSANDRA-15876 relevant unit tests were added (CassandraRelevantPropertiesTest). To verify the new patch we need to ensure that all tests Cassandra pass and also to think about potential update of the mentioned test class. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17103) CEP-15 (C*): Messaging and storage engine integration
[
https://issues.apache.org/jira/browse/CASSANDRA-17103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17574912#comment-17574912
]
David Capwell commented on CASSANDRA-17103:
---
Found out why the error below happens (reported earlier)
{code}
Caused by: java.lang.NullPointerException: null
at
org.apache.cassandra.service.accord.async.AsyncWriter.denormalize(AsyncWriter.java:216)
at
org.apache.cassandra.service.accord.async.AsyncWriter.lambda$denormalize$12(AsyncWriter.java:230)
at java.base/java.util.HashMap$Values.forEach(HashMap.java:977)
at
org.apache.cassandra.service.accord.async.AsyncWriter.denormalize(AsyncWriter.java:230)
at
org.apache.cassandra.service.accord.async.AsyncWriter.save(AsyncWriter.java:251)
at
org.apache.cassandra.service.accord.async.AsyncOperation.run(AsyncOperation.java:100)
at
org.apache.cassandra.service.accord.async.AsyncOperation.callback(AsyncOperation.java:77)
at
org.apache.cassandra.utils.concurrent.ListenerList$CallbackBiConsumerListener.run(ListenerList.java:247)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
... 1 common frames omitted
{code}
We have a NotWitnessed command!
{code}
Command{txnId=[1,1659556902041000,0,127107012], status=NotWitnessed,
txn=null, executeAt=null, deps=[]}
{code}
> CEP-15 (C*): Messaging and storage engine integration
> -
>
> Key: CASSANDRA-17103
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17103
> Project: Cassandra
> Issue Type: New Feature
> Components: Accord
>Reporter: Benedict Elliott Smith
>Priority: Normal
>
> This work encompasses implementing Accord’s storage and networking interfaces
> within Cassandra, so that messages may be sent around the cluster and
> exectuted
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17111) CEP-15: (Accord) Transaction dependencies must include at most the txnId
[
https://issues.apache.org/jira/browse/CASSANDRA-17111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated CASSANDRA-17111:
---
Labels: pull-request-available (was: )
> CEP-15: (Accord) Transaction dependencies must include at most the txnId
>
>
> Key: CASSANDRA-17111
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17111
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Benedict Elliott Smith
>Priority: Normal
> Labels: pull-request-available
>
> Today we must ship the entire transaction as part of the dependencies we
> share. This is very costly. We may ship only the {{txnId,primary shard key}}
> if we modify recovery to first discover the original transaction, or
> invalidate it if it cannot be found.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17679) Make resumable bootstrap feature optional
[
https://issues.apache.org/jira/browse/CASSANDRA-17679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17574904#comment-17574904
]
Jeremiah Jordan commented on CASSANDRA-17679:
-
Is there a reason we need a new option here? We already have the
"-Dcassandra.reset_bootstrap_progress" flag which disables resuming:
[https://github.com/apache/cassandra/blob/cassandra-3.11/src/java/org/apache/cassandra/service/StorageService.java#L1608]
Do we just need to make the already existing flag more discoverable?
> Make resumable bootstrap feature optional
> -
>
> Key: CASSANDRA-17679
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17679
> Project: Cassandra
> Issue Type: New Feature
> Components: Consistency/Streaming
>Reporter: Josh McKenzie
>Assignee: Josh McKenzie
>Priority: Normal
> Fix For: 4.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> From the patch I'm working on:
> {code}
> # In certain environments, operators may want to disable resumable bootstrap
> in order to avoid potential correctness
> # violations or data loss scenarios. Largely this centers around nodes going
> down during bootstrap, tombstones being
> # written, and potential races with repair. By default we leave this on as
> it's been enabled for quite some time,
> # however the option to disable it is more palatable now that we have zero
> copy streaming as that greatly accelerates
> # bootstraps. This defaults to true.
> # resumable_bootstrap_enabled: true
> {code}
> Not really a great fit for guardrails as it's less a "feature to be toggled
> on and off" and more a subset of a specific feature that in certain
> circumstances can lead to issues.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17679) Make resumable bootstrap feature optional
[
https://issues.apache.org/jira/browse/CASSANDRA-17679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17574904#comment-17574904
]
Jeremiah Jordan edited comment on CASSANDRA-17679 at 8/3/22 7:33 PM:
-
Is there a reason we need a new option here? We already have the
"-Dcassandra.reset_bootstrap_progress" flag which disables resuming:
[https://github.com/apache/cassandra/blob/cassandra-3.11/src/java/org/apache/cassandra/service/StorageService.java#L1608]
Do we just need to make the already existing flag more discoverable?
Also I would not mention ZCS as a reason to disable this. ZCS applies pretty
narrowly right now since it only works for whole files. So for the most part
only works if you are using LCS, which is not the default compaction strategy.
was (Author: jjordan):
Is there a reason we need a new option here? We already have the
"-Dcassandra.reset_bootstrap_progress" flag which disables resuming:
[https://github.com/apache/cassandra/blob/cassandra-3.11/src/java/org/apache/cassandra/service/StorageService.java#L1608]
Do we just need to make the already existing flag more discoverable?
> Make resumable bootstrap feature optional
> -
>
> Key: CASSANDRA-17679
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17679
> Project: Cassandra
> Issue Type: New Feature
> Components: Consistency/Streaming
>Reporter: Josh McKenzie
>Assignee: Josh McKenzie
>Priority: Normal
> Fix For: 4.x
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> From the patch I'm working on:
> {code}
> # In certain environments, operators may want to disable resumable bootstrap
> in order to avoid potential correctness
> # violations or data loss scenarios. Largely this centers around nodes going
> down during bootstrap, tombstones being
> # written, and potential races with repair. By default we leave this on as
> it's been enabled for quite some time,
> # however the option to disable it is more palatable now that we have zero
> copy streaming as that greatly accelerates
> # bootstraps. This defaults to true.
> # resumable_bootstrap_enabled: true
> {code}
> Not really a great fit for guardrails as it's less a "feature to be toggled
> on and off" and more a subset of a specific feature that in certain
> circumstances can lead to issues.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17738) Validate that JMX updates any new 4.1 properties (non-duration, non-data rate, non-data storage) in Config class and not local copies which can lead to inconsistenci
[ https://issues.apache.org/jira/browse/CASSANDRA-17738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-17738: Summary: Validate that JMX updates any new 4.1 properties (non-duration, non-data rate, non-data storage) in Config class and not local copies which can lead to inconsistencies between JMX and Settings Virtual Table after startup (was: Validate that JMX updates any new 4.1 properties (which are not moved to the new Config classes; non-duration, non-data rate, non-data storage) ) > Validate that JMX updates any new 4.1 properties (non-duration, non-data > rate, non-data storage) in Config class and not local copies which can lead > to inconsistencies between JMX and Settings Virtual Table after startup > > > Key: CASSANDRA-17738 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17738 > Project: Cassandra > Issue Type: Task > Components: Local/Config >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.1-beta, 4.1 > > > Check that any properties which are added in 4.1 (non-Duration, non-Data > Storage, non-Data Rate) are always updated by JMX in Config class and not > local copies.This will ensure update in Settings Virtual Table after startup > and no inconsistencies between JMX and Settings Virtual Table > I branched the configCompatibilityTest in 4.1 in order to get the lists of > properties in this > [commit|https://github.com/ekaterinadimitrova2/cassandra/commit/f2c02861abf0d6c34257d2fac827562437362137] > - the commit won't get into the codebase, just pasting so people know how > the lists were generated: > _columns_per_table_warn_threshold, partition_keys_in_select_fail_threshold, > group_by_enabled, cdc_block_writes, paxos_variant, > max_top_tombstone_partition_count, data_disk_usage_percentage_fail_threshold, > internode_timeout, compaction_tombstone_warning_threshold, > enable_uuid_sstable_identifiers, user_timestamps_enabled, > paxos_topology_repair_no_dc_checks, scripted_user_defined_functions_enabled, > denylist_max_keys_per_table, auth_cache_warming_enabled, > write_consistency_levels_disallowed, auth_write_consistency_level, > denylist_range_reads_enabled, roles_cache_active_update, > paxos_repair_parallelism, hint_window_persistent_enabled, > paxos_contention_max_wait, table_properties_ignored, > read_before_write_list_operations_enabled, > in_select_cartesian_product_warn_threshold, > paxos_topology_repair_strict_each_quorum, top_partitions_enabled, > secondary_indexes_per_table_fail_threshold, auto_snapshot_ttl, > read_consistency_levels_warned, secondary_indexes_enabled, > internode_error_reporting_exclusions, paxos_contention_min_delta, > default_keyspace_rf, write_consistency_levels_warned, > use_deterministic_table_id, sasi_indexes_enabled, > min_tracked_partition_tombstone_count, > minimum_replication_factor_fail_threshold, table_properties_disallowed, > memtable, keyspaces_warn_threshold, transient_replication_enabled, > user_defined_functions_enabled, user_defined_functions_threads_enabled, > native_transport_max_requests_per_second, > in_select_cartesian_product_fail_threshold, denylist_max_keys_total, > tables_fail_threshold, read_consistency_levels_disallowed, > fields_per_udt_warn_threshold, paxos_on_linearizability_violations, > page_size_warn_threshold, minimum_replication_factor_warn_threshold, > denylist_writes_enabled, traverse_auth_from_root, denylist_consistency_level, > table_properties_warned, materialized_views_enabled, > fields_per_udt_fail_threshold, denylist_reads_enabled, > permissions_cache_active_update, available_processors, > secondary_indexes_per_table_warn_threshold, tables_warn_threshold, > client_error_reporting_exclusions, items_per_collection_warn_threshold, > page_size_fail_threshold, paxos_contention_wait_randomizer, > partition_denylist_enabled, skip_paxos_repair_on_topology_change_keyspaces, > credentials_cache_active_update, failure_detector, > drop_compact_storage_enabled, uncompressed_tables_enabled, > materialized_views_per_table_fail_threshold, keyspaces_fail_threshold, > data_disk_usage_percentage_warn_threshold, repair_state_size, > paxos_contention_min_wait, auth_read_consistency_level, > items_per_collection_fail_threshold, paxos_state_purging, > materialized_views_per_table_warn_threshold, allow_filtering_enabled, > columns_per_table_fail_threshold, paxos_repair_enabled, startup_checks, > compact_tables_enabled, drop_truncate_table_enabled, > max_top_size_partition_count, native_transport_rat
[jira] [Updated] (CASSANDRA-17738) Validate that JMX updates any new 4.1 properties (which are not moved to the new Config classes; non-duration, non-data rate, non-data storage)
[ https://issues.apache.org/jira/browse/CASSANDRA-17738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-17738: Description: Check that any properties which are added in 4.1 (non-Duration, non-Data Storage, non-Data Rate) are always updated by JMX in Config class and not local copies.This will ensure update in Settings Virtual Table after startup and no inconsistencies between JMX and Settings Virtual Table I branched the configCompatibilityTest in 4.1 in order to get the lists of properties in this [commit|https://github.com/ekaterinadimitrova2/cassandra/commit/f2c02861abf0d6c34257d2fac827562437362137] - the commit won't get into the codebase, just pasting so people know how the lists were generated: _columns_per_table_warn_threshold, partition_keys_in_select_fail_threshold, group_by_enabled, cdc_block_writes, paxos_variant, max_top_tombstone_partition_count, data_disk_usage_percentage_fail_threshold, internode_timeout, compaction_tombstone_warning_threshold, enable_uuid_sstable_identifiers, user_timestamps_enabled, paxos_topology_repair_no_dc_checks, scripted_user_defined_functions_enabled, denylist_max_keys_per_table, auth_cache_warming_enabled, write_consistency_levels_disallowed, auth_write_consistency_level, denylist_range_reads_enabled, roles_cache_active_update, paxos_repair_parallelism, hint_window_persistent_enabled, paxos_contention_max_wait, table_properties_ignored, read_before_write_list_operations_enabled, in_select_cartesian_product_warn_threshold, paxos_topology_repair_strict_each_quorum, top_partitions_enabled, secondary_indexes_per_table_fail_threshold, auto_snapshot_ttl, read_consistency_levels_warned, secondary_indexes_enabled, internode_error_reporting_exclusions, paxos_contention_min_delta, default_keyspace_rf, write_consistency_levels_warned, use_deterministic_table_id, sasi_indexes_enabled, min_tracked_partition_tombstone_count, minimum_replication_factor_fail_threshold, table_properties_disallowed, memtable, keyspaces_warn_threshold, transient_replication_enabled, user_defined_functions_enabled, user_defined_functions_threads_enabled, native_transport_max_requests_per_second, in_select_cartesian_product_fail_threshold, denylist_max_keys_total, tables_fail_threshold, read_consistency_levels_disallowed, fields_per_udt_warn_threshold, paxos_on_linearizability_violations, page_size_warn_threshold, minimum_replication_factor_warn_threshold, denylist_writes_enabled, traverse_auth_from_root, denylist_consistency_level, table_properties_warned, materialized_views_enabled, fields_per_udt_fail_threshold, denylist_reads_enabled, permissions_cache_active_update, available_processors, secondary_indexes_per_table_warn_threshold, tables_warn_threshold, client_error_reporting_exclusions, items_per_collection_warn_threshold, page_size_fail_threshold, paxos_contention_wait_randomizer, partition_denylist_enabled, skip_paxos_repair_on_topology_change_keyspaces, credentials_cache_active_update, failure_detector, drop_compact_storage_enabled, uncompressed_tables_enabled, materialized_views_per_table_fail_threshold, keyspaces_fail_threshold, data_disk_usage_percentage_warn_threshold, repair_state_size, paxos_contention_min_wait, auth_read_consistency_level, items_per_collection_fail_threshold, paxos_state_purging, materialized_views_per_table_warn_threshold, allow_filtering_enabled, columns_per_table_fail_threshold, paxos_repair_enabled, startup_checks, compact_tables_enabled, drop_truncate_table_enabled, max_top_size_partition_count, native_transport_rate_limiting_enabled, use_statements_enabled, auto_hints_cleanup_enabled, skip_paxos_repair_on_topology_change, partition_keys_in_select_warn_threshold, read_thresholds_enabled_ was: Check that any properties which are added in 4.1 (non-Duration, non-Data Storage, non-Data Rate) are always updated by JMX and there are no inconsistent validations that might cover bugs. Validate proper update in Settings Virtual Table I branched the configCompatibilityTest in 4.1 in order to get the lists of properties in this [commit|https://github.com/ekaterinadimitrova2/cassandra/commit/f2c02861abf0d6c34257d2fac827562437362137] - the commit won't get into the codebase, just pasting so people know how the lists were generated: _columns_per_table_warn_threshold, partition_keys_in_select_fail_threshold, group_by_enabled, cdc_block_writes, paxos_variant, max_top_tombstone_partition_count, data_disk_usage_percentage_fail_threshold, internode_timeout, compaction_tombstone_warning_threshold, enable_uuid_sstable_identifiers, user_timestamps_enabled, paxos_topology_repair_no_dc_checks, scripted_user_defined_functions_enabled, denylist_max_keys_per_table, auth_cache_warming_enabled, write_consistency_levels_disallowed, auth_write_consistency_level, denylist_range_reads_enabled, roles_cache_
[jira] [Updated] (CASSANDRA-17736) Settings Virtual Table should display the values assigned to a property in the DatabaseDescriptor on startup and not null (as per the yaml)
[ https://issues.apache.org/jira/browse/CASSANDRA-17736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-17736: Description: There are a few properties that after startup do not show their assigned values as per the DatabaseDescriptor assignment but the cassandra.yaml value. They will not be also updated in the virtual table down the road in case they are updated through JMX, nodetool etc. *EDIT: This ticket should serve to check the properties that are not type Duration, Data Storage and Data Rate; also that are not new to 4.1.* I will post a list of who are those later today for convenience. We target all those in Config class (some advanced properties are not broadly advertised in cassandra.yaml intentionally). There is [Settings Virtual Table |https://cassandra.apache.org/doc/trunk/cassandra/new/virtualtables.html#settings-virtual-table] which is supposed to show the values for our config parameters at any time. Especially useful if any property was changed after startup through JMX/nodetool and it doesn't match anymore the value in cassandra.yaml. For this to be possible, we need to ensure that the parameters are always updated in the Config class. It was observed that some are not always updating in Config class, but after startup delegating to other internal variables. This is a bug and this task should review and address any new findings. Classes of interest - [SettingsTable|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/virtual/SettingsTable.java] where you can see how config parameters are listed; [Config|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/config/Config.java] class where our configuration parameters are defined. We need patches 4.0 and above. I suggest you start looking into 4.0 branch and then merge into higher branches. As you won't be checking the data storage, data rate and duration type parameters, there shouldn't be many conflicts on merge. We have a lot of parameters and I suggest you split the list into batches to check and produce patches where/if needed to make the work more incremental and easier to work on and review it. was: There are a few properties that after startup do not show their assigned values as per the DatabaseDescriptor assignment but the cassandra.yaml value. They will not be also updated in the virtual table down the road in case they are updated through JMX, nodetool etc. EDIT: This ticket should serve to check the properties that are not type Duration, Data Storage and Data Rate; also that are not new to 4.1. I will post a list of who are those later today for convenience. We target all those in Config class (some advanced properties are not broadly advertised in cassandra.yaml intentionally). There is [Settings Virtual Table |https://cassandra.apache.org/doc/trunk/cassandra/new/virtualtables.html#settings-virtual-table] which is supposed to show the values for our config parameters at any time. Especially useful if any property was changed after startup through JMX/nodetool and it doesn't match anymore the value in cassandra.yaml. For this to be possible, we need to ensure that the parameters are always updated in the Config class. It was observed that some are not always updating in Config class, but after startup delegating to other internal variables. This is a bug and this task should review and address any new findings. Classes of interest - [SettingsTable|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/virtual/SettingsTable.java] where you can see how config parameters are listed; [Config|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/config/Config.java] class where our configuration parameters are defined. We need patches 4.0 and above. I suggest you start looking into 4.0 branch and then merge into higher branches. As you won't be checking the data storage, data rate and duration type parameters, there shouldn't be many conflicts on merge. We have a lot of parameters and I suggest you split the list into batches to check and produce patches where/if needed to make the work more incremental and easier to work on and review it. > Settings Virtual Table should display the values assigned to a property in > the DatabaseDescriptor on startup and not null (as per the yaml) > --- > > Key: CASSANDRA-17736 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17736 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Ekaterina Dimitrova >Assignee: Atri Sharma >Priority: Normal >
[jira] [Updated] (CASSANDRA-17738) Validate that JMX updates any new 4.1 properties (which are not moved to the new Config classes; non-duration, non-data rate, non-data storage)
[ https://issues.apache.org/jira/browse/CASSANDRA-17738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-17738: Summary: Validate that JMX updates any new 4.1 properties (which are not moved to the new Config classes; non-duration, non-data rate, non-data storage) (was: Validate that JMX updates any new 4.1 properties (which are not moved to the new Config classes; non-duration, non-data rate, non-data storage)) > Validate that JMX updates any new 4.1 properties (which are not moved to the > new Config classes; non-duration, non-data rate, non-data storage) > > > Key: CASSANDRA-17738 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17738 > Project: Cassandra > Issue Type: Task > Components: Local/Config >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.1-beta, 4.1 > > > Check that any properties which are added in 4.1 (non-Duration, non-Data > Storage, non-Data Rate) are always updated by JMX and there are no > inconsistent validations that might cover bugs. Validate proper update in > Settings Virtual Table > I branched the configCompatibilityTest in 4.1 in order to get the lists of > properties in this > [commit|https://github.com/ekaterinadimitrova2/cassandra/commit/f2c02861abf0d6c34257d2fac827562437362137] > - the commit won't get into the codebase, just pasting so people know how > the lists were generated: > _columns_per_table_warn_threshold, partition_keys_in_select_fail_threshold, > group_by_enabled, cdc_block_writes, paxos_variant, > max_top_tombstone_partition_count, data_disk_usage_percentage_fail_threshold, > internode_timeout, compaction_tombstone_warning_threshold, > enable_uuid_sstable_identifiers, user_timestamps_enabled, > paxos_topology_repair_no_dc_checks, scripted_user_defined_functions_enabled, > denylist_max_keys_per_table, auth_cache_warming_enabled, > write_consistency_levels_disallowed, auth_write_consistency_level, > denylist_range_reads_enabled, roles_cache_active_update, > paxos_repair_parallelism, hint_window_persistent_enabled, > paxos_contention_max_wait, table_properties_ignored, > read_before_write_list_operations_enabled, > in_select_cartesian_product_warn_threshold, > paxos_topology_repair_strict_each_quorum, top_partitions_enabled, > secondary_indexes_per_table_fail_threshold, auto_snapshot_ttl, > read_consistency_levels_warned, secondary_indexes_enabled, > internode_error_reporting_exclusions, paxos_contention_min_delta, > default_keyspace_rf, write_consistency_levels_warned, > use_deterministic_table_id, sasi_indexes_enabled, > min_tracked_partition_tombstone_count, > minimum_replication_factor_fail_threshold, table_properties_disallowed, > memtable, keyspaces_warn_threshold, transient_replication_enabled, > user_defined_functions_enabled, user_defined_functions_threads_enabled, > native_transport_max_requests_per_second, > in_select_cartesian_product_fail_threshold, denylist_max_keys_total, > tables_fail_threshold, read_consistency_levels_disallowed, > fields_per_udt_warn_threshold, paxos_on_linearizability_violations, > page_size_warn_threshold, minimum_replication_factor_warn_threshold, > denylist_writes_enabled, traverse_auth_from_root, denylist_consistency_level, > table_properties_warned, materialized_views_enabled, > fields_per_udt_fail_threshold, denylist_reads_enabled, > permissions_cache_active_update, available_processors, > secondary_indexes_per_table_warn_threshold, tables_warn_threshold, > client_error_reporting_exclusions, items_per_collection_warn_threshold, > page_size_fail_threshold, paxos_contention_wait_randomizer, > partition_denylist_enabled, skip_paxos_repair_on_topology_change_keyspaces, > credentials_cache_active_update, failure_detector, > drop_compact_storage_enabled, uncompressed_tables_enabled, > materialized_views_per_table_fail_threshold, keyspaces_fail_threshold, > data_disk_usage_percentage_warn_threshold, repair_state_size, > paxos_contention_min_wait, auth_read_consistency_level, > items_per_collection_fail_threshold, paxos_state_purging, > materialized_views_per_table_warn_threshold, allow_filtering_enabled, > columns_per_table_fail_threshold, paxos_repair_enabled, startup_checks, > compact_tables_enabled, drop_truncate_table_enabled, > max_top_size_partition_count, native_transport_rate_limiting_enabled, > use_statements_enabled, auto_hints_cleanup_enabled, > skip_paxos_repair_on_topology_change, > partition_keys_in_select_warn_threshold, read_thresholds_enabled_ -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (CASSANDRA-17737) Validate that JMX updates in Config class any properties that were moved to the new config classes and not local copies
[ https://issues.apache.org/jira/browse/CASSANDRA-17737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-17737: Description: Check that any properties moved to the new types in 4.1 (Duration, Data Storage, Data Rate) in Config class are always updated by JMX (JMX might update local copies out of Config). This is directly connected to the correct update of the properties in Settings Virtual Table after startup. I branched the configCompatibilityTest in 4.1 in order to get the lists of properties in this [commit|https://github.com/ekaterinadimitrova2/cassandra/commit/f2c02861abf0d6c34257d2fac827562437362137] - the commit won't get into the codebase, just pasting so people know how the lists were generated: *DataRateSpec* - entire_sstable_stream_throughput_outbound, inter_dc_stream_throughput_outbound, entire_sstable_inter_dc_stream_throughput_outbound, compaction_throughput, stream_throughput_outbound - we have setGet* tests and those seem solid, no issues found. *DurationSpec* - gc_log_threshold, permissions_validity, denylist_refresh, request_timeout, hints_flush_period, read_request_timeout, index_summary_resize_interval, streaming_keep_alive_period, max_hint_window, roles_update_interval, user_defined_functions_fail_timeout, write_request_timeout, cdc_free_space_check_interval, roles_validity, internode_streaming_tcp_user_timeout, gc_warn_threshold, range_request_timeout, credentials_update_interval, truncate_request_timeout, cas_contention_timeout, periodic_commitlog_sync_lag_block, streaming_state_expires, repair_request_timeout, permissions_update_interval, dynamic_snitch_reset_interval, internode_tcp_connect_timeout, paxos_purge_grace_period, dynamic_snitch_update_interval, trace_type_query_ttl, denylist_initial_load_retry, commitlog_sync_period, native_transport_idle_timeout, credentials_validity, validation_preview_purge_head_start, repair_state_expires, internode_tcp_user_timeout, trace_type_repair_ttl, cache_load_timeout, commitlog_sync_group_window, slow_query_log_timeout, counter_write_request_timeout, user_defined_functions_warn_timeout *DataStorageSpec* - internode_application_send_queue_reserve_endpoint_capacity, cdc_total_space, networking_cache_size, commitlog_total_space, internode_application_send_queue_capacity, key_cache_size, memtable_heap_space, trickle_fsync_interval, max_hints_size_per_host, internode_application_receive_queue_reserve_endpoint_capacity, native_transport_max_frame_size, coordinator_read_size_warn_threshold , internode_application_receive_queue_reserve_global_capacity, internode_max_message_size, file_cache_size, local_read_size_fail_threshold, data_disk_usage_max_disk_size, memtable_offheap_space, coordinator_read_size_fail_threshold, counter_cache_size, prepared_statements_cache_size, batchlog_replay_throttle, row_index_read_size_fail_threshold, index_summary_capacity, repair_session_space, paxos_cache_size, collection_size_fail_threshold, internode_application_send_queue_reserve_global_capacity, column_index_size, native_transport_receive_queue_capacity, sstable_preemptive_open_interval, max_mutation_size, min_free_space_per_drive, batch_size_fail_threshold, hinted_handoff_throttle, row_index_read_size_warn_threshold, max_value_size, column_index_cache_size, compaction_large_partition_warning_threshold, max_hints_file_size, collection_size_warn_threshold, native_transport_max_request_data_in_flight, internode_socket_receive_buffer_size, internode_application_receive_queue_capacity, internode_socket_send_buffer_size, row_cache_size, min_tracked_partition_size, local_read_size_warn_threshold, commitlog_segment_size, batch_size_warn_threshold, streaming_state_size, native_transport_max_request_data_in_flight_per_ip NOTE: Some of those were checked/fixed in other tickets but I post the full lists for completeness --- *Fixes/commits based on this work:* 1) Fix Setting Virtual Table to update after startup config properties gc_log_threshold_in_ms, gc_warn_threshold_in_ms, conf.index_summary_capacity_in_mb, prepared_statements_cache_size_mb, key_cache_size_in_mb, counter_cache_size_in_mb - [4.0|https://github.com/apache/cassandra/commit/04901913d61f047e806bb4c2bbd530677958a919], [4.1|https://github.com/apache/cassandra/commit/222f66e093b1cf315319b27fd8d0990bc9a9c92c], [trunk|https://github.com/apache/cassandra/commit/0e3bdeab73ad89b7f48fd366b1b2723f5800e784] 2) Fix a null bug we identified in Converters and the disabled value for sstable_preemptive_open_interval_in_mb. In order to start using @Nullable annotation for Config properties, I had to change com.google.code.findbugs scope from provided to default compile one. Looking into its license [https://opensource.org/licenses/BSD-3-Clause] and reading here - [https://www.apache.org/legal/resolved.html] we agreed the change is
[jira] [Updated] (CASSANDRA-17737) Validate that JMX updates in Config class any properties that were moved to the new config classes and not local copies
[ https://issues.apache.org/jira/browse/CASSANDRA-17737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-17737: Summary: Validate that JMX updates in Config class any properties that were moved to the new config classes and not local copies (was: Validate that JMX updates properly any properties that were moved to the new config classes) > Validate that JMX updates in Config class any properties that were moved to > the new config classes and not local copies > --- > > Key: CASSANDRA-17737 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17737 > Project: Cassandra > Issue Type: Task > Components: Local/Config >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 4.0.x, 4.1-beta, 4.1.x, 4.x > > > Check that any properties moved to the new types in 4.1 (Duration, Data > Storage, Data Rate) are always updated by JMX and there are no inconsistent > validations that might cover bugs. Validate proper update in Settings Virtual > Table > I branched the configCompatibilityTest in 4.1 in order to get the lists of > properties in this > [commit|https://github.com/ekaterinadimitrova2/cassandra/commit/f2c02861abf0d6c34257d2fac827562437362137] > - the commit won't get into the codebase, just pasting so people know how > the lists were generated: > *DataRateSpec* - entire_sstable_stream_throughput_outbound, > inter_dc_stream_throughput_outbound, > entire_sstable_inter_dc_stream_throughput_outbound, compaction_throughput, > stream_throughput_outbound - we have setGet* tests and those seem solid, no > issues found. > *DurationSpec* - gc_log_threshold, permissions_validity, denylist_refresh, > request_timeout, hints_flush_period, read_request_timeout, > index_summary_resize_interval, streaming_keep_alive_period, max_hint_window, > roles_update_interval, user_defined_functions_fail_timeout, > write_request_timeout, cdc_free_space_check_interval, roles_validity, > internode_streaming_tcp_user_timeout, gc_warn_threshold, > range_request_timeout, credentials_update_interval, truncate_request_timeout, > cas_contention_timeout, periodic_commitlog_sync_lag_block, > streaming_state_expires, repair_request_timeout, permissions_update_interval, > dynamic_snitch_reset_interval, internode_tcp_connect_timeout, > paxos_purge_grace_period, dynamic_snitch_update_interval, > trace_type_query_ttl, denylist_initial_load_retry, commitlog_sync_period, > native_transport_idle_timeout, credentials_validity, > validation_preview_purge_head_start, repair_state_expires, > internode_tcp_user_timeout, trace_type_repair_ttl, cache_load_timeout, > commitlog_sync_group_window, slow_query_log_timeout, > counter_write_request_timeout, user_defined_functions_warn_timeout > *DataStorageSpec* - > internode_application_send_queue_reserve_endpoint_capacity, cdc_total_space, > networking_cache_size, commitlog_total_space, > internode_application_send_queue_capacity, key_cache_size, > memtable_heap_space, trickle_fsync_interval, max_hints_size_per_host, > internode_application_receive_queue_reserve_endpoint_capacity, > native_transport_max_frame_size, coordinator_read_size_warn_threshold , > internode_application_receive_queue_reserve_global_capacity, > internode_max_message_size, file_cache_size, local_read_size_fail_threshold, > data_disk_usage_max_disk_size, memtable_offheap_space, > coordinator_read_size_fail_threshold, counter_cache_size, > prepared_statements_cache_size, batchlog_replay_throttle, > row_index_read_size_fail_threshold, index_summary_capacity, > repair_session_space, paxos_cache_size, collection_size_fail_threshold, > internode_application_send_queue_reserve_global_capacity, column_index_size, > native_transport_receive_queue_capacity, sstable_preemptive_open_interval, > max_mutation_size, min_free_space_per_drive, batch_size_fail_threshold, > hinted_handoff_throttle, row_index_read_size_warn_threshold, max_value_size, > column_index_cache_size, compaction_large_partition_warning_threshold, > max_hints_file_size, collection_size_warn_threshold, > native_transport_max_request_data_in_flight, > internode_socket_receive_buffer_size, > internode_application_receive_queue_capacity, > internode_socket_send_buffer_size, row_cache_size, > min_tracked_partition_size, local_read_size_warn_threshold, > commitlog_segment_size, batch_size_warn_threshold, streaming_state_size, > native_transport_max_request_data_in_flight_per_ip > NOTE: Some of those were checked/fixed in other tickets but I post the full > lists for completeness > --- > *Fixes/commits based on this work:* > > 1) Fix Setting Virtual Table to update
[jira] [Updated] (CASSANDRA-17781) Secondary index building should not block compaction progress
[ https://issues.apache.org/jira/browse/CASSANDRA-17781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh McKenzie updated CASSANDRA-17781: -- Fix Version/s: 4.2 Source Control Link: https://gitbox.apache.org/repos/asf?p=cassandra.git;a=commit;h=652e49a8bb2f8849ce0ebb5161c30e3889c14608 Resolution: Fixed Status: Resolved (was: Ready to Commit) > Secondary index building should not block compaction progress > - > > Key: CASSANDRA-17781 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17781 > Project: Cassandra > Issue Type: Improvement > Components: Feature/2i Index >Reporter: Josh McKenzie >Assignee: Josh McKenzie >Priority: Normal > Fix For: 4.2 > > > While CASSANDRA-11218 should also address this same problem, that ticket's a > pretty big beautiful hammer and there's a fairly surgical modification we can > make to the 2i building process to mitigate 2i building's ability to block > compaction. > While 11218 approaches this from a "lots of compactions can starve 2i build > and things never finish", what I'd like to address here is the inverse that > building multiple secondary indices in parallel can occupy all available > compactors to a point where we don't have any left to do actual compactions. > Completing 2i build can take several hours and the backlog of compactions > from that time may result in too many uncompacted sstables left to catch up. > A fairly simple fix is to break 2i building out to its own threadpool w/a hot > proppable param you can use to change the # of threads for that build in the > extreme cases where the default of 2 is indufficient. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17781) Secondary index building should not block compaction progress
[ https://issues.apache.org/jira/browse/CASSANDRA-17781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh McKenzie updated CASSANDRA-17781: -- Status: Ready to Commit (was: Review In Progress) > Secondary index building should not block compaction progress > - > > Key: CASSANDRA-17781 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17781 > Project: Cassandra > Issue Type: Improvement > Components: Feature/2i Index >Reporter: Josh McKenzie >Assignee: Josh McKenzie >Priority: Normal > > While CASSANDRA-11218 should also address this same problem, that ticket's a > pretty big beautiful hammer and there's a fairly surgical modification we can > make to the 2i building process to mitigate 2i building's ability to block > compaction. > While 11218 approaches this from a "lots of compactions can starve 2i build > and things never finish", what I'd like to address here is the inverse that > building multiple secondary indices in parallel can occupy all available > compactors to a point where we don't have any left to do actual compactions. > Completing 2i build can take several hours and the backlog of compactions > from that time may result in too many uncompacted sstables left to catch up. > A fairly simple fix is to break 2i building out to its own threadpool w/a hot > proppable param you can use to change the # of threads for that build in the > extreme cases where the default of 2 is indufficient. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated: Add separate thread pool for Secondary Index building so it doesn't block compactions
This is an automated email from the ASF dual-hosted git repository.
jmckenzie pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new 652e49a8bb Add separate thread pool for Secondary Index building so it
doesn't block compactions
652e49a8bb is described below
commit 652e49a8bb2f8849ce0ebb5161c30e3889c14608
Author: Josh McKenzie
AuthorDate: Wed Jul 27 12:57:28 2022 -0400
Add separate thread pool for Secondary Index building so it doesn't block
compactions
Patch by Chris Lohfink; reviewed by Caleb Rackliffe, Josh McKenzie, Sam
Tunnicliffe, and Marcus Eriksson for CASSANDRA-17781
Co-authored-by: Chris Lohfink
Co-authored-by: Josh McKenzie
---
CHANGES.txt | 1 +
src/java/org/apache/cassandra/config/Config.java| 3 +++
.../apache/cassandra/config/DatabaseDescriptor.java | 5 +
.../cassandra/db/compaction/CompactionManager.java | 21 ++---
4 files changed, 27 insertions(+), 3 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 105522af5b..4dda88e314 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.2
+ * Add separate thread pool for Secondary Index building so it doesn't block
compactions (CASSANDRA-17781)
* Added JMX call to getSSTableCountPerTWCSBucket for TWCS (CASSANDRA-17774)
* When doing a host replacement, -Dcassandra.broadcast_interval_ms is used to
know when to check the ring but checks that the ring wasn't changed in
-Dcassandra.ring_delay_ms, changes to ring delay should not depend on when we
publish load stats (CASSANDRA-17776)
* When bootstrap fails, CassandraRoleManager may attempt to do read queries
that fail with "Cannot read from a bootstrapping node", and increments
unavailables counters (CASSANDRA-17754)
diff --git a/src/java/org/apache/cassandra/config/Config.java
b/src/java/org/apache/cassandra/config/Config.java
index a17c3591ce..f7eabff8de 100644
--- a/src/java/org/apache/cassandra/config/Config.java
+++ b/src/java/org/apache/cassandra/config/Config.java
@@ -327,6 +327,9 @@ public class Config
public volatile int concurrent_materialized_view_builders = 1;
public volatile int reject_repair_compaction_threshold = Integer.MAX_VALUE;
+// The number of executors to use for building secondary indexes
+public int concurrent_index_builders = 2;
+
/**
* @deprecated retry support removed on CASSANDRA-10992
*/
diff --git a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
index b60321e131..0af1ef808f 100644
--- a/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
+++ b/src/java/org/apache/cassandra/config/DatabaseDescriptor.java
@@ -2033,6 +2033,11 @@ public class DatabaseDescriptor
return conf.concurrent_validations;
}
+public static int getConcurrentIndexBuilders()
+{
+return conf.concurrent_index_builders;
+}
+
public static void setConcurrentValidations(int value)
{
value = value > 0 ? value : Integer.MAX_VALUE;
diff --git a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
index 925d900ada..49b999e4db 100644
--- a/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
+++ b/src/java/org/apache/cassandra/db/compaction/CompactionManager.java
@@ -129,7 +129,13 @@ public class CompactionManager implements
CompactionManagerMBean
private final CompactionExecutor cacheCleanupExecutor = new
CacheCleanupExecutor();
private final CompactionExecutor viewBuildExecutor = new
ViewBuildExecutor();
-private final CompactionMetrics metrics = new CompactionMetrics(executor,
validationExecutor, viewBuildExecutor);
+// We can't house 2i builds in SecondaryIndexManagement because it could
cause deadlocks with itself, and can cause
+// massive to indefinite pauses if prioritized either before or after
normal compactions so we instead put it in its
+// own pool to prevent either scenario.
+private final SecondaryIndexExecutor secondaryIndexExecutor = new
SecondaryIndexExecutor();
+
+private final CompactionMetrics metrics = new CompactionMetrics(executor,
validationExecutor, viewBuildExecutor, secondaryIndexExecutor);
+
@VisibleForTesting
final Multiset compactingCF =
ConcurrentHashMultiset.create();
@@ -244,6 +250,7 @@ public class CompactionManager implements
CompactionManagerMBean
validationExecutor.shutdown();
viewBuildExecutor.shutdown();
cacheCleanupExecutor.shutdown();
+secondaryIndexExecutor.shutdown();
// interrupt compactions and validations
for (Holder compactionHolder : active.getCompactions())
@@ -254,7 +261,8 @@ public class CompactionManager implements
CompactionManage
[jira] [Updated] (CASSANDRA-17792) Fix org.apache.cassandra.distributed.test.cdc.ToggleCDCOnRepairEnabledTest being flaky on current trunk
[
https://issues.apache.org/jira/browse/CASSANDRA-17792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh McKenzie updated CASSANDRA-17792:
--
Reviewers: Josh McKenzie
> Fix org.apache.cassandra.distributed.test.cdc.ToggleCDCOnRepairEnabledTest
> being flaky on current trunk
> ---
>
> Key: CASSANDRA-17792
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17792
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Assignee: Yifan Cai
>Priority: Normal
> Fix For: 4.x
>
>
> org.apache.cassandra.distributed.test.cdc.ToggleCDCOnRepairEnabledTest is a
> bit flaky on [trunk.
> As [this
> run|https://app.circleci.com/pipelines/github/ekaterinadimitrova2/cassandra?branch=17666]
> shows it was flaky since it was introduced a month ago as part of
> CASSANDRA-17666 but the flakiness is so low that if we don't run it in a loop
> it is hard to hit it.
> Both tests in the test class can fail with the same exception:
> {code:java}
> org.apache.cassandra.distributed.shared.ShutdownException: Uncaught
> exceptions were thrown during test at
> org.apache.cassandra.distributed.impl.AbstractCluster.checkAndResetUncaughtExceptions(AbstractCluster.java:1056)
> at
> org.apache.cassandra.distributed.impl.AbstractCluster.close(AbstractCluster.java:1042)
> at
> org.apache.cassandra.distributed.test.cdc.ToggleCDCOnRepairEnabledTest.testCDCOnRepairEnabled(ToggleCDCOnRepairEnabledTest.java:95)
> at
> org.apache.cassandra.distributed.test.cdc.ToggleCDCOnRepairEnabledTest.testCDCOnRepairIsEnabled(ToggleCDCOnRepairEnabledTest.java:40)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> Suppressed: java.lang.NullPointerException at
> org.apache.cassandra.db.commitlog.CommitLogSegmentManagerCDC$CDCSizeTracker.recalculateOverflowSize(CommitLogSegmentManagerCDC.java:390)
> at org.apache.cassandra.concurrent.FutureTask$1.call(FutureTask.java:81) at
> org.apache.cassandra.concurrent.FutureTask.call(FutureTask.java:47) at
> org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:57) at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:829){code}
> CC [~ycai] , [~jmckenzie]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
