[jira] [Updated] (CASSANDRA-18464) Enable Direct I/O For CommitLog Files
[ https://issues.apache.org/jira/browse/CASSANDRA-18464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxwell Guo updated CASSANDRA-18464: Reviewers: Branimir Lambov, Maxwell Guo (was: Branimir Lambov) > Enable Direct I/O For CommitLog Files > - > > Key: CASSANDRA-18464 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18464 > Project: Cassandra > Issue Type: New Feature > Components: Local/Commit Log >Reporter: Josh McKenzie >Assignee: Amit Pawar >Priority: Normal > Fix For: 5.0.x, 5.x > > Attachments: CommitLogStressTest.patch, > EnableDirectIOForCommitLogUsingNativeAPI.patch, > PeriodicCommitLogStressTest.tar.bz2, SetCommitLogFileSize.patch, > UseDirectIOFeatureForCommitLogFiles.patch, image-2023-06-29-01-12-49-382.png > > > Relocating from [dev@ email > thread.|https://lists.apache.org/thread/j6ny17q2rhkp7jxvwxm69dd6v1dozjrg] > > I shared my investigation about Commitlog I/O issue on large core count > system in my previous email dated July-22 and link to the thread is given > below. > [https://lists.apache.org/thread/xc5ocog2qz2v2gnj4xlw5hbthfqytx2n] > Basically, two solutions looked possible to improve the CommitLog I/O. > # Multi-threaded syncing > # Using Direct-IO through JNA > I worked on 2nd option considering the following benefit compared to the > first one > # Direct I/O read/write throughput is very high compared to non-Direct I/O. > Learnt through FIO benchmarking. > # Reduces kernel file cache uses which in-turn reduces kernel I/O activity > for Commitlog files only. > # Overall CPU usage reduced for flush activity. JVisualvm shows CPU usage < > 30% for Commitlog syncer thread with Direct I/O feature > # Direct I/O implementation is easier compared to multi-threaded > As per the community suggestion, less in code complex is good to have. Direct > I/O enablement looked promising but there was one issue. > Java version 8 does not have native support to enable Direct I/O. So, JNA > library usage is must. The same implementation should also work across other > versions of Java (like 11 and beyond). > I have completed Direct I/O implementation and summary of the attached patch > changes are given below. > # This implementation is not using Java file channels and file is opened > through JNA to use Direct I/O feature. > # New Segment are defined named “DirectIOSegment” for Direct I/O and > “NonDirectIOSegment” for non-direct I/O (NonDirectIOSegment is test purpose > only). > # JNA write call is used to flush the changes. > # New helper functions are defined in NativeLibrary.java and platform > specific file. Currently tested on Linux only. > # Patch allows user to configure optimum block size and alignment if > default values are not OK for CommitLog disk. > # Following configuration options are provided in Cassandra.yaml file > a. use_jna_for_commitlog_io : to use jna feature > b. use_direct_io_for_commitlog : to use Direct I/O feature. > c. direct_io_minimum_block_alignment: 512 (default) > d. nvme_disk_block_size: 32MiB (default and can be changed as per the > required size) > Test matrix is complex so CommitLog related testcases and TPCx-IOT benchmark > was tested. It works with both Java 8 and 11 versions. Compressed and > Encrypted based segments are not supported yet and it can be enabled later > based on the Community feedback. > Following improvement are seen with Direct I/O enablement. > # 32 cores >= ~15% > # 64 cores >= ~80% > Also, another observation would like to share here. Reading Commitlog files > with Direct I/O might help in reducing node bring-up time after the node > crash. > Tested with commit ID: 91f6a9aca8d3c22a03e68aa901a0b154d960ab07 > The attached patch enables Direct I/O feature for Commitlog files. Please > check and share your feedback. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18920) Add jsonl support to sstabledump
[ https://issues.apache.org/jira/browse/CASSANDRA-18920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773901#comment-17773901 ] Maxwell Guo commented on CASSANDRA-18920: - yeah, we know that sstabledump already has a option -l to output the partition in json, the difference between original json output and your jsonl is on "removes whitespace, including newlines, to make the format compact." but the original json is still a json format. So I prefer to add arguments behind -l which like -k [here|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/tools/SSTableExport.java#L84] to show wether is jsonl or original json. [~brandon.williams] [~stefan.miklosovic] wdyt ? > Add jsonl support to sstabledump > > > Key: CASSANDRA-18920 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18920 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Tools >Reporter: Geoff Lay >Assignee: Geoff Lay >Priority: Low > Fix For: 5.x > > > Add a new option to sstabledump to allow the generation of jsonl format. > The -l option for sstabledump does not produce valid jsonl. > [https://jsonlines.org/validator/] > Valid jsonl removes whitespace, including newlines, to make the format > compact. > By adding an option to generate valid jsonl, jsonl can be generated that is > smaller. > Smaller output is better for execution time, storage, and transport. > jsonl also allows the splitting of a file for processing at a known good line > for processing without parsing the file, as each row is a line. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18920) Add jsonl support to sstabledump
[ https://issues.apache.org/jira/browse/CASSANDRA-18920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773899#comment-17773899 ] Geoff Lay commented on CASSANDRA-18920: --- My original intention was to add a new option to prevent breaking a user's pre-existing workflow. For instance: -n Output each row as a jsonline, a compact whitespace free JSON format. However, I am open to implementing it differently. > Add jsonl support to sstabledump > > > Key: CASSANDRA-18920 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18920 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Tools >Reporter: Geoff Lay >Assignee: Geoff Lay >Priority: Low > Fix For: 5.x > > > Add a new option to sstabledump to allow the generation of jsonl format. > The -l option for sstabledump does not produce valid jsonl. > [https://jsonlines.org/validator/] > Valid jsonl removes whitespace, including newlines, to make the format > compact. > By adding an option to generate valid jsonl, jsonl can be generated that is > smaller. > Smaller output is better for execution time, storage, and transport. > jsonl also allows the splitting of a file for processing at a known good line > for processing without parsing the file, as each row is a line. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch asf-staging updated (9fada0ef -> 83d089e1)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git omit 9fada0ef generate docs for db70fb96 new 83d089e1 generate docs for db70fb96 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (9fada0ef) \ N -- N -- N refs/heads/asf-staging (83d089e1) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../developing/cql/dynamic_data_masking.html | 11 --- .../5.0/cassandra/developing/cql/functions.html| 9 +++-- .../developing/cql/dynamic_data_masking.html | 11 --- .../5.1/cassandra/developing/cql/functions.html| 9 +++-- .../developing/cql/dynamic_data_masking.html | 11 --- .../trunk/cassandra/developing/cql/functions.html | 9 +++-- content/search-index.js| 2 +- site-ui/build/ui-bundle.zip| Bin 4881412 -> 4881412 bytes 8 files changed, 46 insertions(+), 16 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14404) Transient Replication & Cheap Quorums: Decouple storage requirements from consensus group size using incremental repair
[ https://issues.apache.org/jira/browse/CASSANDRA-14404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773888#comment-17773888 ] sean commented on CASSANDRA-14404: -- In Cassandra 4.0.X, We have a table with 2 materialized views. When a Cassandra node is down. Some write operations may fail. It works fine in Cassandra 3.11.x. May be related to this change. client exception： com.datastax.oss.driver.api.core.servererrors.WriteFailureException: Cassandra failure during write query at consistency LOCAL_QUORUM (2 responses were required but only 1 replica responded, 2 failed) at com.datastax.oss.driver.api.core.servererrors.WriteFailureException.copy(WriteFailureException.java:142) at com.datastax.oss.driver.internal.core.util.concurrent.CompletableFutures.getUninterruptibly(CompletableFutures.java:149) at com.datastax.oss.driver.internal.core.cql.CqlRequestSyncProcessor.process(CqlRequestSyncProcessor.java:53) at com.datastax.oss.driver.internal.core.cql.CqlRequestSyncProcessor.process(CqlRequestSyncProcessor.java:30) at com.datastax.oss.driver.internal.core.session.DefaultSession.execute(DefaultSession.java:230) at com.datastax.oss.driver.api.core.cql.SyncCqlSession.execute(SyncCqlSession.java:54) Cassandra： [MutationStage-11] [org.apache.cassandra.db.Keyspace] Unknown exception caught while attempting to update MaterializedView! cycling.cycling_task org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level ONE at org.apache.cassandra.exceptions.UnavailableException.create(UnavailableException.java:37) at org.apache.cassandra.locator.ReplicaPlans.assureSufficientLiveReplicas(ReplicaPlans.java:170) at org.apache.cassandra.locator.ReplicaPlans.assureSufficientLiveReplicasForWrite(ReplicaPlans.java:113) at org.apache.cassandra.locator.ReplicaPlans.forWrite(ReplicaPlans.java:354) at org.apache.cassandra.locator.ReplicaPlans.forWrite(ReplicaPlans.java:345) at org.apache.cassandra.locator.ReplicaPlans.forWrite(ReplicaPlans.java:339) at org.apache.cassandra.service.StorageProxy.wrapViewBatchResponseHandler(StorageProxy.java:1312) at org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:1004) at org.apache.cassandra.db.view.TableViews.pushViewReplicaUpdates(TableViews.java:167) at org.apache.cassandra.db.Keyspace.applyInternal(Keyspace.java:647) at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:513) at org.apache.cassandra.db.Mutation.apply(Mutation.java:215) at org.apache.cassandra.db.Mutation.apply(Mutation.java:220) at org.apache.cassandra.db.Mutation.apply(Mutation.java:229) at org.apache.cassandra.service.StorageProxy$4.runMayThrow(StorageProxy.java:1537) at org.apache.cassandra.service.StorageProxy$LocalMutationRunnable.run(StorageProxy.java:2326) > Transient Replication & Cheap Quorums: Decouple storage requirements from > consensus group size using incremental repair > --- > > Key: CASSANDRA-14404 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14404 > Project: Cassandra > Issue Type: New Feature > Components: Consistency/Hints, Consistency/Repair, Feature/2i Index, > Feature/Materialized Views, Legacy/Coordination, Legacy/Core, Legacy/CQL, > Legacy/Distributed Metadata, Legacy/Local Write-Read Paths, Legacy/Testing, > Legacy/Tools >Reporter: Ariel Weisberg >Assignee: Ariel Weisberg >Priority: Normal > Fix For: 5.x > > > Transient Replication is an implementation of [Witness > Replicas|http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.146.3429=rep1=pdf] > that leverages incremental repair to make full replicas consistent with > transient replicas that don't store the entire data set. Witness replicas are > used in real world systems such as Megastore and Spanner to increase > availability inexpensively without having to commit to more full copies of > the database. Transient replicas implement functionality similar to > upgradable and temporary replicas from the paper. > With transient replication the replication factor is increased beyond the > desired level of data redundancy by adding replicas that only store data when > sufficient full replicas are unavailable to store the data. These replicas > are called transient replicas. When incremental repair runs transient > replicas stream any data they have received to full replicas and once the > data is fully replicated it is dropped at the transient replicas. > Cheap quorums are a further set of optimizations on the write path to avoid > writing to transient replicas
[jira] [Comment Edited] (CASSANDRA-18920) Add jsonl support to sstabledump
[ https://issues.apache.org/jira/browse/CASSANDRA-18920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773887#comment-17773887 ] Maxwell Guo edited comment on CASSANDRA-18920 at 10/11/23 3:55 AM: --- Hello, [~lapsuscalami], I want to know wether this ticket is to replace the original -l option to support the output of josnl per partition, right ? if so, I think we can just add a flag which is false by default. if the flag is false , the output is the original output , if true , the output is what you describe above. In this way I think we can keep forward compatible was (Author: maxwellguo): Hello, [~lapsuscalami], I want to know wether this ticket is to replace the original -l option to support the output of josnl per partition, right ? if so, I think we can just add a flag (true / false ) when using the original -l option ,if false , the output is the original output , if true , the output is what you describe above. > Add jsonl support to sstabledump > > > Key: CASSANDRA-18920 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18920 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Tools >Reporter: Geoff Lay >Assignee: Geoff Lay >Priority: Low > Fix For: 5.x > > > Add a new option to sstabledump to allow the generation of jsonl format. > The -l option for sstabledump does not produce valid jsonl. > [https://jsonlines.org/validator/] > Valid jsonl removes whitespace, including newlines, to make the format > compact. > By adding an option to generate valid jsonl, jsonl can be generated that is > smaller. > Smaller output is better for execution time, storage, and transport. > jsonl also allows the splitting of a file for processing at a known good line > for processing without parsing the file, as each row is a line. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18920) Add jsonl support to sstabledump
[ https://issues.apache.org/jira/browse/CASSANDRA-18920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxwell Guo updated CASSANDRA-18920: Change Category: Operability Complexity: Low Hanging Fruit Fix Version/s: 5.x Priority: Low (was: Normal) Status: Open (was: Triage Needed) > Add jsonl support to sstabledump > > > Key: CASSANDRA-18920 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18920 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Tools >Reporter: Geoff Lay >Assignee: Geoff Lay >Priority: Low > Fix For: 5.x > > > Add a new option to sstabledump to allow the generation of jsonl format. > The -l option for sstabledump does not produce valid jsonl. > [https://jsonlines.org/validator/] > Valid jsonl removes whitespace, including newlines, to make the format > compact. > By adding an option to generate valid jsonl, jsonl can be generated that is > smaller. > Smaller output is better for execution time, storage, and transport. > jsonl also allows the splitting of a file for processing at a known good line > for processing without parsing the file, as each row is a line. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18920) Add jsonl support to sstabledump
[ https://issues.apache.org/jira/browse/CASSANDRA-18920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773887#comment-17773887 ] Maxwell Guo commented on CASSANDRA-18920: - Hello, [~lapsuscalami], I want to know wether this ticket is to replace the original -l option to support the output of josnl per partition, right ? if so, I think we can just add a flag (true / false ) when using the original -l option ,if false , the output is the original output , if true , the output is what you describe above. > Add jsonl support to sstabledump > > > Key: CASSANDRA-18920 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18920 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Tools >Reporter: Geoff Lay >Assignee: Geoff Lay >Priority: Normal > > Add a new option to sstabledump to allow the generation of jsonl format. > The -l option for sstabledump does not produce valid jsonl. > [https://jsonlines.org/validator/] > Valid jsonl removes whitespace, including newlines, to make the format > compact. > By adding an option to generate valid jsonl, jsonl can be generated that is > smaller. > Smaller output is better for execution time, storage, and transport. > jsonl also allows the splitting of a file for processing at a known good line > for processing without parsing the file, as each row is a line. > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18798) Appending to list in Accord transactions uses insertion timestamp
[
https://issues.apache.org/jira/browse/CASSANDRA-18798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773851#comment-17773851
]
Henrik Ingo commented on CASSANDRA-18798:
-
After pulling the most recent cep-15-accord branch, it seems this issue is
fixed:
https://github.com/apache/cassandra/blob/cep-15-accord/src/java/org/apache/cassandra/service/accord/AccordKeyspace.java#L361-L363
I'll rerun Jaroslaw's original consistency test tomorrow to verify.
> Appending to list in Accord transactions uses insertion timestamp
> -
>
> Key: CASSANDRA-18798
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18798
> Project: Cassandra
> Issue Type: Bug
> Components: Accord
>Reporter: Jaroslaw Kijanowski
>Assignee: Henrik Ingo
>Priority: Normal
> Attachments: image-2023-09-26-20-05-25-846.png
>
>
> Given the following schema:
> {code:java}
> CREATE KEYSPACE IF NOT EXISTS accord WITH replication = {'class':
> 'SimpleStrategy', 'replication_factor': 3};
> CREATE TABLE IF NOT EXISTS accord.list_append(id int PRIMARY KEY,contents
> LIST);
> TRUNCATE accord.list_append;{code}
> And the following two possible queries executed by 10 threads in parallel:
> {code:java}
> BEGIN TRANSACTION
> LET row = (SELECT * FROM list_append WHERE id = ?);
> SELECT row.contents;
> COMMIT TRANSACTION;"
> BEGIN TRANSACTION
> UPDATE list_append SET contents += ? WHERE id = ?;
> COMMIT TRANSACTION;"
> {code}
> there seems to be an issue with transaction guarantees. Here's an excerpt in
> the edn format from a test.
> {code:java}
> {:type :invoke :process 8 :value [[:append 5 352]] :tid 3 :n 52
> :time 1692607285967116627}
> {:type :invoke :process 9 :value [[:r 5 nil]] :tid 1 :n 54
> :time 1692607286078732473}
> {:type :invoke :process 6 :value [[:append 5 553]] :tid 5 :n 53
> :time 1692607286133833428}
> {:type :invoke :process 7 :value [[:append 5 455]] :tid 4 :n 55
> :time 1692607286149702511}
> {:type :ok :process 8 :value [[:append 5 352]] :tid 3 :n 52
> :time 1692607286156314099}
> {:type :invoke :process 5 :value [[:r 5 nil]] :tid 9 :n 52
> :time 1692607286167090389}
> {:type :ok :process 9 :value [[:r 5 [303 304 604 6 306 509 909 409 912
> 411 514 415 719 419 19 623 22 425 24 926 25 832 130 733 430 533 29 933 333
> 537 934 538 740 139 744 938 544 42 646 749 242 546 547 548 753 450 150 349 48
> 852 352]]] :tid 1 :n 54 :time 1692607286168657534}
> {:type :invoke :process 1 :value [[:r 5 nil]] :tid 0 :n 51
> :time 1692607286201762938}
> {:type :ok :process 7 :value [[:append 5 455]] :tid 4 :n 55
> :time 1692607286245571513}
> {:type :invoke :process 7 :value [[:r 5 nil]] :tid 4 :n 56
> :time 1692607286245655775}
> {:type :ok :process 5 :value [[:r 5 [303 304 604 6 306 509 909 409 912
> 411 514 415 719 419 19 623 22 425 24 926 25 832 130 733 430 533 29 933 333
> 537 934 538 740 139 744 938 544 42 646 749 242 546 547 548 753 450 150 349 48
> 852 352 455]]] :tid 9 :n 52 :time 1692607286253928906}
> {:type :invoke :process 5 :value [[:r 5 nil]] :tid 9 :n 53
> :time 1692607286254095215}
> {:type :ok :process 6 :value [[:append 5 553]] :tid 5 :n 53
> :time 1692607286266263422}
> {:type :ok :process 1 :value [[:r 5 [303 304 604 6 306 509 909 409 912
> 411 514 415 719 419 19 623 22 425 24 926 25 832 130 733 430 533 29 933 333
> 537 934 538 740 139 744 938 544 42 646 749 242 546 547 548 753 450 150 349 48
> 852 352 553 455]]] :tid 0 :n 51 :time 1692607286271617955}
> {:type :ok :process 7 :value [[:r 5 [303 304 604 6 306 509 909 409 912
> 411 514 415 719 419 19 623 22 425 24 926 25 832 130 733 430 533 29 933 333
> 537 934 538 740 139 744 938 544 42 646 749 242 546 547 548 753 450 150 349 48
> 852 352 553 455]]] :tid 4 :n 56 :time 1692607286271816933}
> {:type :ok :process 5 :value [[:r 5 [303 304 604 6 306 509 909 409 912
> 411 514 415 719 419 19 623 22 425 24 926 25 832 130 733 430 533 29 933 333
> 537 934 538 740 139 744 938 544 42 646 749 242 546 547 548 753 450 150 349 48
> 852 352 553 455]]] :tid 9 :n 53 :time 1692607286281483026}
> {:type :invoke :process 9 :value [[:r 5 nil]] :tid 1 :n 56
> :time 1692607286284097561}
> {:type :ok :process 9 :value [[:r 5 [303 304 604 6 306 509 909 409 912
> 411 514 415 719 419 19 623 22 425 24 926 25 832 130 733 430 533 29 933 333
> 537 934 538 740 139 744 938 544 42 646 749 242 546 547 548 753 450 150 349 48
> 852 352 553 455]]] :tid 1 :n 56 :time 1692607286306445242}
> {code}
> Processes process 6 and process 7 are appending the values 553 and 455
>
Re: [PR] [CASSANDRA-18916] Modified Start-Up Validation to Log a Single Report [cassandra-analytics]
5 commented on PR #16: URL: https://github.com/apache/cassandra-analytics/pull/16#issuecomment-1756468193 > Looks good overall. Can you share the a few example report output? Sure! Here's how it looks when the validations are skipped using the `SKIP_STARTUP_VALIDATIONS` system property: ``` ... INFO [Test worker] 2023-10-10 18:39:24,224 StartupValidator.java:97 - Skipped startup validations ... ``` And here's a report logged when the validations do run (individual failure stack traces are logged before as they happen): ``` ... ERROR [Test worker] 2023-10-10 18:33:03,376 StartupValidator.java:101 - Performed startup validations: * org.apache.cassandra.spark.validation.SslValidation: PASSED * org.apache.cassandra.spark.validation.KeyStoreValidation: FAILED * org.apache.cassandra.spark.validation.TrustStoreValidation: PASSED * org.apache.cassandra.spark.validation.SidecarValidation: FAILED * org.apache.cassandra.spark.validation.CassandraValidation: FAILED ERROR [Test worker] 2023-10-10 18:33:03,378 IntegrationTestJob.java:126 - Unexpected exception executing Spark job ... ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18920) Add jsonl support to sstabledump
Geoff Lay created CASSANDRA-18920: - Summary: Add jsonl support to sstabledump Key: CASSANDRA-18920 URL: https://issues.apache.org/jira/browse/CASSANDRA-18920 Project: Cassandra Issue Type: New Feature Components: Legacy/Tools Reporter: Geoff Lay Assignee: Geoff Lay Add a new option to sstabledump to allow the generation of jsonl format. The -l option for sstabledump does not produce valid jsonl. [https://jsonlines.org/validator/] Valid jsonl removes whitespace, including newlines, to make the format compact. By adding an option to generate valid jsonl, jsonl can be generated that is smaller. Smaller output is better for execution time, storage, and transport. jsonl also allows the splitting of a file for processing at a known good line for processing without parsing the file, as each row is a line. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] [CASSANDRA-18916] Modified Start-Up Validation to Log a Single Report [cassandra-analytics]
yifan-c commented on PR #16: URL: https://github.com/apache/cassandra-analytics/pull/16#issuecomment-1756395236 Looks good overall. Can you share the a few example report output? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18919) Test failure: org.apache.cassandra.io.sstable.VerifyTest.testMutateRepair-.jdk11
[
https://issues.apache.org/jira/browse/CASSANDRA-18919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-18919:
Bug Category: Parent values: Correctness(12982)Level 1 values: Test
Failure(12990)
Complexity: Normal
Component/s: Test/unit
Discovered By: User Report
Severity: Normal
Status: Open (was: Triage Needed)
> Test failure:
> org.apache.cassandra.io.sstable.VerifyTest.testMutateRepair-.jdk11
>
>
> Key: CASSANDRA-18919
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18919
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 5.0.x
>
>
>
> {code:java}
> org.apache.cassandra.io.sstable.VerifyTest.testMutateRepair-.jdk11 (from
> org.apache.cassandra.io.sstable.VerifyTest-.jdk11)
> Failing for the past 1 build (Since #60 ) Took 0.42 sec. Failed 1 times
> in the last 16 runs. Flakiness: 6%, Stability: 93% Stacktrace
> java.io.EOFException at
> org.apache.cassandra.io.util.RebufferingInputStream.readByte(RebufferingInputStream.java:180)
> at
> org.apache.cassandra.io.util.RebufferingInputStream.readPrimitiveSlowly(RebufferingInputStream.java:142)
> at
> org.apache.cassandra.io.util.RebufferingInputStream.readLong(RebufferingInputStream.java:231)
> at
> org.apache.cassandra.io.sstable.VerifyTest.testMutateRepair(VerifyTest.java:538)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> Seen here:
> https://ci-cassandra.apache.org/job/Cassandra-5.0/60/testReport/org.apache.cassandra.io.sstable/VerifyTest/testMutateRepair__jdk11/
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18919) Test failure: org.apache.cassandra.io.sstable.VerifyTest.testMutateRepair-.jdk11
Ekaterina Dimitrova created CASSANDRA-18919:
---
Summary: Test failure:
org.apache.cassandra.io.sstable.VerifyTest.testMutateRepair-.jdk11
Key: CASSANDRA-18919
URL: https://issues.apache.org/jira/browse/CASSANDRA-18919
Project: Cassandra
Issue Type: Bug
Reporter: Ekaterina Dimitrova
{code:java}
org.apache.cassandra.io.sstable.VerifyTest.testMutateRepair-.jdk11 (from
org.apache.cassandra.io.sstable.VerifyTest-.jdk11)
Failing for the past 1 build (Since #60 ) Took 0.42 sec. Failed 1 times in
the last 16 runs. Flakiness: 6%, Stability: 93% Stacktrace
java.io.EOFException at
org.apache.cassandra.io.util.RebufferingInputStream.readByte(RebufferingInputStream.java:180)
at
org.apache.cassandra.io.util.RebufferingInputStream.readPrimitiveSlowly(RebufferingInputStream.java:142)
at
org.apache.cassandra.io.util.RebufferingInputStream.readLong(RebufferingInputStream.java:231)
at
org.apache.cassandra.io.sstable.VerifyTest.testMutateRepair(VerifyTest.java:538)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method) at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
{code}
Seen here:
https://ci-cassandra.apache.org/job/Cassandra-5.0/60/testReport/org.apache.cassandra.io.sstable/VerifyTest/testMutateRepair__jdk11/
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18919) Test failure: org.apache.cassandra.io.sstable.VerifyTest.testMutateRepair-.jdk11
[
https://issues.apache.org/jira/browse/CASSANDRA-18919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ekaterina Dimitrova updated CASSANDRA-18919:
Fix Version/s: 5.0.x
> Test failure:
> org.apache.cassandra.io.sstable.VerifyTest.testMutateRepair-.jdk11
>
>
> Key: CASSANDRA-18919
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18919
> Project: Cassandra
> Issue Type: Bug
>Reporter: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 5.0.x
>
>
>
> {code:java}
> org.apache.cassandra.io.sstable.VerifyTest.testMutateRepair-.jdk11 (from
> org.apache.cassandra.io.sstable.VerifyTest-.jdk11)
> Failing for the past 1 build (Since #60 ) Took 0.42 sec. Failed 1 times
> in the last 16 runs. Flakiness: 6%, Stability: 93% Stacktrace
> java.io.EOFException at
> org.apache.cassandra.io.util.RebufferingInputStream.readByte(RebufferingInputStream.java:180)
> at
> org.apache.cassandra.io.util.RebufferingInputStream.readPrimitiveSlowly(RebufferingInputStream.java:142)
> at
> org.apache.cassandra.io.util.RebufferingInputStream.readLong(RebufferingInputStream.java:231)
> at
> org.apache.cassandra.io.sstable.VerifyTest.testMutateRepair(VerifyTest.java:538)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
> Seen here:
> https://ci-cassandra.apache.org/job/Cassandra-5.0/60/testReport/org.apache.cassandra.io.sstable/VerifyTest/testMutateRepair__jdk11/
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18913) Gossip NPE due to shutdown event corrupting empty statuses
[
https://issues.apache.org/jira/browse/CASSANDRA-18913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Capwell updated CASSANDRA-18913:
--
Fix Version/s: 4.0.12
4.1.4
5.0-alpha2
(was: 5.x)
(was: 4.0.x)
(was: 4.1.x)
(was: 5.0.x)
Since Version: 3.0.0
Source Control Link:
https://github.com/apache/cassandra/commit/2bab3f27ba1535203d61497abe6810cdcb4640d0
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> Gossip NPE due to shutdown event corrupting empty statuses
> --
>
> Key: CASSANDRA-18913
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18913
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip, Cluster/Membership
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0.12, 4.1.4, 5.0-alpha2
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> When an instance either disables gossip or shuts down we send a gossip
> shutdown message, peers ignore it if the endpoint isn’t known, else it
> mutates its local copy of the state to mark shutdown…
> When an instance restarts it populates gossip with the endpoints found in
> peers, but the state is empty (not null)
> So, there is a fun timing bug…
> * stop node1
> * start node1; at this point all known endpoints before exist in gossip but
> are empty
> * node2 shutdown (gossip shutdown or node, doesn’t matter)
> * node1 sees the shutdown before gossip messages, and gets corruptted
> * node3 tries to join the cluster, fails due to node1 being corrupted
> There are 2 different patterns the NPE can happen with, in this example node1
> and node3 will have different stack traces
> {code}
> org.apache.cassandra.distributed.shared.ShutdownException: Uncaught
> exceptions were thrown during test
> Suppressed: java.lang.NullPointerException: Unable to get HOST_ID;
> HOST_ID is not defined, given EndpointState: HeartBeatState = HeartBeat:
> generation = 0, version = 2147483647, AppStateMap =
> {STATUS=Value(shutdown,true,37), RPC_READY=Value(false,38),
> STATUS_WITH_PORT=Value(shutdown,true,36)}
> at
> org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1218)
> at
> org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1208)
> at
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:3279)
> at
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:2756)
> at
> org.apache.cassandra.gms.Gossiper.markAsShutdown(Gossiper.java:611)
> at
> org.apache.cassandra.gms.GossipShutdownVerbHandler.doVerb(GossipShutdownVerbHandler.java:39)
> at
> org.apache.cassandra.net.InboundSink.lambda$new$0(InboundSink.java:78)
> Suppressed: java.lang.NullPointerException: Unable to get HOST_ID;
> HOST_ID is not defined, given EndpointState: HeartBeatState = HeartBeat:
> generation = 0, version = 2147483647, AppStateMap =
> {STATUS=Value(shutdown,true,37), RPC_READY=Value(false,38),
> STATUS_WITH_PORT=Value(shutdown,true,36)}
> at
> org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1218)
> at
> org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1208)
> at
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:3279)
> at
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:2756)
> at
> org.apache.cassandra.gms.Gossiper.doOnChangeNotifications(Gossiper.java:1762)
> at
> org.apache.cassandra.service.StorageService.onJoin(StorageService.java:3793)
> at
> org.apache.cassandra.gms.Gossiper.handleMajorStateChange(Gossiper.java:1465)
> at
> org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:1678)
> at
> org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:50)
> at
> org.apache.cassandra.net.InboundSink.lambda$new$0(InboundSink.java:78)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-5.0' into trunk
This is an automated email from the ASF dual-hosted git repository. dcapwell pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit b8b17dc5db7b112ab2b7fe1308c2089cdc38eb1d Merge: 31729b8f92 210ced3b7b Author: David Capwell AuthorDate: Tue Oct 10 14:19:38 2023 -0700 Merge branch 'cassandra-5.0' into trunk CHANGES.txt| 1 + .../org/apache/cassandra/gms/EndpointState.java| 5 + .../org/apache/cassandra/gms/GossipShutdown.java | 66 + .../cassandra/gms/GossipShutdownVerbHandler.java | 7 +- src/java/org/apache/cassandra/gms/Gossiper.java| 31 +- src/java/org/apache/cassandra/net/Verb.java| 5 +- .../test/gossip/GossipShutdownTest.java| 106 + 7 files changed, 214 insertions(+), 7 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (31729b8f92 -> b8b17dc5db)
This is an automated email from the ASF dual-hosted git repository. dcapwell pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git from 31729b8f92 Merge branch 'cassandra-5.0' into trunk new 2bab3f27ba Gossip NPE due to shutdown event corrupting empty statuses new 1920571861 Merge branch 'cassandra-4.0' into cassandra-4.1 new 210ced3b7b Merge branch 'cassandra-4.1' into cassandra-5.0 new b8b17dc5db Merge branch 'cassandra-5.0' into trunk The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + .../org/apache/cassandra/gms/EndpointState.java| 5 + .../org/apache/cassandra/gms/GossipShutdown.java | 66 + .../cassandra/gms/GossipShutdownVerbHandler.java | 7 +- src/java/org/apache/cassandra/gms/Gossiper.java| 31 +- src/java/org/apache/cassandra/net/Verb.java| 5 +- .../test/gossip/GossipShutdownTest.java| 106 + 7 files changed, 214 insertions(+), 7 deletions(-) create mode 100644 src/java/org/apache/cassandra/gms/GossipShutdown.java create mode 100644 test/distributed/org/apache/cassandra/distributed/test/gossip/GossipShutdownTest.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-4.1 updated (e562d6a3a9 -> 1920571861)
This is an automated email from the ASF dual-hosted git repository. dcapwell pushed a change to branch cassandra-4.1 in repository https://gitbox.apache.org/repos/asf/cassandra.git from e562d6a3a9 Merge branch 'cassandra-4.0' into cassandra-4.1 new 2bab3f27ba Gossip NPE due to shutdown event corrupting empty statuses new 1920571861 Merge branch 'cassandra-4.0' into cassandra-4.1 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + .../org/apache/cassandra/gms/EndpointState.java| 5 + src/java/org/apache/cassandra/gms/Gossiper.java| 2 +- .../test/gossip/GossipShutdownTest.java| 106 + 4 files changed, 113 insertions(+), 1 deletion(-) create mode 100644 test/distributed/org/apache/cassandra/distributed/test/gossip/GossipShutdownTest.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-4.1' into cassandra-5.0
This is an automated email from the ASF dual-hosted git repository.
dcapwell pushed a commit to branch cassandra-5.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
commit 210ced3b7b6718b515631b8b1579aa6f6fc2dea3
Merge: 3b9881bfa6 1920571861
Author: David Capwell
AuthorDate: Tue Oct 10 14:15:08 2023 -0700
Merge branch 'cassandra-4.1' into cassandra-5.0
CHANGES.txt| 1 +
.../org/apache/cassandra/gms/EndpointState.java| 5 +
.../org/apache/cassandra/gms/GossipShutdown.java | 66 +
.../cassandra/gms/GossipShutdownVerbHandler.java | 7 +-
src/java/org/apache/cassandra/gms/Gossiper.java| 31 +-
src/java/org/apache/cassandra/net/Verb.java| 5 +-
.../test/gossip/GossipShutdownTest.java| 106 +
7 files changed, 214 insertions(+), 7 deletions(-)
diff --cc CHANGES.txt
index 1e0fe54855,c5f2a1b92c..f40b4020d7
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@@ -1,24 -1,12 +1,25 @@@
-4.1.4
+5.0-alpha2
+ * Fix vector type to support DDM's mask_default function (CASSANDRA-18889)
+ * Remove unnecessary reporter-config3 dependency (CASSANDRA-18907)
+ * Remove support for empty values on the vector data type (CASSANDRA-18876)
+ * Upgrade Dropwizard Metrics to 4.2.19 (CASSANDRA-14667)
+ * Upgrade caffeine cache and fix CIDR permissions cache invalidation
(CASSANDRA-18805)
+ * Remove deprecated properties in CompressionParams (CASSANDRA-18742)
+ * Add support for repair coordinator to retry messages that timeout
(CASSANDRA-18816)
+ * Upgrade slf4j-api to 1.7.36 (CASSANDRA-18882)
+ * Make the output of ON/OFF commands in cqlsh consistent (CASSANDRA-18547)
+ * Do not create sstable files before registering in txn (CASSANDRA-18737)
+ * Do not log stacktrace on mismatched cache and schema version and checksum
error in AutoSavingCache (CASSANDRA-18862)
+ * Remove wrong assertion in SSTableLoader (CASSANDRA-18840)
+ * Fix accessing java.nio.Bits.TOTAL_CAPACITY in Java17 (CASSANDRA-18848)
+ * Remove metrics-reporter-config dependency (CASSANDRA-18743)
+ * Fix SAI's SegmentMetadata min and max primary keys (CASSANDRA-18734)
+ * Remove commons-codec dependency (CASSANDRA-18772)
+Merged from 4.1:
* Internode legacy SSL storage port certificate is not hot reloaded on
update (CASSANDRA-18681)
* Nodetool paxos-only repair is no longer incremental (CASSANDRA-18466)
- * Waiting indefinitely on ReceivedMessage response in
StreamSession#receive() can cause deadlock (CASSANDRA-18733)
- * Allow empty keystore_password in encryption_options (CASSANDRA-18778)
- * Skip ColumnFamilyStore#topPartitions initialization when client or tool
mode (CASSANDRA-18697)
Merged from 4.0:
+ * Gossip NPE due to shutdown event corrupting empty statuses
(CASSANDRA-18913)
- * Fix closing iterator in SecondaryIndexBuilder (CASSANDRA-18361)
* Update hdrhistogram to 2.1.12 (CASSANDRA-18893)
* Improve performance of compactions when table does not have an index
(CASSANDRA-18773)
* JMH improvements - faster build and async profiler (CASSANDRA-18871)
diff --cc src/java/org/apache/cassandra/gms/GossipShutdown.java
index 00,00..02f0f375c4
new file mode 100644
--- /dev/null
+++ b/src/java/org/apache/cassandra/gms/GossipShutdown.java
@@@ -1,0 -1,0 +1,66 @@@
++/*
++ * Licensed to the Apache Software Foundation (ASF) under one
++ * or more contributor license agreements. See the NOTICE file
++ * distributed with this work for additional information
++ * regarding copyright ownership. The ASF licenses this file
++ * to you under the Apache License, Version 2.0 (the
++ * "License"); you may not use this file except in compliance
++ * with the License. You may obtain a copy of the License at
++ *
++ * http://www.apache.org/licenses/LICENSE-2.0
++ *
++ * Unless required by applicable law or agreed to in writing, software
++ * distributed under the License is distributed on an "AS IS" BASIS,
++ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
++ * See the License for the specific language governing permissions and
++ * limitations under the License.
++ */
++
++package org.apache.cassandra.gms;
++
++import java.io.IOException;
++
++import javax.annotation.Nullable;
++
++import org.apache.cassandra.io.IVersionedSerializer;
++import org.apache.cassandra.io.util.DataInputPlus;
++import org.apache.cassandra.io.util.DataOutputPlus;
++import org.apache.cassandra.net.MessagingService;
++
++public class GossipShutdown
++{
++public static final Serializer serializer = new Serializer();
++
++public final EndpointState state;
++
++public GossipShutdown(EndpointState state)
++{
++this.state = state;
++}
++
++public static final class Serializer implements
IVersionedSerializer
++{
++
++@Override
++public void serialize(GossipShutdown t, DataOutputPlus out, int
version) throws IOException
++{
++if
[cassandra] 01/01: Merge branch 'cassandra-4.0' into cassandra-4.1
This is an automated email from the ASF dual-hosted git repository. dcapwell pushed a commit to branch cassandra-4.1 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 1920571861ac2fe953229d171c2e8414788d6ab0 Merge: e562d6a3a9 2bab3f27ba Author: David Capwell AuthorDate: Tue Oct 10 14:05:19 2023 -0700 Merge branch 'cassandra-4.0' into cassandra-4.1 CHANGES.txt| 1 + .../org/apache/cassandra/gms/EndpointState.java| 5 + src/java/org/apache/cassandra/gms/Gossiper.java| 2 +- .../test/gossip/GossipShutdownTest.java| 106 + 4 files changed, 113 insertions(+), 1 deletion(-) diff --cc CHANGES.txt index a8a5ddeaeb,f0d59c2abf..c5f2a1b92c --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,10 -1,6 +1,11 @@@ -4.0.12 +4.1.4 + * Internode legacy SSL storage port certificate is not hot reloaded on update (CASSANDRA-18681) + * Nodetool paxos-only repair is no longer incremental (CASSANDRA-18466) + * Waiting indefinitely on ReceivedMessage response in StreamSession#receive() can cause deadlock (CASSANDRA-18733) + * Allow empty keystore_password in encryption_options (CASSANDRA-18778) + * Skip ColumnFamilyStore#topPartitions initialization when client or tool mode (CASSANDRA-18697) +Merged from 4.0: + * Gossip NPE due to shutdown event corrupting empty statuses (CASSANDRA-18913) - * Synchronize CQLSSTableWriter#build on the Schema.instance object (CASSANDRA-18317) * Fix closing iterator in SecondaryIndexBuilder (CASSANDRA-18361) * Update hdrhistogram to 2.1.12 (CASSANDRA-18893) * Improve performance of compactions when table does not have an index (CASSANDRA-18773) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-5.0 updated (3b9881bfa6 -> 210ced3b7b)
This is an automated email from the ASF dual-hosted git repository. dcapwell pushed a change to branch cassandra-5.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 3b9881bfa6 Fix vector type to support DDM's mask_default function new 2bab3f27ba Gossip NPE due to shutdown event corrupting empty statuses new 1920571861 Merge branch 'cassandra-4.0' into cassandra-4.1 new 210ced3b7b Merge branch 'cassandra-4.1' into cassandra-5.0 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + .../org/apache/cassandra/gms/EndpointState.java| 5 + .../org/apache/cassandra/gms/GossipShutdown.java | 66 + .../cassandra/gms/GossipShutdownVerbHandler.java | 7 +- src/java/org/apache/cassandra/gms/Gossiper.java| 31 +- src/java/org/apache/cassandra/net/Verb.java| 5 +- .../test/gossip/GossipShutdownTest.java| 106 + 7 files changed, 214 insertions(+), 7 deletions(-) create mode 100644 src/java/org/apache/cassandra/gms/GossipShutdown.java create mode 100644 test/distributed/org/apache/cassandra/distributed/test/gossip/GossipShutdownTest.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-4.0 updated: Gossip NPE due to shutdown event corrupting empty statuses
This is an automated email from the ASF dual-hosted git repository.
dcapwell pushed a commit to branch cassandra-4.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-4.0 by this push:
new 2bab3f27ba Gossip NPE due to shutdown event corrupting empty statuses
2bab3f27ba is described below
commit 2bab3f27ba1535203d61497abe6810cdcb4640d0
Author: David Capwell
AuthorDate: Tue Oct 10 09:22:05 2023 -0700
Gossip NPE due to shutdown event corrupting empty statuses
patch by David Capwell; reviewed by Brandon Williams for CASSANDRA-18913
---
CHANGES.txt| 1 +
.../org/apache/cassandra/gms/EndpointState.java| 5 +
src/java/org/apache/cassandra/gms/Gossiper.java| 2 +-
.../cassandra/distributed/shared/ClusterUtils.java | 16
.../test/gossip/GossipShutdownTest.java| 106 +
5 files changed, 129 insertions(+), 1 deletion(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 25691b1cd9..f0d59c2abf 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
4.0.12
+ * Gossip NPE due to shutdown event corrupting empty statuses (CASSANDRA-18913)
* Synchronize CQLSSTableWriter#build on the Schema.instance object
(CASSANDRA-18317)
* Fix closing iterator in SecondaryIndexBuilder (CASSANDRA-18361)
* Update hdrhistogram to 2.1.12 (CASSANDRA-18893)
diff --git a/src/java/org/apache/cassandra/gms/EndpointState.java
b/src/java/org/apache/cassandra/gms/EndpointState.java
index b8d56263e7..782a72207c 100644
--- a/src/java/org/apache/cassandra/gms/EndpointState.java
+++ b/src/java/org/apache/cassandra/gms/EndpointState.java
@@ -191,6 +191,11 @@ public class EndpointState
isAlive = false;
}
+public boolean isStateEmpty()
+{
+return applicationState.get().isEmpty();
+}
+
/**
* @return true if {@link HeartBeatState#isEmpty()} is true and no STATUS
application state exists
*/
diff --git a/src/java/org/apache/cassandra/gms/Gossiper.java
b/src/java/org/apache/cassandra/gms/Gossiper.java
index 2b3c48ed77..f88ee44edf 100644
--- a/src/java/org/apache/cassandra/gms/Gossiper.java
+++ b/src/java/org/apache/cassandra/gms/Gossiper.java
@@ -567,7 +567,7 @@ public class Gossiper implements
IFailureDetectionEventListener, GossiperMBean
{
checkProperThreadForStateMutation();
EndpointState epState = endpointStateMap.get(endpoint);
-if (epState == null)
+if (epState == null || epState.isStateEmpty())
return;
VersionedValue shutdown =
StorageService.instance.valueFactory.shutdown(true);
epState.addApplicationState(ApplicationState.STATUS_WITH_PORT,
shutdown);
diff --git
a/test/distributed/org/apache/cassandra/distributed/shared/ClusterUtils.java
b/test/distributed/org/apache/cassandra/distributed/shared/ClusterUtils.java
index 94256cca1b..d2824ccd12 100644
--- a/test/distributed/org/apache/cassandra/distributed/shared/ClusterUtils.java
+++ b/test/distributed/org/apache/cassandra/distributed/shared/ClusterUtils.java
@@ -136,6 +136,22 @@ public class ClusterUtils
cluster.stream().forEach(ClusterUtils::stopUnchecked);
}
+/**
+ * Create a new instance and add it to the cluster, without starting it.
+ *
+ * @param cluster to add to
+ * @param other config to copy from
+ * @param fn function to add to the config before starting
+ * @param instance type
+ * @return the instance added
+ */
+public static I addInstance(AbstractCluster
cluster,
+ IInstanceConfig other,
+
Consumer fn)
+{
+return addInstance(cluster, other.localDatacenter(),
other.localRack(), fn);
+}
+
/**
* Create a new instance and add it to the cluster, without starting it.
*
diff --git
a/test/distributed/org/apache/cassandra/distributed/test/gossip/GossipShutdownTest.java
b/test/distributed/org/apache/cassandra/distributed/test/gossip/GossipShutdownTest.java
new file mode 100644
index 00..59ac4feeeb
--- /dev/null
+++
b/test/distributed/org/apache/cassandra/distributed/test/gossip/GossipShutdownTest.java
@@ -0,0 +1,106 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR
[jira] [Commented] (CASSANDRA-18916) Start-Up Validation Should Log a Single Report
[ https://issues.apache.org/jira/browse/CASSANDRA-18916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773822#comment-17773822 ] Yuriy Semchyshyn commented on CASSANDRA-18916: -- GitHub PR: https://github.com/apache/cassandra-analytics/pull/16 CircleCI Run: https://app.circleci.com/pipelines/github/5/cassandra-analytics/41 > Start-Up Validation Should Log a Single Report > -- > > Key: CASSANDRA-18916 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18916 > Project: Cassandra > Issue Type: Improvement > Components: Analytics Library >Reporter: Yuriy Semchyshyn >Priority: Normal > Time Spent: 10m > Remaining Estimate: 0h > > In order to improve readability of the log, start-up validation should > produce a complete report and emit it as a single event upon completion -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18838) make stop-server shell out of the box
[ https://issues.apache.org/jira/browse/CASSANDRA-18838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-18838: -- Status: Review In Progress (was: Patch Available) > make stop-server shell out of the box > - > > Key: CASSANDRA-18838 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18838 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Ling Mao >Assignee: Ling Mao >Priority: Normal > Fix For: 5.x > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18838) make stop-server shell out of the box
[ https://issues.apache.org/jira/browse/CASSANDRA-18838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-18838: -- Change Category: Operability Complexity: Normal Reviewers: Stefan Miklosovic Status: Open (was: Triage Needed) > make stop-server shell out of the box > - > > Key: CASSANDRA-18838 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18838 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Ling Mao >Assignee: Ling Mao >Priority: Normal > Fix For: 5.x > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18838) make stop-server shell out of the box
[ https://issues.apache.org/jira/browse/CASSANDRA-18838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-18838: -- Test and Documentation Plan: manual verification that the patch works Status: Patch Available (was: Open) > make stop-server shell out of the box > - > > Key: CASSANDRA-18838 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18838 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Ling Mao >Assignee: Ling Mao >Priority: Normal > Fix For: 5.x > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18849) Test failure: dtest.secondary_indexes_test.TestPreJoinCallback.test_resume
[
https://issues.apache.org/jira/browse/CASSANDRA-18849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-18849:
-
Test and Documentation Plan: run CI
Status: Patch Available (was: Open)
> Test failure: dtest.secondary_indexes_test.TestPreJoinCallback.test_resume
> --
>
> Key: CASSANDRA-18849
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18849
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Ekaterina Dimitrova
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.1.x, 5.0.x, 5.x
>
>
> Since recently, it fails regularly on 4.1, 5.0, and trunk:
> {code:java}
> Error Message
> failed on teardown with "Failed: Unexpected error found in node logs (see
> stdout for full details). Errors: [[node2] 'ERROR
> [Stream-Deserializer-/127.0.0.1:7000-6e2ae2f5] 2023-09-13 12:41:02,461
> StreamSession.java:700 - [Stream #cc2010c0-5232-11ee-bc8b-4945091d03ae]
> Socket closed before session completion, peer 127.0.0.1:7000 is probably
> down.\njava.nio.channels.ClosedChannelException: null\n\tat
> org.apache.cassandra.net.AsyncStreamingInputPlus.reBuffer(AsyncStreamingInputPlus.java:119)\n\tat
>
> org.apache.cassandra.io.util.RebufferingInputStream.readByte(RebufferingInputStream.java:178)\n\tat
>
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:49)\n\tat
>
> org.apache.cassandra.streaming.StreamDeserializingTask.run(StreamDeserializingTask.java:59)\n\tat
>
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)\n\tat
> java.base/java.lang.Thread.run(Thread.java:829)']"
> Stacktrace
> Unexpected error found in node logs (see stdout for full details). Errors:
> [[node2] 'ERROR [Stream-Deserializer-/127.0.0.1:7000-6e2ae2f5] 2023-09-13
> 12:41:02,461 StreamSession.java:700 - [Stream
> #cc2010c0-5232-11ee-bc8b-4945091d03ae] Socket closed before session
> completion, peer 127.0.0.1:7000 is probably
> down.\njava.nio.channels.ClosedChannelException: null\n\tat
> org.apache.cassandra.net.AsyncStreamingInputPlus.reBuffer(AsyncStreamingInputPlus.java:119)\n\tat
>
> org.apache.cassandra.io.util.RebufferingInputStream.readByte(RebufferingInputStream.java:178)\n\tat
>
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:49)\n\tat
>
> org.apache.cassandra.streaming.StreamDeserializingTask.run(StreamDeserializingTask.java:59)\n\tat
>
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)\n\tat
> java.base/java.lang.Thread.run(Thread.java:829)']
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18849) Test failure: dtest.secondary_indexes_test.TestPreJoinCallback.test_resume
[
https://issues.apache.org/jira/browse/CASSANDRA-18849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773801#comment-17773801
]
Brandon Williams commented on CASSANDRA-18849:
--
We just need to
[ignore|https://github.com/driftx/cassandra-dtest/commit/36b5e337a4968d5315640969d35422eb6906e6a8]
this.
||Branch||CI||
|[5.0|https://github.com/driftx/cassandra/tree/CASSANDRA-18918-5.0]|[dtest
repeat|https://app.circleci.com/pipelines/github/driftx/cassandra/1331/workflows/ada4a9bb-7f14-4333-992a-401586060592/jobs/57665]|
> Test failure: dtest.secondary_indexes_test.TestPreJoinCallback.test_resume
> --
>
> Key: CASSANDRA-18849
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18849
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Ekaterina Dimitrova
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.1.x, 5.0.x, 5.x
>
>
> Since recently, it fails regularly on 4.1, 5.0, and trunk:
> {code:java}
> Error Message
> failed on teardown with "Failed: Unexpected error found in node logs (see
> stdout for full details). Errors: [[node2] 'ERROR
> [Stream-Deserializer-/127.0.0.1:7000-6e2ae2f5] 2023-09-13 12:41:02,461
> StreamSession.java:700 - [Stream #cc2010c0-5232-11ee-bc8b-4945091d03ae]
> Socket closed before session completion, peer 127.0.0.1:7000 is probably
> down.\njava.nio.channels.ClosedChannelException: null\n\tat
> org.apache.cassandra.net.AsyncStreamingInputPlus.reBuffer(AsyncStreamingInputPlus.java:119)\n\tat
>
> org.apache.cassandra.io.util.RebufferingInputStream.readByte(RebufferingInputStream.java:178)\n\tat
>
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:49)\n\tat
>
> org.apache.cassandra.streaming.StreamDeserializingTask.run(StreamDeserializingTask.java:59)\n\tat
>
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)\n\tat
> java.base/java.lang.Thread.run(Thread.java:829)']"
> Stacktrace
> Unexpected error found in node logs (see stdout for full details). Errors:
> [[node2] 'ERROR [Stream-Deserializer-/127.0.0.1:7000-6e2ae2f5] 2023-09-13
> 12:41:02,461 StreamSession.java:700 - [Stream
> #cc2010c0-5232-11ee-bc8b-4945091d03ae] Socket closed before session
> completion, peer 127.0.0.1:7000 is probably
> down.\njava.nio.channels.ClosedChannelException: null\n\tat
> org.apache.cassandra.net.AsyncStreamingInputPlus.reBuffer(AsyncStreamingInputPlus.java:119)\n\tat
>
> org.apache.cassandra.io.util.RebufferingInputStream.readByte(RebufferingInputStream.java:178)\n\tat
>
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:49)\n\tat
>
> org.apache.cassandra.streaming.StreamDeserializingTask.run(StreamDeserializingTask.java:59)\n\tat
>
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)\n\tat
> java.base/java.lang.Thread.run(Thread.java:829)']
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-18849) Test failure: dtest.secondary_indexes_test.TestPreJoinCallback.test_resume
[
https://issues.apache.org/jira/browse/CASSANDRA-18849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams reassigned CASSANDRA-18849:
Assignee: Brandon Williams
> Test failure: dtest.secondary_indexes_test.TestPreJoinCallback.test_resume
> --
>
> Key: CASSANDRA-18849
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18849
> Project: Cassandra
> Issue Type: Bug
> Components: Test/dtest/python
>Reporter: Ekaterina Dimitrova
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.1.x, 5.0.x, 5.x
>
>
> Since recently, it fails regularly on 4.1, 5.0, and trunk:
> {code:java}
> Error Message
> failed on teardown with "Failed: Unexpected error found in node logs (see
> stdout for full details). Errors: [[node2] 'ERROR
> [Stream-Deserializer-/127.0.0.1:7000-6e2ae2f5] 2023-09-13 12:41:02,461
> StreamSession.java:700 - [Stream #cc2010c0-5232-11ee-bc8b-4945091d03ae]
> Socket closed before session completion, peer 127.0.0.1:7000 is probably
> down.\njava.nio.channels.ClosedChannelException: null\n\tat
> org.apache.cassandra.net.AsyncStreamingInputPlus.reBuffer(AsyncStreamingInputPlus.java:119)\n\tat
>
> org.apache.cassandra.io.util.RebufferingInputStream.readByte(RebufferingInputStream.java:178)\n\tat
>
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:49)\n\tat
>
> org.apache.cassandra.streaming.StreamDeserializingTask.run(StreamDeserializingTask.java:59)\n\tat
>
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)\n\tat
> java.base/java.lang.Thread.run(Thread.java:829)']"
> Stacktrace
> Unexpected error found in node logs (see stdout for full details). Errors:
> [[node2] 'ERROR [Stream-Deserializer-/127.0.0.1:7000-6e2ae2f5] 2023-09-13
> 12:41:02,461 StreamSession.java:700 - [Stream
> #cc2010c0-5232-11ee-bc8b-4945091d03ae] Socket closed before session
> completion, peer 127.0.0.1:7000 is probably
> down.\njava.nio.channels.ClosedChannelException: null\n\tat
> org.apache.cassandra.net.AsyncStreamingInputPlus.reBuffer(AsyncStreamingInputPlus.java:119)\n\tat
>
> org.apache.cassandra.io.util.RebufferingInputStream.readByte(RebufferingInputStream.java:178)\n\tat
>
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:49)\n\tat
>
> org.apache.cassandra.streaming.StreamDeserializingTask.run(StreamDeserializingTask.java:59)\n\tat
>
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)\n\tat
> java.base/java.lang.Thread.run(Thread.java:829)']
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18918) Test failure: secondary_indexes_test.TestPreJoinCallback.test_resume
[
https://issues.apache.org/jira/browse/CASSANDRA-18918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-18918:
-
Resolution: Duplicate
Status: Resolved (was: Open)
> Test failure: secondary_indexes_test.TestPreJoinCallback.test_resume
>
>
> Key: CASSANDRA-18918
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18918
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 5.0.x, 5.x
>
>
> As seen here:
> https://ci-cassandra.apache.org/job/Cassandra-5.0/60/testReport/dtest-novnode.secondary_indexes_test/TestPreJoinCallback/test_resume/
> {quote}
> failed on teardown with "Failed: Unexpected error found in node logs (see
> stdout for full details). Errors: [[node2] 'ERROR
> [Stream-Deserializer-/127.0.0.1:7000-ffd39504] 2023-10-09 16:12:59,153
> StreamSession.java:702 - [Stream #b69e6f80-66be-11ee-b813-3bca425f16a7]
> Socket closed before session completion, peer 127.0.0.1:7000 is probably
> down.\njava.nio.channels.ClosedChannelException: null\n\tat
> org.apache.cassandra.net.AsyncStreamingInputPlus.reBuffer(AsyncStreamingInputPlus.java:119)\n\tat
>
> org.apache.cassandra.io.util.RebufferingInputStream.readByte(RebufferingInputStream.java:178)\n\tat
>
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:49)\n\tat
>
> org.apache.cassandra.streaming.StreamDeserializingTask.run(StreamDeserializingTask.java:59)\n\tat
>
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)\n\tat
> java.base/java.lang.Thread.run(Thread.java:833)']"
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18905) Index.Group is incorrectly unregistered from the SecondaryIndexManager
[
https://issues.apache.org/jira/browse/CASSANDRA-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773792#comment-17773792
]
Caleb Rackliffe commented on CASSANDRA-18905:
-
The 5.0 patch looks good, although I've left a few nits inline in the PR. It
also looks like we need to adjust
{{IndexGroupMetricsTest#verifyIndexGroupMetrics}} the same way we've adjusted
{{QueryMetricsTest#testMetricRelease}} to reflect the metrics being released
when the last index is dropped.
Is the {{trunk}} PR missing some test fixes? ({{QueryMetricsTest}} et al.)
Finally, a bit of OCD not related to the patch,
{{QueryMetricsTest#testSameIndexNameAcrossKeyspaces}} doesn't need its
{{throws}} clause.
> Index.Group is incorrectly unregistered from the SecondaryIndexManager
> --
>
> Key: CASSANDRA-18905
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18905
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Urgent
> Fix For: 5.0
>
>
> An Index.Group is removed from the SecondaryIndexManager during
> unregisterIndex if it contains no indexes after the index is unregistered.
> The code for removing the group uses the wrong key to remove the group from
> the indexGroups map. It is using the group object rather than the group name
> that is used as the key in the map.
> This means that the group is not added again if a new index is registered
> using that group. The knock on from this is that the
> StorageAttachedIndexGroup unregisters itself from the Tracker when it has no
> indexes after an index is removed. The same group with no tracker is then
> used for new indexes. This group then receives no notifications about sstable
> or memtable updates. The ultimate side effect of this is that, memtables are
> not released, resulting in memory leaks and indexes are not updated with new
> sstables and their associated index files.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18909) [WEBSITE] Update the Metric Reporters documentation page
[ https://issues.apache.org/jira/browse/CASSANDRA-18909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-18909: -- Test and Documentation Plan: renders OK Status: Patch Available (was: In Progress) > [WEBSITE] Update the Metric Reporters documentation page > - > > Key: CASSANDRA-18909 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18909 > Project: Cassandra > Issue Type: Task > Components: Documentation >Reporter: Maxim Muzafarov >Assignee: Maxim Muzafarov >Priority: Normal > Fix For: 5.0.x > > Time Spent: 10m > Remaining Estimate: 0h > > After CASSANDRA-18743 and CASSANDRA-14667 we need to update the website > (documentation) information accordingly. The links to dropwizard metrics must > be updated. The references to the metrics reporters should be added to the > following page: > [https://cassandra.apache.org/doc/latest/cassandra/operating/metrics.html#metric-reporters] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18909) [WEBSITE] Update the Metric Reporters documentation page
[ https://issues.apache.org/jira/browse/CASSANDRA-18909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-18909: -- Reviewers: Stefan Miklosovic Status: Review In Progress (was: Patch Available) > [WEBSITE] Update the Metric Reporters documentation page > - > > Key: CASSANDRA-18909 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18909 > Project: Cassandra > Issue Type: Task > Components: Documentation >Reporter: Maxim Muzafarov >Assignee: Maxim Muzafarov >Priority: Normal > Fix For: 5.0.x > > Time Spent: 10m > Remaining Estimate: 0h > > After CASSANDRA-18743 and CASSANDRA-14667 we need to update the website > (documentation) information accordingly. The links to dropwizard metrics must > be updated. The references to the metrics reporters should be added to the > following page: > [https://cassandra.apache.org/doc/latest/cassandra/operating/metrics.html#metric-reporters] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18909) [WEBSITE] Update the Metric Reporters documentation page
[ https://issues.apache.org/jira/browse/CASSANDRA-18909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated CASSANDRA-18909: Change Category: Code Clarity Complexity: Normal Status: Open (was: Triage Needed) > [WEBSITE] Update the Metric Reporters documentation page > - > > Key: CASSANDRA-18909 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18909 > Project: Cassandra > Issue Type: Task > Components: Documentation >Reporter: Maxim Muzafarov >Assignee: Maxim Muzafarov >Priority: Normal > Fix For: 5.0.x > > Time Spent: 10m > Remaining Estimate: 0h > > After CASSANDRA-18743 and CASSANDRA-14667 we need to update the website > (documentation) information accordingly. The links to dropwizard metrics must > be updated. The references to the metrics reporters should be added to the > following page: > [https://cassandra.apache.org/doc/latest/cassandra/operating/metrics.html#metric-reporters] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18534) Make sstable format configurable per table
[
https://issues.apache.org/jira/browse/CASSANDRA-18534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773778#comment-17773778
]
Stefan Miklosovic commented on CASSANDRA-18534:
---
Interesting, thank you for explaining this. I was clearly approaching this from
a completely different angle. I ll keep thinking about this for a while ...
> Make sstable format configurable per table
> --
>
> Key: CASSANDRA-18534
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18534
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema, Local/SSTable
>Reporter: Branimir Lambov
>Assignee: Maxwell Guo
>Priority: Normal
> Fix For: 5.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Some SSTable format settings need to be configurable per table for better
> efficiency. This includes:
> - {{row_index_granularity}}
> - {{bloom_filter_fp_chance}}
> - {{crc_check_chance}}
> - {{min/max_index_interval}}
> Some of these are currently configurable using direct properties of tables.
> Having them as format properties makes better sense and should also support
> specifying useable combinations of settings, e.g.
> {code:java}
> CREATE TABLE ... WITH sstable_format = "bti-fast";
> CREATE TABLE ... WITH sstable_format = "bti-small";
> {code}
> where {{bti-fast}} and {{bti-small}} can be defined in {{cassandra.yaml}}
> e.g. as
> {code:java}
> sstable.format.options:
> - bti-fast:
> row_index_granularity: 1kiB
> bloom_filter_fp_chance: 0.01
> - bti-small:
> row_index_granularity: 32kiB
> bloom_filter_fp_chance: 0.1
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-14877) StreamCoordinator "leaks" threads
[ https://issues.apache.org/jira/browse/CASSANDRA-14877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated CASSANDRA-14877: Fix Version/s: (was: 5.x) > StreamCoordinator "leaks" threads > - > > Key: CASSANDRA-14877 > URL: https://issues.apache.org/jira/browse/CASSANDRA-14877 > Project: Cassandra > Issue Type: Bug > Components: Legacy/Streaming and Messaging >Reporter: Massimiliano Tomassi >Assignee: Massimiliano Tomassi >Priority: Low > Fix For: 2.1.x, 2.2.x, 3.0.x, 3.11.x > > > Since Cassandra 2.1, streaming sessions are started by running a > StreamSessionConnector task for each session in a dedicated executor (a > static field of StreamCoordinator). > That executor is initialized with > DebuggableThreadPoolExecutor.createWithFixedPoolSize, which means that once > created (up to the given limit of the number of logical cores), its threads > are kept alive for Integer.MAX_VALUE seconds. > This practically means that once a node needs to establish streaming sessions > to n other nodes, it will create Math.min(n, numLogicalCores) > StreamConnectionEstablisher threads that will stay parked forever after > initializing (not completing) the session. > It seems preferable to replace > DebuggableThreadPoolExecutor.createWithFixedPoolSize with > DebuggableThreadPoolExecutor.createWithMaximumPoolSize which allows providing > a saner keep-alive period (e.g. a minute). > That's also what createWithFixedPoolSize's Javadoc recommends: If (most) > threads are expected to be idle most of the time, prefer createWithMaxSize() > instead. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18918) Test failure: secondary_indexes_test.TestPreJoinCallback.test_resume
[
https://issues.apache.org/jira/browse/CASSANDRA-18918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773775#comment-17773775
]
Ekaterina Dimitrova commented on CASSANDRA-18918:
-
Isn't this a duplicate of CASSANDRA-18849?
> Test failure: secondary_indexes_test.TestPreJoinCallback.test_resume
>
>
> Key: CASSANDRA-18918
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18918
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 5.0.x, 5.x
>
>
> As seen here:
> https://ci-cassandra.apache.org/job/Cassandra-5.0/60/testReport/dtest-novnode.secondary_indexes_test/TestPreJoinCallback/test_resume/
> {quote}
> failed on teardown with "Failed: Unexpected error found in node logs (see
> stdout for full details). Errors: [[node2] 'ERROR
> [Stream-Deserializer-/127.0.0.1:7000-ffd39504] 2023-10-09 16:12:59,153
> StreamSession.java:702 - [Stream #b69e6f80-66be-11ee-b813-3bca425f16a7]
> Socket closed before session completion, peer 127.0.0.1:7000 is probably
> down.\njava.nio.channels.ClosedChannelException: null\n\tat
> org.apache.cassandra.net.AsyncStreamingInputPlus.reBuffer(AsyncStreamingInputPlus.java:119)\n\tat
>
> org.apache.cassandra.io.util.RebufferingInputStream.readByte(RebufferingInputStream.java:178)\n\tat
>
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:49)\n\tat
>
> org.apache.cassandra.streaming.StreamDeserializingTask.run(StreamDeserializingTask.java:59)\n\tat
>
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)\n\tat
> java.base/java.lang.Thread.run(Thread.java:833)']"
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18918) Test failure: secondary_indexes_test.TestPreJoinCallback.test_resume
[
https://issues.apache.org/jira/browse/CASSANDRA-18918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773772#comment-17773772
]
Brandon Williams commented on CASSANDRA-18918:
--
Another one we just need to
[ignore|https://github.com/driftx/cassandra-dtest/commit/36b5e337a4968d5315640969d35422eb6906e6a8]
||Branch||CI||
|[5.0|https://github.com/driftx/cassandra/tree/CASSANDRA-18918-5.0]|[dtest
repeat|https://app.circleci.com/pipelines/github/driftx/cassandra/1331/workflows/ada4a9bb-7f14-4333-992a-401586060592/jobs/57665]|
> Test failure: secondary_indexes_test.TestPreJoinCallback.test_resume
>
>
> Key: CASSANDRA-18918
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18918
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 5.0.x, 5.x
>
>
> As seen here:
> https://ci-cassandra.apache.org/job/Cassandra-5.0/60/testReport/dtest-novnode.secondary_indexes_test/TestPreJoinCallback/test_resume/
> {quote}
> failed on teardown with "Failed: Unexpected error found in node logs (see
> stdout for full details). Errors: [[node2] 'ERROR
> [Stream-Deserializer-/127.0.0.1:7000-ffd39504] 2023-10-09 16:12:59,153
> StreamSession.java:702 - [Stream #b69e6f80-66be-11ee-b813-3bca425f16a7]
> Socket closed before session completion, peer 127.0.0.1:7000 is probably
> down.\njava.nio.channels.ClosedChannelException: null\n\tat
> org.apache.cassandra.net.AsyncStreamingInputPlus.reBuffer(AsyncStreamingInputPlus.java:119)\n\tat
>
> org.apache.cassandra.io.util.RebufferingInputStream.readByte(RebufferingInputStream.java:178)\n\tat
>
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:49)\n\tat
>
> org.apache.cassandra.streaming.StreamDeserializingTask.run(StreamDeserializingTask.java:59)\n\tat
>
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)\n\tat
> java.base/java.lang.Thread.run(Thread.java:833)']"
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18808) netty-handler vulnerability: CVE-2023-4586
[
https://issues.apache.org/jira/browse/CASSANDRA-18808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773771#comment-17773771
]
Michael Semb Wever commented on CASSANDRA-18808:
any updates [~norman] ?
> netty-handler vulnerability: CVE-2023-4586
> --
>
> Key: CASSANDRA-18808
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18808
> Project: Cassandra
> Issue Type: Bug
> Components: Consistency/Coordination
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 5.0.x, 5.x
>
>
> This is failing OWASP:
> {noformat}
> Dependency-Check Failure:
> One or more dependencies were identified with vulnerabilities that have a
> CVSS score greater than or equal to '1.0':
> netty-handler-4.1.96.Final.jar: CVE-2023-4586
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18221) CEP-15: (Accord) Define a configuration system for Accord to allow overriding of internal configs and integrate with Cassandra yaml
[
https://issues.apache.org/jira/browse/CASSANDRA-18221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773769#comment-17773769
]
David Capwell commented on CASSANDRA-18221:
---
+1 to C* changes, but have a few comments about Accord side (no PR so pushing
here)
* copy/paste logic (see
https://github.com/jacek-lewandowski/cassandra-accord/compare/b1befa3cc0a8496451bb48ec3bb1c0f56b8c7653..CASSANDRA-18221#diff-0cb0a486433822ffbc29bfa8b6b27352b4d19cf026892f60866ad5c908fb8c11R164-R169).
would be good to refactor
{code}
if (failure instanceof Preempted)
node.agent().metricsEventsListener().onPreempted(txnId);
else if (failure instanceof Timeout)
node.agent().metricsEventsListener().onTimeout(txnId);
else if (failure instanceof Invalidated)
node.agent().metricsEventsListener().onInvalidated(txnId);
{code}
would be nice to have something like
node.agent().metricsEventsListener().onException(txnId, failure) and that does
these if checks by default?
* nit: node.agent().metricsEventsListener().onProgressLogSizeChange(txnId, 1);
this is a SimpleProgressLog thing, so maybe rename to
onSimpleProgressLogSizeChange to denote that its for a specific impl? same
comment for getProgressLogScheduleDelay
* i don't think accord-core/src/main/java/accord/primitives/Timestamp.java
change is correct. toString now returns a bit map but fromString thinks its a
normal integer. so the property that t == fromString(t.toString()) would be
broken... can you add tests if you wish to make such a change?
> CEP-15: (Accord) Define a configuration system for Accord to allow overriding
> of internal configs and integrate with Cassandra yaml
> ---
>
> Key: CASSANDRA-18221
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18221
> Project: Cassandra
> Issue Type: Improvement
> Components: Accord
>Reporter: David Capwell
>Assignee: Jacek Lewandowski
>Priority: Normal
> Labels: pull-request-available
> Fix For: 5.x
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Users are used to modifying cassandra via yaml and JMX (for dynamic configs)
> but accord does not integrate with this right now; we should enhance Accord
> to expose configs that are defined in cassandra yaml.
> As an extension on this, we should figure out which configs are “dynamic” and
> allow overriding via JMX or system vtable.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18918) Test failure: secondary_indexes_test.TestPreJoinCallback.test_resume
[
https://issues.apache.org/jira/browse/CASSANDRA-18918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-18918:
-
Bug Category: Parent values: Correctness(12982)Level 1 values: Test
Failure(12990)
Complexity: Normal
Discovered By: DTest
Fix Version/s: 5.0.x
5.x
Severity: Normal
Assignee: Brandon Williams
Status: Open (was: Triage Needed)
> Test failure: secondary_indexes_test.TestPreJoinCallback.test_resume
>
>
> Key: CASSANDRA-18918
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18918
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Brandon Williams
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 5.0.x, 5.x
>
>
> As seen here:
> https://ci-cassandra.apache.org/job/Cassandra-5.0/60/testReport/dtest-novnode.secondary_indexes_test/TestPreJoinCallback/test_resume/
> {quote}
> failed on teardown with "Failed: Unexpected error found in node logs (see
> stdout for full details). Errors: [[node2] 'ERROR
> [Stream-Deserializer-/127.0.0.1:7000-ffd39504] 2023-10-09 16:12:59,153
> StreamSession.java:702 - [Stream #b69e6f80-66be-11ee-b813-3bca425f16a7]
> Socket closed before session completion, peer 127.0.0.1:7000 is probably
> down.\njava.nio.channels.ClosedChannelException: null\n\tat
> org.apache.cassandra.net.AsyncStreamingInputPlus.reBuffer(AsyncStreamingInputPlus.java:119)\n\tat
>
> org.apache.cassandra.io.util.RebufferingInputStream.readByte(RebufferingInputStream.java:178)\n\tat
>
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:49)\n\tat
>
> org.apache.cassandra.streaming.StreamDeserializingTask.run(StreamDeserializingTask.java:59)\n\tat
>
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)\n\tat
> java.base/java.lang.Thread.run(Thread.java:833)']"
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18918) Test failure: secondary_indexes_test.TestPreJoinCallback.test_resume
Brandon Williams created CASSANDRA-18918:
Summary: Test failure:
secondary_indexes_test.TestPreJoinCallback.test_resume
Key: CASSANDRA-18918
URL: https://issues.apache.org/jira/browse/CASSANDRA-18918
Project: Cassandra
Issue Type: Bug
Components: Feature/2i Index
Reporter: Brandon Williams
As seen here:
https://ci-cassandra.apache.org/job/Cassandra-5.0/60/testReport/dtest-novnode.secondary_indexes_test/TestPreJoinCallback/test_resume/
{quote}
failed on teardown with "Failed: Unexpected error found in node logs (see
stdout for full details). Errors: [[node2] 'ERROR
[Stream-Deserializer-/127.0.0.1:7000-ffd39504] 2023-10-09 16:12:59,153
StreamSession.java:702 - [Stream #b69e6f80-66be-11ee-b813-3bca425f16a7] Socket
closed before session completion, peer 127.0.0.1:7000 is probably
down.\njava.nio.channels.ClosedChannelException: null\n\tat
org.apache.cassandra.net.AsyncStreamingInputPlus.reBuffer(AsyncStreamingInputPlus.java:119)\n\tat
org.apache.cassandra.io.util.RebufferingInputStream.readByte(RebufferingInputStream.java:178)\n\tat
org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:49)\n\tat
org.apache.cassandra.streaming.StreamDeserializingTask.run(StreamDeserializingTask.java:59)\n\tat
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)\n\tat
java.base/java.lang.Thread.run(Thread.java:833)']"
{quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18913) Gossip NPE due to shutdown event corrupting empty statuses
[
https://issues.apache.org/jira/browse/CASSANDRA-18913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773748#comment-17773748
]
David Capwell commented on CASSANDRA-18913:
---
Starting commit
CI Results (pending):
||Branch||Source||Circle CI||Jenkins||
|cassandra-4.0|[branch|https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-18913-cassandra-4.0-5C2AE581-B002-4057-8281-7C893EB64CF9]|[build|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-18913-cassandra-4.0-5C2AE581-B002-4057-8281-7C893EB64CF9]|[build|https://ci-cassandra.apache.org/job/Cassandra-devbranch/2614/]|
|cassandra-4.1|[branch|https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-18913-cassandra-4.1-5C2AE581-B002-4057-8281-7C893EB64CF9]|[build|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-18913-cassandra-4.1-5C2AE581-B002-4057-8281-7C893EB64CF9]|[build|unknown]|
|cassandra-5.0|[branch|https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-18913-cassandra-5.0-5C2AE581-B002-4057-8281-7C893EB64CF9]|[build|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-18913-cassandra-5.0-5C2AE581-B002-4057-8281-7C893EB64CF9]|[build|unknown]|
|trunk|[branch|https://github.com/dcapwell/cassandra/tree/commit_remote_branch/CASSANDRA-18913-trunk-5C2AE581-B002-4057-8281-7C893EB64CF9]|[build|https://app.circleci.com/pipelines/github/dcapwell/cassandra?branch=commit_remote_branch%2FCASSANDRA-18913-trunk-5C2AE581-B002-4057-8281-7C893EB64CF9]|[build|unknown]|
> Gossip NPE due to shutdown event corrupting empty statuses
> --
>
> Key: CASSANDRA-18913
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18913
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip, Cluster/Membership
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> When an instance either disables gossip or shuts down we send a gossip
> shutdown message, peers ignore it if the endpoint isn’t known, else it
> mutates its local copy of the state to mark shutdown…
> When an instance restarts it populates gossip with the endpoints found in
> peers, but the state is empty (not null)
> So, there is a fun timing bug…
> * stop node1
> * start node1; at this point all known endpoints before exist in gossip but
> are empty
> * node2 shutdown (gossip shutdown or node, doesn’t matter)
> * node1 sees the shutdown before gossip messages, and gets corruptted
> * node3 tries to join the cluster, fails due to node1 being corrupted
> There are 2 different patterns the NPE can happen with, in this example node1
> and node3 will have different stack traces
> {code}
> org.apache.cassandra.distributed.shared.ShutdownException: Uncaught
> exceptions were thrown during test
> Suppressed: java.lang.NullPointerException: Unable to get HOST_ID;
> HOST_ID is not defined, given EndpointState: HeartBeatState = HeartBeat:
> generation = 0, version = 2147483647, AppStateMap =
> {STATUS=Value(shutdown,true,37), RPC_READY=Value(false,38),
> STATUS_WITH_PORT=Value(shutdown,true,36)}
> at
> org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1218)
> at
> org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1208)
> at
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:3279)
> at
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:2756)
> at
> org.apache.cassandra.gms.Gossiper.markAsShutdown(Gossiper.java:611)
> at
> org.apache.cassandra.gms.GossipShutdownVerbHandler.doVerb(GossipShutdownVerbHandler.java:39)
> at
> org.apache.cassandra.net.InboundSink.lambda$new$0(InboundSink.java:78)
> Suppressed: java.lang.NullPointerException: Unable to get HOST_ID;
> HOST_ID is not defined, given EndpointState: HeartBeatState = HeartBeat:
> generation = 0, version = 2147483647, AppStateMap =
> {STATUS=Value(shutdown,true,37), RPC_READY=Value(false,38),
> STATUS_WITH_PORT=Value(shutdown,true,36)}
> at
> org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1218)
> at
> org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1208)
> at
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:3279)
> at
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:2756)
> at
> org.apache.cassandra.gms.Gossiper.doOnChangeNotifications(Gossiper.java:1762)
>
[jira] [Updated] (CASSANDRA-18221) CEP-15: (Accord) Define a configuration system for Accord to allow overriding of internal configs and integrate with Cassandra yaml
[ https://issues.apache.org/jira/browse/CASSANDRA-18221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jacek Lewandowski updated CASSANDRA-18221: -- Test and Documentation Plan: run regression tests Status: Patch Available (was: In Progress) > CEP-15: (Accord) Define a configuration system for Accord to allow overriding > of internal configs and integrate with Cassandra yaml > --- > > Key: CASSANDRA-18221 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18221 > Project: Cassandra > Issue Type: Improvement > Components: Accord >Reporter: David Capwell >Assignee: Jacek Lewandowski >Priority: Normal > Labels: pull-request-available > Fix For: 5.x > > Time Spent: 1h > Remaining Estimate: 0h > > Users are used to modifying cassandra via yaml and JMX (for dynamic configs) > but accord does not integrate with this right now; we should enhance Accord > to expose configs that are defined in cassandra yaml. > As an extension on this, we should figure out which configs are “dynamic” and > allow overriding via JMX or system vtable. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18913) Gossip NPE due to shutdown event corrupting empty statuses
[
https://issues.apache.org/jira/browse/CASSANDRA-18913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773695#comment-17773695
]
Brandon Williams edited comment on CASSANDRA-18913 at 10/10/23 2:15 PM:
Everything looks good to me, +1.
edit: modulo the expanded imports Maxwell pointed out which can be fixed on
commit
was (Author: brandon.williams):
Everything looks good to me, +1.
> Gossip NPE due to shutdown event corrupting empty statuses
> --
>
> Key: CASSANDRA-18913
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18913
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip, Cluster/Membership
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> When an instance either disables gossip or shuts down we send a gossip
> shutdown message, peers ignore it if the endpoint isn’t known, else it
> mutates its local copy of the state to mark shutdown…
> When an instance restarts it populates gossip with the endpoints found in
> peers, but the state is empty (not null)
> So, there is a fun timing bug…
> * stop node1
> * start node1; at this point all known endpoints before exist in gossip but
> are empty
> * node2 shutdown (gossip shutdown or node, doesn’t matter)
> * node1 sees the shutdown before gossip messages, and gets corruptted
> * node3 tries to join the cluster, fails due to node1 being corrupted
> There are 2 different patterns the NPE can happen with, in this example node1
> and node3 will have different stack traces
> {code}
> org.apache.cassandra.distributed.shared.ShutdownException: Uncaught
> exceptions were thrown during test
> Suppressed: java.lang.NullPointerException: Unable to get HOST_ID;
> HOST_ID is not defined, given EndpointState: HeartBeatState = HeartBeat:
> generation = 0, version = 2147483647, AppStateMap =
> {STATUS=Value(shutdown,true,37), RPC_READY=Value(false,38),
> STATUS_WITH_PORT=Value(shutdown,true,36)}
> at
> org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1218)
> at
> org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1208)
> at
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:3279)
> at
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:2756)
> at
> org.apache.cassandra.gms.Gossiper.markAsShutdown(Gossiper.java:611)
> at
> org.apache.cassandra.gms.GossipShutdownVerbHandler.doVerb(GossipShutdownVerbHandler.java:39)
> at
> org.apache.cassandra.net.InboundSink.lambda$new$0(InboundSink.java:78)
> Suppressed: java.lang.NullPointerException: Unable to get HOST_ID;
> HOST_ID is not defined, given EndpointState: HeartBeatState = HeartBeat:
> generation = 0, version = 2147483647, AppStateMap =
> {STATUS=Value(shutdown,true,37), RPC_READY=Value(false,38),
> STATUS_WITH_PORT=Value(shutdown,true,36)}
> at
> org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1218)
> at
> org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1208)
> at
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:3279)
> at
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:2756)
> at
> org.apache.cassandra.gms.Gossiper.doOnChangeNotifications(Gossiper.java:1762)
> at
> org.apache.cassandra.service.StorageService.onJoin(StorageService.java:3793)
> at
> org.apache.cassandra.gms.Gossiper.handleMajorStateChange(Gossiper.java:1465)
> at
> org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:1678)
> at
> org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:50)
> at
> org.apache.cassandra.net.InboundSink.lambda$new$0(InboundSink.java:78)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18889) DDM's mask_default function should support vectors
[
https://issues.apache.org/jira/browse/CASSANDRA-18889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773701#comment-17773701
]
Andres de la Peña commented on CASSANDRA-18889:
---
Thanks for the reviews.
Committed to 5.0 as
[3b9881bfa65d2806e3918ba2c73198cb17c7071b|https://github.com/apache/cassandra/commit/3b9881bfa65d2806e3918ba2c73198cb17c7071b]
and merged to
[trunk|https://github.com/apache/cassandra/commit/31729b8f9241e842cdc93f9fbb7169b5aba2ca4e].
> DDM's mask_default function should support vectors
> --
>
> Key: CASSANDRA-18889
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18889
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/Dynamic Data Masking, Feature/Vector Search
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 5.0.x, 5.x
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> DDM uses the CQL function {{mask_default}} as the default masking function.
> This function supports all CQL data types except for {{vector}}, which didn't
> exist when DDM was added.
> We should make {{mask_default}} (and thereof default masking) work with
> vector columns.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18889) DDM's mask_default function should support vectors
[
https://issues.apache.org/jira/browse/CASSANDRA-18889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-18889:
--
Fix Version/s: 5.0-alpha2
5.1
(was: 5.x)
(was: 5.0.x)
Since Version: 5.0-alpha1
Source Control Link:
https://github.com/apache/cassandra/commit/3b9881bfa65d2806e3918ba2c73198cb17c7071b
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> DDM's mask_default function should support vectors
> --
>
> Key: CASSANDRA-18889
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18889
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/Dynamic Data Masking, Feature/Vector Search
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 5.0-alpha2, 5.1
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> DDM uses the CQL function {{mask_default}} as the default masking function.
> This function supports all CQL data types except for {{vector}}, which didn't
> exist when DDM was added.
> We should make {{mask_default}} (and thereof default masking) work with
> vector columns.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (4b84c4332c -> 31729b8f92)
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git from 4b84c4332c Merge branch 'cassandra-5.0' into trunk new 3b9881bfa6 Fix vector type to support DDM's mask_default function new 31729b8f92 Merge branch 'cassandra-5.0' into trunk The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt | 1 + .../cassandra/examples/CQL/ddm_create_table_with_udf.cql | 2 +- doc/modules/cassandra/partials/masking_functions.adoc| 11 ++- src/java/org/apache/cassandra/cql3/CQL3Type.java | 16 src/java/org/apache/cassandra/db/marshal/VectorType.java | 13 + .../masking/ColumnMaskQueryWithDefaultTest.java | 9 - .../masking/ColumnMaskQueryWithReplaceTest.java | 7 +++ .../cassandra/cql3/functions/masking/ColumnMaskTest.java | 16 .../cql3/functions/masking/MaskingFunctionTester.java| 16 .../functions/masking/ReplaceMaskingFunctionTest.java| 5 +++-- .../apache/cassandra/db/marshal/AbstractTypeTest.java| 16 +++- 11 files changed, 106 insertions(+), 6 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-5.0 updated: Fix vector type to support DDM's mask_default function
This is an automated email from the ASF dual-hosted git repository.
adelapena pushed a commit to branch cassandra-5.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-5.0 by this push:
new 3b9881bfa6 Fix vector type to support DDM's mask_default function
3b9881bfa6 is described below
commit 3b9881bfa65d2806e3918ba2c73198cb17c7071b
Author: Andrés de la Peña
AuthorDate: Wed Sep 27 21:56:15 2023 +0100
Fix vector type to support DDM's mask_default function
patch by Andrés de la Peña; reviewed by Berenguer Blasi and Maxwell Guo for
CASSANDRA-18889
---
CHANGES.txt | 1 +
.../cassandra/examples/CQL/ddm_create_table_with_udf.cql | 2 +-
doc/modules/cassandra/partials/masking_functions.adoc| 11 ++-
src/java/org/apache/cassandra/cql3/CQL3Type.java | 16
src/java/org/apache/cassandra/db/marshal/VectorType.java | 13 +
.../masking/ColumnMaskQueryWithDefaultTest.java | 9 -
.../masking/ColumnMaskQueryWithReplaceTest.java | 7 +++
.../cassandra/cql3/functions/masking/ColumnMaskTest.java | 16
.../cql3/functions/masking/MaskingFunctionTester.java| 16
.../functions/masking/ReplaceMaskingFunctionTest.java| 5 +++--
.../apache/cassandra/db/marshal/AbstractTypeTest.java| 16 +++-
11 files changed, 106 insertions(+), 6 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 66f613d26d..1e0fe54855 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
5.0-alpha2
+ * Fix vector type to support DDM's mask_default function (CASSANDRA-18889)
* Remove unnecessary reporter-config3 dependency (CASSANDRA-18907)
* Remove support for empty values on the vector data type (CASSANDRA-18876)
* Upgrade Dropwizard Metrics to 4.2.19 (CASSANDRA-14667)
diff --git a/doc/modules/cassandra/examples/CQL/ddm_create_table_with_udf.cql
b/doc/modules/cassandra/examples/CQL/ddm_create_table_with_udf.cql
index 23db48c1b5..85d6555f3a 100644
--- a/doc/modules/cassandra/examples/CQL/ddm_create_table_with_udf.cql
+++ b/doc/modules/cassandra/examples/CQL/ddm_create_table_with_udf.cql
@@ -2,7 +2,7 @@ CREATE FUNCTION redact(input text)
CALLED ON NULL INPUT
RETURNS text
LANGUAGE java
- AS 'return "redacted";
+ AS 'return "redacted";';
CREATE TABLE patients (
id timeuuid PRIMARY KEY,
diff --git a/doc/modules/cassandra/partials/masking_functions.adoc
b/doc/modules/cassandra/partials/masking_functions.adoc
index 43fb25c38d..a91f168067 100644
--- a/doc/modules/cassandra/partials/masking_functions.adoc
+++ b/doc/modules/cassandra/partials/masking_functions.adoc
@@ -10,7 +10,12 @@ Examples:
`mask_null(123)` -> `null`
-| `mask_default(value)` | Replaces its argument by an arbitrary, fixed default
value of the same type. This will be `\***\***` for text values, zero for
numeric values, `false` for booleans, etc.
+| `mask_default(value)` | Replaces its argument by an arbitrary, fixed default
value of the same type.
+This will be `\***\***` for text values, zero for numeric values, `false` for
booleans, etc.
+
+Variable-length multivalued types such as lists, sets and maps are masked as
empty collections.
+
+Fixed-length multivalued types such as tuples, UDTs and vectors are masked by
replacing each of their values by the default masking value of the value type.
Examples:
@@ -18,6 +23,10 @@ Examples:
`mask_default(123)` -> `0`
+`mask_default((list) [1, 2, 3])` -> `[]`
+
+`mask_default((vector) [1, 2, 3])` -> `[0, 0, 0]`
+
| `mask_replace(value, replacement])` | Replaces the first argument by the
replacement value on the second argument. The replacement value needs to have
the same type as the replaced value.
Examples:
diff --git a/src/java/org/apache/cassandra/cql3/CQL3Type.java
b/src/java/org/apache/cassandra/cql3/CQL3Type.java
index 074a8e515e..6b8b97877d 100644
--- a/src/java/org/apache/cassandra/cql3/CQL3Type.java
+++ b/src/java/org/apache/cassandra/cql3/CQL3Type.java
@@ -20,6 +20,7 @@ package org.apache.cassandra.cql3;
import java.nio.ByteBuffer;
import java.util.ArrayList;
import java.util.List;
+import java.util.Objects;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -566,6 +567,21 @@ public interface CQL3Type
return sb.toString();
}
+@Override
+public boolean equals(Object o)
+{
+if (this == o) return true;
+if (o == null || getClass() != o.getClass()) return false;
+Vector vector = (Vector) o;
+return Objects.equals(type, vector.type);
+}
+
+@Override
+public int hashCode()
+{
+return Objects.hash(type);
+}
+
@Override
public String toString()
{
diff --git a/src/java/org/apache/cassandra/db/marshal/VectorType.java
[cassandra] 01/01: Merge branch 'cassandra-5.0' into trunk
This is an automated email from the ASF dual-hosted git repository. adelapena pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 31729b8f9241e842cdc93f9fbb7169b5aba2ca4e Merge: 4b84c4332c 3b9881bfa6 Author: Andrés de la Peña AuthorDate: Tue Oct 10 15:03:06 2023 +0100 Merge branch 'cassandra-5.0' into trunk CHANGES.txt | 1 + .../cassandra/examples/CQL/ddm_create_table_with_udf.cql | 2 +- doc/modules/cassandra/partials/masking_functions.adoc| 11 ++- src/java/org/apache/cassandra/cql3/CQL3Type.java | 16 src/java/org/apache/cassandra/db/marshal/VectorType.java | 13 + .../masking/ColumnMaskQueryWithDefaultTest.java | 9 - .../masking/ColumnMaskQueryWithReplaceTest.java | 7 +++ .../cassandra/cql3/functions/masking/ColumnMaskTest.java | 16 .../cql3/functions/masking/MaskingFunctionTester.java| 16 .../functions/masking/ReplaceMaskingFunctionTest.java| 5 +++-- .../apache/cassandra/db/marshal/AbstractTypeTest.java| 16 +++- 11 files changed, 106 insertions(+), 6 deletions(-) diff --cc CHANGES.txt index 03ebe157c0,1e0fe54855..1b377ce775 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,8 -1,5 +1,9 @@@ -5.0-alpha2 +5.1 + * Add ELAPSED command to cqlsh (CASSANDRA-18861) + * Add the ability to disable bulk loading of SSTables (CASSANDRA-18781) + * Clean up obsolete functions and simplify cql_version handling in cqlsh (CASSANDRA-18787) +Merged from 5.0: + * Fix vector type to support DDM's mask_default function (CASSANDRA-18889) * Remove unnecessary reporter-config3 dependency (CASSANDRA-18907) * Remove support for empty values on the vector data type (CASSANDRA-18876) * Upgrade Dropwizard Metrics to 4.2.19 (CASSANDRA-14667) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18534) Make sstable format configurable per table
[
https://issues.apache.org/jira/browse/CASSANDRA-18534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Branimir Lambov updated CASSANDRA-18534:
Fix Version/s: 5.0
(was: 5.x)
> Make sstable format configurable per table
> --
>
> Key: CASSANDRA-18534
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18534
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema, Local/SSTable
>Reporter: Branimir Lambov
>Assignee: Maxwell Guo
>Priority: Normal
> Fix For: 5.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Some SSTable format settings need to be configurable per table for better
> efficiency. This includes:
> - {{row_index_granularity}}
> - {{bloom_filter_fp_chance}}
> - {{crc_check_chance}}
> - {{min/max_index_interval}}
> Some of these are currently configurable using direct properties of tables.
> Having them as format properties makes better sense and should also support
> specifying useable combinations of settings, e.g.
> {code:java}
> CREATE TABLE ... WITH sstable_format = "bti-fast";
> CREATE TABLE ... WITH sstable_format = "bti-small";
> {code}
> where {{bti-fast}} and {{bti-small}} can be defined in {{cassandra.yaml}}
> e.g. as
> {code:java}
> sstable.format.options:
> - bti-fast:
> row_index_granularity: 1kiB
> bloom_filter_fp_chance: 0.01
> - bti-small:
> row_index_granularity: 32kiB
> bloom_filter_fp_chance: 0.1
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18534) Make sstable format configurable per table
[
https://issues.apache.org/jira/browse/CASSANDRA-18534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773696#comment-17773696
]
Branimir Lambov commented on CASSANDRA-18534:
-
bq. Also, do you think it is possible and useful to make sstable_format contain
custom parameters?
_All_ of the parameters to the SSTable format are custom, i.e. format-specific.
This is also the qualifying condition for something to be moved into the format
config: if you can imagine an SSTable format that does not need that flag, then
it belongs to the format. E.g. bloom-filter-less formats do not need
{{bloom_filter_fp_chance}}, and (even though they are not a feature of writing
an SSTable) only {{BIG}} requires key cache options. Unless we are certain that
CRC is the only way a format could defend against bit rot, {{check_crc_chance}}
is also a format-specific property.
> Make sstable format configurable per table
> --
>
> Key: CASSANDRA-18534
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18534
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema, Local/SSTable
>Reporter: Branimir Lambov
>Assignee: Maxwell Guo
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Some SSTable format settings need to be configurable per table for better
> efficiency. This includes:
> - {{row_index_granularity}}
> - {{bloom_filter_fp_chance}}
> - {{crc_check_chance}}
> - {{min/max_index_interval}}
> Some of these are currently configurable using direct properties of tables.
> Having them as format properties makes better sense and should also support
> specifying useable combinations of settings, e.g.
> {code:java}
> CREATE TABLE ... WITH sstable_format = "bti-fast";
> CREATE TABLE ... WITH sstable_format = "bti-small";
> {code}
> where {{bti-fast}} and {{bti-small}} can be defined in {{cassandra.yaml}}
> e.g. as
> {code:java}
> sstable.format.options:
> - bti-fast:
> row_index_granularity: 1kiB
> bloom_filter_fp_chance: 0.01
> - bti-small:
> row_index_granularity: 32kiB
> bloom_filter_fp_chance: 0.1
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18913) Gossip NPE due to shutdown event corrupting empty statuses
[
https://issues.apache.org/jira/browse/CASSANDRA-18913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-18913:
-
Status: Ready to Commit (was: Review In Progress)
Everything looks good to me, +1.
> Gossip NPE due to shutdown event corrupting empty statuses
> --
>
> Key: CASSANDRA-18913
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18913
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip, Cluster/Membership
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> When an instance either disables gossip or shuts down we send a gossip
> shutdown message, peers ignore it if the endpoint isn’t known, else it
> mutates its local copy of the state to mark shutdown…
> When an instance restarts it populates gossip with the endpoints found in
> peers, but the state is empty (not null)
> So, there is a fun timing bug…
> * stop node1
> * start node1; at this point all known endpoints before exist in gossip but
> are empty
> * node2 shutdown (gossip shutdown or node, doesn’t matter)
> * node1 sees the shutdown before gossip messages, and gets corruptted
> * node3 tries to join the cluster, fails due to node1 being corrupted
> There are 2 different patterns the NPE can happen with, in this example node1
> and node3 will have different stack traces
> {code}
> org.apache.cassandra.distributed.shared.ShutdownException: Uncaught
> exceptions were thrown during test
> Suppressed: java.lang.NullPointerException: Unable to get HOST_ID;
> HOST_ID is not defined, given EndpointState: HeartBeatState = HeartBeat:
> generation = 0, version = 2147483647, AppStateMap =
> {STATUS=Value(shutdown,true,37), RPC_READY=Value(false,38),
> STATUS_WITH_PORT=Value(shutdown,true,36)}
> at
> org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1218)
> at
> org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1208)
> at
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:3279)
> at
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:2756)
> at
> org.apache.cassandra.gms.Gossiper.markAsShutdown(Gossiper.java:611)
> at
> org.apache.cassandra.gms.GossipShutdownVerbHandler.doVerb(GossipShutdownVerbHandler.java:39)
> at
> org.apache.cassandra.net.InboundSink.lambda$new$0(InboundSink.java:78)
> Suppressed: java.lang.NullPointerException: Unable to get HOST_ID;
> HOST_ID is not defined, given EndpointState: HeartBeatState = HeartBeat:
> generation = 0, version = 2147483647, AppStateMap =
> {STATUS=Value(shutdown,true,37), RPC_READY=Value(false,38),
> STATUS_WITH_PORT=Value(shutdown,true,36)}
> at
> org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1218)
> at
> org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1208)
> at
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:3279)
> at
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:2756)
> at
> org.apache.cassandra.gms.Gossiper.doOnChangeNotifications(Gossiper.java:1762)
> at
> org.apache.cassandra.service.StorageService.onJoin(StorageService.java:3793)
> at
> org.apache.cassandra.gms.Gossiper.handleMajorStateChange(Gossiper.java:1465)
> at
> org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:1678)
> at
> org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:50)
> at
> org.apache.cassandra.net.InboundSink.lambda$new$0(InboundSink.java:78)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18913) Gossip NPE due to shutdown event corrupting empty statuses
[
https://issues.apache.org/jira/browse/CASSANDRA-18913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773642#comment-17773642
]
Brandon Williams edited comment on CASSANDRA-18913 at 10/10/23 1:31 PM:
||Branch||CI||
|[4.0|https://github.com/driftx/cassandra/tree/CASSANDRA-18913-4.0]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/1329/workflows/19db1763-1c84-4567-ac52-84d48608b1ce],
[j11|https://app.circleci.com/pipelines/github/driftx/cassandra/1329/workflows/e3a4dc3a-0208-45e1-b0b6-67ccf1c9e37d]|
|[4.1|https://github.com/driftx/cassandra/tree/CASSANDRA-18913-4.1]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/1328/workflows/6620e30c-3462-4f55-96b6-1e5ab7d0d91d],
[j11|https://app.circleci.com/pipelines/github/driftx/cassandra/1328/workflows/e9be2633-010b-4d04-b2e7-4353c6ec028e]|
|[5.0|https://github.com/driftx/cassandra/tree/CASSANDRA-18913-5.0]|[j11|https://app.circleci.com/pipelines/github/driftx/cassandra/1330/workflows/8a0a955a-8033-4c19-848c-4ce0e55c0b51],
[j17|https://app.circleci.com/pipelines/github/driftx/cassandra/1330/workflows/7ce35fa6-4421-4fb6-8879-e34400c0778e]|
was (Author: brandon.williams):
||Branch||CI||
|[4.0|https://github.com/driftx/cassandra/tree/CASSANDRA-18913-4.0]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/1329/workflows/9615e832-31da-4751-b8ed-859f3752925d],
[j11|https://app.circleci.com/pipelines/github/driftx/cassandra/1329/workflows/e3a4dc3a-0208-45e1-b0b6-67ccf1c9e37d]|
|[4.1|https://github.com/driftx/cassandra/tree/CASSANDRA-18913-4.1]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/1328/workflows/6620e30c-3462-4f55-96b6-1e5ab7d0d91d],
[j11|https://app.circleci.com/pipelines/github/driftx/cassandra/1328/workflows/e9be2633-010b-4d04-b2e7-4353c6ec028e]|
|[5.0|https://github.com/driftx/cassandra/tree/CASSANDRA-18913-5.0]|[j11|https://app.circleci.com/pipelines/github/driftx/cassandra/1330/workflows/8a0a955a-8033-4c19-848c-4ce0e55c0b51],
[j17|https://app.circleci.com/pipelines/github/driftx/cassandra/1330/workflows/7ce35fa6-4421-4fb6-8879-e34400c0778e]|
> Gossip NPE due to shutdown event corrupting empty statuses
> --
>
> Key: CASSANDRA-18913
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18913
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip, Cluster/Membership
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> When an instance either disables gossip or shuts down we send a gossip
> shutdown message, peers ignore it if the endpoint isn’t known, else it
> mutates its local copy of the state to mark shutdown…
> When an instance restarts it populates gossip with the endpoints found in
> peers, but the state is empty (not null)
> So, there is a fun timing bug…
> * stop node1
> * start node1; at this point all known endpoints before exist in gossip but
> are empty
> * node2 shutdown (gossip shutdown or node, doesn’t matter)
> * node1 sees the shutdown before gossip messages, and gets corruptted
> * node3 tries to join the cluster, fails due to node1 being corrupted
> There are 2 different patterns the NPE can happen with, in this example node1
> and node3 will have different stack traces
> {code}
> org.apache.cassandra.distributed.shared.ShutdownException: Uncaught
> exceptions were thrown during test
> Suppressed: java.lang.NullPointerException: Unable to get HOST_ID;
> HOST_ID is not defined, given EndpointState: HeartBeatState = HeartBeat:
> generation = 0, version = 2147483647, AppStateMap =
> {STATUS=Value(shutdown,true,37), RPC_READY=Value(false,38),
> STATUS_WITH_PORT=Value(shutdown,true,36)}
> at
> org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1218)
> at
> org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1208)
> at
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:3279)
> at
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:2756)
> at
> org.apache.cassandra.gms.Gossiper.markAsShutdown(Gossiper.java:611)
> at
> org.apache.cassandra.gms.GossipShutdownVerbHandler.doVerb(GossipShutdownVerbHandler.java:39)
> at
> org.apache.cassandra.net.InboundSink.lambda$new$0(InboundSink.java:78)
> Suppressed: java.lang.NullPointerException: Unable to get HOST_ID;
> HOST_ID is not defined, given EndpointState: HeartBeatState = HeartBeat:
> generation = 0, version = 2147483647, AppStateMap =
> {STATUS=Value(shutdown,true,37), RPC_READY=Value(false,38),
> STATUS_WITH_PORT=Value(shutdown,true,36)}
>
[jira] [Updated] (CASSANDRA-18912) Specify "since" in all Deprecated annotations
[
https://issues.apache.org/jira/browse/CASSANDRA-18912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-18912:
--
Test and Documentation Plan: CI
Status: Patch Available (was: In Progress)
https://github.com/apache/cassandra/pull/2789
> Specify "since" in all Deprecated annotations
> -
>
> Key: CASSANDRA-18912
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18912
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Core
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
> Time Spent: 10m
> Remaining Estimate: 0h
>
> It would be great if we introduced in 5.0 a change in Deprecated annotations
> like this:
> {code}
> @Deprecated(since = "4.0")
> {code}
> or
> {code}
> @Deprecated(since = "3.11")
> {code}
> The reasoning behind this is that as of now, it is pretty cumbersome to
> figure out what can be removed on the next major version. It has to be,
> basically, done manually every time.
> There is also this parameter available:
> {code}
> @Deprecated(forRemoval = true / false)
> {code}
> which indicates whether the annotated element is subject to removal in a
> future version so we do not need to think about this every time if it is
> eligible for deletion in a next major or not.
> We could then have a check which would ensure that we are not releasing a
> next major with some deprecations introduced two majors before.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18913) Gossip NPE due to shutdown event corrupting empty statuses
[
https://issues.apache.org/jira/browse/CASSANDRA-18913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773642#comment-17773642
]
Brandon Williams edited comment on CASSANDRA-18913 at 10/10/23 10:58 AM:
-
||Branch||CI||
|[4.0|https://github.com/driftx/cassandra/tree/CASSANDRA-18913-4.0]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/1329/workflows/9615e832-31da-4751-b8ed-859f3752925d],
[j11|https://app.circleci.com/pipelines/github/driftx/cassandra/1329/workflows/e3a4dc3a-0208-45e1-b0b6-67ccf1c9e37d]|
|[4.1|https://github.com/driftx/cassandra/tree/CASSANDRA-18913-4.1]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/1328/workflows/6620e30c-3462-4f55-96b6-1e5ab7d0d91d],
[j11|https://app.circleci.com/pipelines/github/driftx/cassandra/1328/workflows/e9be2633-010b-4d04-b2e7-4353c6ec028e]|
|[5.0|https://github.com/driftx/cassandra/tree/CASSANDRA-18913-5.0]|[j11|https://app.circleci.com/pipelines/github/driftx/cassandra/1330/workflows/8a0a955a-8033-4c19-848c-4ce0e55c0b51],
[j17|https://app.circleci.com/pipelines/github/driftx/cassandra/1330/workflows/7ce35fa6-4421-4fb6-8879-e34400c0778e]|
was (Author: brandon.williams):
{color:#00}||Branch||CI|| {color}
|[4.0|https://github.com/driftx/cassandra/tree/CASSANDRA-18913-4.0]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/1329/workflows/9615e832-31da-4751-b8ed-859f3752925d],
[j11|https://app.circleci.com/pipelines/github/drift
x/cassandra/1329/workflows/e3a4dc3a-0208-45e1-b0b6-67ccf1c9e37d]|
|[4.1|https://github.com/driftx/cassandra/tree/CASSANDRA-18913-4.1]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/1328/workflows/6620e30c-3462-4f55-96b6-1e5ab7d0d91d],
[j11|https://app.circleci.com/pipelines/github/drift
x/cassandra/1328/workflows/e9be2633-010b-4d04-b2e7-4353c6ec028e]|
|[5.0|https://github.com/driftx/cassandra/tree/CASSANDRA-18913-5.0]|[j11|https://app.circleci.com/pipelines/github/driftx/cassandra/1330/workflows/8a0a955a-8033-4c19-848c-4ce0e55c0b51],
[j17|https://app.circleci.com/pipelines/github/drif
tx/cassandra/1330/workflows/7ce35fa6-4421-4fb6-8879-e34400c0778e]|
> Gossip NPE due to shutdown event corrupting empty statuses
> --
>
> Key: CASSANDRA-18913
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18913
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip, Cluster/Membership
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> When an instance either disables gossip or shuts down we send a gossip
> shutdown message, peers ignore it if the endpoint isn’t known, else it
> mutates its local copy of the state to mark shutdown…
> When an instance restarts it populates gossip with the endpoints found in
> peers, but the state is empty (not null)
> So, there is a fun timing bug…
> * stop node1
> * start node1; at this point all known endpoints before exist in gossip but
> are empty
> * node2 shutdown (gossip shutdown or node, doesn’t matter)
> * node1 sees the shutdown before gossip messages, and gets corruptted
> * node3 tries to join the cluster, fails due to node1 being corrupted
> There are 2 different patterns the NPE can happen with, in this example node1
> and node3 will have different stack traces
> {code}
> org.apache.cassandra.distributed.shared.ShutdownException: Uncaught
> exceptions were thrown during test
> Suppressed: java.lang.NullPointerException: Unable to get HOST_ID;
> HOST_ID is not defined, given EndpointState: HeartBeatState = HeartBeat:
> generation = 0, version = 2147483647, AppStateMap =
> {STATUS=Value(shutdown,true,37), RPC_READY=Value(false,38),
> STATUS_WITH_PORT=Value(shutdown,true,36)}
> at
> org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1218)
> at
> org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1208)
> at
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:3279)
> at
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:2756)
> at
> org.apache.cassandra.gms.Gossiper.markAsShutdown(Gossiper.java:611)
> at
> org.apache.cassandra.gms.GossipShutdownVerbHandler.doVerb(GossipShutdownVerbHandler.java:39)
> at
> org.apache.cassandra.net.InboundSink.lambda$new$0(InboundSink.java:78)
> Suppressed: java.lang.NullPointerException: Unable to get HOST_ID;
> HOST_ID is not defined, given EndpointState: HeartBeatState = HeartBeat:
> generation = 0, version = 2147483647, AppStateMap =
> {STATUS=Value(shutdown,true,37), RPC_READY=Value(false,38),
>
[jira] [Commented] (CASSANDRA-18913) Gossip NPE due to shutdown event corrupting empty statuses
[
https://issues.apache.org/jira/browse/CASSANDRA-18913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773642#comment-17773642
]
Brandon Williams commented on CASSANDRA-18913:
--
{color:#00}||Branch||CI|| {color}
|[4.0|https://github.com/driftx/cassandra/tree/CASSANDRA-18913-4.0]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/1329/workflows/9615e832-31da-4751-b8ed-859f3752925d],
[j11|https://app.circleci.com/pipelines/github/drift
x/cassandra/1329/workflows/e3a4dc3a-0208-45e1-b0b6-67ccf1c9e37d]|
|[4.1|https://github.com/driftx/cassandra/tree/CASSANDRA-18913-4.1]|[j8|https://app.circleci.com/pipelines/github/driftx/cassandra/1328/workflows/6620e30c-3462-4f55-96b6-1e5ab7d0d91d],
[j11|https://app.circleci.com/pipelines/github/drift
x/cassandra/1328/workflows/e9be2633-010b-4d04-b2e7-4353c6ec028e]|
|[5.0|https://github.com/driftx/cassandra/tree/CASSANDRA-18913-5.0]|[j11|https://app.circleci.com/pipelines/github/driftx/cassandra/1330/workflows/8a0a955a-8033-4c19-848c-4ce0e55c0b51],
[j17|https://app.circleci.com/pipelines/github/drif
tx/cassandra/1330/workflows/7ce35fa6-4421-4fb6-8879-e34400c0778e]|
> Gossip NPE due to shutdown event corrupting empty statuses
> --
>
> Key: CASSANDRA-18913
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18913
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip, Cluster/Membership
>Reporter: David Capwell
>Assignee: David Capwell
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> When an instance either disables gossip or shuts down we send a gossip
> shutdown message, peers ignore it if the endpoint isn’t known, else it
> mutates its local copy of the state to mark shutdown…
> When an instance restarts it populates gossip with the endpoints found in
> peers, but the state is empty (not null)
> So, there is a fun timing bug…
> * stop node1
> * start node1; at this point all known endpoints before exist in gossip but
> are empty
> * node2 shutdown (gossip shutdown or node, doesn’t matter)
> * node1 sees the shutdown before gossip messages, and gets corruptted
> * node3 tries to join the cluster, fails due to node1 being corrupted
> There are 2 different patterns the NPE can happen with, in this example node1
> and node3 will have different stack traces
> {code}
> org.apache.cassandra.distributed.shared.ShutdownException: Uncaught
> exceptions were thrown during test
> Suppressed: java.lang.NullPointerException: Unable to get HOST_ID;
> HOST_ID is not defined, given EndpointState: HeartBeatState = HeartBeat:
> generation = 0, version = 2147483647, AppStateMap =
> {STATUS=Value(shutdown,true,37), RPC_READY=Value(false,38),
> STATUS_WITH_PORT=Value(shutdown,true,36)}
> at
> org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1218)
> at
> org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1208)
> at
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:3279)
> at
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:2756)
> at
> org.apache.cassandra.gms.Gossiper.markAsShutdown(Gossiper.java:611)
> at
> org.apache.cassandra.gms.GossipShutdownVerbHandler.doVerb(GossipShutdownVerbHandler.java:39)
> at
> org.apache.cassandra.net.InboundSink.lambda$new$0(InboundSink.java:78)
> Suppressed: java.lang.NullPointerException: Unable to get HOST_ID;
> HOST_ID is not defined, given EndpointState: HeartBeatState = HeartBeat:
> generation = 0, version = 2147483647, AppStateMap =
> {STATUS=Value(shutdown,true,37), RPC_READY=Value(false,38),
> STATUS_WITH_PORT=Value(shutdown,true,36)}
> at
> org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1218)
> at
> org.apache.cassandra.gms.Gossiper.getHostId(Gossiper.java:1208)
> at
> org.apache.cassandra.service.StorageService.handleStateNormal(StorageService.java:3279)
> at
> org.apache.cassandra.service.StorageService.onChange(StorageService.java:2756)
> at
> org.apache.cassandra.gms.Gossiper.doOnChangeNotifications(Gossiper.java:1762)
> at
> org.apache.cassandra.service.StorageService.onJoin(StorageService.java:3793)
> at
> org.apache.cassandra.gms.Gossiper.handleMajorStateChange(Gossiper.java:1465)
> at
> org.apache.cassandra.gms.Gossiper.applyStateLocally(Gossiper.java:1678)
> at
> org.apache.cassandra.gms.GossipDigestAck2VerbHandler.doVerb(GossipDigestAck2VerbHandler.java:50)
> at
> org.apache.cassandra.net.InboundSink.lambda$new$0(InboundSink.java:78)
> {code}
[jira] [Updated] (CASSANDRA-18580) Baseline Metrics for Accord Transactions
[
https://issues.apache.org/jira/browse/CASSANDRA-18580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jacek Lewandowski updated CASSANDRA-18580:
--
Reviewers: Caleb Rackliffe, David Capwell, Henrik Ingo (was: Caleb
Rackliffe)
> Baseline Metrics for Accord Transactions
>
>
> Key: CASSANDRA-18580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18580
> Project: Cassandra
> Issue Type: Improvement
> Components: Accord, Observability/JMX, Observability/Metrics
>Reporter: Caleb Rackliffe
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 8h 40m
> Remaining Estimate: 0h
>
> Based on some conversations w/ [~benedict] and [~dcapwell], this is the
> initial set of metrics that seem both feasible to implement and useful as we
> monitor the health of a cluster performing Accord transactions:
> 1.) Basic latency metrics for transactions up to the point of COMMIT and rate
> metrics for preemption, failure, and timeouts at the coordinator.
> This has already been implemented and split into read and write-specific
> metrics. Our position for now is that metrics around preemption should be
> useful in place of a more difficult-to-define metric around how many
> transactions are completed via recovery.
> 2.) Global cache stats/metrics (i.e. aggregated for all command stores)
> We could, at some point, build metrics scoped to a specific {{CommandStore}},
> but they might be awkward in MBean/JMX space, as command stores would have to
> be identified by ID or key range…the latter possibly being able to change
> across epochs. (An alternative would be just publishing command
> store-specific stats on-demand to a virtual table instead.)
> 3.) Something like a decaying histogram of the number of dependencies per
> transaction (or per partial transaction).
> If this is getting worse over time, it could be useful to know/be a way for
> us to detect that contention is increasing. We should be able to hook this up
> to {{ProgressLog}} notifications. Recording for PartialDeps/PartialTxn (which
> ProgressLog gives us at pre-accept) seems acceptable, given this is a
> directional metric.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18580) Baseline Metrics for Accord Transactions
[
https://issues.apache.org/jira/browse/CASSANDRA-18580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jacek Lewandowski updated CASSANDRA-18580:
--
Source Control Link:
https://github.com/apache/cassandra/commit/79b3dc07e7f01a315df8db3724db4c1064b8eac9
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> Baseline Metrics for Accord Transactions
>
>
> Key: CASSANDRA-18580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18580
> Project: Cassandra
> Issue Type: Improvement
> Components: Accord, Observability/JMX, Observability/Metrics
>Reporter: Caleb Rackliffe
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 8h 40m
> Remaining Estimate: 0h
>
> Based on some conversations w/ [~benedict] and [~dcapwell], this is the
> initial set of metrics that seem both feasible to implement and useful as we
> monitor the health of a cluster performing Accord transactions:
> 1.) Basic latency metrics for transactions up to the point of COMMIT and rate
> metrics for preemption, failure, and timeouts at the coordinator.
> This has already been implemented and split into read and write-specific
> metrics. Our position for now is that metrics around preemption should be
> useful in place of a more difficult-to-define metric around how many
> transactions are completed via recovery.
> 2.) Global cache stats/metrics (i.e. aggregated for all command stores)
> We could, at some point, build metrics scoped to a specific {{CommandStore}},
> but they might be awkward in MBean/JMX space, as command stores would have to
> be identified by ID or key range…the latter possibly being able to change
> across epochs. (An alternative would be just publishing command
> store-specific stats on-demand to a virtual table instead.)
> 3.) Something like a decaying histogram of the number of dependencies per
> transaction (or per partial transaction).
> If this is getting worse over time, it could be useful to know/be a way for
> us to detect that contention is increasing. We should be able to hook this up
> to {{ProgressLog}} notifications. Recording for PartialDeps/PartialTxn (which
> ProgressLog gives us at pre-accept) seems acceptable, given this is a
> directional metric.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18580) Baseline Metrics for Accord Transactions
[
https://issues.apache.org/jira/browse/CASSANDRA-18580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jacek Lewandowski updated CASSANDRA-18580:
--
Status: Ready to Commit (was: Review In Progress)
> Baseline Metrics for Accord Transactions
>
>
> Key: CASSANDRA-18580
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18580
> Project: Cassandra
> Issue Type: Improvement
> Components: Accord, Observability/JMX, Observability/Metrics
>Reporter: Caleb Rackliffe
>Assignee: Jacek Lewandowski
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 8h 40m
> Remaining Estimate: 0h
>
> Based on some conversations w/ [~benedict] and [~dcapwell], this is the
> initial set of metrics that seem both feasible to implement and useful as we
> monitor the health of a cluster performing Accord transactions:
> 1.) Basic latency metrics for transactions up to the point of COMMIT and rate
> metrics for preemption, failure, and timeouts at the coordinator.
> This has already been implemented and split into read and write-specific
> metrics. Our position for now is that metrics around preemption should be
> useful in place of a more difficult-to-define metric around how many
> transactions are completed via recovery.
> 2.) Global cache stats/metrics (i.e. aggregated for all command stores)
> We could, at some point, build metrics scoped to a specific {{CommandStore}},
> but they might be awkward in MBean/JMX space, as command stores would have to
> be identified by ID or key range…the latter possibly being able to change
> across epochs. (An alternative would be just publishing command
> store-specific stats on-demand to a virtual table instead.)
> 3.) Something like a decaying histogram of the number of dependencies per
> transaction (or per partial transaction).
> If this is getting worse over time, it could be useful to know/be a way for
> us to detect that contention is increasing. We should be able to hook this up
> to {{ProgressLog}} notifications. Recording for PartialDeps/PartialTxn (which
> ProgressLog gives us at pre-accept) seems acceptable, given this is a
> directional metric.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cep-15-accord updated (718e24ab95 -> 79b3dc07e7)
This is an automated email from the ASF dual-hosted git repository.
jlewandowski pushed a change to branch cep-15-accord
in repository https://gitbox.apache.org/repos/asf/cassandra.git
from 718e24ab95 Fix Propagate serializer
add 79b3dc07e7 CEP-15: Accord metrics
No new revisions were added by this update.
Summary of changes:
modules/accord | 2 +-
.../apache/cassandra/db/virtual/CachesTable.java | 8 +-
.../metrics/AccordClientRequestMetrics.java| 4 -
.../apache/cassandra/metrics/AccordMetrics.java| 312 +
...stMetrics.java => AccordStateCacheMetrics.java} | 32 ++-
.../cassandra/metrics/CacheAccessMetrics.java | 73 +
.../org/apache/cassandra/metrics/CacheMetrics.java | 89 +-
.../apache/cassandra/metrics/CacheSizeMetrics.java | 63 +
.../apache/cassandra/metrics/RatioGaugeSet.java| 121
.../service/accord/AccordCommandStore.java | 44 ++-
.../service/accord/AccordCommandStores.java| 39 ++-
.../cassandra/service/accord/AccordService.java| 3 +-
.../cassandra/service/accord/AccordStateCache.java | 133 +
.../cassandra/service/accord/api/AccordAgent.java | 8 +
.../distributed/test/accord/AccordMetricsTest.java | 273 ++
.../test/microbench/CachingBenchTest.java | 4 +-
.../org/apache/cassandra/cql3/KeyCacheCqlTest.java | 4 +-
.../compaction/CompactionAccordIteratorsTest.java | 6 +-
.../cassandra/io/sstable/LargePartitionsTest.java | 10 +-
.../apache/cassandra/metrics/CacheMetricsTest.java | 14 +-
.../service/accord/AccordCommandTest.java | 4 +-
.../service/accord/AccordStateCacheTest.java | 179 ++--
.../cassandra/service/accord/AccordTestUtils.java | 6 +-
.../service/accord/async/AsyncLoaderTest.java | 4 +-
.../service/accord/async/AsyncOperationTest.java | 8 +-
25 files changed, 1208 insertions(+), 235 deletions(-)
create mode 100644 src/java/org/apache/cassandra/metrics/AccordMetrics.java
copy src/java/org/apache/cassandra/metrics/{AccordClientRequestMetrics.java =>
AccordStateCacheMetrics.java} (53%)
create mode 100644
src/java/org/apache/cassandra/metrics/CacheAccessMetrics.java
create mode 100644 src/java/org/apache/cassandra/metrics/CacheSizeMetrics.java
create mode 100644 src/java/org/apache/cassandra/metrics/RatioGaugeSet.java
create mode 100644
test/distributed/org/apache/cassandra/distributed/test/accord/AccordMetricsTest.java
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18889) DDM's mask_default function should support vectors
[
https://issues.apache.org/jira/browse/CASSANDRA-18889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andres de la Peña updated CASSANDRA-18889:
--
Status: Ready to Commit (was: Review In Progress)
> DDM's mask_default function should support vectors
> --
>
> Key: CASSANDRA-18889
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18889
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/Dynamic Data Masking, Feature/Vector Search
>Reporter: Andres de la Peña
>Assignee: Andres de la Peña
>Priority: Normal
> Fix For: 5.0.x, 5.x
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> DDM uses the CQL function {{mask_default}} as the default masking function.
> This function supports all CQL data types except for {{vector}}, which didn't
> exist when DDM was added.
> We should make {{mask_default}} (and thereof default masking) work with
> vector columns.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-accord] branch trunk updated: CEP-15: Accord metrics
This is an automated email from the ASF dual-hosted git repository.
jlewandowski pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-accord.git
The following commit(s) were added to refs/heads/trunk by this push:
new 0419858b CEP-15: Accord metrics
0419858b is described below
commit 0419858bd1f6761f08fd1369477f7c142f5bbb4f
Author: Jacek Lewandowski
AuthorDate: Thu Aug 3 16:39:31 2023 +0200
CEP-15: Accord metrics
Patch by Jacek Lewandowski, reviewed by Caleb Rackliffe and Henrik Ingo for
CASSANDRA-18580
---
accord-core/src/main/java/accord/api/Agent.java| 5 ++
.../src/main/java/accord/api/EventsListener.java | 71 ++
.../accord/coordinate/CoordinatePreAccept.java | 16 +
.../accord/coordinate/CoordinateTransaction.java | 3 +
.../src/main/java/accord/coordinate/Recover.java | 24 ++--
.../java/accord/coordinate/RecoverWithRoute.java | 4 +-
.../main/java/accord/impl/SimpleProgressLog.java | 5 +-
.../src/main/java/accord/local/CommandStores.java | 7 +++
.../src/main/java/accord/local/Commands.java | 6 ++
.../src/main/java/accord/primitives/Timestamp.java | 4 +-
10 files changed, 135 insertions(+), 10 deletions(-)
diff --git a/accord-core/src/main/java/accord/api/Agent.java
b/accord-core/src/main/java/accord/api/Agent.java
index 717ae87a..ff8ab047 100644
--- a/accord-core/src/main/java/accord/api/Agent.java
+++ b/accord-core/src/main/java/accord/api/Agent.java
@@ -60,4 +60,9 @@ public interface Agent extends UncaughtExceptionListener
boolean isExpired(TxnId initiated, long now);
Txn emptyTxn(Txn.Kind kind, Seekables keysOrRanges);
+
+default EventsListener metricsEventsListener()
+{
+return EventsListener.NOOP;
+}
}
diff --git a/accord-core/src/main/java/accord/api/EventsListener.java
b/accord-core/src/main/java/accord/api/EventsListener.java
new file mode 100644
index ..67a32942
--- /dev/null
+++ b/accord-core/src/main/java/accord/api/EventsListener.java
@@ -0,0 +1,71 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package accord.api;
+
+import accord.local.Command;
+import accord.primitives.Deps;
+import accord.primitives.Timestamp;
+import accord.primitives.TxnId;
+
+public interface EventsListener
+{
+default void onCommitted(Command cmd)
+{
+}
+
+default void onExecuted(Command cmd)
+{
+}
+
+default void onApplied(Command cmd, long applyStartTimestamp)
+{
+}
+
+default void onFastPathTaken(TxnId txnId, Deps deps)
+{
+}
+
+default void onSlowPathTaken(TxnId txnId, Deps deps)
+{
+}
+
+default void onRecover(TxnId txnId, Timestamp recoveryTimestamp)
+{
+}
+
+default void onPreempted(TxnId txnId)
+{
+}
+
+default void onTimeout(TxnId txnId)
+{
+}
+
+default void onInvalidated(TxnId txnId)
+{
+}
+
+default void onProgressLogSizeChange(TxnId txnId, int delta)
+{
+}
+
+EventsListener NOOP = new EventsListener()
+{
+};
+}
diff --git
a/accord-core/src/main/java/accord/coordinate/CoordinatePreAccept.java
b/accord-core/src/main/java/accord/coordinate/CoordinatePreAccept.java
index 7eb71ced..c1c3de9c 100644
--- a/accord-core/src/main/java/accord/coordinate/CoordinatePreAccept.java
+++ b/accord-core/src/main/java/accord/coordinate/CoordinatePreAccept.java
@@ -216,7 +216,15 @@ abstract class CoordinatePreAccept extends
SettableResult implements Callb
if (extraPreAccept != null)
extraPreAccept.extraPreAcceptIsDone = true;
if (failure instanceof CoordinationFailed)
+{
((CoordinationFailed) failure).set(txnId, route.homeKey());
+if (failure instanceof Timeout)
+node.agent().metricsEventsListener().onTimeout(txnId);
+else if (failure instanceof Preempted)
+node.agent().metricsEventsListener().onPreempted(txnId);
+else if (failure instanceof Invalidated)
+node.agent().metricsEventsListener().onInvalidated(txnId);
+}
super.setFailure(failure);
}
@@ -265,7 +273,15 @@ abstract class
[jira] [Updated] (CASSANDRA-18317) Properly synchronize CQLSSTableWriter#build on the Schema.instance
[
https://issues.apache.org/jira/browse/CASSANDRA-18317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov updated CASSANDRA-18317:
Fix Version/s: 4.0.12
Since Version: 4.0
Source Control Link:
https://github.com/apache/cassandra/commit/26c374da4f03e4a6b64e414805cd92f3eb0a36c6
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Committed to 4.0 with [26c374da4f03e4a6b64e414805cd92f3eb0a36c6
|https://github.com/apache/cassandra/commit/26c374da4f03e4a6b64e414805cd92f3eb0a36c6],
and merged up to
[4.1|https://github.com/apache/cassandra/commit/26c374da4f03e4a6b64e414805cd92f3eb0a36c6],
[5.0|https://github.com/apache/cassandra/commit/8b941a6bdc826207e18a08c09e2ee4166f6e6d0d]
and
[trunk|https://github.com/apache/cassandra/commit/4b84c4332c3043b31fd33c4d8dc2d7f2771c90da].
Not specifying fix versions above 4.0.x since this is a testonly commit for
those branches.
> Properly synchronize CQLSSTableWriter#build on the Schema.instance
> --
>
> Key: CASSANDRA-18317
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18317
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/bulk load
>Reporter: Francisco Guerrero
>Assignee: Francisco Guerrero
>Priority: Normal
> Fix For: 4.0.12
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> The {{CQLSSTableWriter#build}} method should properly synchronize on the
> {{Schema.instance}} class to prevent concurrent Schema operations fail, [when
> the offline tool also updates schema].
> For example, a table creation operation, which modifies the keyspaces tables
> metadata, might end up missing the update when a concurrent call to the
> {{CQLSSTableWriter#build}} method is accessing the {{Schema}} instance.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18317) Properly synchronize CQLSSTableWriter#build on the Schema.instance
[
https://issues.apache.org/jira/browse/CASSANDRA-18317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov updated CASSANDRA-18317:
Status: Ready to Commit (was: Review In Progress)
> Properly synchronize CQLSSTableWriter#build on the Schema.instance
> --
>
> Key: CASSANDRA-18317
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18317
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/bulk load
>Reporter: Francisco Guerrero
>Assignee: Francisco Guerrero
>Priority: Normal
> Time Spent: 1h
> Remaining Estimate: 0h
>
> The {{CQLSSTableWriter#build}} method should properly synchronize on the
> {{Schema.instance}} class to prevent concurrent Schema operations fail, [when
> the offline tool also updates schema].
> For example, a table creation operation, which modifies the keyspaces tables
> metadata, might end up missing the update when a concurrent call to the
> {{CQLSSTableWriter#build}} method is accessing the {{Schema}} instance.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18317) Properly synchronize CQLSSTableWriter#build on the Schema.instance
[
https://issues.apache.org/jira/browse/CASSANDRA-18317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov updated CASSANDRA-18317:
Status: Review In Progress (was: Needs Committer)
> Properly synchronize CQLSSTableWriter#build on the Schema.instance
> --
>
> Key: CASSANDRA-18317
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18317
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/bulk load
>Reporter: Francisco Guerrero
>Assignee: Francisco Guerrero
>Priority: Normal
> Time Spent: 1h
> Remaining Estimate: 0h
>
> The {{CQLSSTableWriter#build}} method should properly synchronize on the
> {{Schema.instance}} class to prevent concurrent Schema operations fail, [when
> the offline tool also updates schema].
> For example, a table creation operation, which modifies the keyspaces tables
> metadata, might end up missing the update when a concurrent call to the
> {{CQLSSTableWriter#build}} method is accessing the {{Schema}} instance.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18317) Properly synchronize CQLSSTableWriter#build on the Schema.instance
[
https://issues.apache.org/jira/browse/CASSANDRA-18317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov updated CASSANDRA-18317:
Status: Patch Available (was: Needs Committer)
> Properly synchronize CQLSSTableWriter#build on the Schema.instance
> --
>
> Key: CASSANDRA-18317
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18317
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/bulk load
>Reporter: Francisco Guerrero
>Assignee: Francisco Guerrero
>Priority: Normal
> Time Spent: 1h
> Remaining Estimate: 0h
>
> The {{CQLSSTableWriter#build}} method should properly synchronize on the
> {{Schema.instance}} class to prevent concurrent Schema operations fail, [when
> the offline tool also updates schema].
> For example, a table creation operation, which modifies the keyspaces tables
> metadata, might end up missing the update when a concurrent call to the
> {{CQLSSTableWriter#build}} method is accessing the {{Schema}} instance.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18317) Properly synchronize CQLSSTableWriter#build on the Schema.instance
[
https://issues.apache.org/jira/browse/CASSANDRA-18317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Petrov updated CASSANDRA-18317:
Status: Needs Committer (was: Patch Available)
> Properly synchronize CQLSSTableWriter#build on the Schema.instance
> --
>
> Key: CASSANDRA-18317
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18317
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/bulk load
>Reporter: Francisco Guerrero
>Assignee: Francisco Guerrero
>Priority: Normal
> Time Spent: 1h
> Remaining Estimate: 0h
>
> The {{CQLSSTableWriter#build}} method should properly synchronize on the
> {{Schema.instance}} class to prevent concurrent Schema operations fail, [when
> the offline tool also updates schema].
> For example, a table creation operation, which modifies the keyspaces tables
> metadata, might end up missing the update when a concurrent call to the
> {{CQLSSTableWriter#build}} method is accessing the {{Schema}} instance.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-5.0' into trunk
This is an automated email from the ASF dual-hosted git repository. ifesdjeen pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 4b84c4332c3043b31fd33c4d8dc2d7f2771c90da Merge: c40e0778b3 8b941a6bdc Author: Alex Petrov AuthorDate: Tue Oct 10 10:52:11 2023 +0200 Merge branch 'cassandra-5.0' into trunk .../sstable/CQLSSTableWriterConcurrencyTest.java | 135 + 1 file changed, 135 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-4.1 updated (ea6461b870 -> e562d6a3a9)
This is an automated email from the ASF dual-hosted git repository. ifesdjeen pushed a change to branch cassandra-4.1 in repository https://gitbox.apache.org/repos/asf/cassandra.git from ea6461b870 Merge branch 'cassandra-4.0' into cassandra-4.1 add 26c374da4f Synchronize CQLSSTableWriter#build on the Schema.instance object add e562d6a3a9 Merge branch 'cassandra-4.0' into cassandra-4.1 No new revisions were added by this update. Summary of changes: .../sstable/CQLSSTableWriterConcurrencyTest.java | 135 + 1 file changed, 135 insertions(+) create mode 100644 test/unit/org/apache/cassandra/io/sstable/CQLSSTableWriterConcurrencyTest.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-4.0 updated (b57c13603a -> 26c374da4f)
This is an automated email from the ASF dual-hosted git repository. ifesdjeen pushed a change to branch cassandra-4.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git from b57c13603a Remove byteman-related files from production sources add 26c374da4f Synchronize CQLSSTableWriter#build on the Schema.instance object No new revisions were added by this update. Summary of changes: CHANGES.txt| 1 + .../cassandra/io/sstable/CQLSSTableWriter.java | 2 +- .../sstable/CQLSSTableWriterConcurrencyTest.java | 135 + 3 files changed, 137 insertions(+), 1 deletion(-) create mode 100644 test/unit/org/apache/cassandra/io/sstable/CQLSSTableWriterConcurrencyTest.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (c40e0778b3 -> 4b84c4332c)
This is an automated email from the ASF dual-hosted git repository. ifesdjeen pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git from c40e0778b3 Merge branch 'cassandra-5.0' into trunk add 26c374da4f Synchronize CQLSSTableWriter#build on the Schema.instance object add e562d6a3a9 Merge branch 'cassandra-4.0' into cassandra-4.1 add 8b941a6bdc Merge branch 'cassandra-4.1' into cassandra-5.0 new 4b84c4332c Merge branch 'cassandra-5.0' into trunk The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../sstable/CQLSSTableWriterConcurrencyTest.java | 135 + 1 file changed, 135 insertions(+) create mode 100644 test/unit/org/apache/cassandra/io/sstable/CQLSSTableWriterConcurrencyTest.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-5.0 updated (5a02e532d4 -> 8b941a6bdc)
This is an automated email from the ASF dual-hosted git repository. ifesdjeen pushed a change to branch cassandra-5.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 5a02e532d4 Merge branch 'cassandra-4.1' into cassandra-5.0 add 26c374da4f Synchronize CQLSSTableWriter#build on the Schema.instance object add e562d6a3a9 Merge branch 'cassandra-4.0' into cassandra-4.1 add 8b941a6bdc Merge branch 'cassandra-4.1' into cassandra-5.0 No new revisions were added by this update. Summary of changes: .../sstable/CQLSSTableWriterConcurrencyTest.java | 135 + 1 file changed, 135 insertions(+) create mode 100644 test/unit/org/apache/cassandra/io/sstable/CQLSSTableWriterConcurrencyTest.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18912) Specify "since" in all Deprecated annotations
[
https://issues.apache.org/jira/browse/CASSANDRA-18912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-18912:
--
Change Category: Code Clarity
Complexity: Normal
Status: Open (was: Triage Needed)
ML thread for reference
https://lists.apache.org/thread/cxmgwcmo4tc03lg4yosf6o0xln61vyjf
> Specify "since" in all Deprecated annotations
> -
>
> Key: CASSANDRA-18912
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18912
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Core
>Reporter: Stefan Miklosovic
>Assignee: Stefan Miklosovic
>Priority: Normal
>
> It would be great if we introduced in 5.0 a change in Deprecated annotations
> like this:
> {code}
> @Deprecated(since = "4.0")
> {code}
> or
> {code}
> @Deprecated(since = "3.11")
> {code}
> The reasoning behind this is that as of now, it is pretty cumbersome to
> figure out what can be removed on the next major version. It has to be,
> basically, done manually every time.
> There is also this parameter available:
> {code}
> @Deprecated(forRemoval = true / false)
> {code}
> which indicates whether the annotated element is subject to removal in a
> future version so we do not need to think about this every time if it is
> eligible for deletion in a next major or not.
> We could then have a check which would ensure that we are not releasing a
> next major with some deprecations introduced two majors before.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18877) remove bytebuddy / byteman from production classpath and remove compress-lzf dependency from build deps
[ https://issues.apache.org/jira/browse/CASSANDRA-18877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-18877: -- Fix Version/s: 4.0.12 4.1.4 5.0-alpha2 5.1 (was: 5.x) (was: 4.0.x) (was: 4.1.x) Source Control Link: https://github.com/apache/cassandra/commit/b57c13603afe699a918edd7ea1e8fdb46f0c1166 Resolution: Fixed Status: Resolved (was: Ready to Commit) > remove bytebuddy / byteman from production classpath and remove compress-lzf > dependency from build deps > --- > > Key: CASSANDRA-18877 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18877 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Stefan Miklosovic >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 4.0.12, 4.1.4, 5.0-alpha2, 5.1 > > Attachments: signature.asc > > Time Spent: 50m > Remaining Estimate: 0h > > I was digging in the project deps and if you compare all libs in "libs" dir > and all libs in "build/lib/jars", there are indeed some differences which are > OK however in build/lib/jars there are also libraries for byteman and > byte-buddy. This is clearly wrong as these dependecies should not be > accessible from the production code, only from tests. > The reason they are accessible in prod code is that there is the class > TestRateLimiter (1). I do not have a clue why that class is in the prod code > in the first place. The only place it is referenced in is here (2) but that > byteman script is not loaded anywhere in tests. I was also checking Python > dtests. > I think this is some leftover or something like "I will keep it here when I > need it", but as nobody seems to do, I strongly advocate for removing it and > making bytebuddy and byteman only test scoped dependencies as it should be. > A reader who pays attention notices that these dependencies are of provided > scope which is a trick to have it compilable but not among the libraries in > the production runtime and it does not do any harm as it is never invoked > from the production code (if it was, it would fail on missing imports) > neverthless this is still an issue which should be addressed. We were doing > something similar with assertj dependency recently. > The second issue is that there is a dependency on compress-lzf in build > dependencies. This is not necessary either as that library was removed from > the repository in (3) but it still somehow leaked to the build process again. > (1) > https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/utils/TestRateLimiter.java > (2) > https://github.com/apache/cassandra/blob/trunk/test/resources/byteman/mutation_limiter.btm > (3) > https://github.com/apache/cassandra/commit/fc92db2b9b56c143516026ba29cecdec37e286bb -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-4.0 updated (016d91a7d7 -> b57c13603a)
This is an automated email from the ASF dual-hosted git repository. smiklosovic pushed a change to branch cassandra-4.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 016d91a7d7 Fix closing iterator in SecondaryIndexBuilder add b57c13603a Remove byteman-related files from production sources No new revisions were added by this update. Summary of changes: build.xml | 8 +-- .../db/streaming/CassandraStreamWriter.java| 2 - .../apache/cassandra/utils/TestRateLimiter.java| 58 -- test/resources/byteman/mutation_limiter.btm| 26 -- 4 files changed, 4 insertions(+), 90 deletions(-) delete mode 100644 src/java/org/apache/cassandra/utils/TestRateLimiter.java delete mode 100644 test/resources/byteman/mutation_limiter.btm - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (e63d3bfc4b -> c40e0778b3)
This is an automated email from the ASF dual-hosted git repository. smiklosovic pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git from e63d3bfc4b Merge branch 'cassandra-5.0' into trunk add b57c13603a Remove byteman-related files from production sources add ea6461b870 Merge branch 'cassandra-4.0' into cassandra-4.1 add 5a02e532d4 Merge branch 'cassandra-4.1' into cassandra-5.0 new c40e0778b3 Merge branch 'cassandra-5.0' into trunk The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .build/cassandra-deps-template.xml | 4 -- .build/checkstyle.xml | 2 +- .build/parent-pom-template.xml | 10 ++-- .../db/streaming/CassandraStreamWriter.java| 2 - .../apache/cassandra/utils/TestRateLimiter.java| 58 -- test/resources/byteman/mutation_limiter.btm| 26 -- 6 files changed, 5 insertions(+), 97 deletions(-) delete mode 100644 src/java/org/apache/cassandra/utils/TestRateLimiter.java delete mode 100644 test/resources/byteman/mutation_limiter.btm - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-5.0 updated (4549b07997 -> 5a02e532d4)
This is an automated email from the ASF dual-hosted git repository. smiklosovic pushed a change to branch cassandra-5.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 4549b07997 Remove unnecessary reporter-config3 dependency add b57c13603a Remove byteman-related files from production sources add ea6461b870 Merge branch 'cassandra-4.0' into cassandra-4.1 add 5a02e532d4 Merge branch 'cassandra-4.1' into cassandra-5.0 No new revisions were added by this update. Summary of changes: .build/cassandra-deps-template.xml | 4 -- .build/checkstyle.xml | 2 +- .build/parent-pom-template.xml | 10 ++-- .../db/streaming/CassandraStreamWriter.java| 2 - .../apache/cassandra/utils/TestRateLimiter.java| 58 -- test/resources/byteman/mutation_limiter.btm| 26 -- 6 files changed, 5 insertions(+), 97 deletions(-) delete mode 100644 src/java/org/apache/cassandra/utils/TestRateLimiter.java delete mode 100644 test/resources/byteman/mutation_limiter.btm - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-5.0' into trunk
This is an automated email from the ASF dual-hosted git repository. smiklosovic pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit c40e0778b3356ab7e40546deac816377606b87cf Merge: e63d3bfc4b 5a02e532d4 Author: Stefan Miklosovic AuthorDate: Tue Oct 10 10:19:12 2023 +0200 Merge branch 'cassandra-5.0' into trunk .build/cassandra-deps-template.xml | 4 -- .build/checkstyle.xml | 2 +- .build/parent-pom-template.xml | 10 ++-- .../db/streaming/CassandraStreamWriter.java| 2 - .../apache/cassandra/utils/TestRateLimiter.java| 58 -- test/resources/byteman/mutation_limiter.btm| 26 -- 6 files changed, 5 insertions(+), 97 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-4.1 updated (59790da813 -> ea6461b870)
This is an automated email from the ASF dual-hosted git repository. smiklosovic pushed a change to branch cassandra-4.1 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 59790da813 Merge branch 'cassandra-4.0' into cassandra-4.1 add b57c13603a Remove byteman-related files from production sources add ea6461b870 Merge branch 'cassandra-4.0' into cassandra-4.1 No new revisions were added by this update. Summary of changes: .build/checkstyle.xml | 2 +- build.xml | 8 +-- .../db/streaming/CassandraStreamWriter.java| 2 - .../apache/cassandra/utils/TestRateLimiter.java| 58 -- test/resources/byteman/mutation_limiter.btm| 26 -- 5 files changed, 5 insertions(+), 91 deletions(-) delete mode 100644 src/java/org/apache/cassandra/utils/TestRateLimiter.java delete mode 100644 test/resources/byteman/mutation_limiter.btm - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18612) java-driver ip clearance
[ https://issues.apache.org/jira/browse/CASSANDRA-18612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-18612: --- Fix Version/s: NA Source Control Link: https://lists.apache.org/thread/ryxcc043m9kkjyx9150hldr42kpcs0r0 Resolution: Fixed Status: Resolved (was: Ready to Commit) > java-driver ip clearance > > > Key: CASSANDRA-18612 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18612 > Project: Cassandra > Issue Type: Sub-task > Components: Messaging/Client >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: NA > > > https://incubator.apache.org/ip-clearance/cassandra-java-driver.html -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18612) java-driver ip clearance
[ https://issues.apache.org/jira/browse/CASSANDRA-18612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-18612: --- Status: Ready to Commit (was: Review In Progress) > java-driver ip clearance > > > Key: CASSANDRA-18612 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18612 > Project: Cassandra > Issue Type: Sub-task > Components: Messaging/Client >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > > https://incubator.apache.org/ip-clearance/cassandra-java-driver.html -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18612) java-driver ip clearance
[ https://issues.apache.org/jira/browse/CASSANDRA-18612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-18612: --- Reviewers: Michael Semb Wever Status: Review In Progress (was: Patch Available) > java-driver ip clearance > > > Key: CASSANDRA-18612 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18612 > Project: Cassandra > Issue Type: Sub-task > Components: Messaging/Client >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > > https://incubator.apache.org/ip-clearance/cassandra-java-driver.html -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18612) java-driver ip clearance
[ https://issues.apache.org/jira/browse/CASSANDRA-18612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-18612: --- Test and Documentation Plan: peer review Status: Patch Available (was: In Progress) > java-driver ip clearance > > > Key: CASSANDRA-18612 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18612 > Project: Cassandra > Issue Type: Sub-task > Components: Messaging/Client >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > > https://incubator.apache.org/ip-clearance/cassandra-java-driver.html -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18612) java-driver ip clearance
[ https://issues.apache.org/jira/browse/CASSANDRA-18612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773590#comment-17773590 ] Michael Semb Wever commented on CASSANDRA-18612: Vote passed: https://lists.apache.org/thread/ryxcc043m9kkjyx9150hldr42kpcs0r0 > java-driver ip clearance > > > Key: CASSANDRA-18612 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18612 > Project: Cassandra > Issue Type: Sub-task > Components: Messaging/Client >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > > https://incubator.apache.org/ip-clearance/cassandra-java-driver.html -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18612) java-driver ip clearance
[ https://issues.apache.org/jira/browse/CASSANDRA-18612?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773587#comment-17773587 ] Michael Semb Wever commented on CASSANDRA-18612: copyright re-assignment was done in https://github.com/datastax/java-driver/pull/1682 > java-driver ip clearance > > > Key: CASSANDRA-18612 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18612 > Project: Cassandra > Issue Type: Sub-task > Components: Messaging/Client >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > > https://incubator.apache.org/ip-clearance/cassandra-java-driver.html -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch asf-site updated (b628c65e -> 9fada0ef)
This is an automated email from the ASF dual-hosted git repository. mck pushed a change to branch asf-site in repository https://gitbox.apache.org/repos/asf/cassandra-website.git discard b628c65e generate docs for bc8bfc13 add 31eba98f Added rustyrazorblade consulting to ecosystem page. add db70fb96 update ecosystem page add 9fada0ef generate docs for db70fb96 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (b628c65e) \ N -- N -- N refs/heads/asf-site (9fada0ef) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: content/_/ecosystem.html | 8 +- .../managing/configuration/cass_yaml_file.html | 19 - .../managing/tools/nodetool/bootstrap.html | 8 +- .../cassandra/managing/tools/nodetool/import.html | 6 +- .../managing/tools/nodetool/nodetool.html | 8 +- .../managing/tools/nodetool/repair_admin.html | 90 +++-- .../managing/configuration/cass_yaml_file.html | 19 - .../doc/5.1/cassandra/managing/tools/cqlsh.html| 32 .../managing/tools/nodetool/bootstrap.html | 8 +- .../cassandra/managing/tools/nodetool/import.html | 6 +- .../managing/tools/nodetool/nodetool.html | 8 +- .../managing/tools/nodetool/repair_admin.html | 90 +++-- .../managing/configuration/cass_yaml_file.html | 19 - .../doc/trunk/cassandra/managing/tools/cqlsh.html | 32 .../managing/tools/nodetool/bootstrap.html | 8 +- .../cassandra/managing/tools/nodetool/import.html | 6 +- .../managing/tools/nodetool/nodetool.html | 8 +- .../managing/tools/nodetool/repair_admin.html | 90 +++-- content/search-index.js| 2 +- .../source/modules/ROOT/pages/ecosystem.adoc | 6 +- site-ui/build/ui-bundle.zip| Bin 4881412 -> 4881412 bytes 21 files changed, 302 insertions(+), 171 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18877) remove bytebuddy / byteman from production classpath and remove compress-lzf dependency from build deps
[ https://issues.apache.org/jira/browse/CASSANDRA-18877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-18877: -- Status: Ready to Commit (was: Review In Progress) > remove bytebuddy / byteman from production classpath and remove compress-lzf > dependency from build deps > --- > > Key: CASSANDRA-18877 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18877 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Stefan Miklosovic >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 4.0.x, 4.1.x, 5.x > > Attachments: signature.asc > > Time Spent: 50m > Remaining Estimate: 0h > > I was digging in the project deps and if you compare all libs in "libs" dir > and all libs in "build/lib/jars", there are indeed some differences which are > OK however in build/lib/jars there are also libraries for byteman and > byte-buddy. This is clearly wrong as these dependecies should not be > accessible from the production code, only from tests. > The reason they are accessible in prod code is that there is the class > TestRateLimiter (1). I do not have a clue why that class is in the prod code > in the first place. The only place it is referenced in is here (2) but that > byteman script is not loaded anywhere in tests. I was also checking Python > dtests. > I think this is some leftover or something like "I will keep it here when I > need it", but as nobody seems to do, I strongly advocate for removing it and > making bytebuddy and byteman only test scoped dependencies as it should be. > A reader who pays attention notices that these dependencies are of provided > scope which is a trick to have it compilable but not among the libraries in > the production runtime and it does not do any harm as it is never invoked > from the production code (if it was, it would fail on missing imports) > neverthless this is still an issue which should be addressed. We were doing > something similar with assertj dependency recently. > The second issue is that there is a dependency on compress-lzf in build > dependencies. This is not necessary either as that library was removed from > the repository in (3) but it still somehow leaked to the build process again. > (1) > https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/utils/TestRateLimiter.java > (2) > https://github.com/apache/cassandra/blob/trunk/test/resources/byteman/mutation_limiter.btm > (3) > https://github.com/apache/cassandra/commit/fc92db2b9b56c143516026ba29cecdec37e286bb -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18877) remove bytebuddy / byteman from production classpath and remove compress-lzf dependency from build deps
[ https://issues.apache.org/jira/browse/CASSANDRA-18877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-18877: -- Reviewers: Michael Semb Wever Status: Review In Progress (was: Needs Committer) > remove bytebuddy / byteman from production classpath and remove compress-lzf > dependency from build deps > --- > > Key: CASSANDRA-18877 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18877 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Stefan Miklosovic >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 4.0.x, 4.1.x, 5.x > > Attachments: signature.asc > > Time Spent: 50m > Remaining Estimate: 0h > > I was digging in the project deps and if you compare all libs in "libs" dir > and all libs in "build/lib/jars", there are indeed some differences which are > OK however in build/lib/jars there are also libraries for byteman and > byte-buddy. This is clearly wrong as these dependecies should not be > accessible from the production code, only from tests. > The reason they are accessible in prod code is that there is the class > TestRateLimiter (1). I do not have a clue why that class is in the prod code > in the first place. The only place it is referenced in is here (2) but that > byteman script is not loaded anywhere in tests. I was also checking Python > dtests. > I think this is some leftover or something like "I will keep it here when I > need it", but as nobody seems to do, I strongly advocate for removing it and > making bytebuddy and byteman only test scoped dependencies as it should be. > A reader who pays attention notices that these dependencies are of provided > scope which is a trick to have it compilable but not among the libraries in > the production runtime and it does not do any harm as it is never invoked > from the production code (if it was, it would fail on missing imports) > neverthless this is still an issue which should be addressed. We were doing > something similar with assertj dependency recently. > The second issue is that there is a dependency on compress-lzf in build > dependencies. This is not necessary either as that library was removed from > the repository in (3) but it still somehow leaked to the build process again. > (1) > https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/utils/TestRateLimiter.java > (2) > https://github.com/apache/cassandra/blob/trunk/test/resources/byteman/mutation_limiter.btm > (3) > https://github.com/apache/cassandra/commit/fc92db2b9b56c143516026ba29cecdec37e286bb -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch asf-staging updated (a2835dd2 -> 9fada0ef)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git discard a2835dd2 generate docs for 31eba98f add db70fb96 update ecosystem page new 9fada0ef generate docs for db70fb96 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (a2835dd2) \ N -- N -- N refs/heads/asf-staging (9fada0ef) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: content/_/ecosystem.html | 5 +- .../managing/tools/nodetool/bootstrap.html | 8 +-- .../managing/tools/nodetool/nodetool.html | 8 +-- .../managing/tools/nodetool/repair_admin.html | 55 +++-- .../managing/tools/nodetool/bootstrap.html | 8 +-- .../managing/tools/nodetool/nodetool.html | 8 +-- .../managing/tools/nodetool/repair_admin.html | 55 +++-- .../managing/tools/nodetool/bootstrap.html | 8 +-- .../managing/tools/nodetool/nodetool.html | 8 +-- .../managing/tools/nodetool/repair_admin.html | 55 +++-- .../source/modules/ROOT/pages/ecosystem.adoc | 4 +- site-ui/build/ui-bundle.zip| Bin 4881412 -> 4881412 bytes 12 files changed, 115 insertions(+), 107 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18898) WEBSITE - Ecosystem page update for AxonOps and link fix
[ https://issues.apache.org/jira/browse/CASSANDRA-18898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-18898: --- Fix Version/s: NA Source Control Link: https://github.com/apache/cassandra-website/commit/db70fb96a5f2dbe1ee53331211533068514b5cc2 Resolution: Fixed Status: Resolved (was: Ready to Commit) > WEBSITE - Ecosystem page update for AxonOps and link fix > > > Key: CASSANDRA-18898 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18898 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: Diogenese Topper >Priority: Normal > Fix For: NA > > > This ticket is to capture the work associated with updating the Apache > Cassandra website's ecosystem page. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18898) WEBSITE - Ecosystem page update for AxonOps and link fix
[ https://issues.apache.org/jira/browse/CASSANDRA-18898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-18898: --- Status: Ready to Commit (was: Review In Progress) > WEBSITE - Ecosystem page update for AxonOps and link fix > > > Key: CASSANDRA-18898 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18898 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: Diogenese Topper >Priority: Normal > > This ticket is to capture the work associated with updating the Apache > Cassandra website's ecosystem page. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18898) WEBSITE - Ecosystem page update for AxonOps and link fix
[ https://issues.apache.org/jira/browse/CASSANDRA-18898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773580#comment-17773580 ] Michael Semb Wever commented on CASSANDRA-18898: committed as https://github.com/apache/cassandra-website/commit/db70fb96a5f2dbe1ee53331211533068514b5cc2 > WEBSITE - Ecosystem page update for AxonOps and link fix > > > Key: CASSANDRA-18898 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18898 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: Diogenese Topper >Priority: Normal > > This ticket is to capture the work associated with updating the Apache > Cassandra website's ecosystem page. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18898) WEBSITE - Ecosystem page update for AxonOps and link fix
[ https://issues.apache.org/jira/browse/CASSANDRA-18898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-18898: --- Reviewers: Michael Semb Wever Status: Review In Progress (was: Patch Available) > WEBSITE - Ecosystem page update for AxonOps and link fix > > > Key: CASSANDRA-18898 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18898 > Project: Cassandra > Issue Type: Task > Components: Documentation/Website >Reporter: Diogenese Topper >Priority: Normal > > This ticket is to capture the work associated with updating the Apache > Cassandra website's ecosystem page. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch trunk updated: update ecosystem page
This is an automated email from the ASF dual-hosted git repository. mck pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-website.git The following commit(s) were added to refs/heads/trunk by this push: new db70fb96 update ecosystem page db70fb96 is described below commit db70fb96a5f2dbe1ee53331211533068514b5cc2 Author: Diogenese Topper <83248625+nonstopd...@users.noreply.github.com> AuthorDate: Mon Oct 2 13:02:12 2023 -0700 update ecosystem page patch by Diogenese Topper; reviewed by Mick Semb Wever for CASSANDRA-18898 --- site-content/source/modules/ROOT/pages/ecosystem.adoc | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/site-content/source/modules/ROOT/pages/ecosystem.adoc b/site-content/source/modules/ROOT/pages/ecosystem.adoc index d96cdbdf..d130dad3 100644 --- a/site-content/source/modules/ROOT/pages/ecosystem.adoc +++ b/site-content/source/modules/ROOT/pages/ecosystem.adoc @@ -325,7 +325,9 @@ https://prestodb.io/docs/current/connector/cassandra.html[Presto,window=blank]: https://anant.us/cassandra/[Anant,window=blank] -https://luna.datastax.com/[DataStax Luna,window=blank] +https://axonops.com/apache-cassandra-support/[AxonOps,window=blank] + +https://www.datastax.com/products/luna[DataStax Luna,window=blank] https://www.datastax.com/services/support/premium-support[DataStax Premium Support,window=blank] - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18534) Make sstable format configurable per table
[
https://issues.apache.org/jira/browse/CASSANDRA-18534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773559#comment-17773559
]
Stefan Miklosovic edited comment on CASSANDRA-18534 at 10/10/23 6:58 AM:
-
I prefer to merge CASSANDRA-18872 first where it will be removed. So it means
that 5.0 will _not_ have crc_check_chance in compression anymore.
Then you might rebase this work against 5.0 where crc_check_chance will not be
in compression anymore and you might tweak FileHandler builder to propagate
sstable format option there to align it.
[~maxwellguo] [~blambov] how do this sound to you?
BTW I think this ticket as a whole needs to have a ML thread. We are changing
CQL here and it would be great to involve more people into this.
What seems to be a little bit "strange" to me is that we chose these properties:
row_index_granularity
bloom_filter_fp_chance
crc_check_chance
min/max_index_interval
But _why exactly these_? It seems to me that we just picked a subset of
configuration options to be available (by some yet-uknown-to-me rule) and we
made it configurable like that. Why is it already pre-defined what properties
might be in sstable_format option? What would be interesting to have is that
you might reference _every parameter_ instead of having a static set to choose
from.
Also, what does have crc_check_chance, for example, to do with _sstable
format_. There is no "format" behind that. crc_check_chance (similarly
bloom_filter_fp_chance), is just a _probability_ with which we do so and so
operation. That is an operational parameter, we are not _formatting an sstable_
as such. Maybe it is just a matter of naming, I just find this to be important
to mention.
Also, do you think it is possible and useful to make sstable_format contain
custom parameters? If we have a way how to specify custom format of an SSTable
by implementing AbstractSSTableFormat, then such format might accept additional
parameters which would be added into sstable_format like this:
{code}
... sstable_format = {"type": "mytype", "myparameter": "abc"}
{code}
That means we would not need to implement every custom parameter out there for
whatever format. The tricky part is that if we allow custom parameters to be
specified, then, on alternation of a schema, it would start to be a different
schema version which would need to be propagated to the cluster.
was (Author: smiklosovic):
I prefer to merge CASSANDRA-18872 first where it will be removed. So it means
that 5.0 will _not_ have crc_check_chance in compression anymore.
Then you might rebase this work against 5.0 where crc_check_chance will not be
in compression anymore and you might tweak FileHandler builder to propagate
sstable format option there to align it.
[~maxwellguo] [~blambov] how do this sound to you?
BTW I think this ticket as a whole needs to have a ML thread. We are changing
CQL here and it would be great to involve more people into this.
What seems to be a little bit "strange" to me is that we chose these properties:
row_index_granularity
bloom_filter_fp_chance
crc_check_chance
min/max_index_interval
But _why exactly these_? Also, what does have crc_check_chance, for example, to
do with _sstable format_. There is no "format" behind that. crc_check_chance
(similarly bloom_filter_fp_chance), is just a _probability_ with which we do so
and so operation. That is an operational parameter, we are not _formatting an
sstable_ as such. Maybe it is just a matter of naming, I just find this to be
important to mention.
Also, do you think it is possible and useful to make sstable_format contain
custom parameters? If we have a way how to specify custom format of an SSTable
by implementing AbstractSSTableFormat, then such format might accept additional
parameters which would be added into sstable_format like this:
{code}
... sstable_format = {"type": "mytype", "myparameter": "abc"}
{code}
That means we would not need to implement every custom parameter out there for
whatever format. The tricky part is that if we allow custom parameters to be
specified, then, on alternation of a schema, it would start to be a different
schema version which would need to be propagated to the cluster.
> Make sstable format configurable per table
> --
>
> Key: CASSANDRA-18534
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18534
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema, Local/SSTable
>Reporter: Branimir Lambov
>Assignee: Maxwell Guo
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Some SSTable format settings need to be configurable per table for better
> efficiency. This includes:
> - {{row_index_granularity}}
> -
[jira] [Comment Edited] (CASSANDRA-18534) Make sstable format configurable per table
[
https://issues.apache.org/jira/browse/CASSANDRA-18534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773559#comment-17773559
]
Stefan Miklosovic edited comment on CASSANDRA-18534 at 10/10/23 6:49 AM:
-
I prefer to merge CASSANDRA-18872 first where it will be removed. So it means
that 5.0 will _not_ have crc_check_chance in compression anymore.
Then you might rebase this work against 5.0 where crc_check_chance will not be
in compression anymore and you might tweak FileHandler builder to propagate
sstable format option there to align it.
[~maxwellguo] [~blambov] how do this sound to you?
BTW I think this ticket as a whole needs to have a ML thread. We are changing
CQL here and it would be great to involve more people into this.
What seems to be a little bit "strange" to me is that we chose these properties:
row_index_granularity
bloom_filter_fp_chance
crc_check_chance
min/max_index_interval
But _why exactly these_? Also, what does have crc_check_chance, for example, to
do with _sstable format_. There is no "format" behind that. crc_check_chance
(similarly bloom_filter_fp_chance), is just a _probability_ with which we do so
and so operation. That is an operational parameter, we are not _formatting an
sstable_ as such. Maybe it is just a matter of naming, I just find this to be
important to mention.
Also, do you think it is possible and useful to make sstable_format contain
custom parameters? If we have a way how to specify custom format of an SSTable
by implementing AbstractSSTableFormat, then such format might accept additional
parameters which would be added into sstable_format like this:
{code}
... sstable_format = {"type": "mytype", "myparameter": "abc"}
{code}
That means we would not need to implement every custom parameter out there for
whatever format. The tricky part is that if we allow custom parameters to be
specified, then, on alternation of a schema, it would start to be a different
schema version which would need to be propagated to the cluster.
was (Author: smiklosovic):
I prefer to merge CASSANDRA-18872 first where it will be removed. So it means
that 5.0 will _not_ have crc_check_chance in compression anymore.
Then you might rebase this work against 5.0 where crc_check_chance will not be
in compression anymore and you might tweak FileHandler builder to propagate
sstable format option there to align it.
[~maxwellguo] [~blambov] how do this sound to you?
BTW I think this ticket as a whole needs to have a ML thread. We are changing
CQL here and it would be great to involve more people into this.
What seems to be a little bit "strange" to me is that we chose these properties:
row_index_granularity
bloom_filter_fp_chance
crc_check_chance
min/max_index_interval
But _why exactly these_? Also, what does have a crc_check_chance, for example,
to do with _sstable format_. There is no "format" behind that. crc_check_chance
(similarly bloom_filter_fp_chance), is just a _probability_ with which we do so
and so operation. That is an operational parameter, we are not _formatting an
sstable_ as such. Maybe it is just a matter of naming, I just find this to be
important to mention.
Also, do you think it is possible and useful to make sstable_format contain
custom parameters? If we have a way how to specify custom format of an SSTable
by implementing AbstractSSTableFormat, then such format might accept additional
parameters which would be added into sstable_format like this:
{code}
... sstable_format = {"type": "mytype", "myparameter": "abc"}
{code}
That means we would not need to implement every custom parameter out there for
whatever format. The tricky part is that if we allow custom parameters to be
specified, then, on alternation of a schema, it would start to be a different
schema version which would need to be propagated to the cluster.
> Make sstable format configurable per table
> --
>
> Key: CASSANDRA-18534
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18534
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema, Local/SSTable
>Reporter: Branimir Lambov
>Assignee: Maxwell Guo
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Some SSTable format settings need to be configurable per table for better
> efficiency. This includes:
> - {{row_index_granularity}}
> - {{bloom_filter_fp_chance}}
> - {{crc_check_chance}}
> - {{min/max_index_interval}}
> Some of these are currently configurable using direct properties of tables.
> Having them as format properties makes better sense and should also support
> specifying useable combinations of settings, e.g.
> {code:java}
> CREATE TABLE ... WITH sstable_format = "bti-fast";
>
[jira] [Comment Edited] (CASSANDRA-18534) Make sstable format configurable per table
[
https://issues.apache.org/jira/browse/CASSANDRA-18534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773559#comment-17773559
]
Stefan Miklosovic edited comment on CASSANDRA-18534 at 10/10/23 6:49 AM:
-
I prefer to merge CASSANDRA-18872 first where it will be removed. So it means
that 5.0 will _not_ have crc_check_chance in compression anymore.
Then you might rebase this work against 5.0 where crc_check_chance will not be
in compression anymore and you might tweak FileHandler builder to propagate
sstable format option there to align it.
[~maxwellguo] [~blambov] how do this sound to you?
BTW I think this ticket as a whole needs to have a ML thread. We are changing
CQL here and it would be great to involve more people into this.
What seems to be a little bit "strange" to me is that we chose these properties:
row_index_granularity
bloom_filter_fp_chance
crc_check_chance
min/max_index_interval
But _why exactly these_? Also, what does have a crc_check_chance, for example,
to do with _sstable format_. There is no "format" behind that. crc_check_chance
(similarly bloom_filter_fp_chance), is just a _probability_ with which we do so
and so operation. That is an operational parameter, we are not _formatting an
sstable_ as such. Maybe it is just a matter of naming, I just find this to be
important to mention.
Also, do you think it is possible and useful to make sstable_format contain
custom parameters? If we have a way how to specify custom format of an SSTable
by implementing AbstractSSTableFormat, then such format might accept additional
parameters which would be added into sstable_format like this:
{code}
... sstable_format = {"type": "mytype", "myparameter": "abc"}
{code}
That means we would not need to implement every custom parameter out there for
whatever format. The tricky part is that if we allow custom parameters to be
specified, then, on alternation of a schema, it would start to be a different
schema version which would need to be propagated to the cluster.
was (Author: smiklosovic):
I prefer to merge CASSANDRA-18872 first where it will be removed. So it means
that 5.0 will _not_ have crc_check_chance in compression anymore.
Then you might rebase this work against 5.0 where crc_check_chance will not be
in compression anymore and you might tweak FileHandler builder to propagate
sstable format option there to align it.
[~maxwellguo] [~blambov] how do this sound to you?
BTW I think this ticket as a whole needs to have a ML thread. We are changing
CQL here and it would be great to involve more people into this.
What seems to be a little bit "strange" to me is that we chose these properties:
row_index_granularity
bloom_filter_fp_chance
crc_check_chance
min/max_index_interval
But _why exactly these_? Also, what does have a crc_check_chance, for example,
to do with _sstable format_. There is no "format" behind that. crc_check_chance
(similarly bloom_filter_fp_chance), is just a _probability_ with which we do so
and so operation. That is an operational parameter, we are not _formatting an
sstable_ as such. Maybe it is just a matter of naming, I just find this to be
important to mention.
> Make sstable format configurable per table
> --
>
> Key: CASSANDRA-18534
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18534
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema, Local/SSTable
>Reporter: Branimir Lambov
>Assignee: Maxwell Guo
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Some SSTable format settings need to be configurable per table for better
> efficiency. This includes:
> - {{row_index_granularity}}
> - {{bloom_filter_fp_chance}}
> - {{crc_check_chance}}
> - {{min/max_index_interval}}
> Some of these are currently configurable using direct properties of tables.
> Having them as format properties makes better sense and should also support
> specifying useable combinations of settings, e.g.
> {code:java}
> CREATE TABLE ... WITH sstable_format = "bti-fast";
> CREATE TABLE ... WITH sstable_format = "bti-small";
> {code}
> where {{bti-fast}} and {{bti-small}} can be defined in {{cassandra.yaml}}
> e.g. as
> {code:java}
> sstable.format.options:
> - bti-fast:
> row_index_granularity: 1kiB
> bloom_filter_fp_chance: 0.01
> - bti-small:
> row_index_granularity: 32kiB
> bloom_filter_fp_chance: 0.1
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18534) Make sstable format configurable per table
[
https://issues.apache.org/jira/browse/CASSANDRA-18534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773559#comment-17773559
]
Stefan Miklosovic commented on CASSANDRA-18534:
---
I prefer to merge CASSANDRA-18872 first where it will be removed. So it means
that 5.0 will _not_ have crc_check_chance in compression anymore.
Then you might rebase this work against 5.0 where crc_check_chance will not be
in compression anymore and you might tweak FileHandler builder to propagate
sstable format option there to align it.
[~maxwellguo] [~blambov] how do this sound to you?
BTW I think this ticket as a whole needs to have a ML thread. We are changing
CQL here and it would be great to involve more people into this.
What seems to be a little bit "strange" to me is that we chose these properties:
row_index_granularity
bloom_filter_fp_chance
crc_check_chance
min/max_index_interval
But _why exactly these_? Also, what does have a crc_check_chance, for example,
to do with _sstable format_. There is no "format" behind that. crc_check_chance
(similarly bloom_filter_fp_chance), is just a _probability_ with which we do so
and so operation. That is an operational parameter, we are not _formatting an
sstable_ as such. Maybe it is just a matter of naming, I just find this to be
important to mention.
> Make sstable format configurable per table
> --
>
> Key: CASSANDRA-18534
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18534
> Project: Cassandra
> Issue Type: Improvement
> Components: Cluster/Schema, Local/SSTable
>Reporter: Branimir Lambov
>Assignee: Maxwell Guo
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Some SSTable format settings need to be configurable per table for better
> efficiency. This includes:
> - {{row_index_granularity}}
> - {{bloom_filter_fp_chance}}
> - {{crc_check_chance}}
> - {{min/max_index_interval}}
> Some of these are currently configurable using direct properties of tables.
> Having them as format properties makes better sense and should also support
> specifying useable combinations of settings, e.g.
> {code:java}
> CREATE TABLE ... WITH sstable_format = "bti-fast";
> CREATE TABLE ... WITH sstable_format = "bti-small";
> {code}
> where {{bti-fast}} and {{bti-small}} can be defined in {{cassandra.yaml}}
> e.g. as
> {code:java}
> sstable.format.options:
> - bti-fast:
> row_index_granularity: 1kiB
> bloom_filter_fp_chance: 0.01
> - bti-small:
> row_index_granularity: 32kiB
> bloom_filter_fp_chance: 0.1
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
