[jira] [Updated] (CASSANDRA-18941) Support max SSTable size in sorted CQLSSTableWriter
[ https://issues.apache.org/jira/browse/CASSANDRA-18941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxwell Guo updated CASSANDRA-18941: Fix Version/s: 5.x > Support max SSTable size in sorted CQLSSTableWriter > --- > > Key: CASSANDRA-18941 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18941 > Project: Cassandra > Issue Type: Improvement > Components: Tool/sstable >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Fix For: 5.x > > Time Spent: 20m > Remaining Estimate: 0h > > The CQLSSTableWriter can produce a series of SSTables with a bounded size in > the unsorted mode. The functionality is missing in the sorted > CQLSSTableWriter. > It will be great to bringing the parity to the sorted mode, so that it is > able to produce a series of size-bounded SSTable, instead of a single but > giant one. > Unlike the unsorted CQLSSTableWriter, the max SSTable size in the sorted > writer does not require buffering. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18941) Support max SSTable size in sorted CQLSSTableWriter
[ https://issues.apache.org/jira/browse/CASSANDRA-18941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxwell Guo updated CASSANDRA-18941: Test and Documentation Plan: unit test Status: Patch Available (was: In Progress) > Support max SSTable size in sorted CQLSSTableWriter > --- > > Key: CASSANDRA-18941 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18941 > Project: Cassandra > Issue Type: Improvement > Components: Tool/sstable >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Time Spent: 20m > Remaining Estimate: 0h > > The CQLSSTableWriter can produce a series of SSTables with a bounded size in > the unsorted mode. The functionality is missing in the sorted > CQLSSTableWriter. > It will be great to bringing the parity to the sorted mode, so that it is > able to produce a series of size-bounded SSTable, instead of a single but > giant one. > Unlike the unsorted CQLSSTableWriter, the max SSTable size in the sorted > writer does not require buffering. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18941) Support max SSTable size in sorted CQLSSTableWriter
[ https://issues.apache.org/jira/browse/CASSANDRA-18941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxwell Guo updated CASSANDRA-18941: Reviewers: Maxwell Guo > Support max SSTable size in sorted CQLSSTableWriter > --- > > Key: CASSANDRA-18941 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18941 > Project: Cassandra > Issue Type: Improvement > Components: Tool/sstable >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Time Spent: 20m > Remaining Estimate: 0h > > The CQLSSTableWriter can produce a series of SSTables with a bounded size in > the unsorted mode. The functionality is missing in the sorted > CQLSSTableWriter. > It will be great to bringing the parity to the sorted mode, so that it is > able to produce a series of size-bounded SSTable, instead of a single but > giant one. > Unlike the unsorted CQLSSTableWriter, the max SSTable size in the sorted > writer does not require buffering. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch asf-staging updated (8018ff370 -> d84e6c9ec)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git omit 8018ff370 generate docs for 1d0135e5 new d84e6c9ec generate docs for 1d0135e5 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (8018ff370) \ N -- N -- N refs/heads/asf-staging (d84e6c9ec) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: site-ui/build/ui-bundle.zip | Bin 4881412 -> 4881412 bytes 1 file changed, 0 insertions(+), 0 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18941) Support max SSTable size in sorted CQLSSTableWriter
[ https://issues.apache.org/jira/browse/CASSANDRA-18941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifan Cai updated CASSANDRA-18941: -- Change Category: Operability Complexity: Normal Status: Open (was: Triage Needed) PR: https://github.com/apache/cassandra/pull/2824 CI: https://app.circleci.com/pipelines/github/yifan-c/cassandra?branch=CASSANDRA-18941%2Ftrunk > Support max SSTable size in sorted CQLSSTableWriter > --- > > Key: CASSANDRA-18941 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18941 > Project: Cassandra > Issue Type: Improvement > Components: Tool/sstable >Reporter: Yifan Cai >Assignee: Yifan Cai >Priority: Normal > Time Spent: 10m > Remaining Estimate: 0h > > The CQLSSTableWriter can produce a series of SSTables with a bounded size in > the unsorted mode. The functionality is missing in the sorted > CQLSSTableWriter. > It will be great to bringing the parity to the sorted mode, so that it is > able to produce a series of size-bounded SSTable, instead of a single but > giant one. > Unlike the unsorted CQLSSTableWriter, the max SSTable size in the sorted > writer does not require buffering. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18941) Support max SSTable size in sorted CQLSSTableWriter
Yifan Cai created CASSANDRA-18941: - Summary: Support max SSTable size in sorted CQLSSTableWriter Key: CASSANDRA-18941 URL: https://issues.apache.org/jira/browse/CASSANDRA-18941 Project: Cassandra Issue Type: Improvement Components: Tool/sstable Reporter: Yifan Cai Assignee: Yifan Cai The CQLSSTableWriter can produce a series of SSTables with a bounded size in the unsorted mode. The functionality is missing in the sorted CQLSSTableWriter. It will be great to bringing the parity to the sorted mode, so that it is able to produce a series of size-bounded SSTable, instead of a single but giant one. Unlike the unsorted CQLSSTableWriter, the max SSTable size in the sorted writer does not require buffering. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18940) SAI post-filtering reads don't update local table latency metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-18940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-18940: Fix Version/s: 5.0-alpha2 5.0 5.1 > SAI post-filtering reads don't update local table latency metrics > - > > Key: CASSANDRA-18940 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18940 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index, Feature/SAI, Observability/Metrics >Reporter: Caleb Rackliffe >Assignee: Caleb Rackliffe >Priority: Normal > Fix For: 5.0-alpha2, 5.0, 5.1 > > Attachments: > draft_fix_for_SAI_post-filtering_reads_not_updating_local_table_metrics.patch > > > Once an SAI index finds matches (primary keys), it reads the associated rows > and post-filters them to incorporate partial writes, tombstones, etc. > However, those reads are not currently updating the local table latency > metrics. It should be simple enough to attach a metrics recording > transformation to the iterator produced by querying local storage. (I've > attached a patch that should apply cleanly to trunk, but there may be a > better way...) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18940) SAI post-filtering reads don't update local table latency metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-18940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-18940: Bug Category: Parent values: Correctness(12982) Complexity: Normal Discovered By: User Report Severity: Low Status: Open (was: Triage Needed) > SAI post-filtering reads don't update local table latency metrics > - > > Key: CASSANDRA-18940 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18940 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index, Feature/SAI, Observability/Metrics >Reporter: Caleb Rackliffe >Assignee: Caleb Rackliffe >Priority: Normal > Attachments: > draft_fix_for_SAI_post-filtering_reads_not_updating_local_table_metrics.patch > > > Once an SAI index finds matches (primary keys), it reads the associated rows > and post-filters them to incorporate partial writes, tombstones, etc. > However, those reads are not currently updating the local table latency > metrics. It should be simple enough to attach a metrics recording > transformation to the iterator produced by querying local storage. (I've > attached a patch that should apply cleanly to trunk, but there may be a > better way...) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18940) SAI post-filtering reads don't update local table latency metrics
Caleb Rackliffe created CASSANDRA-18940: --- Summary: SAI post-filtering reads don't update local table latency metrics Key: CASSANDRA-18940 URL: https://issues.apache.org/jira/browse/CASSANDRA-18940 Project: Cassandra Issue Type: Bug Components: Feature/2i Index, Feature/SAI, Observability/Metrics Reporter: Caleb Rackliffe Assignee: Caleb Rackliffe Attachments: draft_fix_for_SAI_post-filtering_reads_not_updating_local_table_metrics.patch Once an SAI index finds matches (primary keys), it reads the associated rows and post-filters them to incorporate partial writes, tombstones, etc. However, those reads are not currently updating the local table latency metrics. It should be simple enough to attach a metrics recording transformation to the iterator produced by querying local storage. (I've attached a patch that should apply cleanly to trunk, but there may be a better way...) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18923) BLOG - Apache Cassandra 5.0 Features: Dynamic Data Masking
[ https://issues.apache.org/jira/browse/CASSANDRA-18923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Diogenese Topper updated CASSANDRA-18923: - Reviewers: Michael Semb Wever Status: Review In Progress (was: Patch Available) > BLOG - Apache Cassandra 5.0 Features: Dynamic Data Masking > -- > > Key: CASSANDRA-18923 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18923 > Project: Cassandra > Issue Type: Task > Components: Documentation/Blog >Reporter: Diogenese Topper >Priority: Normal > > This ticket is to capture the work associated with publishing the blog > "Apache Cassandra 5.0 Features: Dynamic Data Masking" > This blog can be published as soon as possible, but if it cannot be published > within a week of the noted publish date *(October 11)*, please contact me, > suggest changes, or correct the date when possible in the pull request for > the appropriate time that the blog will go live (on both the blog.adoc and > the blog post's file). -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18923) BLOG - Apache Cassandra 5.0 Features: Dynamic Data Masking
[ https://issues.apache.org/jira/browse/CASSANDRA-18923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Diogenese Topper updated CASSANDRA-18923: - Status: Ready to Commit (was: Review In Progress) > BLOG - Apache Cassandra 5.0 Features: Dynamic Data Masking > -- > > Key: CASSANDRA-18923 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18923 > Project: Cassandra > Issue Type: Task > Components: Documentation/Blog >Reporter: Diogenese Topper >Priority: Normal > > This ticket is to capture the work associated with publishing the blog > "Apache Cassandra 5.0 Features: Dynamic Data Masking" > This blog can be published as soon as possible, but if it cannot be published > within a week of the noted publish date *(October 11)*, please contact me, > suggest changes, or correct the date when possible in the pull request for > the appropriate time that the blog will go live (on both the blog.adoc and > the blog post's file). -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18923) BLOG - Apache Cassandra 5.0 Features: Dynamic Data Masking
[ https://issues.apache.org/jira/browse/CASSANDRA-18923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Diogenese Topper updated CASSANDRA-18923: - Status: Patch Available (was: Open) > BLOG - Apache Cassandra 5.0 Features: Dynamic Data Masking > -- > > Key: CASSANDRA-18923 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18923 > Project: Cassandra > Issue Type: Task > Components: Documentation/Blog >Reporter: Diogenese Topper >Priority: Normal > > This ticket is to capture the work associated with publishing the blog > "Apache Cassandra 5.0 Features: Dynamic Data Masking" > This blog can be published as soon as possible, but if it cannot be published > within a week of the noted publish date *(October 11)*, please contact me, > suggest changes, or correct the date when possible in the pull request for > the appropriate time that the blog will go live (on both the blog.adoc and > the blog post's file). -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18923) BLOG - Apache Cassandra 5.0 Features: Dynamic Data Masking
[ https://issues.apache.org/jira/browse/CASSANDRA-18923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Diogenese Topper updated CASSANDRA-18923: - Status: In Progress (was: Patch Available) > BLOG - Apache Cassandra 5.0 Features: Dynamic Data Masking > -- > > Key: CASSANDRA-18923 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18923 > Project: Cassandra > Issue Type: Task > Components: Documentation/Blog >Reporter: Diogenese Topper >Priority: Normal > > This ticket is to capture the work associated with publishing the blog > "Apache Cassandra 5.0 Features: Dynamic Data Masking" > This blog can be published as soon as possible, but if it cannot be published > within a week of the noted publish date *(October 11)*, please contact me, > suggest changes, or correct the date when possible in the pull request for > the appropriate time that the blog will go live (on both the blog.adoc and > the blog post's file). -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch asf-site updated (9fada0eff -> 8018ff370)
This is an automated email from the ASF dual-hosted git repository.
mck pushed a change to branch asf-site
in repository https://gitbox.apache.org/repos/asf/cassandra-website.git
discard 9fada0eff generate docs for db70fb96
add 327f02f61 BLOG - Apache Cassandra 5.0 Features: Dynamic Data Masking
add 1d0135e5f ninja-fix, escape asterisk, bold not intended
add 8018ff370 generate docs for 1d0135e5
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (9fada0eff)
\
N -- N -- N refs/heads/asf-site (8018ff370)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
content/_/blog.html| 24 +
...ssandra-5.0-Features-Dynamic-Data-Masking.html} | 325 ++
.../developing/cql/create-custom-index.html| 14 +
.../developing/cql/dynamic_data_masking.html | 11 +-
.../5.0/cassandra/developing/cql/functions.html| 9 +-
.../developing/cql/indexing/sai/sai-faq.html | 8 +-
.../cql/indexing/sai/sai-working-with.html | 2 +-
.../5.0/cassandra/managing/operating/metrics.html | 17 +-
.../cql-commands/compact-subproperties.html} | 666 -
.../developing/cql/create-custom-index.html| 14 +
.../developing/cql/dynamic_data_masking.html | 11 +-
.../5.1/cassandra/developing/cql/functions.html| 9 +-
.../developing/cql/indexing/sai/sai-faq.html | 8 +-
.../cql/indexing/sai/sai-working-with.html | 2 +-
.../5.1/cassandra/managing/operating/metrics.html | 17 +-
.../cql-commands/compact-subproperties.html} | 666 -
.../developing/cql/create-custom-index.html| 14 +
.../developing/cql/dynamic_data_masking.html | 11 +-
.../trunk/cassandra/developing/cql/functions.html | 9 +-
.../developing/cql/indexing/sai/sai-faq.html | 8 +-
.../cql/indexing/sai/sai-working-with.html | 2 +-
.../cassandra/managing/operating/metrics.html | 17 +-
.../cql-commands/compact-subproperties.html} | 666 -
content/search-index.js| 2 +-
site-content/source/modules/ROOT/pages/blog.adoc | 23 +
...assandra-5.0-Features-Dynamic-Data-Masking.adoc | 230 +++
site-ui/build/ui-bundle.zip| Bin 4881412 -> 4881412
bytes
27 files changed, 1721 insertions(+), 1064 deletions(-)
copy content/_/blog/{Apache-Cassandra-4.1-New-SSTable-Identifiers.html =>
Apache-Cassandra-5.0-Features-Dynamic-Data-Masking.html} (55%)
copy content/doc/5.0/cassandra/{overview/faq/index.html =>
reference/cql-commands/compact-subproperties.html} (70%)
copy content/doc/5.1/cassandra/{overview/faq/index.html =>
reference/cql-commands/compact-subproperties.html} (70%)
copy content/doc/{5.1/cassandra/overview/faq/index.html =>
trunk/cassandra/reference/cql-commands/compact-subproperties.html} (70%)
create mode 100644
site-content/source/modules/ROOT/pages/blog/Apache-Cassandra-5.0-Features-Dynamic-Data-Masking.adoc
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18912) Specify "since" in all Deprecated annotations
[
https://issues.apache.org/jira/browse/CASSANDRA-18912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776888#comment-17776888
]
Maxim Muzafarov commented on CASSANDRA-18912:
-
[~smiklosovic] Should we also update the code style page?
website code style page update
[https://github.com/apache/cassandra-website/pull/245/files]
> Specify "since" in all Deprecated annotations
> -
>
> Key: CASSANDRA-18912
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18912
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Core
>Reporter: Stefan Miklosovic
>Assignee: Maxim Muzafarov
>Priority: Normal
> Labels: pull-request-available
> Fix For: 5.0-alpha2, 5.1
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> It would be great if we introduced in 5.0 a change in Deprecated annotations
> like this:
> {code}
> @Deprecated(since = "4.0")
> {code}
> or
> {code}
> @Deprecated(since = "3.11")
> {code}
> The reasoning behind this is that as of now, it is pretty cumbersome to
> figure out what can be removed on the next major version. It has to be,
> basically, done manually every time.
> There is also this parameter available:
> {code}
> @Deprecated(forRemoval = true / false)
> {code}
> which indicates whether the annotated element is subject to removal in a
> future version so we do not need to think about this every time if it is
> eligible for deletion in a next major or not.
> We could then have a check which would ensure that we are not releasing a
> next major with some deprecations introduced two majors before.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18939) Creating a SASI index after creating an SAI index breaks secondary index queries
[
https://issues.apache.org/jira/browse/CASSANDRA-18939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776877#comment-17776877
]
Caleb Rackliffe commented on CASSANDRA-18939:
-
In {{QueryController#getIndex()}}, SASI should be using the version of
{{getBestIndexFor()}} that takes an index type filter, like SAI does.
ex. from SAI's {{QueryController#getContext()}}
{noformat}
Set indexes =
cfs.indexManager.getBestIndexFor(expression, StorageAttachedIndex.class);
{noformat}
> Creating a SASI index after creating an SAI index breaks secondary index
> queries
>
>
> Key: CASSANDRA-18939
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18939
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Jon Haddad
>Priority: Normal
> Fix For: 5.0.x, 5.x
>
>
> In the 5.0 branch, HEAD is e45c1092f91edd63591f562b2120ea6a5fd3edd5, I was
> able to break secondary indexes by doing the following:
> {code}
> cqlsh -e "create KEYSPACE test WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};"
> cqlsh -e "create table test.blah (id int primary key, val text);"
> cqlsh -e "create INDEX on test.blah (val) using 'sai';"
> bin/nodetool flush
> ❯ cqlsh -e "SELECT * from test.blah WHERE val = 'something';"
> id | val
> +-
> (0 rows)
> cqlsh -e "create custom INDEX on test.blah (val) using
> 'org.apache.cassandra.index.sasi.SASIIndex';"
> Warnings :
> SASI indexes are experimental and are not recommended for production use.
> cqlsh -e "SELECT * from test.blah WHERE val = 'something';"
> :1:ReadFailure: Error from server: code=1300 [Replica(s) failed to
> execute read] message="Operation failed - received 0 responses and 1
> failures: UNKNOWN from localhost/127.0.0.1:7000" info={'consistency': 'ONE',
> 'required_responses': 1, 'received_responses': 0, 'failures': 1,
> 'error_code_map': {'127.0.0.1': '0x'}}
> {code}
> Server throws an exception:
> {code}
> ERROR [ReadStage-2] 2023-10-18 12:09:42,391 JVMStabilityInspector.java:70 -
> Exception in thread Thread[ReadStage-2,10,SharedPool]
> java.lang.RuntimeException: java.lang.ClassCastException: class
> org.apache.cassandra.index.sai.StorageAttachedIndex cannot be cast to class
> org.apache.cassandra.index.sasi.SASIIndex
> (org.apache.cassandra.index.sai.StorageAttachedIndex and
> org.apache.cassandra.index.sasi.SASIIndex are in unnamed module of loader
> 'app')
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2585)
> at
> org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:829)
> Caused by: java.lang.ClassCastException: class
> org.apache.cassandra.index.sai.StorageAttachedIndex cannot be cast to class
> org.apache.cassandra.index.sasi.SASIIndex
> (org.apache.cassandra.index.sai.StorageAttachedIndex and
> org.apache.cassandra.index.sasi.SASIIndex are in unnamed module of loader
> 'app')
> at
> org.apache.cassandra.index.sasi.plan.QueryController.getIndex(QueryController.java:96)
> at
> org.apache.cassandra.index.sasi.plan.Operation.analyzeGroup(Operation.java:282)
> at
> org.apache.cassandra.index.sasi.plan.Operation$Builder.complete(Operation.java:433)
> at
> org.apache.cassandra.index.sasi.plan.SASIIndexSearcher.analyze(SASIIndexSearcher.java:65)
> at
> org.apache.cassandra.index.sasi.plan.SASIIndexSearcher.search(SASIIndexSearcher.java:77)
> at
> org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:425)
> at
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:2184)
> at org.apache.cassandra.service
> {code}
> Dropping the SASI index restores correct behavior.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-18939) Creating a SASI index after creating an SAI index breaks secondary index queries
[
https://issues.apache.org/jira/browse/CASSANDRA-18939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe reassigned CASSANDRA-18939:
---
Assignee: Caleb Rackliffe
> Creating a SASI index after creating an SAI index breaks secondary index
> queries
>
>
> Key: CASSANDRA-18939
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18939
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Jon Haddad
>Assignee: Caleb Rackliffe
>Priority: Normal
> Fix For: 5.0.x, 5.x
>
>
> In the 5.0 branch, HEAD is e45c1092f91edd63591f562b2120ea6a5fd3edd5, I was
> able to break secondary indexes by doing the following:
> {code}
> cqlsh -e "create KEYSPACE test WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};"
> cqlsh -e "create table test.blah (id int primary key, val text);"
> cqlsh -e "create INDEX on test.blah (val) using 'sai';"
> bin/nodetool flush
> ❯ cqlsh -e "SELECT * from test.blah WHERE val = 'something';"
> id | val
> +-
> (0 rows)
> cqlsh -e "create custom INDEX on test.blah (val) using
> 'org.apache.cassandra.index.sasi.SASIIndex';"
> Warnings :
> SASI indexes are experimental and are not recommended for production use.
> cqlsh -e "SELECT * from test.blah WHERE val = 'something';"
> :1:ReadFailure: Error from server: code=1300 [Replica(s) failed to
> execute read] message="Operation failed - received 0 responses and 1
> failures: UNKNOWN from localhost/127.0.0.1:7000" info={'consistency': 'ONE',
> 'required_responses': 1, 'received_responses': 0, 'failures': 1,
> 'error_code_map': {'127.0.0.1': '0x'}}
> {code}
> Server throws an exception:
> {code}
> ERROR [ReadStage-2] 2023-10-18 12:09:42,391 JVMStabilityInspector.java:70 -
> Exception in thread Thread[ReadStage-2,10,SharedPool]
> java.lang.RuntimeException: java.lang.ClassCastException: class
> org.apache.cassandra.index.sai.StorageAttachedIndex cannot be cast to class
> org.apache.cassandra.index.sasi.SASIIndex
> (org.apache.cassandra.index.sai.StorageAttachedIndex and
> org.apache.cassandra.index.sasi.SASIIndex are in unnamed module of loader
> 'app')
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2585)
> at
> org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:829)
> Caused by: java.lang.ClassCastException: class
> org.apache.cassandra.index.sai.StorageAttachedIndex cannot be cast to class
> org.apache.cassandra.index.sasi.SASIIndex
> (org.apache.cassandra.index.sai.StorageAttachedIndex and
> org.apache.cassandra.index.sasi.SASIIndex are in unnamed module of loader
> 'app')
> at
> org.apache.cassandra.index.sasi.plan.QueryController.getIndex(QueryController.java:96)
> at
> org.apache.cassandra.index.sasi.plan.Operation.analyzeGroup(Operation.java:282)
> at
> org.apache.cassandra.index.sasi.plan.Operation$Builder.complete(Operation.java:433)
> at
> org.apache.cassandra.index.sasi.plan.SASIIndexSearcher.analyze(SASIIndexSearcher.java:65)
> at
> org.apache.cassandra.index.sasi.plan.SASIIndexSearcher.search(SASIIndexSearcher.java:77)
> at
> org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:425)
> at
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:2184)
> at org.apache.cassandra.service
> {code}
> Dropping the SASI index restores correct behavior.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18905) Index.Group is incorrectly unregistered from the SecondaryIndexManager
[ https://issues.apache.org/jira/browse/CASSANDRA-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-18905: Since Version: 5.0-alpha1 Source Control Link: https://github.com/apache/cassandra/commit/e45c1092f91edd63591f562b2120ea6a5fd3edd5 Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed as [https://github.com/apache/cassandra/commit/e45c1092f91edd63591f562b2120ea6a5fd3edd5] > Index.Group is incorrectly unregistered from the SecondaryIndexManager > -- > > Key: CASSANDRA-18905 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18905 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Urgent > Fix For: 5.0-alpha2, 5.0, 5.1 > > > An Index.Group is removed from the SecondaryIndexManager during > unregisterIndex if it contains no indexes after the index is unregistered. > The code for removing the group uses the wrong key to remove the group from > the indexGroups map. It is using the group object rather than the group name > that is used as the key in the map. > This means that the group is not added again if a new index is registered > using that group. The knock on from this is that the > StorageAttachedIndexGroup unregisters itself from the Tracker when it has no > indexes after an index is removed. The same group with no tracker is then > used for new indexes. This group then receives no notifications about sstable > or memtable updates. The ultimate side effect of this is that, memtables are > not released, resulting in memory leaks and indexes are not updated with new > sstables and their associated index files. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18905) Index.Group is incorrectly unregistered from the SecondaryIndexManager
[ https://issues.apache.org/jira/browse/CASSANDRA-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-18905: Status: Ready to Commit (was: Review In Progress) > Index.Group is incorrectly unregistered from the SecondaryIndexManager > -- > > Key: CASSANDRA-18905 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18905 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Urgent > Fix For: 5.0-alpha2, 5.0, 5.1 > > > An Index.Group is removed from the SecondaryIndexManager during > unregisterIndex if it contains no indexes after the index is unregistered. > The code for removing the group uses the wrong key to remove the group from > the indexGroups map. It is using the group object rather than the group name > that is used as the key in the map. > This means that the group is not added again if a new index is registered > using that group. The knock on from this is that the > StorageAttachedIndexGroup unregisters itself from the Tracker when it has no > indexes after an index is removed. The same group with no tracker is then > used for new indexes. This group then receives no notifications about sstable > or memtable updates. The ultimate side effect of this is that, memtables are > not released, resulting in memory leaks and indexes are not updated with new > sstables and their associated index files. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18905) Index.Group is incorrectly unregistered from the SecondaryIndexManager
[ https://issues.apache.org/jira/browse/CASSANDRA-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-18905: Status: Review In Progress (was: Needs Committer) > Index.Group is incorrectly unregistered from the SecondaryIndexManager > -- > > Key: CASSANDRA-18905 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18905 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Urgent > Fix For: 5.0-alpha2, 5.0, 5.1 > > > An Index.Group is removed from the SecondaryIndexManager during > unregisterIndex if it contains no indexes after the index is unregistered. > The code for removing the group uses the wrong key to remove the group from > the indexGroups map. It is using the group object rather than the group name > that is used as the key in the map. > This means that the group is not added again if a new index is registered > using that group. The knock on from this is that the > StorageAttachedIndexGroup unregisters itself from the Tracker when it has no > indexes after an index is removed. The same group with no tracker is then > used for new indexes. This group then receives no notifications about sstable > or memtable updates. The ultimate side effect of this is that, memtables are > not released, resulting in memory leaks and indexes are not updated with new > sstables and their associated index files. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18939) Creating a SASI index after creating an SAI index breaks secondary index queries
[
https://issues.apache.org/jira/browse/CASSANDRA-18939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-18939:
-
Bug Category: Parent values: Correctness(12982)Level 1 values: Recoverable
Corruption / Loss(12986)
Complexity: Normal
Discovered By: User Report
Fix Version/s: 5.0.x
5.x
Severity: Normal
Status: Open (was: Triage Needed)
> Creating a SASI index after creating an SAI index breaks secondary index
> queries
>
>
> Key: CASSANDRA-18939
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18939
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Jon Haddad
>Priority: Normal
> Fix For: 5.0.x, 5.x
>
>
> In the 5.0 branch, HEAD is e45c1092f91edd63591f562b2120ea6a5fd3edd5, I was
> able to break secondary indexes by doing the following:
> {code}
> cqlsh -e "create KEYSPACE test WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};"
> cqlsh -e "create table test.blah (id int primary key, val text);"
> cqlsh -e "create INDEX on test.blah (val) using 'sai';"
> bin/nodetool flush
> ❯ cqlsh -e "SELECT * from test.blah WHERE val = 'something';"
> id | val
> +-
> (0 rows)
> cqlsh -e "create custom INDEX on test.blah (val) using
> 'org.apache.cassandra.index.sasi.SASIIndex';"
> Warnings :
> SASI indexes are experimental and are not recommended for production use.
> cqlsh -e "SELECT * from test.blah WHERE val = 'something';"
> :1:ReadFailure: Error from server: code=1300 [Replica(s) failed to
> execute read] message="Operation failed - received 0 responses and 1
> failures: UNKNOWN from localhost/127.0.0.1:7000" info={'consistency': 'ONE',
> 'required_responses': 1, 'received_responses': 0, 'failures': 1,
> 'error_code_map': {'127.0.0.1': '0x'}}
> {code}
> Server throws an exception:
> {code}
> ERROR [ReadStage-2] 2023-10-18 12:09:42,391 JVMStabilityInspector.java:70 -
> Exception in thread Thread[ReadStage-2,10,SharedPool]
> java.lang.RuntimeException: java.lang.ClassCastException: class
> org.apache.cassandra.index.sai.StorageAttachedIndex cannot be cast to class
> org.apache.cassandra.index.sasi.SASIIndex
> (org.apache.cassandra.index.sai.StorageAttachedIndex and
> org.apache.cassandra.index.sasi.SASIIndex are in unnamed module of loader
> 'app')
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2585)
> at
> org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:829)
> Caused by: java.lang.ClassCastException: class
> org.apache.cassandra.index.sai.StorageAttachedIndex cannot be cast to class
> org.apache.cassandra.index.sasi.SASIIndex
> (org.apache.cassandra.index.sai.StorageAttachedIndex and
> org.apache.cassandra.index.sasi.SASIIndex are in unnamed module of loader
> 'app')
> at
> org.apache.cassandra.index.sasi.plan.QueryController.getIndex(QueryController.java:96)
> at
> org.apache.cassandra.index.sasi.plan.Operation.analyzeGroup(Operation.java:282)
> at
> org.apache.cassandra.index.sasi.plan.Operation$Builder.complete(Operation.java:433)
> at
> org.apache.cassandra.index.sasi.plan.SASIIndexSearcher.analyze(SASIIndexSearcher.java:65)
> at
> org.apache.cassandra.index.sasi.plan.SASIIndexSearcher.search(SASIIndexSearcher.java:77)
> at
> org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:425)
> at
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:2184)
> at org.apache.cassandra.service
> {code}
> Dropping the SASI index restores correct behavior.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18939) Creating a SASI index after creating an SAI index breaks secondary index queries
[
https://issues.apache.org/jira/browse/CASSANDRA-18939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jon Haddad updated CASSANDRA-18939:
---
Summary: Creating a SASI index after creating an SAI index breaks secondary
index queries (was: Creating a SASI index after creating an SAI index breaks
secondary indexing)
> Creating a SASI index after creating an SAI index breaks secondary index
> queries
>
>
> Key: CASSANDRA-18939
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18939
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Jon Haddad
>Priority: Normal
>
> In the 5.0 branch, HEAD is e45c1092f91edd63591f562b2120ea6a5fd3edd5, I was
> able to break secondary indexes by doing the following:
> {code}
> cqlsh -e "create KEYSPACE test WITH replication = {'class': 'SimpleStrategy',
> 'replication_factor': 1};"
> cqlsh -e "create table test.blah (id int primary key, val text);"
> cqlsh -e "create INDEX on test.blah (val) using 'sai';"
> bin/nodetool flush
> ❯ cqlsh -e "SELECT * from test.blah WHERE val = 'something';"
> id | val
> +-
> (0 rows)
> cqlsh -e "create custom INDEX on test.blah (val) using
> 'org.apache.cassandra.index.sasi.SASIIndex';"
> Warnings :
> SASI indexes are experimental and are not recommended for production use.
> cqlsh -e "SELECT * from test.blah WHERE val = 'something';"
> :1:ReadFailure: Error from server: code=1300 [Replica(s) failed to
> execute read] message="Operation failed - received 0 responses and 1
> failures: UNKNOWN from localhost/127.0.0.1:7000" info={'consistency': 'ONE',
> 'required_responses': 1, 'received_responses': 0, 'failures': 1,
> 'error_code_map': {'127.0.0.1': '0x'}}
> {code}
> Server throws an exception:
> {code}
> ERROR [ReadStage-2] 2023-10-18 12:09:42,391 JVMStabilityInspector.java:70 -
> Exception in thread Thread[ReadStage-2,10,SharedPool]
> java.lang.RuntimeException: java.lang.ClassCastException: class
> org.apache.cassandra.index.sai.StorageAttachedIndex cannot be cast to class
> org.apache.cassandra.index.sasi.SASIIndex
> (org.apache.cassandra.index.sai.StorageAttachedIndex and
> org.apache.cassandra.index.sasi.SASIIndex are in unnamed module of loader
> 'app')
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2585)
> at
> org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:829)
> Caused by: java.lang.ClassCastException: class
> org.apache.cassandra.index.sai.StorageAttachedIndex cannot be cast to class
> org.apache.cassandra.index.sasi.SASIIndex
> (org.apache.cassandra.index.sai.StorageAttachedIndex and
> org.apache.cassandra.index.sasi.SASIIndex are in unnamed module of loader
> 'app')
> at
> org.apache.cassandra.index.sasi.plan.QueryController.getIndex(QueryController.java:96)
> at
> org.apache.cassandra.index.sasi.plan.Operation.analyzeGroup(Operation.java:282)
> at
> org.apache.cassandra.index.sasi.plan.Operation$Builder.complete(Operation.java:433)
> at
> org.apache.cassandra.index.sasi.plan.SASIIndexSearcher.analyze(SASIIndexSearcher.java:65)
> at
> org.apache.cassandra.index.sasi.plan.SASIIndexSearcher.search(SASIIndexSearcher.java:77)
> at
> org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:425)
> at
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:2184)
> at org.apache.cassandra.service
> {code}
> Dropping the SASI index restores correct behavior.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18939) Creating a SASI index after creating an SAI index breaks secondary indexing
Jon Haddad created CASSANDRA-18939:
--
Summary: Creating a SASI index after creating an SAI index breaks
secondary indexing
Key: CASSANDRA-18939
URL: https://issues.apache.org/jira/browse/CASSANDRA-18939
Project: Cassandra
Issue Type: Bug
Components: Feature/2i Index
Reporter: Jon Haddad
In the 5.0 branch, HEAD is e45c1092f91edd63591f562b2120ea6a5fd3edd5, I was able
to break secondary indexes by doing the following:
{code}
cqlsh -e "create KEYSPACE test WITH replication = {'class': 'SimpleStrategy',
'replication_factor': 1};"
cqlsh -e "create table test.blah (id int primary key, val text);"
cqlsh -e "create INDEX on test.blah (val) using 'sai';"
bin/nodetool flush
❯ cqlsh -e "SELECT * from test.blah WHERE val = 'something';"
id | val
+-
(0 rows)
cqlsh -e "create custom INDEX on test.blah (val) using
'org.apache.cassandra.index.sasi.SASIIndex';"
Warnings :
SASI indexes are experimental and are not recommended for production use.
cqlsh -e "SELECT * from test.blah WHERE val = 'something';"
:1:ReadFailure: Error from server: code=1300 [Replica(s) failed to
execute read] message="Operation failed - received 0 responses and 1 failures:
UNKNOWN from localhost/127.0.0.1:7000" info={'consistency': 'ONE',
'required_responses': 1, 'received_responses': 0, 'failures': 1,
'error_code_map': {'127.0.0.1': '0x'}}
{code}
Server throws an exception:
{code}
ERROR [ReadStage-2] 2023-10-18 12:09:42,391 JVMStabilityInspector.java:70 -
Exception in thread Thread[ReadStage-2,10,SharedPool]
java.lang.RuntimeException: java.lang.ClassCastException: class
org.apache.cassandra.index.sai.StorageAttachedIndex cannot be cast to class
org.apache.cassandra.index.sasi.SASIIndex
(org.apache.cassandra.index.sai.StorageAttachedIndex and
org.apache.cassandra.index.sasi.SASIIndex are in unnamed module of loader 'app')
at
org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2585)
at
org.apache.cassandra.concurrent.ExecutionFailure$2.run(ExecutionFailure.java:163)
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:143)
at
io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.ClassCastException: class
org.apache.cassandra.index.sai.StorageAttachedIndex cannot be cast to class
org.apache.cassandra.index.sasi.SASIIndex
(org.apache.cassandra.index.sai.StorageAttachedIndex and
org.apache.cassandra.index.sasi.SASIIndex are in unnamed module of loader 'app')
at
org.apache.cassandra.index.sasi.plan.QueryController.getIndex(QueryController.java:96)
at
org.apache.cassandra.index.sasi.plan.Operation.analyzeGroup(Operation.java:282)
at
org.apache.cassandra.index.sasi.plan.Operation$Builder.complete(Operation.java:433)
at
org.apache.cassandra.index.sasi.plan.SASIIndexSearcher.analyze(SASIIndexSearcher.java:65)
at
org.apache.cassandra.index.sasi.plan.SASIIndexSearcher.search(SASIIndexSearcher.java:77)
at
org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:425)
at
org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:2184)
at org.apache.cassandra.service
{code}
Dropping the SASI index restores correct behavior.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18929) CEP-15: (C*) Implement TopologySorter to prioritise hosts based on DynamicSnitch and/or topology layout
[ https://issues.apache.org/jira/browse/CASSANDRA-18929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Capwell updated CASSANDRA-18929: -- Test and Documentation Plan: tests added Status: Patch Available (was: In Progress) > CEP-15: (C*) Implement TopologySorter to prioritise hosts based on > DynamicSnitch and/or topology layout > --- > > Key: CASSANDRA-18929 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18929 > Project: Cassandra > Issue Type: Improvement > Components: Accord >Reporter: David Capwell >Assignee: David Capwell >Priority: Normal > Fix For: 5.x > > Time Spent: 10m > Remaining Estimate: 0h > > Implement TopologySorter to prioritise hosts based on DynamicSnitch and/or > topology layout -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-5.0' into trunk
This is an automated email from the ASF dual-hosted git repository. maedhroz pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 7d3a8d53123d8ac493d3fd3821ba54ace075f972 Merge: 6c18a6c4f4 e45c1092f9 Author: Caleb Rackliffe AuthorDate: Wed Oct 18 13:49:26 2023 -0500 Merge branch 'cassandra-5.0' into trunk * cassandra-5.0: Correctly remove Index.Group from IndexRegistry CHANGES.txt| 1 + .../org/apache/cassandra/db/lifecycle/Tracker.java | 8 ++- src/java/org/apache/cassandra/index/Index.java | 64 +++-- .../org/apache/cassandra/index/IndexRegistry.java | 26 +-- .../cassandra/index/SecondaryIndexManager.java | 60 +++- .../cassandra/index/SingletonIndexGroup.java | 17 + .../cassandra/index/sai/StorageAttachedIndex.java | 8 ++- .../index/sai/StorageAttachedIndexGroup.java | 17 +++-- .../org/apache/cassandra/index/sasi/SASIIndex.java | 2 +- .../apache/cassandra/index/CustomIndexTest.java| 75 +--- .../org/apache/cassandra/index/StubIndexGroup.java | 6 ++ .../index/sai/cql/IndexGroupLifecycleTest.java | 81 ++ .../index/sai/cql/StorageAttachedIndexDDLTest.java | 6 +- .../index/sai/functional/CompactionTest.java | 5 +- .../index/sai/metrics/IndexGroupMetricsTest.java | 4 +- .../index/sai/metrics/QueryMetricsTest.java| 6 +- .../index/sai/metrics/StateMetricsTest.java| 6 +- 17 files changed, 261 insertions(+), 131 deletions(-) diff --cc CHANGES.txt index 47703336da,9a49af3955..0a8684d7d6 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,8 -1,5 +1,9 @@@ -5.0-alpha2 +5.1 + * Add ELAPSED command to cqlsh (CASSANDRA-18861) + * Add the ability to disable bulk loading of SSTables (CASSANDRA-18781) + * Clean up obsolete functions and simplify cql_version handling in cqlsh (CASSANDRA-18787) +Merged from 5.0: + * Correctly remove Index.Group from IndexRegistry (CASSANDRA-18905) * Fix vector type to support DDM's mask_default function (CASSANDRA-18889) * Remove unnecessary reporter-config3 dependency (CASSANDRA-18907) * Remove support for empty values on the vector data type (CASSANDRA-18876) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-5.0 updated: Correctly remove Index.Group from IndexRegistry
This is an automated email from the ASF dual-hosted git repository.
maedhroz pushed a commit to branch cassandra-5.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-5.0 by this push:
new e45c1092f9 Correctly remove Index.Group from IndexRegistry
e45c1092f9 is described below
commit e45c1092f91edd63591f562b2120ea6a5fd3edd5
Author: Mike Adamson
AuthorDate: Wed Oct 4 11:27:50 2023 +0100
Correctly remove Index.Group from IndexRegistry
The Index.Group was being left in the list indexGroups in the
SecondaryIndexManager because the incorrect
key was being used to remove it from the map
patch by Mike Adamson; reviewed by Caleb Rackliffe and Zhao Yang for
CASSANDRA-18905
Co-authored-by: Zhao Yang
---
CHANGES.txt| 1 +
.../org/apache/cassandra/db/lifecycle/Tracker.java | 8 ++-
src/java/org/apache/cassandra/index/Index.java | 64 +++--
.../org/apache/cassandra/index/IndexRegistry.java | 26 +--
.../cassandra/index/SecondaryIndexManager.java | 60 +++-
.../cassandra/index/SingletonIndexGroup.java | 17 +
.../cassandra/index/sai/StorageAttachedIndex.java | 8 ++-
.../index/sai/StorageAttachedIndexGroup.java | 17 +++--
.../org/apache/cassandra/index/sasi/SASIIndex.java | 2 +-
.../apache/cassandra/index/CustomIndexTest.java| 75 +---
.../org/apache/cassandra/index/StubIndexGroup.java | 6 ++
.../index/sai/cql/IndexGroupLifecycleTest.java | 81 ++
.../index/sai/cql/StorageAttachedIndexDDLTest.java | 6 +-
.../index/sai/functional/CompactionTest.java | 5 +-
.../index/sai/metrics/IndexGroupMetricsTest.java | 4 +-
.../index/sai/metrics/QueryMetricsTest.java| 6 +-
.../index/sai/metrics/StateMetricsTest.java| 6 +-
17 files changed, 261 insertions(+), 131 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 9377b7a421..9a49af3955 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
5.0-alpha2
+ * Correctly remove Index.Group from IndexRegistry (CASSANDRA-18905)
* Fix vector type to support DDM's mask_default function (CASSANDRA-18889)
* Remove unnecessary reporter-config3 dependency (CASSANDRA-18907)
* Remove support for empty values on the vector data type (CASSANDRA-18876)
diff --git a/src/java/org/apache/cassandra/db/lifecycle/Tracker.java
b/src/java/org/apache/cassandra/db/lifecycle/Tracker.java
index 061765bd52..e959c72fa9 100644
--- a/src/java/org/apache/cassandra/db/lifecycle/Tracker.java
+++ b/src/java/org/apache/cassandra/db/lifecycle/Tracker.java
@@ -86,7 +86,7 @@ public class Tracker
{
private static final Logger logger =
LoggerFactory.getLogger(Tracker.class);
-private final Collection subscribers = new
CopyOnWriteArrayList<>();
+private final List subscribers = new
CopyOnWriteArrayList<>();
public final ColumnFamilyStore cfstore;
final AtomicReference view;
@@ -560,6 +560,12 @@ public class Tracker
subscribers.add(consumer);
}
+@VisibleForTesting
+public boolean contains(INotificationConsumer consumer)
+{
+return subscribers.contains(consumer);
+}
+
public void unsubscribe(INotificationConsumer consumer)
{
subscribers.remove(consumer);
diff --git a/src/java/org/apache/cassandra/index/Index.java
b/src/java/org/apache/cassandra/index/Index.java
index f116fdb3e0..fc5f5a6f00 100644
--- a/src/java/org/apache/cassandra/index/Index.java
+++ b/src/java/org/apache/cassandra/index/Index.java
@@ -23,6 +23,7 @@ package org.apache.cassandra.index;
import java.io.UncheckedIOException;
import java.util.Collection;
import java.util.Collections;
+import java.util.Objects;
import java.util.Optional;
import java.util.Set;
import java.util.concurrent.Callable;
@@ -55,8 +56,8 @@ import
org.apache.cassandra.index.transactions.IndexTransaction;
import org.apache.cassandra.io.sstable.Component;
import org.apache.cassandra.io.sstable.Descriptor;
import org.apache.cassandra.io.sstable.ReducingKeyIterator;
-import org.apache.cassandra.io.sstable.SSTableFlushObserver;
import org.apache.cassandra.io.sstable.SSTable;
+import org.apache.cassandra.io.sstable.SSTableFlushObserver;
import org.apache.cassandra.io.sstable.format.SSTableReader;
import org.apache.cassandra.schema.ColumnMetadata;
import org.apache.cassandra.schema.IndexMetadata;
@@ -275,6 +276,17 @@ public interface Index
*/
public void register(IndexRegistry registry);
+/**
+ * Unregister current index when it's removed from system
+ *
+ * @param registry the index registry to register the instance with
+ */
+default void unregister(IndexRegistry registry)
+{
+// for singleton index, the group key is the index itself
+registry.unregisterIndex(this, new Index.Group.Key(this));
+}
+
/**
* If the
[cassandra] branch trunk updated (6c18a6c4f4 -> 7d3a8d5312)
This is an automated email from the ASF dual-hosted git repository. maedhroz pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git from 6c18a6c4f4 Merge branch 'cassandra-5.0' into trunk new e45c1092f9 Correctly remove Index.Group from IndexRegistry new 7d3a8d5312 Merge branch 'cassandra-5.0' into trunk The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt| 1 + .../org/apache/cassandra/db/lifecycle/Tracker.java | 8 ++- src/java/org/apache/cassandra/index/Index.java | 64 +++-- .../org/apache/cassandra/index/IndexRegistry.java | 26 +-- .../cassandra/index/SecondaryIndexManager.java | 60 +++- .../cassandra/index/SingletonIndexGroup.java | 17 + .../cassandra/index/sai/StorageAttachedIndex.java | 8 ++- .../index/sai/StorageAttachedIndexGroup.java | 17 +++-- .../org/apache/cassandra/index/sasi/SASIIndex.java | 2 +- .../apache/cassandra/index/CustomIndexTest.java| 75 +--- .../org/apache/cassandra/index/StubIndexGroup.java | 6 ++ .../index/sai/cql/IndexGroupLifecycleTest.java | 81 ++ .../index/sai/cql/StorageAttachedIndexDDLTest.java | 6 +- .../index/sai/functional/CompactionTest.java | 5 +- .../index/sai/metrics/IndexGroupMetricsTest.java | 4 +- .../index/sai/metrics/QueryMetricsTest.java| 6 +- .../index/sai/metrics/StateMetricsTest.java| 6 +- 17 files changed, 261 insertions(+), 131 deletions(-) create mode 100644 test/unit/org/apache/cassandra/index/sai/cql/IndexGroupLifecycleTest.java - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18938) Map and possibly remove Deprecated code
[ https://issues.apache.org/jira/browse/CASSANDRA-18938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-18938: -- Change Category: Code Clarity Complexity: Normal Fix Version/s: 5.x Status: Open (was: Triage Needed) > Map and possibly remove Deprecated code > --- > > Key: CASSANDRA-18938 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18938 > Project: Cassandra > Issue Type: Task > Components: Build >Reporter: Stefan Miklosovic >Assignee: Stefan Miklosovic >Priority: Normal > Fix For: 5.x > > > This is follow-up of CASSANDRA-18912. We should scan the project and document > what was deprecated and when (based on "since" added to Deprecated > annotation). > Once this is done, we need to identify what should be removed and what should > stay. > It is questionable if this ticket will result in some actual code change. I > think we should firstly just map what was deprecated to have a bigger picture > where we are. Then we can discuss what to actually remove. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18938) Map and possibly remove Deprecated code
Stefan Miklosovic created CASSANDRA-18938: - Summary: Map and possibly remove Deprecated code Key: CASSANDRA-18938 URL: https://issues.apache.org/jira/browse/CASSANDRA-18938 Project: Cassandra Issue Type: Task Components: Build Reporter: Stefan Miklosovic Assignee: Stefan Miklosovic This is follow-up of CASSANDRA-18912. We should scan the project and document what was deprecated and when (based on "since" added to Deprecated annotation). Once this is done, we need to identify what should be removed and what should stay. It is questionable if this ticket will result in some actual code change. I think we should firstly just map what was deprecated to have a bigger picture where we are. Then we can discuss what to actually remove. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18912) Specify "since" in all Deprecated annotations
[
https://issues.apache.org/jira/browse/CASSANDRA-18912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-18912:
--
Fix Version/s: 5.0-alpha2
5.1
Source Control Link:
https://github.com/apache/cassandra/commit/269285213d12f9e549f735b93f77d08d36dbbfb8
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> Specify "since" in all Deprecated annotations
> -
>
> Key: CASSANDRA-18912
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18912
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Core
>Reporter: Stefan Miklosovic
>Assignee: Maxim Muzafarov
>Priority: Normal
> Labels: pull-request-available
> Fix For: 5.0-alpha2, 5.1
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> It would be great if we introduced in 5.0 a change in Deprecated annotations
> like this:
> {code}
> @Deprecated(since = "4.0")
> {code}
> or
> {code}
> @Deprecated(since = "3.11")
> {code}
> The reasoning behind this is that as of now, it is pretty cumbersome to
> figure out what can be removed on the next major version. It has to be,
> basically, done manually every time.
> There is also this parameter available:
> {code}
> @Deprecated(forRemoval = true / false)
> {code}
> which indicates whether the annotated element is subject to removal in a
> future version so we do not need to think about this every time if it is
> eligible for deletion in a next major or not.
> We could then have a check which would ensure that we are not releasing a
> next major with some deprecations introduced two majors before.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-5.0 updated (a4fc03e799 -> 269285213d)
This is an automated email from the ASF dual-hosted git repository. smiklosovic pushed a change to branch cassandra-5.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git from a4fc03e799 Merge branch 'cassandra-4.1' into cassandra-5.0 add 269285213d Add versions into Deprecated annotation No new revisions were added by this update. Summary of changes: .build/checkstyle.xml | 8 + .build/checkstyle_test.xml | 8 + .../org/apache/cassandra/auth/IAuthenticator.java | 8 +- .../cassandra/auth/IInternodeAuthenticator.java| 3 +- .../auth/NetworkPermissionsCacheMBean.java | 3 +- .../cassandra/auth/PasswordAuthenticator.java | 2 +- src/java/org/apache/cassandra/auth/Permission.java | 6 +- src/java/org/apache/cassandra/auth/Resources.java | 9 +- src/java/org/apache/cassandra/auth/Role.java | 3 +- .../cassandra/auth/jmx/AuthorizationProxy.java | 3 +- .../cassandra/concurrent/ExecutorFactory.java | 2 +- .../concurrent/ResizableThreadPoolMXBean.java | 16 +- .../config/CassandraRelevantProperties.java| 15 +- src/java/org/apache/cassandra/config/Config.java | 50 +++-- .../org/apache/cassandra/config/DataRateSpec.java | 3 +- .../cassandra/config/DatabaseDescriptor.java | 3 +- .../statements/schema/AlterTableStatement.java | 18 +- .../org/apache/cassandra/db/ColumnFamilyStore.java | 11 +- .../cassandra/db/ColumnFamilyStoreMBean.java | 13 +- src/java/org/apache/cassandra/db/IMutation.java| 2 +- src/java/org/apache/cassandra/db/LivenessInfo.java | 2 - .../org/apache/cassandra/db/SystemKeyspace.java| 33 ++-- .../db/commitlog/CommitLogSegmentManagerCDC.java | 2 +- .../cassandra/db/compaction/CompactionManager.java | 4 +- .../org/apache/cassandra/db/filter/DataLimits.java | 6 +- .../org/apache/cassandra/db/filter/RowFilter.java | 3 +- .../db/guardrails/GuardrailsConfigProvider.java| 4 +- .../org/apache/cassandra/db/marshal/DateType.java | 4 +- .../apache/cassandra/db/marshal/IntegerType.java | 10 +- .../org/apache/cassandra/db/memtable/Memtable.java | 4 +- src/java/org/apache/cassandra/db/rows/Row.java | 3 +- .../db/rows/ThrottledUnfilteredIterator.java | 2 +- .../cassandra/db/rows/UnfilteredSerializer.java| 4 +- .../org/apache/cassandra/db/view/TableViews.java | 2 +- .../org/apache/cassandra/dht/IPartitioner.java | 3 +- .../cassandra/dht/RangeFetchMapCalculator.java | 2 +- .../org/apache/cassandra/gms/ApplicationState.java | 9 +- .../apache/cassandra/gms/FailureDetectorMBean.java | 12 +- src/java/org/apache/cassandra/gms/Gossiper.java| 3 +- .../org/apache/cassandra/gms/VersionedValue.java | 3 +- .../org/apache/cassandra/hints/HintsService.java | 2 +- .../cassandra/index/sai/disk/SSTableIndex.java | 2 +- .../cassandra/index/sai/disk/format/Version.java | 2 +- .../cassandra/io/sstable/CQLSSTableWriter.java | 4 +- .../cassandra/io/sstable/SSTableRewriter.java | 3 +- .../sstable/format/CompressionInfoComponent.java | 6 +- .../cassandra/io/sstable/format/TOCComponent.java | 2 +- .../cassandra/io/sstable/format/Version.java | 6 +- .../apache/cassandra/io/util/DataOutputPlus.java | 6 +- .../org/apache/cassandra/io/util/FileUtils.java| 48 +++-- .../locator/AbstractReplicationStrategy.java | 3 +- .../apache/cassandra/locator/CloudstackSnitch.java | 4 +- .../locator/DynamicEndpointSnitchMBean.java| 3 +- .../apache/cassandra/locator/TokenMetadata.java| 11 +- .../metrics/CassandraMetricsRegistry.java | 2 +- .../cassandra/metrics/ReadRepairMetrics.java | 6 +- .../apache/cassandra/metrics/ScalingReservoir.java | 2 +- .../apache/cassandra/metrics/StreamingMetrics.java | 3 +- src/java/org/apache/cassandra/net/InboundSink.java | 3 +- .../org/apache/cassandra/net/MessagingService.java | 12 +- .../cassandra/net/MessagingServiceMBean.java | 39 ++-- src/java/org/apache/cassandra/net/Verb.java| 6 +- .../org/apache/cassandra/repair/SharedContext.java | 2 +- .../apache/cassandra/schema/CompressionParams.java | 3 +- .../apache/cassandra/schema/DistributedSchema.java | 2 +- src/java/org/apache/cassandra/schema/Schema.java | 4 +- .../schema/SchemaUpdateHandlerFactoryProvider.java | 4 +- .../org/apache/cassandra/schema/TableMetadata.java | 5 +- .../security/PEMBasedSslContextFactory.java| 10 +- .../cassandra/service/ActiveRepairService.java | 12 +- .../service/ActiveRepairServiceMBean.java | 14 +- .../org/apache/cassandra/service/StorageProxy.java | 10 +- .../cassandra/service/StorageProxyMBean.java | 13 +- .../apache/cassandra/service/StorageService.java | 98 ++ .../cassandra/service/StorageServiceMBean.java | 203 +
[cassandra] branch trunk updated (2feeb39d5a -> 6c18a6c4f4)
This is an automated email from the ASF dual-hosted git repository. smiklosovic pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git from 2feeb39d5a Merge branch 'cassandra-5.0' into trunk add 269285213d Add versions into Deprecated annotation new 6c18a6c4f4 Merge branch 'cassandra-5.0' into trunk The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .build/checkstyle.xml | 8 + .build/checkstyle_test.xml | 8 + .../org/apache/cassandra/auth/IAuthenticator.java | 8 +- .../cassandra/auth/IInternodeAuthenticator.java| 3 +- .../auth/NetworkPermissionsCacheMBean.java | 3 +- .../cassandra/auth/PasswordAuthenticator.java | 2 +- src/java/org/apache/cassandra/auth/Permission.java | 6 +- src/java/org/apache/cassandra/auth/Resources.java | 9 +- src/java/org/apache/cassandra/auth/Role.java | 3 +- .../cassandra/auth/jmx/AuthorizationProxy.java | 3 +- .../cassandra/concurrent/ExecutorFactory.java | 2 +- .../concurrent/ResizableThreadPoolMXBean.java | 16 +- .../config/CassandraRelevantProperties.java| 15 +- src/java/org/apache/cassandra/config/Config.java | 50 +++-- .../org/apache/cassandra/config/DataRateSpec.java | 3 +- .../cassandra/config/DatabaseDescriptor.java | 3 +- .../statements/schema/AlterTableStatement.java | 18 +- .../org/apache/cassandra/db/ColumnFamilyStore.java | 11 +- .../cassandra/db/ColumnFamilyStoreMBean.java | 13 +- src/java/org/apache/cassandra/db/IMutation.java| 2 +- src/java/org/apache/cassandra/db/LivenessInfo.java | 2 - .../org/apache/cassandra/db/SystemKeyspace.java| 33 ++-- .../db/commitlog/CommitLogSegmentManagerCDC.java | 2 +- .../cassandra/db/compaction/CompactionManager.java | 4 +- .../org/apache/cassandra/db/filter/DataLimits.java | 6 +- .../org/apache/cassandra/db/filter/RowFilter.java | 3 +- .../db/guardrails/GuardrailsConfigProvider.java| 4 +- .../org/apache/cassandra/db/marshal/DateType.java | 4 +- .../apache/cassandra/db/marshal/IntegerType.java | 10 +- .../org/apache/cassandra/db/memtable/Memtable.java | 4 +- src/java/org/apache/cassandra/db/rows/Row.java | 3 +- .../db/rows/ThrottledUnfilteredIterator.java | 2 +- .../cassandra/db/rows/UnfilteredSerializer.java| 4 +- .../org/apache/cassandra/db/view/TableViews.java | 2 +- .../org/apache/cassandra/dht/IPartitioner.java | 3 +- .../cassandra/dht/RangeFetchMapCalculator.java | 2 +- .../org/apache/cassandra/gms/ApplicationState.java | 9 +- .../apache/cassandra/gms/FailureDetectorMBean.java | 12 +- src/java/org/apache/cassandra/gms/Gossiper.java| 3 +- .../org/apache/cassandra/gms/VersionedValue.java | 3 +- .../org/apache/cassandra/hints/HintsService.java | 2 +- .../cassandra/index/sai/disk/SSTableIndex.java | 2 +- .../cassandra/index/sai/disk/format/Version.java | 2 +- .../cassandra/io/sstable/CQLSSTableWriter.java | 4 +- .../cassandra/io/sstable/SSTableRewriter.java | 3 +- .../sstable/format/CompressionInfoComponent.java | 6 +- .../cassandra/io/sstable/format/TOCComponent.java | 2 +- .../cassandra/io/sstable/format/Version.java | 6 +- .../apache/cassandra/io/util/DataOutputPlus.java | 6 +- .../org/apache/cassandra/io/util/FileUtils.java| 48 +++-- .../locator/AbstractReplicationStrategy.java | 3 +- .../apache/cassandra/locator/CloudstackSnitch.java | 4 +- .../locator/DynamicEndpointSnitchMBean.java| 3 +- .../apache/cassandra/locator/TokenMetadata.java| 11 +- .../metrics/CassandraMetricsRegistry.java | 2 +- .../cassandra/metrics/ReadRepairMetrics.java | 6 +- .../apache/cassandra/metrics/ScalingReservoir.java | 2 +- .../apache/cassandra/metrics/StreamingMetrics.java | 3 +- src/java/org/apache/cassandra/net/InboundSink.java | 3 +- .../org/apache/cassandra/net/MessagingService.java | 12 +- .../cassandra/net/MessagingServiceMBean.java | 39 ++-- src/java/org/apache/cassandra/net/Verb.java| 6 +- .../org/apache/cassandra/repair/SharedContext.java | 2 +- .../apache/cassandra/schema/CompressionParams.java | 3 +- .../apache/cassandra/schema/DistributedSchema.java | 2 +- src/java/org/apache/cassandra/schema/Schema.java | 4 +- .../schema/SchemaUpdateHandlerFactoryProvider.java | 4 +- .../org/apache/cassandra/schema/TableMetadata.java | 5 +- .../security/PEMBasedSslContextFactory.java| 10 +- .../cassandra/service/ActiveRepairService.java | 12 +- .../service/ActiveRepairServiceMBean.java | 14 +-
[cassandra] 01/01: Merge branch 'cassandra-5.0' into trunk
This is an automated email from the ASF dual-hosted git repository. smiklosovic pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 6c18a6c4f4eb2249a4bc546a720c91b328411699 Merge: 2feeb39d5a 269285213d Author: Stefan Miklosovic AuthorDate: Wed Oct 18 20:03:29 2023 +0200 Merge branch 'cassandra-5.0' into trunk .build/checkstyle.xml | 8 + .build/checkstyle_test.xml | 8 + .../org/apache/cassandra/auth/IAuthenticator.java | 8 +- .../cassandra/auth/IInternodeAuthenticator.java| 3 +- .../auth/NetworkPermissionsCacheMBean.java | 3 +- .../cassandra/auth/PasswordAuthenticator.java | 2 +- src/java/org/apache/cassandra/auth/Permission.java | 6 +- src/java/org/apache/cassandra/auth/Resources.java | 9 +- src/java/org/apache/cassandra/auth/Role.java | 3 +- .../cassandra/auth/jmx/AuthorizationProxy.java | 3 +- .../cassandra/concurrent/ExecutorFactory.java | 2 +- .../concurrent/ResizableThreadPoolMXBean.java | 16 +- .../config/CassandraRelevantProperties.java| 15 +- src/java/org/apache/cassandra/config/Config.java | 50 +++-- .../org/apache/cassandra/config/DataRateSpec.java | 3 +- .../cassandra/config/DatabaseDescriptor.java | 3 +- .../statements/schema/AlterTableStatement.java | 18 +- .../org/apache/cassandra/db/ColumnFamilyStore.java | 11 +- .../cassandra/db/ColumnFamilyStoreMBean.java | 13 +- src/java/org/apache/cassandra/db/IMutation.java| 2 +- src/java/org/apache/cassandra/db/LivenessInfo.java | 2 - .../org/apache/cassandra/db/SystemKeyspace.java| 33 ++-- .../db/commitlog/CommitLogSegmentManagerCDC.java | 2 +- .../cassandra/db/compaction/CompactionManager.java | 4 +- .../org/apache/cassandra/db/filter/DataLimits.java | 6 +- .../org/apache/cassandra/db/filter/RowFilter.java | 3 +- .../db/guardrails/GuardrailsConfigProvider.java| 4 +- .../org/apache/cassandra/db/marshal/DateType.java | 4 +- .../apache/cassandra/db/marshal/IntegerType.java | 10 +- .../org/apache/cassandra/db/memtable/Memtable.java | 4 +- src/java/org/apache/cassandra/db/rows/Row.java | 3 +- .../db/rows/ThrottledUnfilteredIterator.java | 2 +- .../cassandra/db/rows/UnfilteredSerializer.java| 4 +- .../org/apache/cassandra/db/view/TableViews.java | 2 +- .../org/apache/cassandra/dht/IPartitioner.java | 3 +- .../cassandra/dht/RangeFetchMapCalculator.java | 2 +- .../org/apache/cassandra/gms/ApplicationState.java | 9 +- .../apache/cassandra/gms/FailureDetectorMBean.java | 12 +- src/java/org/apache/cassandra/gms/Gossiper.java| 3 +- .../org/apache/cassandra/gms/VersionedValue.java | 3 +- .../org/apache/cassandra/hints/HintsService.java | 2 +- .../cassandra/index/sai/disk/SSTableIndex.java | 2 +- .../cassandra/index/sai/disk/format/Version.java | 2 +- .../cassandra/io/sstable/CQLSSTableWriter.java | 4 +- .../cassandra/io/sstable/SSTableRewriter.java | 3 +- .../sstable/format/CompressionInfoComponent.java | 6 +- .../cassandra/io/sstable/format/TOCComponent.java | 2 +- .../cassandra/io/sstable/format/Version.java | 6 +- .../apache/cassandra/io/util/DataOutputPlus.java | 6 +- .../org/apache/cassandra/io/util/FileUtils.java| 48 +++-- .../locator/AbstractReplicationStrategy.java | 3 +- .../apache/cassandra/locator/CloudstackSnitch.java | 4 +- .../locator/DynamicEndpointSnitchMBean.java| 3 +- .../apache/cassandra/locator/TokenMetadata.java| 11 +- .../metrics/CassandraMetricsRegistry.java | 2 +- .../cassandra/metrics/ReadRepairMetrics.java | 6 +- .../apache/cassandra/metrics/ScalingReservoir.java | 2 +- .../apache/cassandra/metrics/StreamingMetrics.java | 3 +- src/java/org/apache/cassandra/net/InboundSink.java | 3 +- .../org/apache/cassandra/net/MessagingService.java | 12 +- .../cassandra/net/MessagingServiceMBean.java | 39 ++-- src/java/org/apache/cassandra/net/Verb.java| 6 +- .../org/apache/cassandra/repair/SharedContext.java | 2 +- .../apache/cassandra/schema/CompressionParams.java | 3 +- .../apache/cassandra/schema/DistributedSchema.java | 2 +- src/java/org/apache/cassandra/schema/Schema.java | 4 +- .../schema/SchemaUpdateHandlerFactoryProvider.java | 4 +- .../org/apache/cassandra/schema/TableMetadata.java | 5 +- .../security/PEMBasedSslContextFactory.java| 10 +- .../cassandra/service/ActiveRepairService.java | 12 +- .../service/ActiveRepairServiceMBean.java | 14 +- .../org/apache/cassandra/service/StorageProxy.java | 10 +- .../cassandra/service/StorageProxyMBean.java | 13 +- .../apache/cassandra/service/StorageService.java | 98 ++ .../cassandra/service/StorageServiceMBean.java | 203 +
[jira] [Updated] (CASSANDRA-18905) Index.Group is incorrectly unregistered from the SecondaryIndexManager
[ https://issues.apache.org/jira/browse/CASSANDRA-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-18905: Fix Version/s: 5.0-alpha2 > Index.Group is incorrectly unregistered from the SecondaryIndexManager > -- > > Key: CASSANDRA-18905 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18905 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Urgent > Fix For: 5.0-alpha2, 5.0, 5.1 > > > An Index.Group is removed from the SecondaryIndexManager during > unregisterIndex if it contains no indexes after the index is unregistered. > The code for removing the group uses the wrong key to remove the group from > the indexGroups map. It is using the group object rather than the group name > that is used as the key in the map. > This means that the group is not added again if a new index is registered > using that group. The knock on from this is that the > StorageAttachedIndexGroup unregisters itself from the Tracker when it has no > indexes after an index is removed. The same group with no tracker is then > used for new indexes. This group then receives no notifications about sstable > or memtable updates. The ultimate side effect of this is that, memtables are > not released, resulting in memory leaks and indexes are not updated with new > sstables and their associated index files. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18905) Index.Group is incorrectly unregistered from the SecondaryIndexManager
[ https://issues.apache.org/jira/browse/CASSANDRA-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-18905: Reviewers: Caleb Rackliffe, Zhao Yang (was: Caleb Rackliffe) > Index.Group is incorrectly unregistered from the SecondaryIndexManager > -- > > Key: CASSANDRA-18905 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18905 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Urgent > Fix For: 5.0, 5.1 > > > An Index.Group is removed from the SecondaryIndexManager during > unregisterIndex if it contains no indexes after the index is unregistered. > The code for removing the group uses the wrong key to remove the group from > the indexGroups map. It is using the group object rather than the group name > that is used as the key in the map. > This means that the group is not added again if a new index is registered > using that group. The knock on from this is that the > StorageAttachedIndexGroup unregisters itself from the Tracker when it has no > indexes after an index is removed. The same group with no tracker is then > used for new indexes. This group then receives no notifications about sstable > or memtable updates. The ultimate side effect of this is that, memtables are > not released, resulting in memory leaks and indexes are not updated with new > sstables and their associated index files. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cep-21-tcm-review updated: JAVADOC: Startup and Discovery
This is an automated email from the ASF dual-hosted git repository.
jlewandowski pushed a commit to branch cep-21-tcm-review
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cep-21-tcm-review by this push:
new 064f07b13d JAVADOC: Startup and Discovery
064f07b13d is described below
commit 064f07b13df7a9fd8b0a4b24a8d53fa290bbf2bf
Author: Jacek Lewandowski
AuthorDate: Wed Oct 18 19:48:44 2023 +0200
JAVADOC: Startup and Discovery
---
src/java/org/apache/cassandra/tcm/Discovery.java | 8 ++
src/java/org/apache/cassandra/tcm/Startup.java | 34 +---
2 files changed, 38 insertions(+), 4 deletions(-)
diff --git a/src/java/org/apache/cassandra/tcm/Discovery.java
b/src/java/org/apache/cassandra/tcm/Discovery.java
index 5b94e328ae..992855de3c 100644
--- a/src/java/org/apache/cassandra/tcm/Discovery.java
+++ b/src/java/org/apache/cassandra/tcm/Discovery.java
@@ -67,6 +67,14 @@ public class Discovery
return discover(5);
}
+/**
+ * The discovery process starts with the configured seeds - a message is
sent to them asking for the list of the
+ * known CMS members. If a seed node knows about CMS members, it responds
with a collection of only those nodes.
+ * Otherwise, the seed node responds with a collection of all known peers.
They are accumulated as new target
+ * nodes to query for CMS members in the next round. The process completes
when either:
+ * - a response containing the CMS members is received
+ * - a deadline or a specified maximum number of rounds is reached
+ */
public DiscoveredNodes discover(int rounds)
{
boolean res = state.compareAndSet(State.NOT_STARTED,
State.IN_PROGRESS);
diff --git a/src/java/org/apache/cassandra/tcm/Startup.java
b/src/java/org/apache/cassandra/tcm/Startup.java
index f52a21f9fd..b341eafd05 100644
--- a/src/java/org/apache/cassandra/tcm/Startup.java
+++ b/src/java/org/apache/cassandra/tcm/Startup.java
@@ -147,6 +147,17 @@ public class Startup
}
}
+/**
+ * Discovers or elects CMS nodes and proceeds with the initialization.
+ *
+ * First, the node tries to discover CMS members in the cluster (see
{@link Discovery#discover(int)}). If no CMS
+ * members could be found, the node either tries to elect itself as a CMS
member or waits for another node to
+ * do so (self-election is attempted in case the node's broadcast
address/port is the lowest among the discovered
+ * peers).
+ *
+ * Then, the method keeps trying to fetch the log from any of the
discovered nodes until the log is fetched and
+ * some transformations are applied, which is indicated by increasing the
current epoch.
+ */
public static void initializeForDiscovery(Runnable initMessaging)
{
initMessaging.run();
@@ -275,20 +286,35 @@ public class Startup
enum StartupMode
{
/**
- * The node will initialize as a non-CMS node.
+ * The node will initialize as a non-CMS node. The startup mode is
selected when the node was started before
+ * and has witnessed some CMS transformations.
*/
NORMAL,
+
/**
- * The node will transition from the gossip protocol to CMS.
+ * The node will transition from the gossip protocol to CMS. The
startup mode is selected when the node was
+ * started before, but it has not witnessed any transformation yet.
*/
UPGRADE,
+
+/**
+ * Whether this node becomes a member of CMS or not is determined by
the election procedure (see {@link Election}).
+ * This startup mode is selected when the node was not started before,
and it does not satisfy the conditions
+ * for {@link #FIRST_CMS}.
+ */
VOTE,
+
/**
- * The node will start as the first node from the CMS
+ * The node will start as the first node from the CMS. The startup
mode is selected when the node was
+ * not started before, the only seed refers to this node, and the
listen address of this node is loopback.
+ * This startup mode is used for local development and CCM based
clusters where the only seed configured for
+ * the first node is self loopback address.
*/
FIRST_CMS,
+
/**
- * The node will use the existing {@code ClusterMetadata} provided
through a file
+ * The node will use the existing {@code ClusterMetadata} provided
through a file - it happens when
+ * {@link
CassandraRelevantProperties.TCM_UNSAFE_BOOT_WITH_CLUSTERMETADATA} is set.
*/
BOOT_WITH_CLUSTERMETADATA;
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18221) CEP-15: (Accord) Define a configuration system for Accord to allow overriding of internal configs and integrate with Cassandra yaml
[ https://issues.apache.org/jira/browse/CASSANDRA-18221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-18221: Reviewers: David Capwell (was: Caleb Rackliffe, David Capwell) Status: Review In Progress (was: Patch Available) > CEP-15: (Accord) Define a configuration system for Accord to allow overriding > of internal configs and integrate with Cassandra yaml > --- > > Key: CASSANDRA-18221 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18221 > Project: Cassandra > Issue Type: Improvement > Components: Accord >Reporter: David Capwell >Assignee: Jacek Lewandowski >Priority: Normal > Labels: pull-request-available > Fix For: 5.x > > Time Spent: 1h > Remaining Estimate: 0h > > Users are used to modifying cassandra via yaml and JMX (for dynamic configs) > but accord does not integrate with this right now; we should enhance Accord > to expose configs that are defined in cassandra yaml. > As an extension on this, we should figure out which configs are “dynamic” and > allow overriding via JMX or system vtable. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18221) CEP-15: (Accord) Define a configuration system for Accord to allow overriding of internal configs and integrate with Cassandra yaml
[ https://issues.apache.org/jira/browse/CASSANDRA-18221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Caleb Rackliffe updated CASSANDRA-18221: Status: Ready to Commit (was: Review In Progress) > CEP-15: (Accord) Define a configuration system for Accord to allow overriding > of internal configs and integrate with Cassandra yaml > --- > > Key: CASSANDRA-18221 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18221 > Project: Cassandra > Issue Type: Improvement > Components: Accord >Reporter: David Capwell >Assignee: Jacek Lewandowski >Priority: Normal > Labels: pull-request-available > Fix For: 5.x > > Time Spent: 1h > Remaining Estimate: 0h > > Users are used to modifying cassandra via yaml and JMX (for dynamic configs) > but accord does not integrate with this right now; we should enhance Accord > to expose configs that are defined in cassandra yaml. > As an extension on this, we should figure out which configs are “dynamic” and > allow overriding via JMX or system vtable. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18905) Index.Group is incorrectly unregistered from the SecondaryIndexManager
[ https://issues.apache.org/jira/browse/CASSANDRA-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776824#comment-17776824 ] Caleb Rackliffe commented on CASSANDRA-18905: - Thanks [~adelapena]. I'm going to go ahead and commit this... > Index.Group is incorrectly unregistered from the SecondaryIndexManager > -- > > Key: CASSANDRA-18905 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18905 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Urgent > Fix For: 5.0, 5.1 > > > An Index.Group is removed from the SecondaryIndexManager during > unregisterIndex if it contains no indexes after the index is unregistered. > The code for removing the group uses the wrong key to remove the group from > the indexGroups map. It is using the group object rather than the group name > that is used as the key in the map. > This means that the group is not added again if a new index is registered > using that group. The knock on from this is that the > StorageAttachedIndexGroup unregisters itself from the Tracker when it has no > indexes after an index is removed. The same group with no tracker is then > used for new indexes. This group then receives no notifications about sstable > or memtable updates. The ultimate side effect of this is that, memtables are > not released, resulting in memory leaks and indexes are not updated with new > sstables and their associated index files. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch asf-staging updated (9e99f5f52 -> 8018ff370)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git discard 9e99f5f52 generate docs for 1d0135e5 new 8018ff370 generate docs for 1d0135e5 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (9e99f5f52) \ N -- N -- N refs/heads/asf-staging (8018ff370) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: site-ui/build/ui-bundle.zip | Bin 4881412 -> 4881412 bytes 1 file changed, 0 insertions(+), 0 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18905) Index.Group is incorrectly unregistered from the SecondaryIndexManager
[
https://issues.apache.org/jira/browse/CASSANDRA-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776792#comment-17776792
]
Andres de la Peña commented on CASSANDRA-18905:
---
The test failures don't seem caused by the changes:
* {{testServiceTopPartitionsSingleTable}} is CASSANDRA-17798.
* {{simulationTest}} is not reported on Butler nor JIRA but can be reproduced
on
[5.0|https://app.circleci.com/pipelines/github/adelapena/cassandra/3253/workflows/e48b49e9-cf36-412a-a811-d813031e6f01/jobs/83735/tests]
and
[trunk|https://app.circleci.com/pipelines/github/adelapena/cassandra/3254/workflows/69f451ef-fb39-48e4-b1d1-40ee4141b0c1/jobs/83739/tests].
* {{explicitEndpointIgnore}} is not reported on Butler nor JIRA but can be
reproduced on
[trunk|https://app.circleci.com/pipelines/github/adelapena/cassandra/3254/workflows/69f451ef-fb39-48e4-b1d1-40ee4141b0c1/jobs/83738/tests].
I'll open tickets for the two unreported test failures.
> Index.Group is incorrectly unregistered from the SecondaryIndexManager
> --
>
> Key: CASSANDRA-18905
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18905
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/2i Index
>Reporter: Mike Adamson
>Assignee: Mike Adamson
>Priority: Urgent
> Fix For: 5.0, 5.1
>
>
> An Index.Group is removed from the SecondaryIndexManager during
> unregisterIndex if it contains no indexes after the index is unregistered.
> The code for removing the group uses the wrong key to remove the group from
> the indexGroups map. It is using the group object rather than the group name
> that is used as the key in the map.
> This means that the group is not added again if a new index is registered
> using that group. The knock on from this is that the
> StorageAttachedIndexGroup unregisters itself from the Tracker when it has no
> indexes after an index is removed. The same group with no tracker is then
> used for new indexes. This group then receives no notifications about sstable
> or memtable updates. The ultimate side effect of this is that, memtables are
> not released, resulting in memory leaks and indexes are not updated with new
> sstables and their associated index files.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cep-21-tcm updated: Add implementation overview doc
This is an automated email from the ASF dual-hosted git repository. samt pushed a commit to branch cep-21-tcm in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/cep-21-tcm by this push: new 357ca7c2e9 Add implementation overview doc 357ca7c2e9 is described below commit 357ca7c2e9e0515dd8f68821a81cbe820b4bf026 Author: Alex Petrov AuthorDate: Wed Oct 18 17:36:57 2023 +0100 Add implementation overview doc --- .../org/apache/cassandra/tcm/TCM_implementation.md | 57 ++ 1 file changed, 57 insertions(+) diff --git a/src/java/org/apache/cassandra/tcm/TCM_implementation.md b/src/java/org/apache/cassandra/tcm/TCM_implementation.md new file mode 100644 index 00..58cf01d5e3 --- /dev/null +++ b/src/java/org/apache/cassandra/tcm/TCM_implementation.md @@ -0,0 +1,57 @@ +# TCM Implementation Details + +This document will walk you through the core classes involved in Transactional Cluster Metadata. It describes a process of a node bringup into the existing TCM cluster. Each section will be prefixed by the header holding key classes that are used/described in the setion. + +## Startup: Discovery, Vote + +Boot process in TCM is very similar to the previously existing one, but is now split into several different classes rather than being mostly in `StorageService`. At first, `ClusterMetadataService` is initialized using `Startup#initialize`. Node determines its startup mode, which will be `Vote` in a usual case, which means that the node will initialize itself as a non-CMS node and will attempt to discover an existing CMS service or, failing that, participate in a vote to establish a new o [...] + +Node then continues startup, and eventually gets to `StorageService#initServer`, where it, among other things, gossips with CMS nodes to get a fresh view of the cluster for FD purposes, and then waits for Gossip to settle (again, for FD purposes). + +## Registration: ClusterMetadata, Transformation, RemoteProcessor + +Before joining the ring, the node has to register in order to obtain `NodeId`, which happens in `Register#maybeRegister`. Registration happens by committing a `Register` transformation using `ClusterMetadataService#commit` method. `Register` and other transformations are side-effect free functions mapping an instance of immutable `ClusterMetadata` to next `ClusterMetadata`. `ClusterMetadata` holds all information about cluster: directory of registered nodes, schema, node states and data [...] + +Since the node executing register is not a CMS node, it is going to use a `RemoteProcessor` in order to perform this commit. `RemoteProcessor` is a simple RPC tool that serializes transformation and attempts to execute it by contacting CMS nodes and sending them `TCM_COMMIT_REQ`. + +## Commit Request: PaxosBackedProcessor, DistributedMetadataLogKeyspace, Retry + +When a CMS node receives a commit request, it deserializes and attempts to execute the transformation using `PaxosBackedProcessor`. Paxos backed processor stores an entire cluster metadata log in the `cluster_metadata.``distributed_metadata_log` table. It performs a simple CAS LWT that attempts to append a new entry to the log with an `Epoch` that is strictly consecutive to the last one. `Epoch` is a monotonically incrementing counter of `ClusterMetadata` versions. + +Both remote and paxos-backed processors are using `Retry` class for managing retries. Remote processor sets a deadline for its retries using `tcm_await_timeout`. CMS-local processor permits itself to use at most `tcm_rpc_timeout` for its attempts to retry. + +`PaxosBackedProcessor` then attempts to execute `Transformation`. Result of the execution can be either `Success` or `Reject`. `Reject`s are not persisted in the log, and are linearized using a read that confirms that transformation was executed against the highest epoch. Examples of `Reject`s are validation errors, exceptions encountered while attempting to execute transformation, etc. For example, `Register` would return a rejection if a node with the same IP address already exists in [...] + +## Commit Response: Entry, LocalLog + +After `PaxosBackedProcessor` suceeds with committing the entry to the distributed log, it broadcasts the commit result that contains `Entry` holding newly appended transformation to the rest of the cluster using `Replicator` (which simply iterates all nodes in the directory, informing them about the new epoch). This operation does not need to be reliable and has no retries. In other words, if a node was down during CMS attempt to replicate entries to it, it will inevitably learn about th [...] + +Along with committed `Entry`, the response from CMS to the peer which submitted it also contains all entries that will allow the node that has initiated the commit to fully catch up to the epoch enacted by the committed transformation. + +When `RemoteProcessor` receives a
[jira] [Commented] (CASSANDRA-18798) Appending to list in Accord transactions uses insertion timestamp
[
https://issues.apache.org/jira/browse/CASSANDRA-18798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776780#comment-17776780
]
Henrik Ingo commented on CASSANDRA-18798:
-
Okay actually bug was in my own code after all. I had lost 10 microseconds
granularity. Seems to work now, pushed a commit, taking a break.
> Appending to list in Accord transactions uses insertion timestamp
> -
>
> Key: CASSANDRA-18798
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18798
> Project: Cassandra
> Issue Type: Bug
> Components: Accord
>Reporter: Jaroslaw Kijanowski
>Assignee: Henrik Ingo
>Priority: Normal
> Fix For: 5.0-alpha2
>
> Attachments: image-2023-09-26-20-05-25-846.png
>
>
> Given the following schema:
> {code:java}
> CREATE KEYSPACE IF NOT EXISTS accord WITH replication = {'class':
> 'SimpleStrategy', 'replication_factor': 3};
> CREATE TABLE IF NOT EXISTS accord.list_append(id int PRIMARY KEY,contents
> LIST);
> TRUNCATE accord.list_append;{code}
> And the following two possible queries executed by 10 threads in parallel:
> {code:java}
> BEGIN TRANSACTION
> LET row = (SELECT * FROM list_append WHERE id = ?);
> SELECT row.contents;
> COMMIT TRANSACTION;"
> BEGIN TRANSACTION
> UPDATE list_append SET contents += ? WHERE id = ?;
> COMMIT TRANSACTION;"
> {code}
> there seems to be an issue with transaction guarantees. Here's an excerpt in
> the edn format from a test.
> {code:java}
> {:type :invoke :process 8 :value [[:append 5 352]] :tid 3 :n 52
> :time 1692607285967116627}
> {:type :invoke :process 9 :value [[:r 5 nil]] :tid 1 :n 54
> :time 1692607286078732473}
> {:type :invoke :process 6 :value [[:append 5 553]] :tid 5 :n 53
> :time 1692607286133833428}
> {:type :invoke :process 7 :value [[:append 5 455]] :tid 4 :n 55
> :time 1692607286149702511}
> {:type :ok :process 8 :value [[:append 5 352]] :tid 3 :n 52
> :time 1692607286156314099}
> {:type :invoke :process 5 :value [[:r 5 nil]] :tid 9 :n 52
> :time 1692607286167090389}
> {:type :ok :process 9 :value [[:r 5 [303 304 604 6 306 509 909 409 912
> 411 514 415 719 419 19 623 22 425 24 926 25 832 130 733 430 533 29 933 333
> 537 934 538 740 139 744 938 544 42 646 749 242 546 547 548 753 450 150 349 48
> 852 352]]] :tid 1 :n 54 :time 1692607286168657534}
> {:type :invoke :process 1 :value [[:r 5 nil]] :tid 0 :n 51
> :time 1692607286201762938}
> {:type :ok :process 7 :value [[:append 5 455]] :tid 4 :n 55
> :time 1692607286245571513}
> {:type :invoke :process 7 :value [[:r 5 nil]] :tid 4 :n 56
> :time 1692607286245655775}
> {:type :ok :process 5 :value [[:r 5 [303 304 604 6 306 509 909 409 912
> 411 514 415 719 419 19 623 22 425 24 926 25 832 130 733 430 533 29 933 333
> 537 934 538 740 139 744 938 544 42 646 749 242 546 547 548 753 450 150 349 48
> 852 352 455]]] :tid 9 :n 52 :time 1692607286253928906}
> {:type :invoke :process 5 :value [[:r 5 nil]] :tid 9 :n 53
> :time 1692607286254095215}
> {:type :ok :process 6 :value [[:append 5 553]] :tid 5 :n 53
> :time 1692607286266263422}
> {:type :ok :process 1 :value [[:r 5 [303 304 604 6 306 509 909 409 912
> 411 514 415 719 419 19 623 22 425 24 926 25 832 130 733 430 533 29 933 333
> 537 934 538 740 139 744 938 544 42 646 749 242 546 547 548 753 450 150 349 48
> 852 352 553 455]]] :tid 0 :n 51 :time 1692607286271617955}
> {:type :ok :process 7 :value [[:r 5 [303 304 604 6 306 509 909 409 912
> 411 514 415 719 419 19 623 22 425 24 926 25 832 130 733 430 533 29 933 333
> 537 934 538 740 139 744 938 544 42 646 749 242 546 547 548 753 450 150 349 48
> 852 352 553 455]]] :tid 4 :n 56 :time 1692607286271816933}
> {:type :ok :process 5 :value [[:r 5 [303 304 604 6 306 509 909 409 912
> 411 514 415 719 419 19 623 22 425 24 926 25 832 130 733 430 533 29 933 333
> 537 934 538 740 139 744 938 544 42 646 749 242 546 547 548 753 450 150 349 48
> 852 352 553 455]]] :tid 9 :n 53 :time 1692607286281483026}
> {:type :invoke :process 9 :value [[:r 5 nil]] :tid 1 :n 56
> :time 1692607286284097561}
> {:type :ok :process 9 :value [[:r 5 [303 304 604 6 306 509 909 409 912
> 411 514 415 719 419 19 623 22 425 24 926 25 832 130 733 430 533 29 933 333
> 537 934 538 740 139 744 938 544 42 646 749 242 546 547 548 753 450 150 349 48
> 852 352 553 455]]] :tid 1 :n 56 :time 1692607286306445242}
> {code}
> Processes process 6 and process 7 are appending the values 553 and 455
> respectively. 455 succeeded and a read by process 5 confirms that. But then
> also 553 is appended and a read by
[jira] [Updated] (CASSANDRA-18937) Two accord transactions have the exact same transaction id
[
https://issues.apache.org/jira/browse/CASSANDRA-18937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Henrik Ingo updated CASSANDRA-18937:
Resolution: Invalid
Status: Resolved (was: Triage Needed)
Closing: Bug was in code added by myself.
> Two accord transactions have the exact same transaction id
> --
>
> Key: CASSANDRA-18937
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18937
> Project: Cassandra
> Issue Type: Bug
> Components: Accord
>Reporter: Henrik Ingo
>Priority: Normal
>
> When testing solutions for CASSANDRA-18798 I noticed that two independent
> transactions running at the same time in two parallel threads ended up having
> the exact same transaction id:
> {code}
> public void testListAddition() throws Exception
> {
> SHARED_CLUSTER.schemaChange("CREATE TABLE " + currentTable + " (k int
> PRIMARY KEY, l list)");
> SHARED_CLUSTER.forEach(node -> node.runOnInstance(() ->
> AccordService.instance().setCacheSize(0)));
> CountDownLatch latch = CountDownLatch.newCountDownLatch(1);
> Vector completionOrder = new Vector<>();
> try
> {
> for (int i=0; i<100; i++)
> {
> ForkJoinTask add1 = ForkJoinPool.commonPool().submit(() ->
> {
> latch.awaitThrowUncheckedOnInterrupt();
> SHARED_CLUSTER.get(1).executeInternal("BEGIN TRANSACTION
> " +
> "UPDATE " + currentTable + " SET l = l + [1]
> WHERE k = 1; " +
> "COMMIT TRANSACTION");
> completionOrder.add(1);
> });
> ForkJoinTask add2 = ForkJoinPool.commonPool().submit(() ->
> {
> try {
> Thread.sleep(0);
> {code}
> Adding some logging in TxnWrite.java reveals the two threads ave identical
> executeAt and unix timestamps:
> {noformat}
> lastmicros 0
> DEBUG [node2_Messaging-EventLoop-3-4] node2 2023-10-18 18:26:08,954
> AccordVerbHandler.java:54 - Receiving Apply{kind:Minimal,
> txnId:[10,1697642767659000,10,1], deps:[distributed_test_keyspace:[(-Inf,-1],
> (-1,9223372036854775805], (9223372036854775805,+Inf]]]:{}, {},
> executeAt:[10,1697642767659000,10,1],
> writes:TxnWrites{executeAt:[10,1697642767659000,10,1],
> keys:[distributed_test_keyspace:DecoratedKey(-4069959284402364209,
> 0001)], write:TxnWrite{}}, result:accord.api.Result$1@253c102e} from
> /127.0.0.1:7012
> raw 0 (NO_LAST_EXECUTED_HLC=-9223372036854775808
> lastExecutedTimestamp [0,0,0,0]
> lastmicros 1697642767659000
> raw -9223372036854775808 (NO_LAST_EXECUTED_HLC=-9223372036854775808
> lastExecutedTimestamp [10,1697642767659000,10,1]
> DEBUG [node2_CommandStore[1]:1] node2 2023-10-18 18:26:09,023
> AccordMessageSink.java:167 - Replying ACCORD_APPLY_RSP ApplyApplied to
> /127.0.0.1:7012
> lastmicros 0
> raw 0 (NO_LAST_EXECUTED_HLC=-9223372036854775808
> lastExecutedTimestamp [0,0,0,0]
> lastmicros 1697642767659000
> raw -9223372036854775808 (NO_LAST_EXECUTED_HLC=-9223372036854775808
> lastExecutedTimestamp [10,1697642767659000,10,1]
> timestamp 1697642767659000executeAt[10,1697642767659000,10,1]
> timestamp 1697642767659000executeAt[10,1697642767659000,10,1]
> {noformat}
> Increasing the Thread.sleep() to9 or 10 helps so that the transactions have
> different IDs.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18932) Harry-found CorruptSSTableException / RT Closer issue when reading entire partition
[
https://issues.apache.org/jira/browse/CASSANDRA-18932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated CASSANDRA-18932:
Bug Category: Parent values: Correctness(12982)
Complexity: Challenging
Component/s: Local/SSTable
Discovered By: Workload Replay
Severity: Critical
Status: Open (was: Triage Needed)
> Harry-found CorruptSSTableException / RT Closer issue when reading entire
> partition
> ---
>
> Key: CASSANDRA-18932
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18932
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Alex Petrov
>Assignee: Maxim Muzafarov
>Priority: Normal
> Attachments: node1_.zip, operation.log.zip
>
>
> While testing some new machinery for Harry, I have encountered a new RT
> closer / SSTable Corruption issue. I have grounds to believe this was
> introduced during the last year.
> Issue seems to happen because of intricate interleaving of flushes with
> writes and deletes.
> {code:java}
> ERROR [ReadStage-2] 2023-10-16 18:47:06,696 JVMStabilityInspector.java:76 -
> Exception in thread Thread[ReadStage-2,5,SharedPool]
> org.apache.cassandra.io.sstable.CorruptSSTableException: Corrupted:
> RandomAccessReader:BufferManagingRebufferer.Aligned:CompressedChunkReader.Mmap(/Users/ifesdjeen/foss/java/apache-cassandra-4.0/data/data1/harry/table_1-07c35a606c0a11eeae7a4f6ca489eb0c/nc-5-big-Data.db
> - LZ4Compressor, chunk length 16384, data length 232569)
> at
> org.apache.cassandra.io.sstable.AbstractSSTableIterator$AbstractReader.hasNext(AbstractSSTableIterator.java:381)
> at
> org.apache.cassandra.io.sstable.AbstractSSTableIterator.hasNext(AbstractSSTableIterator.java:242)
> at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:95)
> at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> at
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133)
> at
> org.apache.cassandra.utils.MergeIterator$Candidate.advance(MergeIterator.java:376)
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.advance(MergeIterator.java:188)
> at
> org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:157)
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:534)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIterators$UnfilteredRowMergeIterator.computeNext(UnfilteredRowIterators.java:402)
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:95)
> at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> at
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133)
> at
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:151)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:101)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:86)
> at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:343)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:201)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:186)
> at
> org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:48)
> at
> org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:346)
> at
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:2186)
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2581)
> at
>
[jira] [Updated] (CASSANDRA-18905) Index.Group is incorrectly unregistered from the SecondaryIndexManager
[ https://issues.apache.org/jira/browse/CASSANDRA-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Adamson updated CASSANDRA-18905: - Status: Needs Committer (was: Review In Progress) > Index.Group is incorrectly unregistered from the SecondaryIndexManager > -- > > Key: CASSANDRA-18905 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18905 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Urgent > Fix For: 5.0, 5.1 > > > An Index.Group is removed from the SecondaryIndexManager during > unregisterIndex if it contains no indexes after the index is unregistered. > The code for removing the group uses the wrong key to remove the group from > the indexGroups map. It is using the group object rather than the group name > that is used as the key in the map. > This means that the group is not added again if a new index is registered > using that group. The knock on from this is that the > StorageAttachedIndexGroup unregisters itself from the Tracker when it has no > indexes after an index is removed. The same group with no tracker is then > used for new indexes. This group then receives no notifications about sstable > or memtable updates. The ultimate side effect of this is that, memtables are > not released, resulting in memory leaks and indexes are not updated with new > sstables and their associated index files. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18937) Two accord transactions have the exact same transaction id
Henrik Ingo created CASSANDRA-18937:
---
Summary: Two accord transactions have the exact same transaction id
Key: CASSANDRA-18937
URL: https://issues.apache.org/jira/browse/CASSANDRA-18937
Project: Cassandra
Issue Type: Bug
Components: Accord
Reporter: Henrik Ingo
When testing solutions for CASSANDRA-18798 I noticed that two independent
transactions running at the same time in two parallel threads ended up having
the exact same transaction id:
{code}
public void testListAddition() throws Exception
{
SHARED_CLUSTER.schemaChange("CREATE TABLE " + currentTable + " (k int
PRIMARY KEY, l list)");
SHARED_CLUSTER.forEach(node -> node.runOnInstance(() ->
AccordService.instance().setCacheSize(0)));
CountDownLatch latch = CountDownLatch.newCountDownLatch(1);
Vector completionOrder = new Vector<>();
try
{
for (int i=0; i<100; i++)
{
ForkJoinTask add1 = ForkJoinPool.commonPool().submit(() -> {
latch.awaitThrowUncheckedOnInterrupt();
SHARED_CLUSTER.get(1).executeInternal("BEGIN TRANSACTION " +
"UPDATE " + currentTable + " SET l = l + [1] WHERE
k = 1; " +
"COMMIT TRANSACTION");
completionOrder.add(1);
});
ForkJoinTask add2 = ForkJoinPool.commonPool().submit(() -> {
try {
Thread.sleep(0);
{code}
Adding some logging in TxnWrite.java reveals the two threads ave identical
executeAt and unix timestamps:
{noformat}
lastmicros 0
DEBUG [node2_Messaging-EventLoop-3-4] node2 2023-10-18 18:26:08,954
AccordVerbHandler.java:54 - Receiving Apply{kind:Minimal,
txnId:[10,1697642767659000,10,1], deps:[distributed_test_keyspace:[(-Inf,-1],
(-1,9223372036854775805], (9223372036854775805,+Inf]]]:{}, {},
executeAt:[10,1697642767659000,10,1],
writes:TxnWrites{executeAt:[10,1697642767659000,10,1],
keys:[distributed_test_keyspace:DecoratedKey(-4069959284402364209, 0001)],
write:TxnWrite{}}, result:accord.api.Result$1@253c102e} from /127.0.0.1:7012
raw 0 (NO_LAST_EXECUTED_HLC=-9223372036854775808
lastExecutedTimestamp [0,0,0,0]
lastmicros 1697642767659000
raw -9223372036854775808 (NO_LAST_EXECUTED_HLC=-9223372036854775808
lastExecutedTimestamp [10,1697642767659000,10,1]
DEBUG [node2_CommandStore[1]:1] node2 2023-10-18 18:26:09,023
AccordMessageSink.java:167 - Replying ACCORD_APPLY_RSP ApplyApplied to
/127.0.0.1:7012
lastmicros 0
raw 0 (NO_LAST_EXECUTED_HLC=-9223372036854775808
lastExecutedTimestamp [0,0,0,0]
lastmicros 1697642767659000
raw -9223372036854775808 (NO_LAST_EXECUTED_HLC=-9223372036854775808
lastExecutedTimestamp [10,1697642767659000,10,1]
timestamp 1697642767659000executeAt[10,1697642767659000,10,1]
timestamp 1697642767659000executeAt[10,1697642767659000,10,1]
{noformat}
Increasing the Thread.sleep() to9 or 10 helps so that the transactions have
different IDs.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18799) Remove org.caffinitas.ohc:ohc-core-j8 dependency
[
https://issues.apache.org/jira/browse/CASSANDRA-18799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776713#comment-17776713
]
Maxim Muzafarov commented on CASSANDRA-18799:
-
Hi [~aweisberg], I appreciate you analyzing this ticket and pointing out the
things we should pay additional attention to. So, the issue originally came
from the idea to remove unused dependencies (like the {{{}ohc-core-j8{}}})
after dropping the {{jdk8}} support, but it quickly turned out that the ohc
library version needs to be updated as well. You mentioned the UnsTest above, I
can reproduce the `testDecrementIncrement` failure on my arm64 environment with
an exception "java.lang.InternalError: a fault occurred in a recent unsafe
memory access operation in compiled Java code".
However, my biggest concern is related to the JNA dependency on the ohc
library. We are now using version {{0.5.1}} of the ohc library in Cassandra.
The ohc {{0.5.1}} depends on version {{4.1.0}} [1] of the {{net.java.dev.jna}}
. For the Cassandra project, JNA has been updated since the 4.0 release version
up to JNA {{5.6.0}} in CASSANDRA-16212 to support running nodes on the
{{arm64}} architecture, so I assume that JNA 4.1.0 is no longer in Casssandra's
classpath. Considering the fact that the JNANativeAllocator is a default one
[2] for the ohc, and I haven't found any places where the
{{'org.caffinitas.ohc.allocator="unsafe"'}} is specified we could possibly get
a runtime error on the arm64 architecture when Config.row_cache_size is set
greater than zero (meaning the ohc is enabled), because the JNA 5.6.0 and JNA
4.1.0 are not compatible, and the breaking changes [3] exist in between. This,
in turn, could be very dangerous for production environments.
The 0.7.4. ohc depends on JNA 5.10.0, so I guess the above problem shouldn't be
raised after the update. If I'm wrong in my concerns, please correct me. I also
support your point about having reliable eviction policy tests, so we can
easily trust the test results and improve if any concerns arise.
So the question here is, should we move the ohc under the ASF first or do the
update?
WDYT?
[1] [https://github.com/snazy/ohc/blob/0.5.1/pom.xml#L81]
[2]
[https://github.com/snazy/ohc/blob/0.5.1/ohc-core/src/main/java/org/caffinitas/ohc/linked/Uns.java#L192]
[3]
[https://github.com/java-native-access/jna/blob/5.6.0/CHANGES.md#breaking-changes]
The stack trace of the `testDecrementIncrement` is for visibility:
{code:java}
java.lang.InternalError: a fault occurred in a recent unsafe memory access
operation in compiled Java code
at org.testng.Assert.failNotEquals(Assert.java:1037)
at org.testng.Assert.assertEqualsImpl(Assert.java:140)
at org.testng.Assert.assertEquals(Assert.java:122)
at org.testng.Assert.assertEquals(Assert.java:797)
at
org.caffinitas.ohc.linked.UnsTest.testDecrementIncrement(UnsTest.java:395)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
at
org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:133)
at org.testng.internal.TestInvoker.invokeMethod(TestInvoker.java:598)
at
org.testng.internal.TestInvoker.invokeTestMethod(TestInvoker.java:173)
at org.testng.internal.MethodRunner.runInSequence(MethodRunner.java:46)
at
org.testng.internal.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:824)
at
org.testng.internal.TestInvoker.invokeTestMethods(TestInvoker.java:146)
at
org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:146)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:128)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
at org.testng.TestRunner.privateRun(TestRunner.java:794)
at org.testng.TestRunner.run(TestRunner.java:596)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:377)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:371)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:332)
at org.testng.SuiteRunner.run(SuiteRunner.java:276)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:53)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:96)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1212)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1134)
at org.testng.TestNG.runSuites(TestNG.java:1063)
at org.testng.TestNG.run(TestNG.java:1031)
at
[jira] [Updated] (CASSANDRA-18936) CI jvm-dtest-upgrade breakage
[ https://issues.apache.org/jira/browse/CASSANDRA-18936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-18936: --- Test and Documentation Plan: ci-cassandra Status: Patch Available (was: Open) > CI jvm-dtest-upgrade breakage > - > > Key: CASSANDRA-18936 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18936 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 5.0.x, 5.x > > > git clean failing when reusing the dtest sub clone. > ref: https://the-asf.slack.com/archives/CK23JSY2K/p1697465832946299 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18936) CI jvm-dtest-upgrade breakage
[ https://issues.apache.org/jira/browse/CASSANDRA-18936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Semb Wever updated CASSANDRA-18936: --- Bug Category: Parent values: Correctness(12982) Complexity: Normal Discovered By: DTest Fix Version/s: 5.0.x 5.x Severity: Critical Assignee: Michael Semb Wever Status: Open (was: Triage Needed) > CI jvm-dtest-upgrade breakage > - > > Key: CASSANDRA-18936 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18936 > Project: Cassandra > Issue Type: Bug > Components: CI >Reporter: Michael Semb Wever >Assignee: Michael Semb Wever >Priority: Normal > Fix For: 5.0.x, 5.x > > > git clean failing when reusing the dtest sub clone. > ref: https://the-asf.slack.com/archives/CK23JSY2K/p1697465832946299 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch asf-staging updated (94d0484c1 -> 9e99f5f52)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git discard 94d0484c1 generate docs for 1d0135e5 new 9e99f5f52 generate docs for 1d0135e5 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (94d0484c1) \ N -- N -- N refs/heads/asf-staging (9e99f5f52) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: content/search-index.js | 2 +- site-ui/build/ui-bundle.zip | Bin 4881412 -> 4881412 bytes 2 files changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cep-21-tcm-review updated (fbe391130a -> 3ea1c0e898)
This is an automated email from the ASF dual-hosted git repository. jlewandowski pushed a change to branch cep-21-tcm-review in repository https://gitbox.apache.org/repos/asf/cassandra.git from fbe391130a Fix ClusterMetadataService.state javadoc add 3ea1c0e898 fix ClusterMetadataService doc No new revisions were added by this update. Summary of changes: src/java/org/apache/cassandra/tcm/ClusterMetadataService.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch asf-staging updated (28f6b431f -> 94d0484c1)
This is an automated email from the ASF dual-hosted git repository.
git-site-role pushed a change to branch asf-staging
in repository https://gitbox.apache.org/repos/asf/cassandra-website.git
discard 28f6b431f generate docs for db70fb96
add 327f02f61 BLOG - Apache Cassandra 5.0 Features: Dynamic Data Masking
add 1d0135e5f ninja-fix, escape asterisk, bold not intended
new 94d0484c1 generate docs for 1d0135e5
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (28f6b431f)
\
N -- N -- N refs/heads/asf-staging (94d0484c1)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
content/_/blog.html| 24 ++
...ssandra-5.0-Features-Dynamic-Data-Masking.html} | 325 ++---
content/search-index.js| 2 +-
site-content/source/modules/ROOT/pages/blog.adoc | 23 ++
...assandra-5.0-Features-Dynamic-Data-Masking.adoc | 230 +++
site-ui/build/ui-bundle.zip| Bin 4881412 -> 4881412
bytes
6 files changed, 497 insertions(+), 107 deletions(-)
copy content/_/blog/{Apache-Cassandra-4.1-New-SSTable-Identifiers.html =>
Apache-Cassandra-5.0-Features-Dynamic-Data-Masking.html} (55%)
create mode 100644
site-content/source/modules/ROOT/pages/blog/Apache-Cassandra-5.0-Features-Dynamic-Data-Masking.adoc
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-builds] branch trunk updated (d92589b -> 088cca4)
This is an automated email from the ASF dual-hosted git repository. smiklosovic pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-builds.git from d92589b Switch to use in-tree for Cassandra-5.0 and Cassandra-5.1 pipelines (CASSANDRA-18665) add 088cca4 ninja fix: temporarily comment out Cassandra version check in build scripts (CASSANDRA-18133) No new revisions were added by this update. Summary of changes: docker/build-debs.sh | 2 +- docker/build-rpms.sh | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch trunk updated: ninja-fix, escape asterisk, bold not intended
This is an automated email from the ASF dual-hosted git repository. mck pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-website.git The following commit(s) were added to refs/heads/trunk by this push: new 1d0135e5 ninja-fix, escape asterisk, bold not intended 1d0135e5 is described below commit 1d0135e5f55b2ae3f3bfe41b5178441815fc1e12 Author: mck AuthorDate: Wed Oct 18 14:47:32 2023 +0200 ninja-fix, escape asterisk, bold not intended --- .../pages/blog/Apache-Cassandra-5.0-Features-Dynamic-Data-Masking.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/site-content/source/modules/ROOT/pages/blog/Apache-Cassandra-5.0-Features-Dynamic-Data-Masking.adoc b/site-content/source/modules/ROOT/pages/blog/Apache-Cassandra-5.0-Features-Dynamic-Data-Masking.adoc index 039bd60d..dcf7f599 100644 --- a/site-content/source/modules/ROOT/pages/blog/Apache-Cassandra-5.0-Features-Dynamic-Data-Masking.adoc +++ b/site-content/source/modules/ROOT/pages/blog/Apache-Cassandra-5.0-Features-Dynamic-Data-Masking.adoc @@ -21,7 +21,7 @@ The data can be partially redacted, for example showing only the four last digit DDM is based on a set of https://cassandra.apache.org/doc/5.0/cassandra/developing/cql/functions.html[CQL built-in (native) functions^] that obscure sensitive information. The available functions are: * **mask_null**: Replaces the first argument with a null column. The returned value is always a non-existent column, and not a not-null column representing a null value. -* **mask_default**: Replaces its argument by an arbitrary, fixed default value of the same type. This will be for text values, zero for numeric values, false for booleans, etc. +* **mask_default**: Replaces its argument by an arbitrary, fixed default value of the same type. This will be \*\*\*\* for text values, zero for numeric values, false for booleans, etc. * **mask_replace**: Replaces the first argument by the replacement value on the second argument. The replacement value needs to have the same type as the replaced value. * **mask_inner**: Returns a copy of the first text, varchar or ascii argument, replacing each character except the first and last ones by a padding character. * **mask_outer**: Returns a copy of the first text, varchar or ascii argument, replacing the first and last character by a padding character. - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra-website] branch trunk updated: BLOG - Apache Cassandra 5.0 Features: Dynamic Data Masking
This is an automated email from the ASF dual-hosted git repository. mck pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-website.git The following commit(s) were added to refs/heads/trunk by this push: new 327f02f6 BLOG - Apache Cassandra 5.0 Features: Dynamic Data Masking 327f02f6 is described below commit 327f02f610b9e7b664a316ac7c29754550b17095 Author: Diogenese Topper <83248625+nonstopd...@users.noreply.github.com> AuthorDate: Wed Oct 11 17:45:56 2023 -0700 BLOG - Apache Cassandra 5.0 Features: Dynamic Data Masking patch by Diogenese Topper; reviewed by Mick Semb Wever for CASSANDRA-18923 --- site-content/source/modules/ROOT/pages/blog.adoc | 23 +++ ...assandra-5.0-Features-Dynamic-Data-Masking.adoc | 230 + 2 files changed, 253 insertions(+) diff --git a/site-content/source/modules/ROOT/pages/blog.adoc b/site-content/source/modules/ROOT/pages/blog.adoc index 8e826ac7..585b99e2 100644 --- a/site-content/source/modules/ROOT/pages/blog.adoc +++ b/site-content/source/modules/ROOT/pages/blog.adoc @@ -8,6 +8,29 @@ NOTES FOR CONTENT CREATORS - Replace post tile, date, description and link to you post. +//start card +[openblock,card shadow relative test] + +[openblock,card-header] +-- +[discrete] +=== Apache Cassandra 5.0 Features: Dynamic Data Masking +[discrete] + October 11, 2023 +-- +[openblock,card-content] +-- +Apache Cassandra 5.0 adds Dynamic Data Masking for secure data retrieval via masking functions and permissions. +[openblock,card-btn card-btn--blog] + +[.btn.btn--alt] +xref:blog/Apache-Cassandra-5.0-Features-Dynamic-Data-Masking.adoc[Read More] + + +-- + +//end card + //start card [openblock,card shadow relative test] diff --git a/site-content/source/modules/ROOT/pages/blog/Apache-Cassandra-5.0-Features-Dynamic-Data-Masking.adoc b/site-content/source/modules/ROOT/pages/blog/Apache-Cassandra-5.0-Features-Dynamic-Data-Masking.adoc new file mode 100644 index ..039bd60d --- /dev/null +++ b/site-content/source/modules/ROOT/pages/blog/Apache-Cassandra-5.0-Features-Dynamic-Data-Masking.adoc @@ -0,0 +1,230 @@ += Apache Cassandra 5.0 Features: Dynamic Data Masking +:page-layout: single-post +:page-role: blog-post +:page-post-date: October 11, 2023 +:page-post-author: Andrés de la Peña +:description: New dynamic data masking capabilities are a feature available in the coming Apache Cassandra 5.0. +:keywords: + +__Apache Cassandra 5.0 is the project’s major release for 2023, and it promises some of the biggest changes for Cassandra to-date. After more than a decade of engineering work dedicated to stabilizing and building Cassandra as a distributed database, we now look forward to introducing a host of exciting features and enhancements that empower users to take their data-driven applications to the next level - including machine learning and artificial intelligence.__ + +__This blog series aims to give a deeper dive into some of the key features of Cassandra 5.0.__ + +Cassandra 5.0 introduces new dynamic data masking (DDM) capabilities which allows you to obscure sensitive information using a concept called masked columns. DDM doesn't change the stored data. Instead, it just presents the data in its redacted form during SELECT queries. + +DDM aims to provide some degree of protection against accidental data exposure. For example, the column masks can prevent undesired data exposure when a user shares the results of a query with someone. Or it can limit what users with read access can actually see. However, it's important to know that anyone with direct access to the sstable files will be able to read the clear data. + +The data can be partially redacted, for example showing only the four last digits of a credit card number. This partial view of the data can be enough to allow users to do some checks on the data without seeing it entirely. The data can also be fully redacted, making it clear for users that the information is sensitive. + +== Masking functions + +DDM is based on a set of https://cassandra.apache.org/doc/5.0/cassandra/developing/cql/functions.html[CQL built-in (native) functions^] that obscure sensitive information. The available functions are: + +* **mask_null**: Replaces the first argument with a null column. The returned value is always a non-existent column, and not a not-null column representing a null value. +* **mask_default**: Replaces its argument by an arbitrary, fixed default value of the same type. This will be for text values, zero for numeric values, false for booleans, etc. +* **mask_replace**: Replaces the first argument by the replacement value on the second argument. The replacement value needs to have the same type as the replaced value. +* **mask_inner**: Returns a copy of the first text, varchar or ascii argument, replacing each character except the first and last ones
[jira] [Updated] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[
https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-18935:
--
Test and Documentation Plan: CI
Status: Patch Available (was: In Progress)
Patches are available. I will build it all in next days.
> Fix nodetool enable/disablebinary to correctly set rpc
> --
>
> Key: CASSANDRA-18935
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18935
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core, Legacy/CQL
>Reporter: Cameron Zemek
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
> Attachments: 18935-3.11.patch
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
>
> {code:java}
> if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) ||
> (nativeFlag == null && DatabaseDescriptor.startNativeTransport()))
> {
> startNativeTransport();
> StorageService.instance.setRpcReady(true);
> } {code}
> The startup code here only sets RpcReady if native transport is enabled. If
> you call
> {code:java}
> nodetool enablebinary{code}
> then this flag doesn't get set.
> But with the change from CASSANDRA-13043 it requires RpcReady set to true in
> order to get a leader for the counter update.
> Not sure what the correct fix is here, seems to only really use this flag for
> counters. So thinking perhaps the fix is to just move this outside the if
> condition.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18912) Specify "since" in all Deprecated annotations
[
https://issues.apache.org/jira/browse/CASSANDRA-18912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-18912:
--
Reviewers: Brandon Williams, Maxim Muzafarov
Status: Review In Progress (was: Needs Committer)
> Specify "since" in all Deprecated annotations
> -
>
> Key: CASSANDRA-18912
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18912
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Core
>Reporter: Stefan Miklosovic
>Assignee: Maxim Muzafarov
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> It would be great if we introduced in 5.0 a change in Deprecated annotations
> like this:
> {code}
> @Deprecated(since = "4.0")
> {code}
> or
> {code}
> @Deprecated(since = "3.11")
> {code}
> The reasoning behind this is that as of now, it is pretty cumbersome to
> figure out what can be removed on the next major version. It has to be,
> basically, done manually every time.
> There is also this parameter available:
> {code}
> @Deprecated(forRemoval = true / false)
> {code}
> which indicates whether the annotated element is subject to removal in a
> future version so we do not need to think about this every time if it is
> eligible for deletion in a next major or not.
> We could then have a check which would ensure that we are not releasing a
> next major with some deprecations introduced two majors before.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18912) Specify "since" in all Deprecated annotations
[
https://issues.apache.org/jira/browse/CASSANDRA-18912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-18912:
--
Status: Ready to Commit (was: Review In Progress)
> Specify "since" in all Deprecated annotations
> -
>
> Key: CASSANDRA-18912
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18912
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Core
>Reporter: Stefan Miklosovic
>Assignee: Maxim Muzafarov
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> It would be great if we introduced in 5.0 a change in Deprecated annotations
> like this:
> {code}
> @Deprecated(since = "4.0")
> {code}
> or
> {code}
> @Deprecated(since = "3.11")
> {code}
> The reasoning behind this is that as of now, it is pretty cumbersome to
> figure out what can be removed on the next major version. It has to be,
> basically, done manually every time.
> There is also this parameter available:
> {code}
> @Deprecated(forRemoval = true / false)
> {code}
> which indicates whether the annotated element is subject to removal in a
> future version so we do not need to think about this every time if it is
> eligible for deletion in a next major or not.
> We could then have a check which would ensure that we are not releasing a
> next major with some deprecations introduced two majors before.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18838) make stop-server shell out of the box
[ https://issues.apache.org/jira/browse/CASSANDRA-18838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stefan Miklosovic updated CASSANDRA-18838: -- Fix Version/s: 3.0.30 3.11.17 4.0.12 4.1.4 5.0-alpha2 (was: 3.0.x) (was: 3.11.x) (was: 5.x) (was: 4.0.x) (was: 4.1.x) Source Control Link: https://github.com/apache/cassandra/commit/f27c6c8e6ef021141130e5b314ff10a429563030 Resolution: Fixed Status: Resolved (was: Ready to Commit) > make stop-server shell out of the box > - > > Key: CASSANDRA-18838 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18838 > Project: Cassandra > Issue Type: Improvement > Components: Legacy/Tools >Reporter: Ling Mao >Assignee: Ling Mao >Priority: Normal > Fix For: 3.0.30, 3.11.17, 4.0.12, 4.1.4, 5.0-alpha2 > > Time Spent: 1h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-4.1 updated (1920571861 -> ede18e6c9f)
This is an automated email from the ASF dual-hosted git repository. smiklosovic pushed a change to branch cassandra-4.1 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 1920571861 Merge branch 'cassandra-4.0' into cassandra-4.1 add f27c6c8e6e Implement the logic in bin/stop-server add 6212b0aaa5 Merge branch 'cassandra-3.0' into cassandra-3.11 add 85285fa0f9 Merge branch 'cassandra-3.11' into cassandra-4.0 add ede18e6c9f Merge branch 'cassandra-4.0' into cassandra-4.1 No new revisions were added by this update. Summary of changes: CHANGES.txt | 1 + bin/stop-server | 65 ++--- redhat/cassandra.spec | 2 +- redhat/noboolean/cassandra.spec | 2 +- 4 files changed, 57 insertions(+), 13 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] 01/01: Merge branch 'cassandra-5.0' into trunk
This is an automated email from the ASF dual-hosted git repository. smiklosovic pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 2feeb39d5a223da3b46805ce656844c44fb9a8cc Merge: 3d15be1d5e a4fc03e799 Author: Stefan Miklosovic AuthorDate: Wed Oct 18 12:57:21 2023 +0200 Merge branch 'cassandra-5.0' into trunk CHANGES.txt | 1 + bin/stop-server | 65 ++--- redhat/cassandra.spec | 2 +- redhat/noboolean/cassandra.spec | 2 +- 4 files changed, 57 insertions(+), 13 deletions(-) diff --cc CHANGES.txt index 0e92350f19,9377b7a421..47703336da --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -30,8 -25,8 +30,9 @@@ Merged from 4.0 * JMH improvements - faster build and async profiler (CASSANDRA-18871) * Enable 3rd party JDK installations for Debian package (CASSANDRA-18844) Merged from 3.11: + * Revert CASSANDRA-18543 (CASSANDRA-18854) Merged from 3.0: - * Implement the logic in bin/stop-server (CASSANDRA-18838) ++ * Implement the logic in bin/stop-server (CASSANDRA-18838) * Upgrade snappy-java to 1.1.10.4 (CASSANDRA-18878) * Add cqlshrc.sample and credentials.sample into Debian package (CASSANDRA-18818) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.11 updated (058621a446 -> 6212b0aaa5)
This is an automated email from the ASF dual-hosted git repository. smiklosovic pushed a change to branch cassandra-3.11 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 058621a446 Merge branch 'cassandra-3.0' into cassandra-3.11 add f27c6c8e6e Implement the logic in bin/stop-server add 6212b0aaa5 Merge branch 'cassandra-3.0' into cassandra-3.11 No new revisions were added by this update. Summary of changes: CHANGES.txt | 1 + bin/stop-server | 65 ++- redhat/cassandra.spec | 2 +- 3 files changed, 56 insertions(+), 12 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-4.0 updated (2bab3f27ba -> 85285fa0f9)
This is an automated email from the ASF dual-hosted git repository. smiklosovic pushed a change to branch cassandra-4.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 2bab3f27ba Gossip NPE due to shutdown event corrupting empty statuses add f27c6c8e6e Implement the logic in bin/stop-server add 6212b0aaa5 Merge branch 'cassandra-3.0' into cassandra-3.11 add 85285fa0f9 Merge branch 'cassandra-3.11' into cassandra-4.0 No new revisions were added by this update. Summary of changes: CHANGES.txt | 1 + bin/stop-server | 65 ++--- redhat/cassandra.spec | 2 +- redhat/noboolean/cassandra.spec | 2 +- 4 files changed, 57 insertions(+), 13 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-5.0 updated (802bd5fe13 -> a4fc03e799)
This is an automated email from the ASF dual-hosted git repository. smiklosovic pushed a change to branch cassandra-5.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 802bd5fe13 ninja-fix – reusing git clone under build needs reset and permissions add f27c6c8e6e Implement the logic in bin/stop-server add 6212b0aaa5 Merge branch 'cassandra-3.0' into cassandra-3.11 add 85285fa0f9 Merge branch 'cassandra-3.11' into cassandra-4.0 add ede18e6c9f Merge branch 'cassandra-4.0' into cassandra-4.1 add a4fc03e799 Merge branch 'cassandra-4.1' into cassandra-5.0 No new revisions were added by this update. Summary of changes: CHANGES.txt | 1 + bin/stop-server | 65 ++--- redhat/cassandra.spec | 2 +- redhat/noboolean/cassandra.spec | 2 +- 4 files changed, 57 insertions(+), 13 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch trunk updated (3d15be1d5e -> 2feeb39d5a)
This is an automated email from the ASF dual-hosted git repository. smiklosovic pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git from 3d15be1d5e Merge branch 'cassandra-5.0' into trunk add f27c6c8e6e Implement the logic in bin/stop-server add 6212b0aaa5 Merge branch 'cassandra-3.0' into cassandra-3.11 add 85285fa0f9 Merge branch 'cassandra-3.11' into cassandra-4.0 add ede18e6c9f Merge branch 'cassandra-4.0' into cassandra-4.1 add a4fc03e799 Merge branch 'cassandra-4.1' into cassandra-5.0 new 2feeb39d5a Merge branch 'cassandra-5.0' into trunk The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt | 1 + bin/stop-server | 65 ++--- redhat/cassandra.spec | 2 +- redhat/noboolean/cassandra.spec | 2 +- 4 files changed, 57 insertions(+), 13 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cassandra-3.0 updated (dc7234134c -> f27c6c8e6e)
This is an automated email from the ASF dual-hosted git repository. smiklosovic pushed a change to branch cassandra-3.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git from dc7234134c Upgrade snappy-java to 1.1.10.4 add f27c6c8e6e Implement the logic in bin/stop-server No new revisions were added by this update. Summary of changes: CHANGES.txt | 1 + bin/stop-server | 65 ++- redhat/cassandra.spec | 2 +- 3 files changed, 56 insertions(+), 12 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18905) Index.Group is incorrectly unregistered from the SecondaryIndexManager
[ https://issues.apache.org/jira/browse/CASSANDRA-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776610#comment-17776610 ] Mike Adamson edited comment on CASSANDRA-18905 at 10/18/23 10:56 AM: - Latest test runs are here: |[5.0|https://github.com/apache/cassandra/pull/2769]|[CircleCI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/331/workflows/f04324d8-33a2-47a8-b544-3296cf8b53d1]| |[trunk|https://github.com/apache/cassandra/pull/2771]|[CircleCI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/332/workflows/946e28f4-2dec-4384-ac38-de011093f6c6]| The 5.0 unit test failure appears unrelated to these changes and broke on 11 & 17 test runs. It doesn't appear to be repeatable locally. was (Author: mike_tr_adamson): Latest test runs are here: |[5.0|https://github.com/apache/cassandra/pull/2769]|[CircleCI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/331/workflows/f04324d8-33a2-47a8-b544-3296cf8b53d1]| |[trunk|https://github.com/apache/cassandra/pull/2771]|[CircleCI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/332/workflows/946e28f4-2dec-4384-ac38-de011093f6c6]| > Index.Group is incorrectly unregistered from the SecondaryIndexManager > -- > > Key: CASSANDRA-18905 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18905 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Urgent > Fix For: 5.0, 5.1 > > > An Index.Group is removed from the SecondaryIndexManager during > unregisterIndex if it contains no indexes after the index is unregistered. > The code for removing the group uses the wrong key to remove the group from > the indexGroups map. It is using the group object rather than the group name > that is used as the key in the map. > This means that the group is not added again if a new index is registered > using that group. The knock on from this is that the > StorageAttachedIndexGroup unregisters itself from the Tracker when it has no > indexes after an index is removed. The same group with no tracker is then > used for new indexes. This group then receives no notifications about sstable > or memtable updates. The ultimate side effect of this is that, memtables are > not released, resulting in memory leaks and indexes are not updated with new > sstables and their associated index files. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18905) Index.Group is incorrectly unregistered from the SecondaryIndexManager
[ https://issues.apache.org/jira/browse/CASSANDRA-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776610#comment-17776610 ] Mike Adamson commented on CASSANDRA-18905: -- Latest test runs are here: |[5.0|https://github.com/apache/cassandra/pull/2769]|[CircleCI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/331/workflows/f04324d8-33a2-47a8-b544-3296cf8b53d1]| |[trunk|https://github.com/apache/cassandra/pull/2771]|[CircleCI|https://app.circleci.com/pipelines/github/mike-tr-adamson/cassandra/332/workflows/946e28f4-2dec-4384-ac38-de011093f6c6]| > Index.Group is incorrectly unregistered from the SecondaryIndexManager > -- > > Key: CASSANDRA-18905 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18905 > Project: Cassandra > Issue Type: Bug > Components: Feature/2i Index >Reporter: Mike Adamson >Assignee: Mike Adamson >Priority: Urgent > Fix For: 5.0, 5.1 > > > An Index.Group is removed from the SecondaryIndexManager during > unregisterIndex if it contains no indexes after the index is unregistered. > The code for removing the group uses the wrong key to remove the group from > the indexGroups map. It is using the group object rather than the group name > that is used as the key in the map. > This means that the group is not added again if a new index is registered > using that group. The knock on from this is that the > StorageAttachedIndexGroup unregisters itself from the Tracker when it has no > indexes after an index is removed. The same group with no tracker is then > used for new indexes. This group then receives no notifications about sstable > or memtable updates. The ultimate side effect of this is that, memtables are > not released, resulting in memory leaks and indexes are not updated with new > sstables and their associated index files. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[cassandra] branch cep-21-tcm-review updated: Fix ClusterMetadataService.state javadoc
This is an automated email from the ASF dual-hosted git repository.
blerer pushed a commit to branch cep-21-tcm-review
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cep-21-tcm-review by this push:
new fbe391130a Fix ClusterMetadataService.state javadoc
fbe391130a is described below
commit fbe391130aa0946d7d8389e957e96520c6638842
Author: Benjamin Lerer
AuthorDate: Wed Oct 18 11:57:09 2023 +0200
Fix ClusterMetadataService.state javadoc
---
.../cassandra/tcm/ClusterMetadataService.java | 37 --
1 file changed, 34 insertions(+), 3 deletions(-)
diff --git a/src/java/org/apache/cassandra/tcm/ClusterMetadataService.java
b/src/java/org/apache/cassandra/tcm/ClusterMetadataService.java
index d01c63b419..3b7afc273b 100644
--- a/src/java/org/apache/cassandra/tcm/ClusterMetadataService.java
+++ b/src/java/org/apache/cassandra/tcm/ClusterMetadataService.java
@@ -137,8 +137,15 @@ public class ClusterMetadataService
private final AtomicBoolean commitsPaused = new AtomicBoolean();
/**
- * Returns the state of the {@code ClusteMetadataService}.
- * @return the state of the {@code ClusteMetadataService}.
+ * Returns the current state of the {@code ClusteMetadataService}.
+ *
+ * The state can be:
+ *
+ * LOCAL: the node is a member of CMS
+ * REMOTE: the node is not a CMS nember
+ * GOSSIP: the node has not been upgraded to TCM yet
+ * RESET: the node is being forced to reset the local metadata
+ *
*/
public static State state()
{
@@ -860,8 +867,32 @@ public class ClusterMetadataService
}
}
+/**
+ * A state of the {@code ClusterMetadataService} of the node. A state can
be:
+ *
+ * the node is a member of CMS
+ * the node is not a CMS nember
+ * the node has not been upgraded to TCM yet
+ * the node is being forced to reset the local metadata
+ *
+ */
public enum State
{
-LOCAL, REMOTE, GOSSIP, RESET
+/**
+ * The node is a member of the CMS
+ */
+LOCAL,
+/**
+ * The node is not a member of the CMS
+ */
+REMOTE,
+/**
+ * The node has not been upgraded to TCM yet
+ */
+GOSSIP,
+/**
+ * The node is being forced to reset the local metadata
+ */
+RESET
}
}
\ No newline at end of file
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13911) IllegalStateException thrown by UPI.Serializer.hasNext() for some SELECT queries
[
https://issues.apache.org/jira/browse/CASSANDRA-13911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776585#comment-17776585
]
Aleksey Yeschenko commented on CASSANDRA-13911:
---
Indeed, if hidden is the right word for it. This was a pretty nasty one.
> IllegalStateException thrown by UPI.Serializer.hasNext() for some SELECT
> queries
>
>
> Key: CASSANDRA-13911
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13911
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Coordination
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Normal
> Fix For: 3.0.15, 3.11.1
>
>
> Certain combinations of rows, in presence of per partition limit (set
> explicitly in 3.6+ or implicitly to 1 via DISTINCT) cause
> {{UnfilteredPartitionIterators.Serializer.hasNext()}} to throw
> {{IllegalStateException}} .
> Relevant code snippet:
> {code}
> // We can't answer this until the previously returned iterator has been fully
> consumed,
> // so complain if that's not the case.
> if (next != null && next.hasNext())
> throw new IllegalStateException("Cannot call hasNext() until the previous
> iterator has been fully consumed");
> {code}
> Since {{UnfilteredPartitionIterators.Serializer}} and
> {{UnfilteredRowIteratorSerializer.serializer}} deserialize partitions/rows
> lazily, it is required for correct operation of the partition iterator to
> have the previous partition fully consumed, so that deserializing the next
> one can start from the correct position in the byte buffer. However, that
> condition won’t always be satisfied, as there are legitimate combinations of
> rows that do not consume every row in every partition.
> For example, look at [this
> dtest|https://github.com/iamaleksey/cassandra-dtest/commits/13911].
> In case we end up with a following pattern of rows:
> {code}
> node1, partition 0 | 0
> node2, partition 0 | x x
> {code}
> , where {{x}} and {{x}} a row tombstones for rows 1 and 2, it’s sufficient
> for {{MergeIterator}} to only look at row 0 in partition from node1 and at
> row tombstone 1 from node2 to satisfy the per partition limit of 1. The
> stopping merge result counter will stop iteration right there, leaving row
> tombstone 2 from node2 unvisited and not deseiralized. Switching to the next
> partition will in turn trigger the {{IllegalStateException}} because we
> aren’t done yet.
> The stopping counter is behaving correctly, so is the {{MergeIterator}}. I’ll
> note that simply removing that condition is not enough to fix the problem
> properly - it’d just cause us to deseiralize garbage, trying to deserialize a
> new partition from a position in the bytebuffer that precedes remaining rows
> in the previous partition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[
https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776582#comment-17776582
]
Stefan Miklosovic commented on CASSANDRA-18935:
---
I just renamed the ticket.
> Fix nodetool enable/disablebinary to correctly set rpc
> --
>
> Key: CASSANDRA-18935
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18935
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core, Legacy/CQL
>Reporter: Cameron Zemek
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
> Attachments: 18935-3.11.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
>
> {code:java}
> if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) ||
> (nativeFlag == null && DatabaseDescriptor.startNativeTransport()))
> {
> startNativeTransport();
> StorageService.instance.setRpcReady(true);
> } {code}
> The startup code here only sets RpcReady if native transport is enabled. If
> you call
> {code:java}
> nodetool enablebinary{code}
> then this flag doesn't get set.
> But with the change from CASSANDRA-13043 it requires RpcReady set to true in
> order to get a leader for the counter update.
> Not sure what the correct fix is here, seems to only really use this flag for
> counters. So thinking perhaps the fix is to just move this outside the if
> condition.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18935) Fix nodetool enable/disablebinary to correctly set rpc
[
https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-18935:
--
Summary: Fix nodetool enable/disablebinary to correctly set rpc (was:
Unable to write if native transport is disabled on startup)
> Fix nodetool enable/disablebinary to correctly set rpc
> --
>
> Key: CASSANDRA-18935
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18935
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core, Legacy/CQL
>Reporter: Cameron Zemek
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
> Attachments: 18935-3.11.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
>
> {code:java}
> if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) ||
> (nativeFlag == null && DatabaseDescriptor.startNativeTransport()))
> {
> startNativeTransport();
> StorageService.instance.setRpcReady(true);
> } {code}
> The startup code here only sets RpcReady if native transport is enabled. If
> you call
> {code:java}
> nodetool enablebinary{code}
> then this flag doesn't get set.
> But with the change from CASSANDRA-13043 it requires RpcReady set to true in
> order to get a leader for the counter update.
> Not sure what the correct fix is here, seems to only really use this flag for
> counters. So thinking perhaps the fix is to just move this outside the if
> condition.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Unable to write if native transport is disabled on startup
[
https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776569#comment-17776569
]
Maxwell Guo commented on CASSANDRA-18935:
-
So it seems that we need to create a bug fix for nodetool enablebinary and
disablebinary:)
> Unable to write if native transport is disabled on startup
> --
>
> Key: CASSANDRA-18935
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18935
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core, Legacy/CQL
>Reporter: Cameron Zemek
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
> Attachments: 18935-3.11.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
>
> {code:java}
> if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) ||
> (nativeFlag == null && DatabaseDescriptor.startNativeTransport()))
> {
> startNativeTransport();
> StorageService.instance.setRpcReady(true);
> } {code}
> The startup code here only sets RpcReady if native transport is enabled. If
> you call
> {code:java}
> nodetool enablebinary{code}
> then this flag doesn't get set.
> But with the change from CASSANDRA-13043 it requires RpcReady set to true in
> order to get a leader for the counter update.
> Not sure what the correct fix is here, seems to only really use this flag for
> counters. So thinking perhaps the fix is to just move this outside the if
> condition.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18221) CEP-15: (Accord) Define a configuration system for Accord to allow overriding of internal configs and integrate with Cassandra yaml
[ https://issues.apache.org/jira/browse/CASSANDRA-18221?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776566#comment-17776566 ] Jacek Lewandowski commented on CASSANDRA-18221: --- [~maedhroz] are you going to review the PRs as well or I can merge it? > CEP-15: (Accord) Define a configuration system for Accord to allow overriding > of internal configs and integrate with Cassandra yaml > --- > > Key: CASSANDRA-18221 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18221 > Project: Cassandra > Issue Type: Improvement > Components: Accord >Reporter: David Capwell >Assignee: Jacek Lewandowski >Priority: Normal > Labels: pull-request-available > Fix For: 5.x > > Time Spent: 1h > Remaining Estimate: 0h > > Users are used to modifying cassandra via yaml and JMX (for dynamic configs) > but accord does not integrate with this right now; we should enhance Accord > to expose configs that are defined in cassandra yaml. > As an extension on this, we should figure out which configs are “dynamic” and > allow overriding via JMX or system vtable. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Unable to write if native transport is disabled on startup
[
https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776565#comment-17776565
]
Brandon Williams commented on CASSANDRA-18935:
--
LGTM
> Unable to write if native transport is disabled on startup
> --
>
> Key: CASSANDRA-18935
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18935
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core, Legacy/CQL
>Reporter: Cameron Zemek
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
> Attachments: 18935-3.11.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
>
> {code:java}
> if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) ||
> (nativeFlag == null && DatabaseDescriptor.startNativeTransport()))
> {
> startNativeTransport();
> StorageService.instance.setRpcReady(true);
> } {code}
> The startup code here only sets RpcReady if native transport is enabled. If
> you call
> {code:java}
> nodetool enablebinary{code}
> then this flag doesn't get set.
> But with the change from CASSANDRA-13043 it requires RpcReady set to true in
> order to get a leader for the counter update.
> Not sure what the correct fix is here, seems to only really use this flag for
> counters. So thinking perhaps the fix is to just move this outside the if
> condition.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18935) Unable to write if native transport is disabled on startup
[
https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776564#comment-17776564
]
Stefan Miklosovic edited comment on CASSANDRA-18935 at 10/18/23 8:52 AM:
-
pinging [~brandon.williams] if this patch makes sense to him
https://github.com/apache/cassandra/pull/2817
was (Author: smiklosovic):
poking [~brandon.williams] if this patch makes sense to him
https://github.com/apache/cassandra/pull/2817
> Unable to write if native transport is disabled on startup
> --
>
> Key: CASSANDRA-18935
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18935
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core, Legacy/CQL
>Reporter: Cameron Zemek
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
> Attachments: 18935-3.11.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
>
> {code:java}
> if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) ||
> (nativeFlag == null && DatabaseDescriptor.startNativeTransport()))
> {
> startNativeTransport();
> StorageService.instance.setRpcReady(true);
> } {code}
> The startup code here only sets RpcReady if native transport is enabled. If
> you call
> {code:java}
> nodetool enablebinary{code}
> then this flag doesn't get set.
> But with the change from CASSANDRA-13043 it requires RpcReady set to true in
> order to get a leader for the counter update.
> Not sure what the correct fix is here, seems to only really use this flag for
> counters. So thinking perhaps the fix is to just move this outside the if
> condition.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Unable to write if native transport is disabled on startup
[
https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776564#comment-17776564
]
Stefan Miklosovic commented on CASSANDRA-18935:
---
poking [~brandon.williams] if this patch makes sense to him
https://github.com/apache/cassandra/pull/2817
> Unable to write if native transport is disabled on startup
> --
>
> Key: CASSANDRA-18935
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18935
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core, Legacy/CQL
>Reporter: Cameron Zemek
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
> Attachments: 18935-3.11.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
>
> {code:java}
> if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) ||
> (nativeFlag == null && DatabaseDescriptor.startNativeTransport()))
> {
> startNativeTransport();
> StorageService.instance.setRpcReady(true);
> } {code}
> The startup code here only sets RpcReady if native transport is enabled. If
> you call
> {code:java}
> nodetool enablebinary{code}
> then this flag doesn't get set.
> But with the change from CASSANDRA-13043 it requires RpcReady set to true in
> order to get a leader for the counter update.
> Not sure what the correct fix is here, seems to only really use this flag for
> counters. So thinking perhaps the fix is to just move this outside the if
> condition.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18935) Unable to write if native transport is disabled on startup
[
https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-18935:
-
Summary: Unable to write if native transport is disabled on startup (was:
Unable to write to counter table if native transport is disabled on startup)
> Unable to write if native transport is disabled on startup
> --
>
> Key: CASSANDRA-18935
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18935
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core, Legacy/CQL
>Reporter: Cameron Zemek
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
> Attachments: 18935-3.11.patch
>
>
>
> {code:java}
> if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) ||
> (nativeFlag == null && DatabaseDescriptor.startNativeTransport()))
> {
> startNativeTransport();
> StorageService.instance.setRpcReady(true);
> } {code}
> The startup code here only sets RpcReady if native transport is enabled. If
> you call
> {code:java}
> nodetool enablebinary{code}
> then this flag doesn't get set.
> But with the change from CASSANDRA-13043 it requires RpcReady set to true in
> order to get a leader for the counter update.
> Not sure what the correct fix is here, seems to only really use this flag for
> counters. So thinking perhaps the fix is to just move this outside the if
> condition.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Unable to write to counter table if native transport is disabled on startup
[
https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776559#comment-17776559
]
Brandon Williams commented on CASSANDRA-18935:
--
bq. why not set the flag when doing nodetool enablebinary if we found the flag
is not true
Yes, that's the correct fix, having enablebinary also call setRpcReady(true).
Given that the call appears idempotent, we could move it into the start/stop
native transport calls directly. It's probably a vestige of having thrift and
the native protocol.
> Unable to write to counter table if native transport is disabled on startup
> ---
>
> Key: CASSANDRA-18935
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18935
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core, Legacy/CQL
>Reporter: Cameron Zemek
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
> Attachments: 18935-3.11.patch
>
>
>
> {code:java}
> if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) ||
> (nativeFlag == null && DatabaseDescriptor.startNativeTransport()))
> {
> startNativeTransport();
> StorageService.instance.setRpcReady(true);
> } {code}
> The startup code here only sets RpcReady if native transport is enabled. If
> you call
> {code:java}
> nodetool enablebinary{code}
> then this flag doesn't get set.
> But with the change from CASSANDRA-13043 it requires RpcReady set to true in
> order to get a leader for the counter update.
> Not sure what the correct fix is here, seems to only really use this flag for
> counters. So thinking perhaps the fix is to just move this outside the if
> condition.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18935) Unable to write to counter table if native transport is disabled on startup
[
https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Miklosovic updated CASSANDRA-18935:
--
Bug Category: Parent values: Correctness(12982)
Complexity: Normal
Component/s: Legacy/Core
Legacy/CQL
Discovered By: Adhoc Test
Fix Version/s: 3.0.x
3.11.x
4.0.x
4.1.x
5.x
Severity: Normal
Assignee: Stefan Miklosovic
Status: Open (was: Triage Needed)
> Unable to write to counter table if native transport is disabled on startup
> ---
>
> Key: CASSANDRA-18935
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18935
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Core, Legacy/CQL
>Reporter: Cameron Zemek
>Assignee: Stefan Miklosovic
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x, 4.1.x, 5.x
>
> Attachments: 18935-3.11.patch
>
>
>
> {code:java}
> if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) ||
> (nativeFlag == null && DatabaseDescriptor.startNativeTransport()))
> {
> startNativeTransport();
> StorageService.instance.setRpcReady(true);
> } {code}
> The startup code here only sets RpcReady if native transport is enabled. If
> you call
> {code:java}
> nodetool enablebinary{code}
> then this flag doesn't get set.
> But with the change from CASSANDRA-13043 it requires RpcReady set to true in
> order to get a leader for the counter update.
> Not sure what the correct fix is here, seems to only really use this flag for
> counters. So thinking perhaps the fix is to just move this outside the if
> condition.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18935) Unable to write to counter table if native transport is disabled on startup
[
https://issues.apache.org/jira/browse/CASSANDRA-18935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776558#comment-17776558
]
Stefan Miklosovic commented on CASSANDRA-18935:
---
I think there is also another issue
if you start it all just fine and then you do nodetool disablebinary it will
not set rpc to false.
so then this
https://github.com/apache/cassandra/commit/36bdc253193318ceaf5beb9bc5e869f6af590cb1
it will still think rpc is true / that native transport is running there
> Unable to write to counter table if native transport is disabled on startup
> ---
>
> Key: CASSANDRA-18935
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18935
> Project: Cassandra
> Issue Type: Bug
>Reporter: Cameron Zemek
>Priority: Normal
> Attachments: 18935-3.11.patch
>
>
>
> {code:java}
> if ((nativeFlag != null && Boolean.parseBoolean(nativeFlag)) ||
> (nativeFlag == null && DatabaseDescriptor.startNativeTransport()))
> {
> startNativeTransport();
> StorageService.instance.setRpcReady(true);
> } {code}
> The startup code here only sets RpcReady if native transport is enabled. If
> you call
> {code:java}
> nodetool enablebinary{code}
> then this flag doesn't get set.
> But with the change from CASSANDRA-13043 it requires RpcReady set to true in
> order to get a leader for the counter update.
> Not sure what the correct fix is here, seems to only really use this flag for
> counters. So thinking perhaps the fix is to just move this outside the if
> condition.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-13911) IllegalStateException thrown by UPI.Serializer.hasNext() for some SELECT queries
[
https://issues.apache.org/jira/browse/CASSANDRA-13911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776534#comment-17776534
]
Szymon Miezal commented on CASSANDRA-13911:
---
Possibly the answer is hidden in the two following tests (committed at
https://github.com/apache/cassandra-dtest/commit/b0f34e3a6b41c3be5f86dcb8db4f32de9436b071):
*
[https://github.com/apache/cassandra-dtest/blob/0a8fea03d2f66463478ec4a49387bfb035dd403a/consistency_test.py#L1073]
*
[https://github.com/apache/cassandra-dtest/blob/0a8fea03d2f66463478ec4a49387bfb035dd403a/consistency_test.py#L1009]
> IllegalStateException thrown by UPI.Serializer.hasNext() for some SELECT
> queries
>
>
> Key: CASSANDRA-13911
> URL: https://issues.apache.org/jira/browse/CASSANDRA-13911
> Project: Cassandra
> Issue Type: Bug
> Components: Legacy/Coordination
>Reporter: Aleksey Yeschenko
>Assignee: Aleksey Yeschenko
>Priority: Normal
> Fix For: 3.0.15, 3.11.1
>
>
> Certain combinations of rows, in presence of per partition limit (set
> explicitly in 3.6+ or implicitly to 1 via DISTINCT) cause
> {{UnfilteredPartitionIterators.Serializer.hasNext()}} to throw
> {{IllegalStateException}} .
> Relevant code snippet:
> {code}
> // We can't answer this until the previously returned iterator has been fully
> consumed,
> // so complain if that's not the case.
> if (next != null && next.hasNext())
> throw new IllegalStateException("Cannot call hasNext() until the previous
> iterator has been fully consumed");
> {code}
> Since {{UnfilteredPartitionIterators.Serializer}} and
> {{UnfilteredRowIteratorSerializer.serializer}} deserialize partitions/rows
> lazily, it is required for correct operation of the partition iterator to
> have the previous partition fully consumed, so that deserializing the next
> one can start from the correct position in the byte buffer. However, that
> condition won’t always be satisfied, as there are legitimate combinations of
> rows that do not consume every row in every partition.
> For example, look at [this
> dtest|https://github.com/iamaleksey/cassandra-dtest/commits/13911].
> In case we end up with a following pattern of rows:
> {code}
> node1, partition 0 | 0
> node2, partition 0 | x x
> {code}
> , where {{x}} and {{x}} a row tombstones for rows 1 and 2, it’s sufficient
> for {{MergeIterator}} to only look at row 0 in partition from node1 and at
> row tombstone 1 from node2 to satisfy the per partition limit of 1. The
> stopping merge result counter will stop iteration right there, leaving row
> tombstone 2 from node2 unvisited and not deseiralized. Switching to the next
> partition will in turn trigger the {{IllegalStateException}} because we
> aren’t done yet.
> The stopping counter is behaving correctly, so is the {{MergeIterator}}. I’ll
> note that simply removing that condition is not enough to fix the problem
> properly - it’d just cause us to deseiralize garbage, trying to deserialize a
> new partition from a position in the bytebuffer that precedes remaining rows
> in the previous partition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18936) CI jvm-dtest-upgrade breakage
Michael Semb Wever created CASSANDRA-18936: -- Summary: CI jvm-dtest-upgrade breakage Key: CASSANDRA-18936 URL: https://issues.apache.org/jira/browse/CASSANDRA-18936 Project: Cassandra Issue Type: Bug Components: CI Reporter: Michael Semb Wever git clean failing when reusing the dtest sub clone. ref: https://the-asf.slack.com/archives/CK23JSY2K/p1697465832946299 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18912) Specify "since" in all Deprecated annotations
[
https://issues.apache.org/jira/browse/CASSANDRA-18912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776520#comment-17776520
]
Stefan Miklosovic edited comment on CASSANDRA-18912 at 10/18/23 7:05 AM:
-
j17 5.0
https://app.circleci.com/pipelines/github/instaclustr/cassandra/3347/workflows/47fab436-eb1a-4e14-a8c7-c8681475c447
j17 trunk
https://app.circleci.com/pipelines/github/instaclustr/cassandra/3349/workflows/4e1d5e11-92be-4815-90bf-1acc2bd9ca74
I dont think we need Java 11 here. This is literally just putting annotations
with comments into the code.
+1
was (Author: smiklosovic):
j17 5.0
https://app.circleci.com/pipelines/github/instaclustr/cassandra/3347/workflows/47fab436-eb1a-4e14-a8c7-c8681475c447
j17 trunk
https://app.circleci.com/pipelines/github/instaclustr/cassandra/3349/workflows/4e1d5e11-92be-4815-90bf-1acc2bd9ca74
I dont think we need Java 11 here. This is literally just putting annotations
with comments into the code.
> Specify "since" in all Deprecated annotations
> -
>
> Key: CASSANDRA-18912
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18912
> Project: Cassandra
> Issue Type: Improvement
> Components: Legacy/Core
>Reporter: Stefan Miklosovic
>Assignee: Maxim Muzafarov
>Priority: Normal
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> It would be great if we introduced in 5.0 a change in Deprecated annotations
> like this:
> {code}
> @Deprecated(since = "4.0")
> {code}
> or
> {code}
> @Deprecated(since = "3.11")
> {code}
> The reasoning behind this is that as of now, it is pretty cumbersome to
> figure out what can be removed on the next major version. It has to be,
> basically, done manually every time.
> There is also this parameter available:
> {code}
> @Deprecated(forRemoval = true / false)
> {code}
> which indicates whether the annotated element is subject to removal in a
> future version so we do not need to think about this every time if it is
> eligible for deletion in a next major or not.
> We could then have a check which would ensure that we are not releasing a
> next major with some deprecations introduced two majors before.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18330) Delivery of CEP-21: Transactional Cluster Metadata
[ https://issues.apache.org/jira/browse/CASSANDRA-18330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benjamin Lerer updated CASSANDRA-18330: --- Reviewers: Benjamin Lerer, Ekaterina Dimitrova, Jacek Lewandowski > Delivery of CEP-21: Transactional Cluster Metadata > -- > > Key: CASSANDRA-18330 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18330 > Project: Cassandra > Issue Type: Epic > Components: Cluster/Membership, Cluster/Schema >Reporter: Sam Tunnicliffe >Assignee: Sam Tunnicliffe >Priority: Normal > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18934) Downgrade to 4.1 fails due to schema changes
[
https://issues.apache.org/jira/browse/CASSANDRA-18934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17776507#comment-17776507
]
Claude Warren commented on CASSANDRA-18934:
---
I discovered similar issues while attempting to write a downgrader from v 4 to
3.1 as noted in CASSANDRA-8928 and associated pull request 2045.
CASSANDRA-8928 is part of CASSANDRA-18300. I believe that this should also be
part of that epic.
The design for the Downgrader was to have the Downgrader be able to downgrade
to the earliest format of the previous version so 5,0 should downgrade to 4.0.
Then the standard roll forward from 4.0 to 4.x should work correctly.
> Downgrade to 4.1 fails due to schema changes
>
>
> Key: CASSANDRA-18934
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18934
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Startup and Shutdown
>Reporter: David Capwell
>Priority: Normal
> Fix For: 5.0.x, 5.x
>
>
> We are required to support 5.0 downgrading to 4.1 as a migration step, but we
> don’t have tests to show this is working… I wrote a quick test to make sure a
> change we needed in Accord wouldn’t block the downgrade and see that we fail
> right now.
> {code}
> ERROR 20:56:39 Exiting due to error while processing commit log during
> initialization.
> org.apache.cassandra.db.commitlog.CommitLogReadHandler$CommitLogReadException:
> Unexpected error deserializing mutation; saved to
> /var/folders/h1/s_3p1x3s3hl0hltbpck67m0hgn/T/mutation418421767150092dat.
> This may be caused by replaying a mutation against a table with the same
> name but incompatible schema. Exception follows: java.lang.RuntimeException:
> Unknown column compaction_properties during deserialization
> at
> org.apache.cassandra.db.commitlog.CommitLogReader.readMutation(CommitLogReader.java:464)
> at
> org.apache.cassandra.db.commitlog.CommitLogReader.readSection(CommitLogReader.java:397)
> at
> org.apache.cassandra.db.commitlog.CommitLogReader.readCommitLogSegment(CommitLogReader.java:244)
> at
> org.apache.cassandra.db.commitlog.CommitLogReader.readCommitLogSegment(CommitLogReader.java:147)
> at
> org.apache.cassandra.db.commitlog.CommitLogReplayer.replayFiles(CommitLogReplayer.java:191)
> at
> org.apache.cassandra.db.commitlog.CommitLog.recoverFiles(CommitLog.java:223)
> at
> org.apache.cassandra.db.commitlog.CommitLog.recoverSegmentsOnDisk(CommitLog.java:204)
> {code}
> This was caused by a schema change in CASSANDRA-18061
> {code}
> /*
> * Licensed to the Apache Software Foundation (ASF) under one
> * or more contributor license agreements. See the NOTICE file
> * distributed with this work for additional information
> * regarding copyright ownership. The ASF licenses this file
> * to you under the Apache License, Version 2.0 (the
> * "License"); you may not use this file except in compliance
> * with the License. You may obtain a copy of the License at
> *
> * http://www.apache.org/licenses/LICENSE-2.0
> *
> * Unless required by applicable law or agreed to in writing, software
> * distributed under the License is distributed on an "AS IS" BASIS,
> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> * See the License for the specific language governing permissions and
> * limitations under the License.
> */
> package org.apache.cassandra.distributed.upgrade;
> import java.io.IOException;
> import java.io.File;
> import java.util.concurrent.atomic.AtomicBoolean;
> import org.junit.Test;
> import org.apache.cassandra.distributed.api.IUpgradeableInstance;
> public class DowngradeTest extends UpgradeTestBase
> {
> @Test
> public void test() throws Throwable
> {
> AtomicBoolean first = new AtomicBoolean(true);
> new TestCase()
> .nodes(1)
> .withConfig(c -> {
> if (first.compareAndSet(true, false))
> c.set("storage_compatibility_mode", "CASSANDRA_4");
> })
> .downgradeTo(v41)
> .setup(cluster -> {})
> // Uncomment if you want to test what happens after reading the commit log,
> which fails right now
> //.runBeforeNodeRestart((cluster, nodeId) -> {
> //IUpgradeableInstance inst = cluster.get(nodeId);
> //File f = new File((String)
> inst.config().get("commitlog_directory"));
> //deleteRecursive(f);
> //})
> .runAfterClusterUpgrade(cluster -> {})
> .run();
> }
> private void deleteRecursive(File f)
> {
> if (f.isDirectory())
> {
> File[] children = f.listFiles();
> if (children != null)
> {
> for (File c : children)
>
