Re: [PR] JAVA-3142: Ability to optionally specify remote dcs for deterministic failovers when remote dcs are used in query plan [cassandra-java-driver]
absurdfarce commented on PR #1896: URL: https://github.com/apache/cassandra-java-driver/pull/1896#issuecomment-2097402276 Apologies @nitinitt , I know this has taken a long time to get to... I really appreciate your patience. Overall I'm pretty positive on the change but I'm a bit concerned about one aspect of the implementation. As currently written the changes intermingle the "preferred DC" and "no preferred DC" cases when (maybe) adding remote DC failover nodes. I'd rather make that division much cleaner so that it's very clear which part of the code does what. I've taken a stab at [a set of changes on top of this PR](https://github.com/absurdfarce/cassandra-java-driver/pull/1) (as of this writing) to show that what might look like. Would you mind taking a look and letting me know what you think? Thanks again! And thanks to @adutra and @tolbertam for their reviews as well! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19619) Enforce contract for internal metrics naming
[
https://issues.apache.org/jira/browse/CASSANDRA-19619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843911#comment-17843911
]
Caleb Rackliffe edited comment on CASSANDRA-19619 at 5/7/24 4:01 AM:
-
CI runs in progress...will post soon...
EDIT: {{trunk}} run looks clean (see [^ci_summary.html]). Still waiting on
5.0...
was (Author: maedhroz):
CI runs in progress...will post soon...
> Enforce contract for internal metrics naming
>
>
> Key: CASSANDRA-19619
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19619
> Project: Cassandra
> Issue Type: Task
> Components: Observability/Metrics
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Normal
> Fix For: 5.0-rc, 5.0.x, 5.x
>
> Attachments: ci_summary.html
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Metrics have their internal representation which uniquely identifies a
> particular metric name. The name is formed as {{. type>..}}. For some metrics, the scope includes a
> metric name that is not necessary for uniqueness, this can be simplified.
> For example,
> {code}
> // AS IS
> org.apache.cassandra.metrics.StorageAttachedIndex.BalancedTreeIntersectionEarlyExits.my_keyspace.my_table.ColumnQueryMetrics.BalancedTreeIntersectionEarlyExits
> // TO BE
> org.apache.cassandra.metrics.StorageAttachedIndex.BalancedTreeIntersectionEarlyExits.my_keyspace.my_table.ColumnQueryMetrics
> {code}
> The metrics are filtered based on knowledge of how they are formatted, so
> having a metric scope without a built-in metric name also simplifies the way
> we filter metrics.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19619) Enforce contract for internal metrics naming
[
https://issues.apache.org/jira/browse/CASSANDRA-19619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-19619:
Attachment: ci_summary.html
> Enforce contract for internal metrics naming
>
>
> Key: CASSANDRA-19619
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19619
> Project: Cassandra
> Issue Type: Task
> Components: Observability/Metrics
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Normal
> Fix For: 5.0-rc, 5.0.x, 5.x
>
> Attachments: ci_summary.html
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Metrics have their internal representation which uniquely identifies a
> particular metric name. The name is formed as {{. type>..}}. For some metrics, the scope includes a
> metric name that is not necessary for uniqueness, this can be simplified.
> For example,
> {code}
> // AS IS
> org.apache.cassandra.metrics.StorageAttachedIndex.BalancedTreeIntersectionEarlyExits.my_keyspace.my_table.ColumnQueryMetrics.BalancedTreeIntersectionEarlyExits
> // TO BE
> org.apache.cassandra.metrics.StorageAttachedIndex.BalancedTreeIntersectionEarlyExits.my_keyspace.my_table.ColumnQueryMetrics
> {code}
> The metrics are filtered based on knowledge of how they are formatted, so
> having a metric scope without a built-in metric name also simplifies the way
> we filter metrics.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] JAVA-3142: Ability to optionally specify remote dcs for deterministic failovers when remote dcs are used in query plan [cassandra-java-driver]
tolbertam commented on code in PR #1896:
URL:
https://github.com/apache/cassandra-java-driver/pull/1896#discussion_r1591774161
##
core/src/main/java/com/datastax/oss/driver/api/core/config/TypedDriverOption.java:
##
@@ -892,6 +892,16 @@ public String toString() {
DefaultDriverOption.LOAD_BALANCING_DC_FAILOVER_ALLOW_FOR_LOCAL_CONSISTENCY_LEVELS,
GenericType.BOOLEAN);
+ /**
+ * Ordered preference list of remote dc's optionally supplied for automatic
failover and included
Review Comment:
```suggestion
* Ordered preference list of remote dcs optionally supplied for automatic
failover and included
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
Re: [PR] JAVA-3142: Ability to optionally specify remote dcs for deterministic failovers when remote dcs are used in query plan [cassandra-java-driver]
tolbertam commented on code in PR #1896:
URL:
https://github.com/apache/cassandra-java-driver/pull/1896#discussion_r1591773768
##
core/src/main/java/com/datastax/oss/driver/api/core/config/DefaultDriverOption.java:
##
@@ -982,7 +982,13 @@ public enum DefaultDriverOption implements DriverOption {
* Value-type: {@link java.time.Duration}
*/
SSL_KEYSTORE_RELOAD_INTERVAL("advanced.ssl-engine-factory.keystore-reload-interval"),
- ;
+ /**
+ * Ordered preference list of remote dc's optionally supplied for automatic
failover.
Review Comment:
Tiny suggestion:
```suggestion
* Ordered preference list of remote dcs optionally supplied for automatic
failover.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-accord) 01/01: retry new system checkpoint
This is an automated email from the ASF dual-hosted git repository.
aweisberg pushed a commit to branch retry-new-system
in repository https://gitbox.apache.org/repos/asf/cassandra-accord.git
commit 892f19d7714bc7128d84b7e635acc6621ce476c4
Author: Ariel Weisberg
AuthorDate: Mon May 6 18:12:30 2024 -0400
retry new system checkpoint
---
accord-core/src/main/java/accord/local/Node.java | 18 ++--
.../main/java/accord/topology/TopologyManager.java | 120 +
accord-core/src/test/java/accord/Utils.java| 8 ++
.../java/accord/topology/TopologyManagerTest.java | 61 +--
4 files changed, 149 insertions(+), 58 deletions(-)
diff --git a/accord-core/src/main/java/accord/local/Node.java
b/accord-core/src/main/java/accord/local/Node.java
index 1605f43..62331d0 100644
--- a/accord-core/src/main/java/accord/local/Node.java
+++ b/accord-core/src/main/java/accord/local/Node.java
@@ -34,14 +34,7 @@ import java.util.function.Function;
import java.util.function.LongSupplier;
import java.util.function.Supplier;
import java.util.function.ToLongFunction;
-import javax.annotation.Nonnull;
-import javax.annotation.Nullable;
-import accord.coordinate.CoordinateEphemeralRead;
-import accord.coordinate.CoordinationAdapter;
-import accord.coordinate.CoordinationAdapter.Factory.Step;
-import accord.utils.DeterministicSet;
-import accord.utils.Invariants;
import com.google.common.annotations.VisibleForTesting;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -58,7 +51,10 @@ import accord.api.RoutingKey;
import accord.api.Scheduler;
import accord.api.TopologySorter;
import accord.config.LocalConfig;
+import accord.coordinate.CoordinateEphemeralRead;
import accord.coordinate.CoordinateTransaction;
+import accord.coordinate.CoordinationAdapter;
+import accord.coordinate.CoordinationAdapter.Factory.Step;
import accord.coordinate.MaybeRecover;
import accord.coordinate.Outcome;
import accord.coordinate.RecoverWithRoute;
@@ -85,12 +81,16 @@ import accord.primitives.Unseekables;
import accord.topology.Shard;
import accord.topology.Topology;
import accord.topology.TopologyManager;
+import accord.utils.DeterministicSet;
+import accord.utils.Invariants;
import accord.utils.MapReduceConsume;
import accord.utils.RandomSource;
import accord.utils.async.AsyncChain;
import accord.utils.async.AsyncExecutor;
import accord.utils.async.AsyncResult;
import accord.utils.async.AsyncResults;
+import javax.annotation.Nonnull;
+import javax.annotation.Nullable;
import net.nicoulaj.compilecommand.annotations.Inline;
import static accord.utils.Invariants.illegalState;
@@ -179,7 +179,8 @@ public class Node implements ConfigurationService.Listener,
NodeTimeService
this.localRequestHandler = localRequestHandler;
this.configService = configService;
this.coordinationAdapters = coordinationAdapters;
-this.topology = new TopologyManager(topologySorter, id);
+this.topology = new TopologyManager(topologySorter, id, scheduler,
nowTimeUnit);
+topology.scheduleTopologyUpdateWatchdog();
this.nowSupplier = nowSupplier;
this.nowTimeUnit = nowTimeUnit;
this.now = new
AtomicReference<>(Timestamp.fromValues(topology.epoch(),
nowSupplier.getAsLong(), id));
@@ -330,6 +331,7 @@ public class Node implements ConfigurationService.Listener,
NodeTimeService
public void shutdown()
{
commandStores.shutdown();
+topology.shutdown();
}
public Timestamp uniqueNow()
diff --git a/accord-core/src/main/java/accord/topology/TopologyManager.java
b/accord-core/src/main/java/accord/topology/TopologyManager.java
index bd4b25a..d2f609e 100644
--- a/accord-core/src/main/java/accord/topology/TopologyManager.java
+++ b/accord-core/src/main/java/accord/topology/TopologyManager.java
@@ -23,14 +23,18 @@ import java.util.BitSet;
import java.util.List;
import java.util.Set;
import java.util.TreeSet;
+import java.util.concurrent.TimeUnit;
import java.util.function.Function;
import java.util.function.Supplier;
+import java.util.function.ToLongFunction;
import com.google.common.annotations.VisibleForTesting;
import accord.api.ConfigurationService.EpochReady;
import accord.api.RoutingKey;
+import accord.api.Scheduler;
import accord.api.TopologySorter;
+import accord.coordinate.Timeout;
import accord.coordinate.TopologyMismatch;
import accord.coordinate.tracking.QuorumTracker;
import accord.local.CommandStore;
@@ -44,13 +48,14 @@ import accord.utils.Invariants;
import accord.utils.async.AsyncChain;
import accord.utils.async.AsyncResult;
import accord.utils.async.AsyncResults;
-
import javax.annotation.Nullable;
+import javax.annotation.concurrent.GuardedBy;
import static accord.coordinate.tracking.RequestStatus.Success;
import static accord.primitives.AbstractRanges.UnionMode.MERGE_ADJACENT;
import static accord.primitives.Routables.Slice.Minimal;
import static
(cassandra-accord) branch retry-new-system created (now 892f19d)
This is an automated email from the ASF dual-hosted git repository. aweisberg pushed a change to branch retry-new-system in repository https://gitbox.apache.org/repos/asf/cassandra-accord.git at 892f19d retry new system checkpoint This branch includes the following new commits: new 892f19d retry new system checkpoint The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch dependabot/pip/dot-build/ci/jinja2-3.1.4 created (now 0368d5cf7c)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/pip/dot-build/ci/jinja2-3.1.4 in repository https://gitbox.apache.org/repos/asf/cassandra.git at 0368d5cf7c Bump jinja2 from 3.1.3 to 3.1.4 in /.build/ci No new revisions were added by this update. - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19619) Enforce contract for internal metrics naming
[
https://issues.apache.org/jira/browse/CASSANDRA-19619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843911#comment-17843911
]
Caleb Rackliffe commented on CASSANDRA-19619:
-
CI runs in progress...will post soon...
> Enforce contract for internal metrics naming
>
>
> Key: CASSANDRA-19619
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19619
> Project: Cassandra
> Issue Type: Task
> Components: Observability/Metrics
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Normal
> Fix For: 5.0-rc, 5.0.x, 5.x
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Metrics have their internal representation which uniquely identifies a
> particular metric name. The name is formed as {{. type>..}}. For some metrics, the scope includes a
> metric name that is not necessary for uniqueness, this can be simplified.
> For example,
> {code}
> // AS IS
> org.apache.cassandra.metrics.StorageAttachedIndex.BalancedTreeIntersectionEarlyExits.my_keyspace.my_table.ColumnQueryMetrics.BalancedTreeIntersectionEarlyExits
> // TO BE
> org.apache.cassandra.metrics.StorageAttachedIndex.BalancedTreeIntersectionEarlyExits.my_keyspace.my_table.ColumnQueryMetrics
> {code}
> The metrics are filtered based on knowledge of how they are formatted, so
> having a metric scope without a built-in metric name also simplifies the way
> we filter metrics.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-19341) Relation and Restriction hierarchies are too complex and error prone
[
https://issues.apache.org/jira/browse/CASSANDRA-19341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843844#comment-17843844
]
Ekaterina Dimitrova edited comment on CASSANDRA-19341 at 5/6/24 8:50 PM:
-
This was the latest CI run:
https://app.circleci.com/pipelines/github/blerer/cassandra/401/workflows/befac1e9-d706-436c-aa18-bb5141ec4436
https://app.circleci.com/pipelines/github/blerer/cassandra/401/workflows/befac1e9-d706-436c-aa18-bb5141ec4436/jobs/11079/tests
- environmental, seen before. Considering the same tests passed with 11, we
did not see the need to rerun that job
The rest are known issues:
test_counter_node_down - CASSANDRA-19346
test_change_durable_writes - CASSANDRA-19601
test_cleanup - CASSANDRA-18908
org.apache.cassandra.simulator.test.HarrySimulatorTest - CASSANDRA-19279
New unit tests were not run in a loop as they are pretty deterministic.
Those that changed were changing primarily error messages text.
Thank you everyone!
was (Author: e.dimitrova):
This was the latest CI run:
https://app.circleci.com/pipelines/github/blerer/cassandra/401/workflows/befac1e9-d706-436c-aa18-bb5141ec4436
https://app.circleci.com/pipelines/github/blerer/cassandra/401/workflows/befac1e9-d706-436c-aa18-bb5141ec4436/jobs/11079/tests
- environmental, seen before. Considering the same tests passed with 11, we
did not see the need to rerun that job
The rest are known issues:
test_counter_node_down - CASSANDRA-19346
test_change_durable_writes - CASSANDRA-19601
test_cleanup - CASSANDRA-18908
org.apache.cassandra.simulator.test.HarrySimulatorTest - CASSANDRA-19279
New unit tests were not run in a loop as they are pretty deterministic.
Thank you everyone!
> Relation and Restriction hierarchies are too complex and error prone
>
>
> Key: CASSANDRA-19341
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19341
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL/Interpreter
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Normal
> Fix For: 5.1
>
> Time Spent: 32.5h
> Remaining Estimate: 0h
>
> The {{Relation}} and {{Restriction}} hierarchy have been designed when C* was
> only supporting a limited amount of operators and columns expressions (single
> column, multi-column and token expressions). Over time they have grown in
> complexity making the code harder to understand and modify and error prone.
> Their design is also resulting in unnecessary limitations that could be
> easily lifted, like the ability to accept different predicates on the same
> column.
> Today adding a new operator requires the addition of a lot of glue code and
> surgical changes accross the CQL layer. Making patch for features such as
> CASSANDRA-18584 much complex than it should be.
> The goal of this ticket is to simplify the {{Relation}} and {{Restriction}}
> hierarchies and modify operator class so that adding new operators requires
> only changes to the {{Operator}} class and ANTLR file.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19620) Refactor ColumnCondition post CASSANDRA-19341
[ https://issues.apache.org/jira/browse/CASSANDRA-19620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-19620: Description: CASSANDRA-19341 added ColumnsExpression in restrictions. This ticket's target is to extend the kinds of ColumnsExpression and use them in conditions, too. Also, it should add a few other simplifications around ColumnCondition, which were suggested by [~blerer] was: CASSANDRA-19341 added ColumnsExpression in restrictions. This ticket's target is to extend the kinds of ColumnsExpression and use them in conditions, too. Also, it should add a few other simplifications around ColumnCondition, suggested by [~blerer] > Refactor ColumnCondition post CASSANDRA-19341 > - > > Key: CASSANDRA-19620 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19620 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Interpreter >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 5.x > > > CASSANDRA-19341 added ColumnsExpression in restrictions. > This ticket's target is to extend the kinds of ColumnsExpression and use them > in conditions, too. > Also, it should add a few other simplifications around ColumnCondition, which > were suggested by [~blerer] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19620) Refactor ColumnCondition post CASSANDRA-19341
[ https://issues.apache.org/jira/browse/CASSANDRA-19620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ekaterina Dimitrova updated CASSANDRA-19620: Change Category: Code Clarity Complexity: Normal Component/s: CQL/Interpreter Fix Version/s: 5.x Status: Open (was: Triage Needed) > Refactor ColumnCondition post CASSANDRA-19341 > - > > Key: CASSANDRA-19620 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19620 > Project: Cassandra > Issue Type: Improvement > Components: CQL/Interpreter >Reporter: Ekaterina Dimitrova >Assignee: Ekaterina Dimitrova >Priority: Normal > Fix For: 5.x > > > CASSANDRA-19341 added ColumnsExpression in restrictions. > This ticket's target is to extend the kinds of ColumnsExpression and use them > in conditions, too. > Also, it should add a few other simplifications around ColumnCondition, > suggested by [~blerer] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-19620) Refactor ColumnCondition post CASSANDRA-19341
Ekaterina Dimitrova created CASSANDRA-19620: --- Summary: Refactor ColumnCondition post CASSANDRA-19341 Key: CASSANDRA-19620 URL: https://issues.apache.org/jira/browse/CASSANDRA-19620 Project: Cassandra Issue Type: Improvement Reporter: Ekaterina Dimitrova Assignee: Ekaterina Dimitrova CASSANDRA-19341 added ColumnsExpression in restrictions. This ticket's target is to extend the kinds of ColumnsExpression and use them in conditions, too. Also, it should add a few other simplifications around ColumnCondition, suggested by [~blerer] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[PR] JAVA-3118: Add support for vector data type in Schema Builder, QueryBuilder [cassandra-java-driver]
SiyaoIsHiding opened a new pull request, #1931: URL: https://github.com/apache/cassandra-java-driver/pull/1931 The current PR is only about the `ANN OF` in the `QueryBuiler`, not anything else yet. I would appreciate it if this PR undergo an initial review, which will help my future development in `SchemaBuilder`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19341) Relation and Restriction hierarchies are too complex and error prone
[
https://issues.apache.org/jira/browse/CASSANDRA-19341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843844#comment-17843844
]
Ekaterina Dimitrova commented on CASSANDRA-19341:
-
This was the latest CI run:
https://app.circleci.com/pipelines/github/blerer/cassandra/401/workflows/befac1e9-d706-436c-aa18-bb5141ec4436
https://app.circleci.com/pipelines/github/blerer/cassandra/401/workflows/befac1e9-d706-436c-aa18-bb5141ec4436/jobs/11079/tests
- environmental, seen before. Considering the same tests passed with 11, we
did not see the need to rerun that job
The rest are known issues:
test_counter_node_down - CASSANDRA-19346
test_change_durable_writes - CASSANDRA-19601
test_cleanup - CASSANDRA-18908
org.apache.cassandra.simulator.test.HarrySimulatorTest - CASSANDRA-19279
New unit tests were not run in a loop as they are pretty deterministic.
Thank you everyone!
> Relation and Restriction hierarchies are too complex and error prone
>
>
> Key: CASSANDRA-19341
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19341
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL/Interpreter
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Normal
> Fix For: 5.1
>
> Time Spent: 32.5h
> Remaining Estimate: 0h
>
> The {{Relation}} and {{Restriction}} hierarchy have been designed when C* was
> only supporting a limited amount of operators and columns expressions (single
> column, multi-column and token expressions). Over time they have grown in
> complexity making the code harder to understand and modify and error prone.
> Their design is also resulting in unnecessary limitations that could be
> easily lifted, like the ability to accept different predicates on the same
> column.
> Today adding a new operator requires the addition of a lot of glue code and
> surgical changes accross the CQL layer. Making patch for features such as
> CASSANDRA-18584 much complex than it should be.
> The goal of this ticket is to simplify the {{Relation}} and {{Restriction}}
> hierarchies and modify operator class so that adding new operators requires
> only changes to the {{Operator}} class and ANTLR file.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19619) Enforce contract for internal metrics naming
[
https://issues.apache.org/jira/browse/CASSANDRA-19619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843835#comment-17843835
]
Michael Semb Wever commented on CASSANDRA-19619:
+1 (so long CI is ok)
> Enforce contract for internal metrics naming
>
>
> Key: CASSANDRA-19619
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19619
> Project: Cassandra
> Issue Type: Task
> Components: Observability/Metrics
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Normal
> Fix For: 5.0-rc, 5.0.x, 5.x
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Metrics have their internal representation which uniquely identifies a
> particular metric name. The name is formed as {{. type>..}}. For some metrics, the scope includes a
> metric name that is not necessary for uniqueness, this can be simplified.
> For example,
> {code}
> // AS IS
> org.apache.cassandra.metrics.StorageAttachedIndex.BalancedTreeIntersectionEarlyExits.my_keyspace.my_table.ColumnQueryMetrics.BalancedTreeIntersectionEarlyExits
> // TO BE
> org.apache.cassandra.metrics.StorageAttachedIndex.BalancedTreeIntersectionEarlyExits.my_keyspace.my_table.ColumnQueryMetrics
> {code}
> The metrics are filtered based on knowledge of how they are formatted, so
> having a metric scope without a built-in metric name also simplifies the way
> we filter metrics.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19335) Default nodetool tablestats to Human-Readable Output
[
https://issues.apache.org/jira/browse/CASSANDRA-19335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843829#comment-17843829
]
Brandon Williams commented on CASSANDRA-19335:
--
First let me say you don't have to use docker, I personally don't and just
setup a python venv locally.
In both of your first two attempts, the real issue is that C* isn't working:
{noformat}
Node node1 should be running before waiting for log
message, but C* process is terminated.
{noformat}
I would check that you can run C* manually from /home/cassandra/cassandra.
Your last attempt used the wrong cassandra-dir but it probably would have ended
up the same way, so troubleshooting C* to get it working is the next step.
> Default nodetool tablestats to Human-Readable Output
>
>
> Key: CASSANDRA-19335
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19335
> Project: Cassandra
> Issue Type: Improvement
> Components: Tool/nodetool
>Reporter: Leo Toff
>Assignee: Leo Toff
>Priority: Low
> Fix For: 5.x
>
> Attachments: c-19335-dtest-bootstraptest-fail.txt,
> c-19335-dtest-ccm-fail.txt, c-19335-dtest-compactiontest-fail.txt
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> *Current Behavior*
> The current implementation of nodetool tablestats in Apache Cassandra outputs
> statistics in a format that is not immediately human-readable. This output
> primarily includes raw byte counts, which require additional calculation or
> conversion to be easily understood by users. This can be inefficient and
> time-consuming, especially for users who frequently monitor these statistics
> for performance tuning or maintenance purposes.
> *Proposed Change*
> We propose that nodetool tablestats should, by default, provide its output in
> a human-readable format. This change would involve converting byte counts
> into more understandable units (KiB, MiB, GiB). The tool could still retain
> the option to display raw data for those who need it, perhaps through a flag
> such as --no-human-readable or --raw.
> *Considerations*
> The change should maintain backward compatibility, ensuring that scripts or
> tools relying on the current output format can continue to function correctly.
> We should provide adequate documentation and examples of both the new default
> output and how to access the raw data format, if needed.
> *Alignment*
> Discussion in the dev mailing list:
> [https://lists.apache.org/thread/mlp715kxho5b6f1ql9omlzmmnh4qfby9]
> *Related work*
> Previous work in the series:
> # https://issues.apache.org/jira/browse/CASSANDRA-19015
> # https://issues.apache.org/jira/browse/CASSANDRA-19104
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19429) Remove lock contention generated by getCapacity function in SSTableReader
[
https://issues.apache.org/jira/browse/CASSANDRA-19429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843813#comment-17843813
]
Stefan Miklosovic commented on CASSANDRA-19429:
---
I think that size and capacity are trully two different concepts as already
explained by Brandon and myself and I believe that having capacity set to 0
practically disables a respective cache.
CI for 4.1 looks good with the below patch. I identified couple more places
where we could use reading the capacity from DatabaseDescriptor rather than
calling getCapacity() on the cache to see if it is bigger than zero to proceed.
One just needs to be cautious to not forget to update DatabaseDescriptor when
capacity is set to zero, e.g. from JMX etc if we are going to use DD as the
source of truth whether a cache is enabled or not.
For other branches, this ticket, as I write this, specifies only 4.0 and 4.1 ax
fix versions but that should be extended up to trunk.
This is all being said if there is an appetite to do this across all versions
instead of just fixing the trace messages as suggested by Jon. The bellow build
tells me that we are on something here and it would be just a matter of
applying this elsewhere.
[CASSANDRA-19429-4.1|https://github.com/instaclustr/cassandra/tree/CASSANDRA-19429-4.1]
{noformat}
java11_pre-commit_tests
java11_separate_tests
java8_pre-commit_tests
✓ j8_build 6m 53s
✓ j8_cqlsh_dtests_py3 6m 53s
✓ j8_cqlsh_dtests_py3118m 26s
✓ j8_cqlsh_dtests_py311_vnode 6m 53s
✓ j8_cqlsh_dtests_py38 8m 3s
✓ j8_cqlsh_dtests_py38_vnode 7m 40s
✓ j8_cqlsh_dtests_py3_vnode8m 16s
✓ j8_cqlshlib_cython_tests13m 32s
✓ j8_cqlshlib_tests 11m 44s
✓ j8_dtests 33m 23s
✓ j8_dtests_vnode 38m 26s
✓ j8_jvm_dtests 19m 30s
✓ j8_jvm_dtests_vnode 12m 23s
✓ j8_simulator_dtests 5m 51s
✓ j11_jvm_dtests_vnode11m 50s
✓ j11_jvm_dtests 19m 47s
✓ j11_dtests_vnode 35m 6s
✓ j11_dtests 34m 46s
✓ j11_cqlshlib_tests 6m 35s
✓ j11_cqlshlib_cython_tests9m 27s
✓ j11_cqlsh_dtests_py3_vnode 5m 47s
✓ j11_cqlsh_dtests_py38_vnode 6m 7s
✓ j11_cqlsh_dtests_py385m 41s
✓ j11_cqlsh_dtests_py311_vnode 5m 50s
✓ j11_cqlsh_dtests_py311 5m 56s
✓ j11_cqlsh_dtests_py3 5m 38s
✕ j8_unit_tests9m 53s
org.apache.cassandra.cql3.MemtableSizeTest testSize[skiplist]
org.apache.cassandra.net.ConnectionTest testMessageDeliveryOnReconnect
✕ j8_utests_system_keyspace_directory 11m 5s
org.apache.cassandra.cql3.MemtableSizeTest testSize[skiplist]
✕ j11_unit_tests 8m 12s
org.apache.cassandra.cql3.MemtableSizeTest testSize[skiplist]
java8_separate_tests
{noformat}
[java11_pre-commit_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/4285/workflows/7470a92f-1cb3-487d-9d0a-dd1f781a79e8]
[java11_separate_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/4285/workflows/2116e61d-1e6a-4589-85e7-1980acfbdb05]
[java8_pre-commit_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/4285/workflows/30501a49-c2b0-4aaf-a504-f087f27e88f7]
[java8_separate_tests|https://app.circleci.com/pipelines/github/instaclustr/cassandra/4285/workflows/232134f1-5956-4346-afa3-bd556b6c5f60]
> Remove lock contention generated by getCapacity function in SSTableReader
> -
>
> Key: CASSANDRA-19429
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19429
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Dipietro Salvatore
>Assignee: Dipietro Salvatore
>Priority: Normal
> Fix For: 4.0.x, 4.1.x
>
> Attachments: Screenshot 2024-02-26 at 10.27.10.png, Screenshot
> 2024-02-27 at 11.29.41.png, Screenshot 2024-03-19 at 15.22.50.png,
> asprof_cass4.1.3__lock_20240216052912lock.html,
> image-2024-03-08-15-51-30-439.png, image-2024-03-08-15-52-07-902.png
>
> Time
[jira] [Updated] (CASSANDRA-19341) Relation and Restriction hierarchies are too complex and error prone
[
https://issues.apache.org/jira/browse/CASSANDRA-19341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-19341:
---
Source Control Link:
https://github.com/apache/cassandra/commit/534da538d229a5b4fc1d9ef7cc8b37ead455191c
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Patch committed into trunk at 534da538d229a5b4fc1d9ef7cc8b37ead455191c.
DTest committed into the trunk branch at
6dd7def99467952d1df6a451d2e74861cf6599de
> Relation and Restriction hierarchies are too complex and error prone
>
>
> Key: CASSANDRA-19341
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19341
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL/Interpreter
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Normal
> Fix For: 5.1
>
> Time Spent: 32.5h
> Remaining Estimate: 0h
>
> The {{Relation}} and {{Restriction}} hierarchy have been designed when C* was
> only supporting a limited amount of operators and columns expressions (single
> column, multi-column and token expressions). Over time they have grown in
> complexity making the code harder to understand and modify and error prone.
> Their design is also resulting in unnecessary limitations that could be
> easily lifted, like the ability to accept different predicates on the same
> column.
> Today adding a new operator requires the addition of a lot of glue code and
> surgical changes accross the CQL layer. Making patch for features such as
> CASSANDRA-18584 much complex than it should be.
> The goal of this ticket is to simplify the {{Relation}} and {{Restriction}}
> hierarchies and modify operator class so that adding new operators requires
> only changes to the {{Operator}} class and ANTLR file.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19341) Relation and Restriction hierarchies are too complex and error prone
[
https://issues.apache.org/jira/browse/CASSANDRA-19341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-19341:
---
Fix Version/s: 5.1
(was: 5.x)
> Relation and Restriction hierarchies are too complex and error prone
>
>
> Key: CASSANDRA-19341
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19341
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL/Interpreter
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Normal
> Fix For: 5.1
>
> Time Spent: 32.5h
> Remaining Estimate: 0h
>
> The {{Relation}} and {{Restriction}} hierarchy have been designed when C* was
> only supporting a limited amount of operators and columns expressions (single
> column, multi-column and token expressions). Over time they have grown in
> complexity making the code harder to understand and modify and error prone.
> Their design is also resulting in unnecessary limitations that could be
> easily lifted, like the ability to accept different predicates on the same
> column.
> Today adding a new operator requires the addition of a lot of glue code and
> surgical changes accross the CQL layer. Making patch for features such as
> CASSANDRA-18584 much complex than it should be.
> The goal of this ticket is to simplify the {{Relation}} and {{Restriction}}
> hierarchies and modify operator class so that adding new operators requires
> only changes to the {{Operator}} class and ANTLR file.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19341) Relation and Restriction hierarchies are too complex and error prone
[
https://issues.apache.org/jira/browse/CASSANDRA-19341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Benjamin Lerer updated CASSANDRA-19341:
---
Status: Ready to Commit (was: Review In Progress)
> Relation and Restriction hierarchies are too complex and error prone
>
>
> Key: CASSANDRA-19341
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19341
> Project: Cassandra
> Issue Type: Improvement
> Components: CQL/Interpreter
>Reporter: Benjamin Lerer
>Assignee: Benjamin Lerer
>Priority: Normal
> Fix For: 5.x
>
> Time Spent: 32.5h
> Remaining Estimate: 0h
>
> The {{Relation}} and {{Restriction}} hierarchy have been designed when C* was
> only supporting a limited amount of operators and columns expressions (single
> column, multi-column and token expressions). Over time they have grown in
> complexity making the code harder to understand and modify and error prone.
> Their design is also resulting in unnecessary limitations that could be
> easily lifted, like the ability to accept different predicates on the same
> column.
> Today adding a new operator requires the addition of a lot of glue code and
> surgical changes accross the CQL layer. Making patch for features such as
> CASSANDRA-18584 much complex than it should be.
> The goal of this ticket is to simplify the {{Relation}} and {{Restriction}}
> hierarchies and modify operator class so that adding new operators requires
> only changes to the {{Operator}} class and ANTLR file.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-dtest) branch trunk updated: Align tests with CASSANDRA-19341
This is an automated email from the ASF dual-hosted git repository.
blerer pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra-dtest.git
The following commit(s) were added to refs/heads/trunk by this push:
new 6dd7def9 Align tests with CASSANDRA-19341
6dd7def9 is described below
commit 6dd7def99467952d1df6a451d2e74861cf6599de
Author: Benjamin Lerer
AuthorDate: Thu Mar 21 17:19:58 2024 +0100
Align tests with CASSANDRA-19341
patch by Benjamin Lerer; review by Berenguer Blasi, Caleb Rackliffe and
Ekaterina Dimitrova for CASSANDRA-19341
---
cql_test.py | 23 ---
json_test.py | 4 +---
2 files changed, 17 insertions(+), 10 deletions(-)
diff --git a/cql_test.py b/cql_test.py
index 99d3088c..30c5507d 100644
--- a/cql_test.py
+++ b/cql_test.py
@@ -1219,14 +1219,23 @@ class TestCQLSlowQuery(CQLTester):
TestCQLSlowQuery._assert_logs(node, session, table,
query="SELECT * FROM {} WHERE k >= 0 AND
k < 100 ALLOW FILTERING",
logged_query="SELECT \* FROM ks.{} WHERE
k >= 0 AND k < 100")
-TestCQLSlowQuery._assert_logs(node, session, table,
- query="SELECT * FROM {} WHERE k <= 100
AND k > 0 ALLOW FILTERING",
- logged_query="SELECT \* FROM ks.{} WHERE
k > 0 AND k <= 100")
-TestCQLSlowQuery._assert_logs(node, session, table,
- query="SELECT * FROM {} WHERE k < 100
AND k >= 0 ALLOW FILTERING",
- logged_query="SELECT \* FROM ks.{} WHERE
k >= 0 AND k < 100")
-# test logging of slow queries with restriciton on regular column
+if node.cluster.version() >= '5.1':
+TestCQLSlowQuery._assert_logs(node, session, table,
+ query="SELECT * FROM {} WHERE k <=
100 AND k > 0 ALLOW FILTERING",
+ logged_query="SELECT \* FROM ks.{}
WHERE k <= 100 AND k > 0")
+TestCQLSlowQuery._assert_logs(node, session, table,
+ query="SELECT * FROM {} WHERE k <
100 AND k >= 0 ALLOW FILTERING",
+ logged_query="SELECT \* FROM ks.{}
WHERE k < 100 AND k >= 0")
+else:
+TestCQLSlowQuery._assert_logs(node, session, table,
+ query="SELECT * FROM {} WHERE k <=
100 AND k > 0 ALLOW FILTERING",
+ logged_query="SELECT \* FROM ks.{}
WHERE k > 0 AND k <= 100")
+TestCQLSlowQuery._assert_logs(node, session, table,
+ query="SELECT * FROM {} WHERE k <
100 AND k >= 0 ALLOW FILTERING",
+ logged_query="SELECT \* FROM ks.{}
WHERE k >= 0 AND k < 100")
+
+# test logging of slow queries with restriction on regular column
TestCQLSlowQuery._assert_logs(node, session, table,
query="SELECT * FROM {} WHERE v = 1
ALLOW FILTERING",
logged_query="SELECT \* FROM ks.{} WHERE
v = 1")
diff --git a/json_test.py b/json_test.py
index 4f2e32b5..94c09f55 100644
--- a/json_test.py
+++ b/json_test.py
@@ -1238,10 +1238,10 @@ class TestJsonFullRowInsertSelect(Tester):
"""
run_func_docstring(tester=self, test_func=self.test_simple_schema)
+@since("2.0", max_version="5.0")
def test_pkey_requirement(self):
"""
Create schema:
-
>>> cqlsh('''
... CREATE TABLE primitive_type_test (
... key1 text PRIMARY KEY,
@@ -1261,9 +1261,7 @@ class TestJsonFullRowInsertSelect(Tester):
... col14 varint,
... col15 boolean)
... ''')
-
Try to create a JSON row with the pkey omitted from the column list,
and omitted from the JSON data:
-
>>> cqlsh_err_print('''INSERT INTO primitive_type_test JSON
'{"col1": "bar"}' ''')
:2:InvalidRequest: Error from server: code=2200 [Invalid
query] message="Invalid null value in condition for column key1"
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19619) Enforce contract for internal metrics naming
[
https://issues.apache.org/jira/browse/CASSANDRA-19619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843802#comment-17843802
]
Caleb Rackliffe commented on CASSANDRA-19619:
-
I'll review and run CI for ya...
> Enforce contract for internal metrics naming
>
>
> Key: CASSANDRA-19619
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19619
> Project: Cassandra
> Issue Type: Task
> Components: Observability/Metrics
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Normal
> Fix For: 5.0-rc, 5.0.x, 5.x
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Metrics have their internal representation which uniquely identifies a
> particular metric name. The name is formed as {{. type>..}}. For some metrics, the scope includes a
> metric name that is not necessary for uniqueness, this can be simplified.
> For example,
> {code}
> // AS IS
> org.apache.cassandra.metrics.StorageAttachedIndex.BalancedTreeIntersectionEarlyExits.my_keyspace.my_table.ColumnQueryMetrics.BalancedTreeIntersectionEarlyExits
> // TO BE
> org.apache.cassandra.metrics.StorageAttachedIndex.BalancedTreeIntersectionEarlyExits.my_keyspace.my_table.ColumnQueryMetrics
> {code}
> The metrics are filtered based on knowledge of how they are formatted, so
> having a metric scope without a built-in metric name also simplifies the way
> we filter metrics.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19619) Enforce contract for internal metrics naming
[
https://issues.apache.org/jira/browse/CASSANDRA-19619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-19619:
Reviewers: Caleb Rackliffe
Status: Review In Progress (was: Patch Available)
> Enforce contract for internal metrics naming
>
>
> Key: CASSANDRA-19619
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19619
> Project: Cassandra
> Issue Type: Task
> Components: Observability/Metrics
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Normal
> Fix For: 5.0-rc, 5.0.x, 5.x
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Metrics have their internal representation which uniquely identifies a
> particular metric name. The name is formed as {{. type>..}}. For some metrics, the scope includes a
> metric name that is not necessary for uniqueness, this can be simplified.
> For example,
> {code}
> // AS IS
> org.apache.cassandra.metrics.StorageAttachedIndex.BalancedTreeIntersectionEarlyExits.my_keyspace.my_table.ColumnQueryMetrics.BalancedTreeIntersectionEarlyExits
> // TO BE
> org.apache.cassandra.metrics.StorageAttachedIndex.BalancedTreeIntersectionEarlyExits.my_keyspace.my_table.ColumnQueryMetrics
> {code}
> The metrics are filtered based on knowledge of how they are formatted, so
> having a metric scope without a built-in metric name also simplifies the way
> we filter metrics.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19619) Enforce contract for internal metrics naming
[
https://issues.apache.org/jira/browse/CASSANDRA-19619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Caleb Rackliffe updated CASSANDRA-19619:
Test and Documentation Plan: n/a (shouldn't be any client-facing changes)
Status: Patch Available (was: In Progress)
> Enforce contract for internal metrics naming
>
>
> Key: CASSANDRA-19619
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19619
> Project: Cassandra
> Issue Type: Task
> Components: Observability/Metrics
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Normal
> Fix For: 5.0-rc, 5.0.x, 5.x
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Metrics have their internal representation which uniquely identifies a
> particular metric name. The name is formed as {{. type>..}}. For some metrics, the scope includes a
> metric name that is not necessary for uniqueness, this can be simplified.
> For example,
> {code}
> // AS IS
> org.apache.cassandra.metrics.StorageAttachedIndex.BalancedTreeIntersectionEarlyExits.my_keyspace.my_table.ColumnQueryMetrics.BalancedTreeIntersectionEarlyExits
> // TO BE
> org.apache.cassandra.metrics.StorageAttachedIndex.BalancedTreeIntersectionEarlyExits.my_keyspace.my_table.ColumnQueryMetrics
> {code}
> The metrics are filtered based on knowledge of how they are formatted, so
> having a metric scope without a built-in metric name also simplifies the way
> we filter metrics.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19557) ShallowIndexedEntry scenario: the same IndexInfo is read multiple times, per every read row
[
https://issues.apache.org/jira/browse/CASSANDRA-19557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-19557:
-
Fix Version/s: 5.0-beta2
5.0
5.1
(was: 5.x)
(was: 5.0.x)
Source Control Link:
https://github.com/apache/cassandra/commit/049160e70a22bfcc9c751a1fe3afec1e5c329f4c
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Committed, thanks for the patch and review!
> ShallowIndexedEntry scenario: the same IndexInfo is read multiple times, per
> every read row
> ---
>
> Key: CASSANDRA-19557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19557
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Dmitry Konstantinov
>Assignee: Dmitry Konstantinov
>Priority: Normal
> Fix For: 5.0-beta2, 5.0, 5.1
>
> Attachments: 19557-4.1.patch, 19557-5.0.patch
>
>
> When we read rows from a large partition stored in an SSTable and
> ShallowIndexedEntry is used - the same IndexInfo entity is read from disk
> multiple times, it happens per every read row.
> The following stacktrace shows the execution path:
> {code:java}
> at
> org.apache.cassandra.db.RowIndexEntry$ShallowInfoRetriever.fetchIndex(RowIndexEntry.java:742)
> at
> org.apache.cassandra.db.RowIndexEntry$FileIndexInfoRetriever.columnsIndex(RowIndexEntry.java:792)
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$IndexState.index(AbstractSSTableIterator.java:528)
>
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$IndexState.currentIndex(AbstractSSTableIterator.java:523)
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$IndexState.isPastCurrentBlock(AbstractSSTableIterator.java:513)
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$IndexState.updateBlock(AbstractSSTableIterator.java:487)
> <=== here we retrieve the current index entry
> at
> org.apache.cassandra.db.columniterator.SSTableIterator$ForwardIndexedReader.computeNext(SSTableIterator.java:290)
> <== here we iterate over rows
> at
> org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.hasNextInternal(SSTableIterator.java:182)
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$Reader.hasNext(AbstractSSTableIterator.java:342)
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator.hasNext(AbstractSSTableIterator.java:224)
> at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133)
> at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:100)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:110)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:48)
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133)
> at
> org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:74)
> at
> org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:76)
> at
> org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:27)
> at
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:97)
> at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:303)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:191)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:181)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:177)
> at
> org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:48)
> at org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:308)
> at
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1991)
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2277)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:165)
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:137)
> at
[jira] [Updated] (CASSANDRA-19557) ShallowIndexedEntry scenario: the same IndexInfo is read multiple times, per every read row
[
https://issues.apache.org/jira/browse/CASSANDRA-19557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-19557:
-
Status: Ready to Commit (was: Review In Progress)
> ShallowIndexedEntry scenario: the same IndexInfo is read multiple times, per
> every read row
> ---
>
> Key: CASSANDRA-19557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19557
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Dmitry Konstantinov
>Assignee: Dmitry Konstantinov
>Priority: Normal
> Fix For: 5.0.x, 5.x
>
> Attachments: 19557-4.1.patch, 19557-5.0.patch
>
>
> When we read rows from a large partition stored in an SSTable and
> ShallowIndexedEntry is used - the same IndexInfo entity is read from disk
> multiple times, it happens per every read row.
> The following stacktrace shows the execution path:
> {code:java}
> at
> org.apache.cassandra.db.RowIndexEntry$ShallowInfoRetriever.fetchIndex(RowIndexEntry.java:742)
> at
> org.apache.cassandra.db.RowIndexEntry$FileIndexInfoRetriever.columnsIndex(RowIndexEntry.java:792)
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$IndexState.index(AbstractSSTableIterator.java:528)
>
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$IndexState.currentIndex(AbstractSSTableIterator.java:523)
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$IndexState.isPastCurrentBlock(AbstractSSTableIterator.java:513)
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$IndexState.updateBlock(AbstractSSTableIterator.java:487)
> <=== here we retrieve the current index entry
> at
> org.apache.cassandra.db.columniterator.SSTableIterator$ForwardIndexedReader.computeNext(SSTableIterator.java:290)
> <== here we iterate over rows
> at
> org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.hasNextInternal(SSTableIterator.java:182)
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$Reader.hasNext(AbstractSSTableIterator.java:342)
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator.hasNext(AbstractSSTableIterator.java:224)
> at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133)
> at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:100)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:110)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:48)
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133)
> at
> org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:74)
> at
> org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:76)
> at
> org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:27)
> at
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:97)
> at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:303)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:191)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:181)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:177)
> at
> org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:48)
> at org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:308)
> at
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1991)
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2277)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:165)
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:137)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:119)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:829)
> {code}
> This Cassandra logic was originally written for the case when there is a
> small number of index entries and all of them are fetched in memory, so it
> was cheap to retrieve the IndexInfo again and again for
[jira] [Updated] (CASSANDRA-19557) ShallowIndexedEntry scenario: the same IndexInfo is read multiple times, per every read row
[
https://issues.apache.org/jira/browse/CASSANDRA-19557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-19557:
-
Test and Documentation Plan: run CI
Status: Patch Available (was: Open)
> ShallowIndexedEntry scenario: the same IndexInfo is read multiple times, per
> every read row
> ---
>
> Key: CASSANDRA-19557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19557
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Dmitry Konstantinov
>Assignee: Dmitry Konstantinov
>Priority: Normal
> Fix For: 5.0.x, 5.x
>
> Attachments: 19557-4.1.patch, 19557-5.0.patch
>
>
> When we read rows from a large partition stored in an SSTable and
> ShallowIndexedEntry is used - the same IndexInfo entity is read from disk
> multiple times, it happens per every read row.
> The following stacktrace shows the execution path:
> {code:java}
> at
> org.apache.cassandra.db.RowIndexEntry$ShallowInfoRetriever.fetchIndex(RowIndexEntry.java:742)
> at
> org.apache.cassandra.db.RowIndexEntry$FileIndexInfoRetriever.columnsIndex(RowIndexEntry.java:792)
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$IndexState.index(AbstractSSTableIterator.java:528)
>
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$IndexState.currentIndex(AbstractSSTableIterator.java:523)
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$IndexState.isPastCurrentBlock(AbstractSSTableIterator.java:513)
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$IndexState.updateBlock(AbstractSSTableIterator.java:487)
> <=== here we retrieve the current index entry
> at
> org.apache.cassandra.db.columniterator.SSTableIterator$ForwardIndexedReader.computeNext(SSTableIterator.java:290)
> <== here we iterate over rows
> at
> org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.hasNextInternal(SSTableIterator.java:182)
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$Reader.hasNext(AbstractSSTableIterator.java:342)
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator.hasNext(AbstractSSTableIterator.java:224)
> at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133)
> at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:100)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:110)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:48)
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133)
> at
> org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:74)
> at
> org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:76)
> at
> org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:27)
> at
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:97)
> at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:303)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:191)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:181)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:177)
> at
> org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:48)
> at org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:308)
> at
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1991)
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2277)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:165)
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:137)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:119)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:829)
> {code}
> This Cassandra logic was originally written for the case when there is a
> small number of index entries and all of them are fetched in memory, so it
> was cheap
[jira] [Updated] (CASSANDRA-19557) ShallowIndexedEntry scenario: the same IndexInfo is read multiple times, per every read row
[
https://issues.apache.org/jira/browse/CASSANDRA-19557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-19557:
-
Reviewers: Berenguer Blasi, Brandon Williams, Brandon Williams
Berenguer Blasi, Brandon Williams, Brandon Williams (was:
Berenguer Blasi, Brandon Williams)
Status: Review In Progress (was: Patch Available)
> ShallowIndexedEntry scenario: the same IndexInfo is read multiple times, per
> every read row
> ---
>
> Key: CASSANDRA-19557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19557
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Dmitry Konstantinov
>Assignee: Dmitry Konstantinov
>Priority: Normal
> Fix For: 5.0.x, 5.x
>
> Attachments: 19557-4.1.patch, 19557-5.0.patch
>
>
> When we read rows from a large partition stored in an SSTable and
> ShallowIndexedEntry is used - the same IndexInfo entity is read from disk
> multiple times, it happens per every read row.
> The following stacktrace shows the execution path:
> {code:java}
> at
> org.apache.cassandra.db.RowIndexEntry$ShallowInfoRetriever.fetchIndex(RowIndexEntry.java:742)
> at
> org.apache.cassandra.db.RowIndexEntry$FileIndexInfoRetriever.columnsIndex(RowIndexEntry.java:792)
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$IndexState.index(AbstractSSTableIterator.java:528)
>
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$IndexState.currentIndex(AbstractSSTableIterator.java:523)
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$IndexState.isPastCurrentBlock(AbstractSSTableIterator.java:513)
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$IndexState.updateBlock(AbstractSSTableIterator.java:487)
> <=== here we retrieve the current index entry
> at
> org.apache.cassandra.db.columniterator.SSTableIterator$ForwardIndexedReader.computeNext(SSTableIterator.java:290)
> <== here we iterate over rows
> at
> org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.hasNextInternal(SSTableIterator.java:182)
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$Reader.hasNext(AbstractSSTableIterator.java:342)
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator.hasNext(AbstractSSTableIterator.java:224)
> at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133)
> at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:100)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:110)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:48)
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133)
> at
> org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:74)
> at
> org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:76)
> at
> org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:27)
> at
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:97)
> at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:303)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:191)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:181)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:177)
> at
> org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:48)
> at org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:308)
> at
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1991)
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2277)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:165)
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:137)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:119)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:829)
> {code}
> This Cassandra logic was
(cassandra) branch cassandra-5.0 updated: Avoid reading of the same IndexInfo from disk many times for a large partition when ShallowIndexedEntry is used
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch cassandra-5.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-5.0 by this push:
new 049160e70a Avoid reading of the same IndexInfo from disk many times
for a large partition when ShallowIndexedEntry is used
049160e70a is described below
commit 049160e70a22bfcc9c751a1fe3afec1e5c329f4c
Author: Brandon Williams
AuthorDate: Wed Apr 24 16:11:50 2024 -0700
Avoid reading of the same IndexInfo from disk many times for a large
partition when ShallowIndexedEntry is used
Patch by Dmitry Konstantinov; reviewed by bereng and brandonwilliams for
CASSANDRA-19557
---
CHANGES.txt | 1 +
.../apache/cassandra/io/sstable/format/big/IndexState.java | 13 -
2 files changed, 13 insertions(+), 1 deletion(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 45500b642e..8af3d01ac9 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
5.0-beta2
+ * Avoid reading of the same IndexInfo from disk many times for a large
partition (CASSANDRA-19557)
* Resolve the oldest hints just from descriptors and current writer if
available (CASSANDRA-19600)
* Optionally fail writes when SAI refuses to index a term value exceeding
configured term max size (CASSANDRA-19493)
* Vector search can restrict on clustering keys when filtering isn't required
(CASSANDRA-19544)
diff --git
a/src/java/org/apache/cassandra/io/sstable/format/big/IndexState.java
b/src/java/org/apache/cassandra/io/sstable/format/big/IndexState.java
index f738f8ab7c..754b34b065 100644
--- a/src/java/org/apache/cassandra/io/sstable/format/big/IndexState.java
+++ b/src/java/org/apache/cassandra/io/sstable/format/big/IndexState.java
@@ -41,6 +41,9 @@ public class IndexState implements AutoCloseable
private int currentIndexIdx;
+private int cachedIndexIdx = Integer.MIN_VALUE;
+private IndexInfo cachedIndexInfo;
+
// Marks the beginning of the block corresponding to currentIndexIdx.
private DataPosition mark;
@@ -139,7 +142,15 @@ public class IndexState implements AutoCloseable
public IndexInfo index(int i) throws IOException
{
-return indexInfoRetriever.columnsIndex(i);
+// during an iteration we retrieve the same IndexInfo many times
sequentially, for each row
+// caching of the last retreived IndexInfo can save a lot of IO in
case of ShallowIndexedEntry
+if (i == cachedIndexIdx)
+{
+return cachedIndexInfo;
+}
+cachedIndexInfo = indexInfoRetriever.columnsIndex(i);
+cachedIndexIdx = i;
+return cachedIndexInfo;
}
// Finds the index of the first block containing the provided bound,
starting at the provided index.
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) 01/01: Merge branch 'cassandra-5.0' into trunk
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 68a42b23c47556059dca749aec8872cf6b82269e Merge: 505f06a4ae 049160e70a Author: Brandon Williams AuthorDate: Mon May 6 10:28:33 2024 -0500 Merge branch 'cassandra-5.0' into trunk CHANGES.txt | 1 + .../apache/cassandra/io/sstable/format/big/IndexState.java | 13 - 2 files changed, 13 insertions(+), 1 deletion(-) diff --cc CHANGES.txt index ddb6a57103,8af3d01ac9..6be56fcf3e --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,44 -1,5 +1,45 @@@ -5.0-beta2 +5.1 + * Raise priority of TCM internode messages during critical operations (CASSANDRA-19517) + * Add nodetool command to unregister LEFT nodes (CASSANDRA-19581) + * Add cluster metadata id to gossip syn messages (CASSANDRA-19613) + * Reduce heap usage occupied by the metrics (CASSANDRA-19567) + * Improve handling of transient replicas during range movements (CASSANDRA-19344) + * Enable debounced internode log requests to be cancelled at shutdown (CASSANDRA-19514) + * Correctly set last modified epoch when combining multistep operations into a single step (CASSANDRA-19538) + * Add new TriggersPolicy configuration to allow operators to disable triggers (CASSANDRA-19532) + * Use Transformation.Kind.id in local and distributed log tables (CASSANDRA-19516) + * Remove period field from ClusterMetadata and metadata log tables (CASSANDRA-19482) + * Enrich system_views.pending_hints vtable with hints sizes (CASSANDRA-19486) + * Expose all dropwizard metrics in virtual tables (CASSANDRA-14572) + * Ensured that PropertyFileSnitchTest do not overwrite cassandra-toploogy.properties (CASSANDRA-19502) + * Add option for MutualTlsAuthenticator to restrict the certificate validity period (CASSANDRA-18951) + * Fix StorageService::constructRangeToEndpointMap for non-distributed keyspaces (CASSANDRA-19255) + * Group nodetool cms commands into single command group (CASSANDRA-19393) + * Register the measurements of the bootstrap process as Dropwizard metrics (CASSANDRA-19447) + * Add LIST SUPERUSERS CQL statement (CASSANDRA-19417) + * Modernize CQLSH datetime conversions (CASSANDRA-18879) + * Harry model and in-JVM tests for partition-restricted 2i queries (CASSANDRA-18275) + * Refactor cqlshmain global constants (CASSANDRA-19201) + * Remove native_transport_port_ssl (CASSANDRA-19397) + * Make nodetool reconfigurecms sync by default and add --cancel to be able to cancel ongoing reconfigurations (CASSANDRA-19216) + * Expose auth mode in system_views.clients, nodetool clientstats, metrics (CASSANDRA-19366) + * Remove sealed_periods and last_sealed_period tables (CASSANDRA-19189) + * Improve setup and initialisation of LocalLog/LogSpec (CASSANDRA-19271) + * Refactor structure of caching metrics and expose auth cache metrics via JMX (CASSANDRA-17062) + * Allow CQL client certificate authentication to work without sending an AUTHENTICATE request (CASSANDRA-18857) + * Extend nodetool tpstats and system_views.thread_pools with detailed pool parameters (CASSANDRA-19289) + * Remove dependency on Sigar in favor of OSHI (CASSANDRA-16565) + * Simplify the bind marker and Term logic (CASSANDRA-18813) + * Limit cassandra startup to supported JDKs, allow higher JDKs by setting CASSANDRA_JDK_UNSUPPORTED (CASSANDRA-18688) + * Standardize nodetool tablestats formatting of data units (CASSANDRA-19104) + * Make nodetool tablestats use number of significant digits for time and average values consistently (CASSANDRA-19015) + * Upgrade jackson to 2.15.3 and snakeyaml to 2.1 (CASSANDRA-18875) + * Transactional Cluster Metadata [CEP-21] (CASSANDRA-18330) + * Add ELAPSED command to cqlsh (CASSANDRA-18861) + * Add the ability to disable bulk loading of SSTables (CASSANDRA-18781) + * Clean up obsolete functions and simplify cql_version handling in cqlsh (CASSANDRA-18787) +Merged from 5.0: + * Avoid reading of the same IndexInfo from disk many times for a large partition (CASSANDRA-19557) * Resolve the oldest hints just from descriptors and current writer if available (CASSANDRA-19600) * Optionally fail writes when SAI refuses to index a term value exceeding configured term max size (CASSANDRA-19493) * Vector search can restrict on clustering keys when filtering isn't required (CASSANDRA-19544) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch trunk updated (505f06a4ae -> 68a42b23c4)
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git from 505f06a4ae Merge branch 'cassandra-5.0' into trunk new 049160e70a Avoid reading of the same IndexInfo from disk many times for a large partition when ShallowIndexedEntry is used new 68a42b23c4 Merge branch 'cassandra-5.0' into trunk The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt | 1 + .../apache/cassandra/io/sstable/format/big/IndexState.java | 13 - 2 files changed, 13 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19557) ShallowIndexedEntry scenario: the same IndexInfo is read multiple times, per every read row
[
https://issues.apache.org/jira/browse/CASSANDRA-19557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843772#comment-17843772
]
Brandon Williams commented on CASSANDRA-19557:
--
5.0 under j11 had some timeouts but hit the concurrency limit and they didn't
fail under j17 so I don't think they are concerning. j17 also hit
CASSANDRA-19612.
trunk under j11 hit CASSANDRA-19601, CASSANDRA-19279 and a timeout that didn't
recur. j17 also encountered CASSANDRA-19601, CASSANDRA-18098, and a timeout
under the concurrency limit.
None of this is a concern, and I am +1 here but unfortunately could not
convince myself (as much as I would like to) that this is a bug to fix from
3.11 forward. That said, I will begin committing to 5.0.
> ShallowIndexedEntry scenario: the same IndexInfo is read multiple times, per
> every read row
> ---
>
> Key: CASSANDRA-19557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19557
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Dmitry Konstantinov
>Assignee: Dmitry Konstantinov
>Priority: Normal
> Fix For: 5.0.x, 5.x
>
> Attachments: 19557-4.1.patch, 19557-5.0.patch
>
>
> When we read rows from a large partition stored in an SSTable and
> ShallowIndexedEntry is used - the same IndexInfo entity is read from disk
> multiple times, it happens per every read row.
> The following stacktrace shows the execution path:
> {code:java}
> at
> org.apache.cassandra.db.RowIndexEntry$ShallowInfoRetriever.fetchIndex(RowIndexEntry.java:742)
> at
> org.apache.cassandra.db.RowIndexEntry$FileIndexInfoRetriever.columnsIndex(RowIndexEntry.java:792)
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$IndexState.index(AbstractSSTableIterator.java:528)
>
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$IndexState.currentIndex(AbstractSSTableIterator.java:523)
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$IndexState.isPastCurrentBlock(AbstractSSTableIterator.java:513)
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$IndexState.updateBlock(AbstractSSTableIterator.java:487)
> <=== here we retrieve the current index entry
> at
> org.apache.cassandra.db.columniterator.SSTableIterator$ForwardIndexedReader.computeNext(SSTableIterator.java:290)
> <== here we iterate over rows
> at
> org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.hasNextInternal(SSTableIterator.java:182)
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$Reader.hasNext(AbstractSSTableIterator.java:342)
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator.hasNext(AbstractSSTableIterator.java:224)
> at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133)
> at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:100)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:110)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:48)
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133)
> at
> org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:74)
> at
> org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:76)
> at
> org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:27)
> at
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:97)
> at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:303)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:191)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:181)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:177)
> at
> org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:48)
> at org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:308)
> at
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1991)
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2277)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:165)
> at
>
[jira] [Comment Edited] (CASSANDRA-19619) Enforce contract for internal metrics naming
[
https://issues.apache.org/jira/browse/CASSANDRA-19619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843767#comment-17843767
]
Maxim Muzafarov edited comment on CASSANDRA-19619 at 5/6/24 2:57 PM:
-
[~e.dimitrova] yes, we've agreed with [~maedhroz] to include these changes into
the 5.0. The changes should be pretty straightforward, but verification is
needed. I'm trying to run everything locally so far as I've exhausted my OS
credits on CircleCi.
was (Author: mmuzaf):
[~e.dimitrova] yes, we've agreed with [~maedhroz] to include these changes into
the 5.0. The changes should be pretty straightforward, but verification is
needed. I'm trying to run everything locally so far as I'm exhausted my OS
credits on CircleCi.
> Enforce contract for internal metrics naming
>
>
> Key: CASSANDRA-19619
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19619
> Project: Cassandra
> Issue Type: Task
> Components: Observability/Metrics
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Normal
> Fix For: 5.0-rc, 5.0.x, 5.x
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Metrics have their internal representation which uniquely identifies a
> particular metric name. The name is formed as {{. type>..}}. For some metrics, the scope includes a
> metric name that is not necessary for uniqueness, this can be simplified.
> For example,
> {code}
> // AS IS
> org.apache.cassandra.metrics.StorageAttachedIndex.BalancedTreeIntersectionEarlyExits.my_keyspace.my_table.ColumnQueryMetrics.BalancedTreeIntersectionEarlyExits
> // TO BE
> org.apache.cassandra.metrics.StorageAttachedIndex.BalancedTreeIntersectionEarlyExits.my_keyspace.my_table.ColumnQueryMetrics
> {code}
> The metrics are filtered based on knowledge of how they are formatted, so
> having a metric scope without a built-in metric name also simplifies the way
> we filter metrics.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19619) Enforce contract for internal metrics naming
[
https://issues.apache.org/jira/browse/CASSANDRA-19619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843767#comment-17843767
]
Maxim Muzafarov commented on CASSANDRA-19619:
-
[~e.dimitrova] yes, we've agreed with [~maedhroz] to include these changes into
the 5.0. The changes should be pretty straightforward, but verification is
needed. I'm trying to run everything locally so far as I'm exhausted my OS
credits on CircleCi.
> Enforce contract for internal metrics naming
>
>
> Key: CASSANDRA-19619
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19619
> Project: Cassandra
> Issue Type: Task
> Components: Observability/Metrics
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Normal
> Fix For: 5.0-rc, 5.0.x, 5.x
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Metrics have their internal representation which uniquely identifies a
> particular metric name. The name is formed as {{. type>..}}. For some metrics, the scope includes a
> metric name that is not necessary for uniqueness, this can be simplified.
> For example,
> {code}
> // AS IS
> org.apache.cassandra.metrics.StorageAttachedIndex.BalancedTreeIntersectionEarlyExits.my_keyspace.my_table.ColumnQueryMetrics.BalancedTreeIntersectionEarlyExits
> // TO BE
> org.apache.cassandra.metrics.StorageAttachedIndex.BalancedTreeIntersectionEarlyExits.my_keyspace.my_table.ColumnQueryMetrics
> {code}
> The metrics are filtered based on knowledge of how they are formatted, so
> having a metric scope without a built-in metric name also simplifies the way
> we filter metrics.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19565) SIGSEGV on Cassandra v4.1.4
[
https://issues.apache.org/jira/browse/CASSANDRA-19565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-19565:
-
Reviewers: Berenguer Blasi (was: Berenguer Blasi, Brandon Williams)
> SIGSEGV on Cassandra v4.1.4
> ---
>
> Key: CASSANDRA-19565
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19565
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Thomas De Keulenaer
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0.14, 4.1.6, 5.0-beta2, 5.0, 5.1
>
> Attachments: cassandra_57_debian_jdk11_amd64_attempt1.log.xz,
> cassandra_57_redhat_jdk11_amd64_attempt1.log.xz, hs_err_pid1116450.log
>
>
> Hello,
> Since upgrading to v4.1. we cannat run CAssandra any more. Each start
> immediately crashes:
> {{Apr 17 08:58:34 SVALD108 cassandra[1116450]: # A fatal error has been
> detected by the Java Runtime Environment:
> Apr 17 08:58:34 SVALD108 cassandra[1116450]: # SIGSEGV (0xb) at
> pc=0x7fccaab4d152, pid=1116450, tid=1116451}}
> I have added the log from the coe dump.
> This issue is perhaps related to
> https://davecturner.github.io/2021/08/30/seven-year-old-segfault.html ?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19565) SIGSEGV on Cassandra v4.1.4
[
https://issues.apache.org/jira/browse/CASSANDRA-19565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-19565:
-
Fix Version/s: 4.0.14
4.1.6
5.0-beta2
5.0
5.1
(was: 5.x)
(was: 4.0.x)
(was: 4.1.x)
(was: 5.0.x)
Since Version: NA
Source Control Link:
https://github.com/apache/cassandra/commit/f12f928dee2291d0ca98a20c788d545e6ffb623b
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Committed, thanks for the review!
> SIGSEGV on Cassandra v4.1.4
> ---
>
> Key: CASSANDRA-19565
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19565
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Thomas De Keulenaer
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0.14, 4.1.6, 5.0-beta2, 5.0, 5.1
>
> Attachments: cassandra_57_debian_jdk11_amd64_attempt1.log.xz,
> cassandra_57_redhat_jdk11_amd64_attempt1.log.xz, hs_err_pid1116450.log
>
>
> Hello,
> Since upgrading to v4.1. we cannat run CAssandra any more. Each start
> immediately crashes:
> {{Apr 17 08:58:34 SVALD108 cassandra[1116450]: # A fatal error has been
> detected by the Java Runtime Environment:
> Apr 17 08:58:34 SVALD108 cassandra[1116450]: # SIGSEGV (0xb) at
> pc=0x7fccaab4d152, pid=1116450, tid=1116451}}
> I have added the log from the coe dump.
> This issue is perhaps related to
> https://davecturner.github.io/2021/08/30/seven-year-old-segfault.html ?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19565) SIGSEGV on Cassandra v4.1.4
[
https://issues.apache.org/jira/browse/CASSANDRA-19565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-19565:
-
Status: Ready to Commit (was: Review In Progress)
> SIGSEGV on Cassandra v4.1.4
> ---
>
> Key: CASSANDRA-19565
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19565
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Thomas De Keulenaer
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Attachments: cassandra_57_debian_jdk11_amd64_attempt1.log.xz,
> cassandra_57_redhat_jdk11_amd64_attempt1.log.xz, hs_err_pid1116450.log
>
>
> Hello,
> Since upgrading to v4.1. we cannat run CAssandra any more. Each start
> immediately crashes:
> {{Apr 17 08:58:34 SVALD108 cassandra[1116450]: # A fatal error has been
> detected by the Java Runtime Environment:
> Apr 17 08:58:34 SVALD108 cassandra[1116450]: # SIGSEGV (0xb) at
> pc=0x7fccaab4d152, pid=1116450, tid=1116451}}
> I have added the log from the coe dump.
> This issue is perhaps related to
> https://davecturner.github.io/2021/08/30/seven-year-old-segfault.html ?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19565) SIGSEGV on Cassandra v4.1.4
[
https://issues.apache.org/jira/browse/CASSANDRA-19565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Brandon Williams updated CASSANDRA-19565:
-
Reviewers: Berenguer Blasi, Brandon Williams (was: Berenguer Blasi)
Status: Review In Progress (was: Patch Available)
> SIGSEGV on Cassandra v4.1.4
> ---
>
> Key: CASSANDRA-19565
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19565
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Thomas De Keulenaer
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Attachments: cassandra_57_debian_jdk11_amd64_attempt1.log.xz,
> cassandra_57_redhat_jdk11_amd64_attempt1.log.xz, hs_err_pid1116450.log
>
>
> Hello,
> Since upgrading to v4.1. we cannat run CAssandra any more. Each start
> immediately crashes:
> {{Apr 17 08:58:34 SVALD108 cassandra[1116450]: # A fatal error has been
> detected by the Java Runtime Environment:
> Apr 17 08:58:34 SVALD108 cassandra[1116450]: # SIGSEGV (0xb) at
> pc=0x7fccaab4d152, pid=1116450, tid=1116451}}
> I have added the log from the coe dump.
> This issue is perhaps related to
> https://davecturner.github.io/2021/08/30/seven-year-old-segfault.html ?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch cassandra-5.0 updated (326bf4b3f5 -> 1902195d99)
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a change to branch cassandra-5.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 326bf4b3f5 Resolve the oldest hints just from descriptors and current writer if available new f12f928dee Bring dirs/ownership/perms in line with Debian package new 103e9cca1a Merge branch 'cassandra-4.0' into cassandra-4.1 new 1902195d99 Merge branch 'cassandra-4.1' into cassandra-5.0 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt | 1 + redhat/cassandra.spec | 11 --- 2 files changed, 5 insertions(+), 7 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) 01/01: Merge branch 'cassandra-4.1' into cassandra-5.0
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch cassandra-5.0 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 1902195d651ba6405665a79da92c10bb881f Merge: 326bf4b3f5 103e9cca1a Author: Brandon Williams AuthorDate: Mon May 6 08:37:01 2024 -0500 Merge branch 'cassandra-4.1' into cassandra-5.0 CHANGES.txt | 1 + redhat/cassandra.spec | 11 --- 2 files changed, 5 insertions(+), 7 deletions(-) diff --cc CHANGES.txt index 5a53faf257,0095e2d078..45500b642e --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -39,9 -9,7 +39,10 @@@ Merged from 4.1 * Fix hints delivery for a node going down repeatedly (CASSANDRA-19495) * Do not go to disk for reading hints file sizes (CASSANDRA-19477) * Fix system_views.settings to handle array types (CASSANDRA-19475) + * Memoize Cassandra verion and add a backoff interval for failed schema pulls (CASSANDRA-18902) + * Fix StackOverflowError on ALTER after many previous schema changes (CASSANDRA-19166) Merged from 4.0: ++ * Bring Redhat packge dirs/ownership/perms in line with Debian package (CASSANDRA-19565) * Make nodetool import congruent with the documentation by not relying on the folder structure of the imported SSTable files (CASSANDRA-19401) * IR may leak SSTables with pending repair when coming from streaming (CASSANDRA-19182) * Streaming exception race creates corrupt transaction log files that prevent restart (CASSANDRA-18736) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) 01/01: Merge branch 'cassandra-4.0' into cassandra-4.1
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch cassandra-4.1 in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 103e9cca1abe66905188d5ac0137f785b22b Merge: 6b13426562 f12f928dee Author: Brandon Williams AuthorDate: Mon May 6 08:35:07 2024 -0500 Merge branch 'cassandra-4.0' into cassandra-4.1 CHANGES.txt | 5 + redhat/cassandra.spec | 7 ++- 2 files changed, 7 insertions(+), 5 deletions(-) diff --cc CHANGES.txt index 4c7d59183e,a506c9e9e0..0095e2d078 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -1,10 -1,8 +1,15 @@@ -4.0.14 ++4.1.6 ++Merged from 4.0: + * Bring Redhat packge dirs/ownership/perms in line with Debian package (CASSANDRA-19565) + + -4.0.13 +4.1.5 + * Make queries visible to the "system_views.queries" virtual table at the coordinator level (CASSANDRA-19577) + * Concurrent equivalent schema updates lead to unresolved disagreement (CASSANDRA-19578) + * Fix hints delivery for a node going down repeatedly (CASSANDRA-19495) + * Do not go to disk for reading hints file sizes (CASSANDRA-19477) + * Fix system_views.settings to handle array types (CASSANDRA-19475) +Merged from 4.0: * Make nodetool import congruent with the documentation by not relying on the folder structure of the imported SSTable files (CASSANDRA-19401) * IR may leak SSTables with pending repair when coming from streaming (CASSANDRA-19182) * Streaming exception race creates corrupt transaction log files that prevent restart (CASSANDRA-18736) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch trunk updated (a9dbb89419 -> 505f06a4ae)
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a change to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git from a9dbb89419 remove unused import new f12f928dee Bring dirs/ownership/perms in line with Debian package new 103e9cca1a Merge branch 'cassandra-4.0' into cassandra-4.1 new 1902195d99 Merge branch 'cassandra-4.1' into cassandra-5.0 new 505f06a4ae Merge branch 'cassandra-5.0' into trunk The 4 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt | 1 + redhat/cassandra.spec | 11 --- 2 files changed, 5 insertions(+), 7 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch cassandra-4.1 updated (6b13426562 -> 103e9cca1a)
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a change to branch cassandra-4.1 in repository https://gitbox.apache.org/repos/asf/cassandra.git from 6b13426562 Prepare debian changelog for 4.1.5 new f12f928dee Bring dirs/ownership/perms in line with Debian package new 103e9cca1a Merge branch 'cassandra-4.0' into cassandra-4.1 The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: CHANGES.txt | 5 + redhat/cassandra.spec | 7 ++- 2 files changed, 7 insertions(+), 5 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) 01/01: Merge branch 'cassandra-5.0' into trunk
This is an automated email from the ASF dual-hosted git repository. brandonwilliams pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git commit 505f06a4ae5a8c80e2fad75a78eda95cfae742ce Merge: a9dbb89419 1902195d99 Author: Brandon Williams AuthorDate: Mon May 6 08:37:11 2024 -0500 Merge branch 'cassandra-5.0' into trunk CHANGES.txt | 1 + redhat/cassandra.spec | 11 --- 2 files changed, 5 insertions(+), 7 deletions(-) diff --cc CHANGES.txt index fb1ae2f8bd,45500b642e..ddb6a57103 --- a/CHANGES.txt +++ b/CHANGES.txt @@@ -78,9 -38,11 +78,10 @@@ Merged from 4.1 * Fix hints delivery for a node going down repeatedly (CASSANDRA-19495) * Do not go to disk for reading hints file sizes (CASSANDRA-19477) * Fix system_views.settings to handle array types (CASSANDRA-19475) - * Memoize Cassandra verion and add a backoff interval for failed schema pulls (CASSANDRA-18902) * Fix StackOverflowError on ALTER after many previous schema changes (CASSANDRA-19166) + * Memoize Cassandra verion (CASSANDRA-18902) Merged from 4.0: + * Bring Redhat packge dirs/ownership/perms in line with Debian package (CASSANDRA-19565) * Make nodetool import congruent with the documentation by not relying on the folder structure of the imported SSTable files (CASSANDRA-19401) * IR may leak SSTables with pending repair when coming from streaming (CASSANDRA-19182) * Streaming exception race creates corrupt transaction log files that prevent restart (CASSANDRA-18736) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch cassandra-4.0 updated: Bring dirs/ownership/perms in line with Debian package
This is an automated email from the ASF dual-hosted git repository.
brandonwilliams pushed a commit to branch cassandra-4.0
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/cassandra-4.0 by this push:
new f12f928dee Bring dirs/ownership/perms in line with Debian package
f12f928dee is described below
commit f12f928dee2291d0ca98a20c788d545e6ffb623b
Author: Brandon Williams
AuthorDate: Mon Apr 29 10:16:51 2024 -0500
Bring dirs/ownership/perms in line with Debian package
Patch by brandonwilliams; reviewed by bereng for CASSANDRA-19565
---
CHANGES.txt | 4
redhat/cassandra.spec | 7 ++-
2 files changed, 6 insertions(+), 5 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 452751cfc4..a506c9e9e0 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,3 +1,7 @@
+4.0.14
+ * Bring Redhat packge dirs/ownership/perms in line with Debian package
(CASSANDRA-19565)
+
+
4.0.13
* Make nodetool import congruent with the documentation by not relying on the
folder structure of the imported SSTable files (CASSANDRA-19401)
* IR may leak SSTables with pending repair when coming from streaming
(CASSANDRA-19182)
diff --git a/redhat/cassandra.spec b/redhat/cassandra.spec
index 13b6cf42b6..775aa2a208 100644
--- a/redhat/cassandra.spec
+++ b/redhat/cassandra.spec
@@ -82,10 +82,7 @@ mkdir -p %{buildroot}/%{_sysconfdir}/security/limits.d
mkdir -p %{buildroot}/%{_sysconfdir}/default
mkdir -p %{buildroot}/usr/sbin
mkdir -p %{buildroot}/usr/bin
-mkdir -p %{buildroot}/var/lib/%{username}/commitlog
-mkdir -p %{buildroot}/var/lib/%{username}/data
-mkdir -p %{buildroot}/var/lib/%{username}/saved_caches
-mkdir -p %{buildroot}/var/lib/%{username}/hints
+mkdir -p %{buildroot}/var/lib/%{username}
mkdir -p %{buildroot}/var/run/%{username}
mkdir -p %{buildroot}/var/log/%{username}
( cd pylib && %{__python} setup.py install --no-compile --root %{buildroot}; )
@@ -161,7 +158,7 @@ exit 0
%{_sysconfdir}/security/limits.d/%{username}.conf
/usr/share/%{username}*
%config(noreplace) /%{_sysconfdir}/%{username}
-%attr(755,%{username},%{username}) %config(noreplace) /var/lib/%{username}/*
+%attr(755,%{username},%{username}) %config(noreplace) /var/lib/%{username}
%attr(755,%{username},%{username}) /var/log/%{username}*
%attr(755,%{username},%{username}) /var/run/%{username}*
%{python_sitelib}/cqlshlib/
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19556) Guardrail to block DDL/DCL queries
[ https://issues.apache.org/jira/browse/CASSANDRA-19556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843726#comment-17843726 ] Stefan Miklosovic commented on CASSANDRA-19556: --- https://lists.apache.org/thread/zsw12lnsoy5h3334s8hrwsh8lfz5pfh4 > Guardrail to block DDL/DCL queries > -- > > Key: CASSANDRA-19556 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19556 > Project: Cassandra > Issue Type: New Feature > Components: Feature/Guardrails >Reporter: Yuqi Yan >Assignee: Yuqi Yan >Priority: Normal > Fix For: 5.x > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Sometimes we want to block DDL/DCL queries to stop new schemas being created > or roles created. (e.g. when doing live-upgrade) > For DDL guardrail current implementation won't block the query if it's no-op > (e.g. CREATE TABLE...IF NOT EXISTS, but table already exists, etc. The > guardrail check is added in apply() right after all the existence check) > I don't have preference on either block every DDL query or check whether if > it's no-op here. Just we have some users always run CREATE..IF NOT EXISTS.. > at startup, which is no-op but will be blocked by this guardrail and failed > to start. > > 4.1 PR: [https://github.com/apache/cassandra/pull/3248] > trunk PR: [https://github.com/apache/cassandra/pull/3275] > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19619) Enforce contract for internal metrics naming
[
https://issues.apache.org/jira/browse/CASSANDRA-19619?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843725#comment-17843725
]
Ekaterina Dimitrova commented on CASSANDRA-19619:
-
[~mmuzaf], this ticket is marked 5.0-rc blocker? Was this a mistake or is it a
blocker?
> Enforce contract for internal metrics naming
>
>
> Key: CASSANDRA-19619
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19619
> Project: Cassandra
> Issue Type: Task
> Components: Observability/Metrics
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Normal
> Fix For: 5.0-rc, 5.0.x, 5.x
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Metrics have their internal representation which uniquely identifies a
> particular metric name. The name is formed as {{. type>..}}. For some metrics, the scope includes a
> metric name that is not necessary for uniqueness, this can be simplified.
> For example,
> {code}
> // AS IS
> org.apache.cassandra.metrics.StorageAttachedIndex.BalancedTreeIntersectionEarlyExits.my_keyspace.my_table.ColumnQueryMetrics.BalancedTreeIntersectionEarlyExits
> // TO BE
> org.apache.cassandra.metrics.StorageAttachedIndex.BalancedTreeIntersectionEarlyExits.my_keyspace.my_table.ColumnQueryMetrics
> {code}
> The metrics are filtered based on knowledge of how they are formatted, so
> having a metric scope without a built-in metric name also simplifies the way
> we filter metrics.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19556) Guardrail to block DDL/DCL queries
[ https://issues.apache.org/jira/browse/CASSANDRA-19556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843709#comment-17843709 ] Josh McKenzie commented on CASSANDRA-19556: --- bq. I think that having it more granular is just overkill Maybe. But there's a lot of users with a lot of different experiences on the project; please hit up the dev list w/a [DISCUSS] thread and see if there's a clear consensus. > Guardrail to block DDL/DCL queries > -- > > Key: CASSANDRA-19556 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19556 > Project: Cassandra > Issue Type: New Feature > Components: Feature/Guardrails >Reporter: Yuqi Yan >Assignee: Yuqi Yan >Priority: Normal > Fix For: 5.x > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Sometimes we want to block DDL/DCL queries to stop new schemas being created > or roles created. (e.g. when doing live-upgrade) > For DDL guardrail current implementation won't block the query if it's no-op > (e.g. CREATE TABLE...IF NOT EXISTS, but table already exists, etc. The > guardrail check is added in apply() right after all the existence check) > I don't have preference on either block every DDL query or check whether if > it's no-op here. Just we have some users always run CREATE..IF NOT EXISTS.. > at startup, which is no-op but will be blocked by this guardrail and failed > to start. > > 4.1 PR: [https://github.com/apache/cassandra/pull/3248] > trunk PR: [https://github.com/apache/cassandra/pull/3275] > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-website) branch asf-staging updated (a6fcb4a3e -> fd21e11a5)
This is an automated email from the ASF dual-hosted git repository. git-site-role pushed a change to branch asf-staging in repository https://gitbox.apache.org/repos/asf/cassandra-website.git discard a6fcb4a3e generate docs for cc1c7113 add 3b8cf04d9 fix: remove the paragraph tag from "ecosystem.adoc" new fd21e11a5 generate docs for 3b8cf04d This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (a6fcb4a3e) \ N -- N -- N refs/heads/asf-staging (fd21e11a5) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: content/_/ecosystem.html | 2 +- content/search-index.js| 2 +- .../source/modules/ROOT/pages/ecosystem.adoc | 2 +- site-ui/build/ui-bundle.zip| Bin 4883646 -> 4883646 bytes 4 files changed, 3 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra-website) branch trunk updated: fix: remove the paragraph tag from "ecosystem.adoc"
This is an automated email from the ASF dual-hosted git repository. mck pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra-website.git The following commit(s) were added to refs/heads/trunk by this push: new 3b8cf04d9 fix: remove the paragraph tag from "ecosystem.adoc" 3b8cf04d9 is described below commit 3b8cf04d9a163fb701b1557984100d31bacaa35d Author: Abhishek Patel <> AuthorDate: Mon Apr 29 13:51:03 2024 +0530 fix: remove the paragraph tag from "ecosystem.adoc" --- site-content/source/modules/ROOT/pages/ecosystem.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/site-content/source/modules/ROOT/pages/ecosystem.adoc b/site-content/source/modules/ROOT/pages/ecosystem.adoc index d130dad31..4bee7f900 100644 --- a/site-content/source/modules/ROOT/pages/ecosystem.adoc +++ b/site-content/source/modules/ROOT/pages/ecosystem.adoc @@ -18,7 +18,7 @@ https://aiven.io/cassandra[Aiven for Apache Cassandra,window=_blank]: Aiven for https://aws.amazon.com/keyspaces/[Amazon Keyspaces (for Apache Cassandra),window=_blank]: Scalable, highly available, and managed Apache Cassandra–compatible database service. -https://docs.microsoft.com/en-us/azure/cosmos-db/cassandra-introduction[Azure Cosmos DB Cassandra API,window=_blank]: Enables you to interact with data stored in https://docs.microsoft.com/en-us/azure/cosmos-db/introduction[Azure Cosmos DB,window=_blank] using the Cassandra Query Language (CQL) , Cassandra-based tools (like cqlsh) and Cassandra client drivers that you're already familiar with. +https://docs.microsoft.com/en-us/azure/cosmos-db/cassandra-introduction[Azure Cosmos DB Cassandra API,window=_blank]: Enables you to interact with data stored in https://docs.microsoft.com/en-us/azure/cosmos-db/introduction[Azure Cosmos DB,window=_blank] using the Cassandra Query Language (CQL) , Cassandra-based tools (like cqlsh) and Cassandra client drivers that you're already familiar with. https://azure.microsoft.com/en-us/services/managed-instance-apache-cassandra/[Azure Managed Instance for Apache Cassandra,window=_blank]: Azure Managed Instance for Apache Cassandra is a service offering moderate management, elasticity, and instance-based pricing for Cassandra data. Go beyond traditional lift and shift by expanding your Cassandra workloads to the cloud and keep control over what matters to you. - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch trunk updated: remove unused import
This is an automated email from the ASF dual-hosted git repository. marcuse pushed a commit to branch trunk in repository https://gitbox.apache.org/repos/asf/cassandra.git The following commit(s) were added to refs/heads/trunk by this push: new a9dbb89419 remove unused import a9dbb89419 is described below commit a9dbb8941973bcdb6ad72047ae92a42480987d73 Author: Marcus Eriksson AuthorDate: Mon May 6 09:30:22 2024 +0200 remove unused import --- src/java/org/apache/cassandra/service/paxos/cleanup/PaxosCleanup.java | 1 - 1 file changed, 1 deletion(-) diff --git a/src/java/org/apache/cassandra/service/paxos/cleanup/PaxosCleanup.java b/src/java/org/apache/cassandra/service/paxos/cleanup/PaxosCleanup.java index 331adef64f..23be99f000 100644 --- a/src/java/org/apache/cassandra/service/paxos/cleanup/PaxosCleanup.java +++ b/src/java/org/apache/cassandra/service/paxos/cleanup/PaxosCleanup.java @@ -33,7 +33,6 @@ import org.apache.cassandra.dht.Token; import org.apache.cassandra.gms.EndpointState; import org.apache.cassandra.locator.InetAddressAndPort; import org.apache.cassandra.repair.SharedContext; -import org.apache.cassandra.schema.Keyspaces; import org.apache.cassandra.schema.Schema; import org.apache.cassandra.schema.TableId; import org.apache.cassandra.schema.TableMetadata; - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19517) Raise priority of TCM internode messages during critical operations
[
https://issues.apache.org/jira/browse/CASSANDRA-19517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-19517:
Fix Version/s: 5.1-alpha1
Source Control Link:
https://github.com/apache/cassandra/commit/67139d5c334e58fdc8d9f09f9288155448666cd3
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
> Raise priority of TCM internode messages during critical operations
> ---
>
> Key: CASSANDRA-19517
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19517
> Project: Cassandra
> Issue Type: Improvement
> Components: Transactional Cluster Metadata
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>Priority: Normal
> Fix For: 5.1-alpha1
>
> Attachments: ci_summary-1.html, ci_summary.html, result_details.tar.gz
>
>
> In a busy cluster, TCM messages may not get propagated throughout the
> cluster, since they will be ordered together with other P1 messages (for
> {{TCM_}} prefixed verbs), and with P2 with all Paxos operations.
> To avoid this, and make sure we can continue cluster metadata changes, all
> {{TCM_}}-prefixed verbs should have {{P0}} priority, just like Gossip
> messages used to. All Paxos messages that involve distributed metadata
> keyspace should now get an {{URGENT}} flag, which will instruct internode
> messaging to schedule them on the {{URGENT_MESSAGES}} connection.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19517) Raise priority of TCM internode messages during critical operations
[
https://issues.apache.org/jira/browse/CASSANDRA-19517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-19517:
Reviewers: Marcus Eriksson, Sam Tunnicliffe (was: Marcus Eriksson)
Status: Review In Progress (was: Patch Available)
> Raise priority of TCM internode messages during critical operations
> ---
>
> Key: CASSANDRA-19517
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19517
> Project: Cassandra
> Issue Type: Improvement
> Components: Transactional Cluster Metadata
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>Priority: Normal
> Attachments: ci_summary-1.html, ci_summary.html, result_details.tar.gz
>
>
> In a busy cluster, TCM messages may not get propagated throughout the
> cluster, since they will be ordered together with other P1 messages (for
> {{TCM_}} prefixed verbs), and with P2 with all Paxos operations.
> To avoid this, and make sure we can continue cluster metadata changes, all
> {{TCM_}}-prefixed verbs should have {{P0}} priority, just like Gossip
> messages used to. All Paxos messages that involve distributed metadata
> keyspace should now get an {{URGENT}} flag, which will instruct internode
> messaging to schedule them on the {{URGENT_MESSAGES}} connection.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19517) Raise priority of TCM internode messages during critical operations
[
https://issues.apache.org/jira/browse/CASSANDRA-19517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcus Eriksson updated CASSANDRA-19517:
Status: Ready to Commit (was: Review In Progress)
> Raise priority of TCM internode messages during critical operations
> ---
>
> Key: CASSANDRA-19517
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19517
> Project: Cassandra
> Issue Type: Improvement
> Components: Transactional Cluster Metadata
>Reporter: Alex Petrov
>Assignee: Alex Petrov
>Priority: Normal
> Attachments: ci_summary-1.html, ci_summary.html, result_details.tar.gz
>
>
> In a busy cluster, TCM messages may not get propagated throughout the
> cluster, since they will be ordered together with other P1 messages (for
> {{TCM_}} prefixed verbs), and with P2 with all Paxos operations.
> To avoid this, and make sure we can continue cluster metadata changes, all
> {{TCM_}}-prefixed verbs should have {{P0}} priority, just like Gossip
> messages used to. All Paxos messages that involve distributed metadata
> keyspace should now get an {{URGENT}} flag, which will instruct internode
> messaging to schedule them on the {{URGENT_MESSAGES}} connection.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch trunk updated: Raise priority of TCM internode messages during critical operations
This is an automated email from the ASF dual-hosted git repository.
marcuse pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new 67139d5c33 Raise priority of TCM internode messages during critical
operations
67139d5c33 is described below
commit 67139d5c334e58fdc8d9f09f9288155448666cd3
Author: Alex Petrov
AuthorDate: Tue Mar 26 11:52:17 2024 +0100
Raise priority of TCM internode messages during critical operations
Patch by Alex Petrov; reviewed by Sam Tunnicliffe and marcuse for
CASSANDRA-19517.
---
CHANGES.txt| 1 +
src/java/org/apache/cassandra/net/Message.java | 21 +-
src/java/org/apache/cassandra/net/MessageFlag.java | 4 +-
.../org/apache/cassandra/net/MessagingService.java | 8 +++
.../apache/cassandra/net/OutboundConnections.java | 7 +-
src/java/org/apache/cassandra/net/Verb.java| 30 -
.../org/apache/cassandra/schema/Keyspaces.java | 9 +++
.../org/apache/cassandra/service/paxos/Paxos.java | 5 ++
.../cassandra/service/paxos/PaxosCommit.java | 8 ++-
.../service/paxos/PaxosCommitAndPrepare.java | 4 +-
.../cassandra/service/paxos/PaxosPrepare.java | 2 +-
.../service/paxos/PaxosPrepareRefresh.java | 2 +-
.../cassandra/service/paxos/PaxosPropose.java | 2 +-
.../cassandra/service/paxos/PaxosRepair.java | 3 +-
.../service/paxos/cleanup/PaxosCleanup.java| 11 ++--
.../paxos/cleanup/PaxosCleanupComplete.java| 8 ++-
.../service/paxos/cleanup/PaxosCleanupRequest.java | 12 ++--
.../service/paxos/cleanup/PaxosCleanupSession.java | 6 +-
.../paxos/cleanup/PaxosFinishPrepareCleanup.java | 4 +-
.../paxos/cleanup/PaxosStartPrepareCleanup.java| 4 +-
.../cassandra/distributed/impl/Instance.java | 13
.../test/ring/CMSUrgentMessagesTest.java | 74 ++
22 files changed, 191 insertions(+), 47 deletions(-)
diff --git a/CHANGES.txt b/CHANGES.txt
index 12787c4fb3..fb1ae2f8bd 100644
--- a/CHANGES.txt
+++ b/CHANGES.txt
@@ -1,4 +1,5 @@
5.1
+ * Raise priority of TCM internode messages during critical operations
(CASSANDRA-19517)
* Add nodetool command to unregister LEFT nodes (CASSANDRA-19581)
* Add cluster metadata id to gossip syn messages (CASSANDRA-19613)
* Reduce heap usage occupied by the metrics (CASSANDRA-19567)
diff --git a/src/java/org/apache/cassandra/net/Message.java
b/src/java/org/apache/cassandra/net/Message.java
index b0a0b9c48d..7d7799a186 100644
--- a/src/java/org/apache/cassandra/net/Message.java
+++ b/src/java/org/apache/cassandra/net/Message.java
@@ -217,6 +217,15 @@ public class Message
return outWithParam(nextId(), verb, expiresAtNanos, payload, 0, null,
null);
}
+public static Message out(Verb verb, T payload, boolean isUrgent)
+{
+assert !verb.isResponse();
+if (isUrgent)
+return outWithFlag(verb, payload, MessageFlag.URGENT);
+else
+return out(verb, payload);
+}
+
public static Message outWithFlag(Verb verb, T payload, MessageFlag
flag)
{
assert !verb.isResponse();
@@ -305,7 +314,10 @@ public class Message
/** Builds a response Message with provided payload, and all the right
fields inferred from request Message */
public Message responseWith(T payload)
{
-return outWithParam(id(), verb().responseVerb, expiresAtNanos(),
payload, null, null);
+Message msg = outWithParam(id(), verb().responseVerb,
expiresAtNanos(), payload, null, null);
+if (header.hasFlag(MessageFlag.URGENT))
+msg = msg.withFlag(MessageFlag.URGENT);
+return msg;
}
/** Builds a response Message with no payload, and all the right fields
inferred from request Message */
@@ -485,6 +497,11 @@ public class Message
this.params = params;
}
+public boolean hasFlag(MessageFlag messageFlag)
+{
+return messageFlag.isIn(flags);
+}
+
Header withFrom(InetAddressAndPort from)
{
return new Header(id, epoch, verb, from, createdAtNanos,
expiresAtNanos, flags, params);
@@ -936,7 +953,7 @@ public class Message
serializeParams(header.params, out, version);
}
-private Header deserializeHeader(DataInputPlus in, InetAddressAndPort
peer, int version) throws IOException
+public Header deserializeHeader(DataInputPlus in, InetAddressAndPort
peer, int version) throws IOException
{
long id = in.readUnsignedVInt();
Epoch epoch = Epoch.EMPTY;
diff --git a/src/java/org/apache/cassandra/net/MessageFlag.java
b/src/java/org/apache/cassandra/net/MessageFlag.java
index 441b06b6a3..1c2db557c3 100644
--- a/src/java/org/apache/cassandra/net/MessageFlag.java
+++
[jira] [Updated] (CASSANDRA-19615) Merge pre-existing schema with the system defined one during upgrade
[ https://issues.apache.org/jira/browse/CASSANDRA-19615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcus Eriksson updated CASSANDRA-19615: Source Control Link: https://github.com/apache/cassandra/commit/4d29a1c75e4c7a5493d5601c7a907b75e042c5f9 Resolution: Fixed Status: Resolved (was: Ready to Commit) and committed with the comment fixed > Merge pre-existing schema with the system defined one during upgrade > > > Key: CASSANDRA-19615 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19615 > Project: Cassandra > Issue Type: Improvement > Components: Transactional Cluster Metadata >Reporter: Marcus Eriksson >Assignee: Marcus Eriksson >Priority: Normal > Fix For: 5.1-alpha1 > > Attachments: ci_summary.html > > Time Spent: 20m > Remaining Estimate: 0h > > When upgrading we should merge the pre-existing schema with the > system-defined schema. For example, if a table was defined in 5.0 in > system_distributed, but then removed from SystemDistributedKeyspace.java in > 5.1 we should still be able to read it (until manually dropped). -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
(cassandra) branch trunk updated: Merge pre-existing schema with the system defined one during upgrade
This is an automated email from the ASF dual-hosted git repository.
marcuse pushed a commit to branch trunk
in repository https://gitbox.apache.org/repos/asf/cassandra.git
The following commit(s) were added to refs/heads/trunk by this push:
new 4d29a1c75e Merge pre-existing schema with the system defined one
during upgrade
4d29a1c75e is described below
commit 4d29a1c75e4c7a5493d5601c7a907b75e042c5f9
Author: Marcus Eriksson
AuthorDate: Thu May 2 09:21:50 2024 +0200
Merge pre-existing schema with the system defined one during upgrade
Patch by marcuse; reviewed by Sam Tunnicliffe for CASSANDRA-19615
---
.../apache/cassandra/schema/DistributedSchema.java | 28 +-
.../tcm/transformations/cms/Initialize.java| 6 -
2 files changed, 27 insertions(+), 7 deletions(-)
diff --git a/src/java/org/apache/cassandra/schema/DistributedSchema.java
b/src/java/org/apache/cassandra/schema/DistributedSchema.java
index a837b0773d..3bac327c17 100644
--- a/src/java/org/apache/cassandra/schema/DistributedSchema.java
+++ b/src/java/org/apache/cassandra/schema/DistributedSchema.java
@@ -114,16 +114,32 @@ public class DistributedSchema implements
MetadataValue
public static DistributedSchema fromSystemTables(Keyspaces keyspaces,
Set knownDatacenters)
{
if
(!keyspaces.containsKeyspace(SchemaConstants.METADATA_KEYSPACE_NAME))
-keyspaces =
keyspaces.withAddedOrReplaced(Keyspaces.of(DistributedMetadataLogKeyspace.initialMetadata(knownDatacenters),
-
TraceKeyspace.metadata(),
-
SystemDistributedKeyspace.metadata(),
-
AuthKeyspace.metadata()));
+{
+Keyspaces kss =
Keyspaces.of(DistributedMetadataLogKeyspace.initialMetadata(knownDatacenters),
+ TraceKeyspace.metadata(),
+ SystemDistributedKeyspace.metadata(),
+ AuthKeyspace.metadata());
+for (KeyspaceMetadata ksm : keyspaces) // on disk keyspaces
+kss = kss.withAddedOrUpdated(kss.get(ksm.name)
+.map(k -> merged(k, ksm))
+.orElse(ksm));
+keyspaces = kss;
+}
return new DistributedSchema(keyspaces, Epoch.UPGRADE_GOSSIP);
}
-public void initializeKeyspaceInstances(DistributedSchema prev)
+/**
+ * merges any tables in `mergeFrom` to `mergeTo` unless they already exist
there.
+ */
+private static KeyspaceMetadata merged(KeyspaceMetadata mergeTo,
KeyspaceMetadata mergeFrom)
{
-initializeKeyspaceInstances(prev, true);
+Tables newTables = mergeTo.tables;
+for (TableMetadata metadata : mergeFrom.tables)
+{
+if (!newTables.containsTable(metadata.id))
+newTables = newTables.with(metadata);
+}
+return mergeTo.withSwapped(newTables);
}
public void initializeKeyspaceInstances(DistributedSchema prev, boolean
loadSSTables)
diff --git
a/src/java/org/apache/cassandra/tcm/transformations/cms/Initialize.java
b/src/java/org/apache/cassandra/tcm/transformations/cms/Initialize.java
index 8103e14f92..11c2ed4d31 100644
--- a/src/java/org/apache/cassandra/tcm/transformations/cms/Initialize.java
+++ b/src/java/org/apache/cassandra/tcm/transformations/cms/Initialize.java
@@ -69,7 +69,11 @@ public class Initialize extends ForceSnapshot
public Result execute(ClusterMetadata prev)
{
ClusterMetadata next = baseState;
-DistributedSchema initialSchema = new
DistributedSchema(setUpDistributedSystemKeyspaces(next));
+// when upgrading the schema is read from disk and set correctly in
baseState, but we still need to
+// make sure that lastModified is bumped, otherwise the keyspace
instances are not created
+DistributedSchema initialSchema = new
DistributedSchema(next.schema.getKeyspaces().get(SystemDistributedKeyspace.NAME).isEmpty()
+?
setUpDistributedSystemKeyspaces(next)
+:
next.schema.getKeyspaces());
ClusterMetadata.Transformer transformer =
next.transformer().with(initialSchema);
return Transformation.success(transformer,
MetaStrategy.affectedRanges(prev));
}
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19557) ShallowIndexedEntry scenario: the same IndexInfo is read multiple times, per every read row
[
https://issues.apache.org/jira/browse/CASSANDRA-19557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843648#comment-17843648
]
Berenguer Blasi commented on CASSANDRA-19557:
-
It's difficult to keep track of trunk failures, but code if pretty simple and
failures LGTM iiuc. +1
> ShallowIndexedEntry scenario: the same IndexInfo is read multiple times, per
> every read row
> ---
>
> Key: CASSANDRA-19557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19557
> Project: Cassandra
> Issue Type: Bug
> Components: Local/SSTable
>Reporter: Dmitry Konstantinov
>Assignee: Dmitry Konstantinov
>Priority: Normal
> Fix For: 5.0.x, 5.x
>
> Attachments: 19557-4.1.patch, 19557-5.0.patch
>
>
> When we read rows from a large partition stored in an SSTable and
> ShallowIndexedEntry is used - the same IndexInfo entity is read from disk
> multiple times, it happens per every read row.
> The following stacktrace shows the execution path:
> {code:java}
> at
> org.apache.cassandra.db.RowIndexEntry$ShallowInfoRetriever.fetchIndex(RowIndexEntry.java:742)
> at
> org.apache.cassandra.db.RowIndexEntry$FileIndexInfoRetriever.columnsIndex(RowIndexEntry.java:792)
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$IndexState.index(AbstractSSTableIterator.java:528)
>
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$IndexState.currentIndex(AbstractSSTableIterator.java:523)
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$IndexState.isPastCurrentBlock(AbstractSSTableIterator.java:513)
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$IndexState.updateBlock(AbstractSSTableIterator.java:487)
> <=== here we retrieve the current index entry
> at
> org.apache.cassandra.db.columniterator.SSTableIterator$ForwardIndexedReader.computeNext(SSTableIterator.java:290)
> <== here we iterate over rows
> at
> org.apache.cassandra.db.columniterator.SSTableIterator$ForwardReader.hasNextInternal(SSTableIterator.java:182)
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$Reader.hasNext(AbstractSSTableIterator.java:342)
> at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator.hasNext(AbstractSSTableIterator.java:224)
> at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133)
> at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:100)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:110)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorWithLowerBound.computeNext(UnfilteredRowIteratorWithLowerBound.java:48)
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> at org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:133)
> at
> org.apache.cassandra.db.transform.UnfilteredRows.isEmpty(UnfilteredRows.java:74)
> at
> org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:76)
> at
> org.apache.cassandra.db.partitions.PurgeFunction.applyToPartition(PurgeFunction.java:27)
> at
> org.apache.cassandra.db.transform.BasePartitions.hasNext(BasePartitions.java:97)
> at
> org.apache.cassandra.db.partitions.UnfilteredPartitionIterators$Serializer.serialize(UnfilteredPartitionIterators.java:303)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.build(ReadResponse.java:191)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:181)
> at
> org.apache.cassandra.db.ReadResponse$LocalDataResponse.(ReadResponse.java:177)
> at
> org.apache.cassandra.db.ReadResponse.createDataResponse(ReadResponse.java:48)
> at org.apache.cassandra.db.ReadCommand.createResponse(ReadCommand.java:308)
> at
> org.apache.cassandra.service.StorageProxy$LocalReadRunnable.runMayThrow(StorageProxy.java:1991)
> at
> org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:2277)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:165)
> at
> org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:137)
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:119)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:829)
> {code}
> This Cassandra logic was originally written for the case when there is a
> small number of index entries and all of them are
[jira] [Commented] (CASSANDRA-19565) SIGSEGV on Cassandra v4.1.4
[
https://issues.apache.org/jira/browse/CASSANDRA-19565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17843647#comment-17843647
]
Berenguer Blasi commented on CASSANDRA-19565:
-
Nice one and everything on devbranch :-) +1
> SIGSEGV on Cassandra v4.1.4
> ---
>
> Key: CASSANDRA-19565
> URL: https://issues.apache.org/jira/browse/CASSANDRA-19565
> Project: Cassandra
> Issue Type: Bug
> Components: Packaging
>Reporter: Thomas De Keulenaer
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.0.x, 4.1.x, 5.0.x, 5.x
>
> Attachments: cassandra_57_debian_jdk11_amd64_attempt1.log.xz,
> cassandra_57_redhat_jdk11_amd64_attempt1.log.xz, hs_err_pid1116450.log
>
>
> Hello,
> Since upgrading to v4.1. we cannat run CAssandra any more. Each start
> immediately crashes:
> {{Apr 17 08:58:34 SVALD108 cassandra[1116450]: # A fatal error has been
> detected by the Java Runtime Environment:
> Apr 17 08:58:34 SVALD108 cassandra[1116450]: # SIGSEGV (0xb) at
> pc=0x7fccaab4d152, pid=1116450, tid=1116451}}
> I have added the log from the coe dump.
> This issue is perhaps related to
> https://davecturner.github.io/2021/08/30/seven-year-old-segfault.html ?
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
