[jira] [Commented] (CASSANDRA-19578) Concurrent equivalent schema updates lead to unresolved disagreement
[ https://issues.apache.org/jira/browse/CASSANDRA-19578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17842181#comment-17842181 ] Jordan West commented on CASSANDRA-19578: - 4.1 is when it was introduced but afaik the issue wasn't fixed so its likely broken in 5.0.x as well -- or any code where TCM isn't merged post 4.1 > Concurrent equivalent schema updates lead to unresolved disagreement > > > Key: CASSANDRA-19578 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19578 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: Chris Lohfink >Priority: Normal > Fix For: 4.1.x, 5.0.x > > > As part of CASSANDRA-17819 a check for empty schema changes was added to the > updateSchema. This only looks at the _logical_ schema difference of the > schemas, but the changes made to the system_schema keyspace are the ones that > actually are involved in the digest. > If two nodes issue the same CREATE statement the difference from the > keyspace.diff would be empty but the timestamps on the mutations would be > different, leading to a pseudo schema disagreement which will never resolve > until resetlocalschema or nodes being bounced. > Only impacts 4.1 > test and fix : > https://github.com/clohfink/cassandra/commit/ba915f839089006ac6d08494ef19dc010bcd6411 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-19578) Concurrent equivalent schema updates lead to unresolved disagreement
[ https://issues.apache.org/jira/browse/CASSANDRA-19578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17839276#comment-17839276 ] Jordan West commented on CASSANDRA-19578: - +1. We should definitely get this into a 4.1.5 quickly as its a pretty significant operational bug. We should get a full test run on this branch going so we can merge. > Concurrent equivalent schema updates lead to unresolved disagreement > > > Key: CASSANDRA-19578 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19578 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: Chris Lohfink >Priority: Normal > Fix For: 4.1.5, 5.0-beta2 > > > As part of CASSANDRA-17819 a check for empty schema changes was added to the > updateSchema. This only looks at the _logical_ schema difference of the > schemas, but the changes made to the system_schema keyspace are the ones that > actually are involved in the digest. > If two nodes issue the same CREATE statement the difference from the > keyspace.diff would be empty but the timestamps on the mutations would be > different, leading to a pseudo schema disagreement which will never resolve > until resetlocalschema or nodes being bounced. > Only impacts 4.1 > test and fix : > https://github.com/clohfink/cassandra/commit/ba915f839089006ac6d08494ef19dc010bcd6411 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19578) Concurrent equivalent schema updates lead to unresolved disagreement
[ https://issues.apache.org/jira/browse/CASSANDRA-19578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-19578: Fix Version/s: 4.1.5 5.0-beta2 > Concurrent equivalent schema updates lead to unresolved disagreement > > > Key: CASSANDRA-19578 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19578 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: Chris Lohfink >Priority: Normal > Fix For: 4.1.5, 5.0-beta2 > > > As part of CASSANDRA-17819 a check for empty schema changes was added to the > updateSchema. This only looks at the _logical_ schema difference of the > schemas, but the changes made to the system_schema keyspace are the ones that > actually are involved in the digest. > If two nodes issue the same CREATE statement the difference from the > keyspace.diff would be empty but the timestamps on the mutations would be > different, leading to a pseudo schema disagreement which will never resolve > until resetlocalschema or nodes being bounced. > Only impacts 4.1 > test and fix : > https://github.com/clohfink/cassandra/commit/ba915f839089006ac6d08494ef19dc010bcd6411 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-19578) Concurrent equivalent schema updates lead to unresolved disagreement
[ https://issues.apache.org/jira/browse/CASSANDRA-19578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-19578: Bug Category: Parent values: Correctness(12982)Level 1 values: Recoverable Corruption / Loss(12986) Complexity: Normal Discovered By: User Report Severity: Critical Status: Open (was: Triage Needed) > Concurrent equivalent schema updates lead to unresolved disagreement > > > Key: CASSANDRA-19578 > URL: https://issues.apache.org/jira/browse/CASSANDRA-19578 > Project: Cassandra > Issue Type: Bug > Components: Cluster/Schema >Reporter: Chris Lohfink >Priority: Normal > > As part of CASSANDRA-17819 a check for empty schema changes was added to the > updateSchema. This only looks at the _logical_ schema difference of the > schemas, but the changes made to the system_schema keyspace are the ones that > actually are involved in the digest. > If two nodes issue the same CREATE statement the difference from the > keyspace.diff would be empty but the timestamps on the mutations would be > different, leading to a pseudo schema disagreement which will never resolve > until resetlocalschema or nodes being bounced. > Only impacts 4.1 > test and fix : > https://github.com/clohfink/cassandra/commit/ba915f839089006ac6d08494ef19dc010bcd6411 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18624) Make Corretto Crypto Provider the Default
[ https://issues.apache.org/jira/browse/CASSANDRA-18624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17748266#comment-17748266 ] Jordan West commented on CASSANDRA-18624: - Thanks [~smiklosovic]. I don't think that JMX is too much of a concern even if it did use the JRE implementation. The issue is performance on the read / write path and openining connections, etc (customer facing, higher throughput events). > Make Corretto Crypto Provider the Default > - > > Key: CASSANDRA-18624 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18624 > Project: Cassandra > Issue Type: Improvement > Components: Dependencies >Reporter: Jordan West >Assignee: Ayushi Singh >Priority: Normal > Fix For: 5.x > > Attachments: image.png > > Time Spent: 28h 20m > Remaining Estimate: 0h > > [Amazon Corretto Crypto Provider| > https://github.com/corretto/amazon-corretto-crypto-provider] is an > alternative provider of TLS and cryptographic functions that has significant > performance benefits for Cassandra. It is Apache 2.0 licensed and has been > deployed in several existing large fleets. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18624) Make Corretto Crypto Provider the Default
[ https://issues.apache.org/jira/browse/CASSANDRA-18624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747207#comment-17747207 ] Jordan West edited comment on CASSANDRA-18624 at 7/26/23 3:04 PM: -- My thoughts: * Shipping it even if not the default is better than not shipping it at all. At least then those who determine its safe can opt-in to the performance benefit * Shipping it on by default would be preferred because ideally we are safer and faster out of the box. However, I wouldn't do this at the cost of breaking upgrades. The few upgrades I would see as acceptable to break are ones where the cluster is highly configured (e.g. the user tuned the algorithms in a non-standard way we don't document or recommend). In those cases I think the user will be responsible for ensuring the upgrade doesn't break (we have to do this internally for example in a few places). My understanding is *if* ACCP doesn't implement something the priority list causes a fallback to the JRE implementation. But it *prefers* ACCP. We've also been running ACCP in production for years without issue. We did notice a performance impact immediately when trying to deploy 4.1 without it. Its evident in the graph shared in the ticket and in flame graphs we took. was (Author: jrwest): My thoughts: * Shipping it even if not the default is better than not shipping it at all. At least then those who determine its safe can opt-in to the performance benefit * Shipping it on by default would be preferred because ideally we are safer and faster out of the box. However, I wouldn't do this at the cost of breaking upgrades. The few upgrades I would see as acceptable to break are ones where the cluster is highly configured (e.g. the user tuned the algorithms in a non-standard way we don't document or recommend). In those cases I think the user will be responsible for ensuring the upgrade doesn't break (we have to do this internally for example in a few places). My understanding is *if* ACCP doesn't implement something the priority list causes a fallback to the JRE implementation. But it *prefers* ACCP. > Make Corretto Crypto Provider the Default > - > > Key: CASSANDRA-18624 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18624 > Project: Cassandra > Issue Type: Improvement > Components: Dependencies >Reporter: Jordan West >Assignee: Ayushi Singh >Priority: Normal > Fix For: 5.x > > Attachments: image.png > > Time Spent: 28h > Remaining Estimate: 0h > > [Amazon Corretto Crypto Provider| > https://github.com/corretto/amazon-corretto-crypto-provider] is an > alternative provider of TLS and cryptographic functions that has significant > performance benefits for Cassandra. It is Apache 2.0 licensed and has been > deployed in several existing large fleets. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18624) Make Corretto Crypto Provider the Default
[ https://issues.apache.org/jira/browse/CASSANDRA-18624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747207#comment-17747207 ] Jordan West edited comment on CASSANDRA-18624 at 7/26/23 2:35 PM: -- My thoughts: * Shipping it even if not the default is better than not shipping it at all. At least then those who determine its safe can opt-in to the performance benefit * Shipping it on by default would be preferred because ideally we are safer and faster out of the box. However, I wouldn't do this at the cost of breaking upgrades. The few upgrades I would see as acceptable to break are ones where the cluster is highly configured (e.g. the user tuned the algorithms in a non-standard way we don't document or recommend). In those cases I think the user will be responsible for ensuring the upgrade doesn't break (we have to do this internally for example in a few places). My understanding is *if* ACCP doesn't implement something the priority list causes a fallback to the JRE implementation. But it *prefers* ACCP. was (Author: jrwest): My thoughts: * Shipping it even if not the default is better than not shipping it at all. At least then those who determine its safe can opt-in to the performance benefit * Shipping it on by default would be preferred because ideally we are safer and faster out of the box. However, I wouldn't do this at the cost of breaking upgrades. The few upgrades I would see as acceptable to break are ones where the cluster is highly configured (e.g. the user tuned the algorithms in a non-standard way we don't document or recommend). In those cases I think the user will be responsible for ensuring the upgrade doesn't break (we have to do this internally for example in a few places). > Make Corretto Crypto Provider the Default > - > > Key: CASSANDRA-18624 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18624 > Project: Cassandra > Issue Type: Improvement > Components: Dependencies >Reporter: Jordan West >Assignee: Ayushi Singh >Priority: Normal > Fix For: 5.x > > Attachments: image.png > > Time Spent: 28h > Remaining Estimate: 0h > > [Amazon Corretto Crypto Provider| > https://github.com/corretto/amazon-corretto-crypto-provider] is an > alternative provider of TLS and cryptographic functions that has significant > performance benefits for Cassandra. It is Apache 2.0 licensed and has been > deployed in several existing large fleets. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18624) Make Corretto Crypto Provider the Default
[ https://issues.apache.org/jira/browse/CASSANDRA-18624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17747207#comment-17747207 ] Jordan West commented on CASSANDRA-18624: - My thoughts: * Shipping it even if not the default is better than not shipping it at all. At least then those who determine its safe can opt-in to the performance benefit * Shipping it on by default would be preferred because ideally we are safer and faster out of the box. However, I wouldn't do this at the cost of breaking upgrades. The few upgrades I would see as acceptable to break are ones where the cluster is highly configured (e.g. the user tuned the algorithms in a non-standard way we don't document or recommend). In those cases I think the user will be responsible for ensuring the upgrade doesn't break (we have to do this internally for example in a few places). > Make Corretto Crypto Provider the Default > - > > Key: CASSANDRA-18624 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18624 > Project: Cassandra > Issue Type: Improvement > Components: Dependencies >Reporter: Jordan West >Assignee: Ayushi Singh >Priority: Normal > Fix For: 5.x > > Attachments: image.png > > Time Spent: 28h > Remaining Estimate: 0h > > [Amazon Corretto Crypto Provider| > https://github.com/corretto/amazon-corretto-crypto-provider] is an > alternative provider of TLS and cryptographic functions that has significant > performance benefits for Cassandra. It is Apache 2.0 licensed and has been > deployed in several existing large fleets. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18624) Make Corretto Crypto Provider the Default
[ https://issues.apache.org/jira/browse/CASSANDRA-18624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17745292#comment-17745292 ] Jordan West commented on CASSANDRA-18624: - My comment is similar to [~jolynch], while it may not support ARM we shouldn't punish x86's users by giving them worse performance by default as long as we don't break things for ARM users. > Make Corretto Crypto Provider the Default > - > > Key: CASSANDRA-18624 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18624 > Project: Cassandra > Issue Type: Improvement > Components: Dependencies >Reporter: Jordan West >Assignee: Ayushi Singh >Priority: Normal > Attachments: image.png > > Time Spent: 9h 50m > Remaining Estimate: 0h > > [Amazon Corretto Crypto Provider| > https://github.com/corretto/amazon-corretto-crypto-provider] is an > alternative provider of TLS and cryptographic functions that has significant > performance benefits for Cassandra. It is Apache 2.0 licensed and has been > deployed in several existing large fleets. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18651) When starting up after a failed repair was detected Cassandra should clean up automatically
Jordan West created CASSANDRA-18651: --- Summary: When starting up after a failed repair was detected Cassandra should clean up automatically Key: CASSANDRA-18651 URL: https://issues.apache.org/jira/browse/CASSANDRA-18651 Project: Cassandra Issue Type: Improvement Reporter: Jordan West When a repair fails and is not restarted using "resume" the operator has to manually clear out the data directory before restarting. Instead Cassandra could detect this case (or does detect this case) and can clean up itself. This removes one error prone step for the operator or control plane. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18624) Make Corretto Crypto Provider the Default
[ https://issues.apache.org/jira/browse/CASSANDRA-18624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17737178#comment-17737178 ] Jordan West commented on CASSANDRA-18624: - Added a graph that show the performance benefit of turning on ACCP in a recent benchmark we performed. The peak p99s were regularly above 200ms. After they are consistently around 100-130ms. > Make Corretto Crypto Provider the Default > - > > Key: CASSANDRA-18624 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18624 > Project: Cassandra > Issue Type: Improvement > Components: Dependencies >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Attachments: image.png > > > [Amazon Corretto Crypto Provider| > https://github.com/corretto/amazon-corretto-crypto-provider] is an > alternative provider of TLS and cryptographic functions that has significant > performance benefits for Cassandra. It is Apache 2.0 licensed and has been > deployed in several existing large fleets. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18624) Make Corretto Crypto Provider the Default
[ https://issues.apache.org/jira/browse/CASSANDRA-18624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-18624: Attachment: image.png > Make Corretto Crypto Provider the Default > - > > Key: CASSANDRA-18624 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18624 > Project: Cassandra > Issue Type: Improvement > Components: Dependencies >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Attachments: image.png > > > [Amazon Corretto Crypto Provider| > https://github.com/corretto/amazon-corretto-crypto-provider] is an > alternative provider of TLS and cryptographic functions that has significant > performance benefits for Cassandra. It is Apache 2.0 licensed and has been > deployed in several existing large fleets. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18624) Make Corretto Crypto Provider the Default
[ https://issues.apache.org/jira/browse/CASSANDRA-18624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-18624: Change Category: Performance Complexity: Low Hanging Fruit Component/s: Dependencies Assignee: Jordan West Status: Open (was: Triage Needed) > Make Corretto Crypto Provider the Default > - > > Key: CASSANDRA-18624 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18624 > Project: Cassandra > Issue Type: Improvement > Components: Dependencies >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > [Amazon Corretto Crypto Provider| > https://github.com/corretto/amazon-corretto-crypto-provider] is an > alternative provider of TLS and cryptographic functions that has significant > performance benefits for Cassandra. It is Apache 2.0 licensed and has been > deployed in several existing large fleets. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18624) Make Corretto Crypto Provider the Default
Jordan West created CASSANDRA-18624: --- Summary: Make Corretto Crypto Provider the Default Key: CASSANDRA-18624 URL: https://issues.apache.org/jira/browse/CASSANDRA-18624 Project: Cassandra Issue Type: Improvement Reporter: Jordan West [Amazon Corretto Crypto Provider| https://github.com/corretto/amazon-corretto-crypto-provider] is an alternative provider of TLS and cryptographic functions that has significant performance benefits for Cassandra. It is Apache 2.0 licensed and has been deployed in several existing large fleets. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18352) Add Option to Timebox write timestamps
[ https://issues.apache.org/jira/browse/CASSANDRA-18352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-18352: Fix Version/s: 5.0 Source Control Link: https://github.com/apache/cassandra/commit/2ff1ad4788a1e29b99f81f75b2966b7951ba8250 Resolution: Fixed Status: Resolved (was: Ready to Commit) Commited as https://github.com/apache/cassandra/commit/2ff1ad4788a1e29b99f81f75b2966b7951ba8250 > Add Option to Timebox write timestamps > -- > > Key: CASSANDRA-18352 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18352 > Project: Cassandra > Issue Type: New Feature > Components: CQL/Semantics >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Fix For: 5.0 > > Time Spent: 2.5h > Remaining Estimate: 0h > > In several cases it is desirable to have client provided timestamps generated > at the application-level. This can be error prone, however. In particular, > applications can choose timestamps that may be nonsensical for a given > application. One dangerous manifestation of this is the "doomstone" (a > tombstone far in the future of any realistic write). This feature would allow > either operators or users to specify a minimum and maximum timebound of > "reasonable" timestamps. The default would be negative infinity, positive > infinity to maintain backwards compatibility. Writes that are USING TIMESTAMP > with a timestamp outside of the timebox will see an exception. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18352) Add Option to Timebox write timestamps
[ https://issues.apache.org/jira/browse/CASSANDRA-18352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-18352: Status: Ready to Commit (was: Review In Progress) > Add Option to Timebox write timestamps > -- > > Key: CASSANDRA-18352 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18352 > Project: Cassandra > Issue Type: New Feature > Components: CQL/Semantics >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Time Spent: 2.5h > Remaining Estimate: 0h > > In several cases it is desirable to have client provided timestamps generated > at the application-level. This can be error prone, however. In particular, > applications can choose timestamps that may be nonsensical for a given > application. One dangerous manifestation of this is the "doomstone" (a > tombstone far in the future of any realistic write). This feature would allow > either operators or users to specify a minimum and maximum timebound of > "reasonable" timestamps. The default would be negative infinity, positive > infinity to maintain backwards compatibility. Writes that are USING TIMESTAMP > with a timestamp outside of the timebox will see an exception. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18352) Add Option to Timebox write timestamps
[ https://issues.apache.org/jira/browse/CASSANDRA-18352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-18352: Reviewers: Andres de la Peña, Brandon Williams Status: Review In Progress (was: Patch Available) > Add Option to Timebox write timestamps > -- > > Key: CASSANDRA-18352 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18352 > Project: Cassandra > Issue Type: New Feature > Components: CQL/Semantics >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Time Spent: 2.5h > Remaining Estimate: 0h > > In several cases it is desirable to have client provided timestamps generated > at the application-level. This can be error prone, however. In particular, > applications can choose timestamps that may be nonsensical for a given > application. One dangerous manifestation of this is the "doomstone" (a > tombstone far in the future of any realistic write). This feature would allow > either operators or users to specify a minimum and maximum timebound of > "reasonable" timestamps. The default would be negative infinity, positive > infinity to maintain backwards compatibility. Writes that are USING TIMESTAMP > with a timestamp outside of the timebox will see an exception. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18352) Add Option to Timebox write timestamps
[ https://issues.apache.org/jira/browse/CASSANDRA-18352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17722947#comment-17722947 ] Jordan West commented on CASSANDRA-18352: - [~adelapena] added reason field. if the changes look good to you I will go ahead and merge. > Add Option to Timebox write timestamps > -- > > Key: CASSANDRA-18352 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18352 > Project: Cassandra > Issue Type: New Feature > Components: CQL/Semantics >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Time Spent: 2.5h > Remaining Estimate: 0h > > In several cases it is desirable to have client provided timestamps generated > at the application-level. This can be error prone, however. In particular, > applications can choose timestamps that may be nonsensical for a given > application. One dangerous manifestation of this is the "doomstone" (a > tombstone far in the future of any realistic write). This feature would allow > either operators or users to specify a minimum and maximum timebound of > "reasonable" timestamps. The default would be negative infinity, positive > infinity to maintain backwards compatibility. Writes that are USING TIMESTAMP > with a timestamp outside of the timebox will see an exception. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18352) Add Option to Timebox write timestamps
[
https://issues.apache.org/jira/browse/CASSANDRA-18352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17720092#comment-17720092
]
Jordan West commented on CASSANDRA-18352:
-
[~adelapena] added your two commits and some tests in {{DurationSpecTest}}.
Rebased and added the repeated tests.
[j11|https://app.circleci.com/pipelines/github/jrwest/cassandra/165/workflows/0b0afa99-fe16-4e18-8109-c7822a16a14e]
[j8|https://app.circleci.com/pipelines/github/jrwest/cassandra/165/workflows/591c3fdf-ac62-4777-9312-fe245f1a6936]
> Add Option to Timebox write timestamps
> --
>
> Key: CASSANDRA-18352
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18352
> Project: Cassandra
> Issue Type: New Feature
> Components: CQL/Semantics
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> In several cases it is desirable to have client provided timestamps generated
> at the application-level. This can be error prone, however. In particular,
> applications can choose timestamps that may be nonsensical for a given
> application. One dangerous manifestation of this is the "doomstone" (a
> tombstone far in the future of any realistic write). This feature would allow
> either operators or users to specify a minimum and maximum timebound of
> "reasonable" timestamps. The default would be negative infinity, positive
> infinity to maintain backwards compatibility. Writes that are USING TIMESTAMP
> with a timestamp outside of the timebox will see an exception.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18352) Add Option to Timebox write timestamps
[ https://issues.apache.org/jira/browse/CASSANDRA-18352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711642#comment-17711642 ] Jordan West edited comment on CASSANDRA-18352 at 4/13/23 2:29 AM: -- [~adelapena] thank you for the commits w/ the review. I took both of them. Test run on a rebased trunk and with your patches: [j8|https://app.circleci.com/pipelines/github/jrwest/cassandra/163/workflows/2e9c9758-7fcc-41d7-8874-164bebc5f542] [j11|https://app.circleci.com/pipelines/github/jrwest/cassandra/163/workflows/1c88d752-7c95-4ad7-85f9-602fe4a2d28d] was (Author: jrwest): [~adelapena] thank you for the commits w/ the review. I took both of them. Test run on a rebased trunk and with your patches: [j8|https://app.circleci.com/pipelines/github/jrwest/cassandra/163/workflows/2e9c9758-7fcc-41d7-8874-164bebc5f542] [ j11|https://app.circleci.com/pipelines/github/jrwest/cassandra/163/workflows/1c88d752-7c95-4ad7-85f9-602fe4a2d28d] > Add Option to Timebox write timestamps > -- > > Key: CASSANDRA-18352 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18352 > Project: Cassandra > Issue Type: New Feature > Components: CQL/Semantics >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Time Spent: 2h 20m > Remaining Estimate: 0h > > In several cases it is desirable to have client provided timestamps generated > at the application-level. This can be error prone, however. In particular, > applications can choose timestamps that may be nonsensical for a given > application. One dangerous manifestation of this is the "doomstone" (a > tombstone far in the future of any realistic write). This feature would allow > either operators or users to specify a minimum and maximum timebound of > "reasonable" timestamps. The default would be negative infinity, positive > infinity to maintain backwards compatibility. Writes that are USING TIMESTAMP > with a timestamp outside of the timebox will see an exception. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18352) Add Option to Timebox write timestamps
[ https://issues.apache.org/jira/browse/CASSANDRA-18352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17711642#comment-17711642 ] Jordan West commented on CASSANDRA-18352: - [~adelapena] thank you for the commits w/ the review. I took both of them. Test run on a rebased trunk and with your patches: [j8|https://app.circleci.com/pipelines/github/jrwest/cassandra/163/workflows/2e9c9758-7fcc-41d7-8874-164bebc5f542] [ j11|https://app.circleci.com/pipelines/github/jrwest/cassandra/163/workflows/1c88d752-7c95-4ad7-85f9-602fe4a2d28d] > Add Option to Timebox write timestamps > -- > > Key: CASSANDRA-18352 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18352 > Project: Cassandra > Issue Type: New Feature > Components: CQL/Semantics >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Time Spent: 2h 20m > Remaining Estimate: 0h > > In several cases it is desirable to have client provided timestamps generated > at the application-level. This can be error prone, however. In particular, > applications can choose timestamps that may be nonsensical for a given > application. One dangerous manifestation of this is the "doomstone" (a > tombstone far in the future of any realistic write). This feature would allow > either operators or users to specify a minimum and maximum timebound of > "reasonable" timestamps. The default would be negative infinity, positive > infinity to maintain backwards compatibility. Writes that are USING TIMESTAMP > with a timestamp outside of the timebox will see an exception. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18352) Add Option to Timebox write timestamps
[ https://issues.apache.org/jira/browse/CASSANDRA-18352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17708606#comment-17708606 ] Jordan West commented on CASSANDRA-18352: - Addressed review comments from [~adelapena] (thanks!). New test runs running now and linked below: [j8|https://app.circleci.com/pipelines/github/jrwest/cassandra/161/workflows/75522b14-eab8-4162-8027-131aac728420] [j11 |https://app.circleci.com/pipelines/github/jrwest/cassandra/161/workflows/efbd9663-b753-43e4-b456-26776d72ef7c] > Add Option to Timebox write timestamps > -- > > Key: CASSANDRA-18352 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18352 > Project: Cassandra > Issue Type: New Feature > Components: CQL/Semantics >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Time Spent: 40m > Remaining Estimate: 0h > > In several cases it is desirable to have client provided timestamps generated > at the application-level. This can be error prone, however. In particular, > applications can choose timestamps that may be nonsensical for a given > application. One dangerous manifestation of this is the "doomstone" (a > tombstone far in the future of any realistic write). This feature would allow > either operators or users to specify a minimum and maximum timebound of > "reasonable" timestamps. The default would be negative infinity, positive > infinity to maintain backwards compatibility. Writes that are USING TIMESTAMP > with a timestamp outside of the timebox will see an exception. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18352) Add Option to Timebox write timestamps
[ https://issues.apache.org/jira/browse/CASSANDRA-18352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17708060#comment-17708060 ] Jordan West edited comment on CASSANDRA-18352 at 4/3/23 5:46 PM: - Pushed the renaming requested by [~adelapena] and kicked off new test runs that are running now. [j8|https://app.circleci.com/pipelines/github/jrwest/cassandra/160/workflows/5d438432-7ca7-4819-ba7b-c787caa1e2a7] [j11|https://app.circleci.com/pipelines/github/jrwest/cassandra/160/workflows/3d3e02fc-7ffc-4a7f-adcc-053a1d29f100] EDIT: haven't pushed the YAML docs yet but I don't expect that to impact testing was (Author: jrwest): Pushed the renaming requested by [~adelapena] and kicked off new test runs that are running now. [j8|https://app.circleci.com/pipelines/github/jrwest/cassandra/160/workflows/5d438432-7ca7-4819-ba7b-c787caa1e2a7] [j11|https://app.circleci.com/pipelines/github/jrwest/cassandra/160/workflows/3d3e02fc-7ffc-4a7f-adcc-053a1d29f100] > Add Option to Timebox write timestamps > -- > > Key: CASSANDRA-18352 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18352 > Project: Cassandra > Issue Type: New Feature > Components: CQL/Semantics >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > In several cases it is desirable to have client provided timestamps generated > at the application-level. This can be error prone, however. In particular, > applications can choose timestamps that may be nonsensical for a given > application. One dangerous manifestation of this is the "doomstone" (a > tombstone far in the future of any realistic write). This feature would allow > either operators or users to specify a minimum and maximum timebound of > "reasonable" timestamps. The default would be negative infinity, positive > infinity to maintain backwards compatibility. Writes that are USING TIMESTAMP > with a timestamp outside of the timebox will see an exception. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18352) Add Option to Timebox write timestamps
[ https://issues.apache.org/jira/browse/CASSANDRA-18352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17708060#comment-17708060 ] Jordan West commented on CASSANDRA-18352: - Pushed the renaming requested by [~adelapena] and kicked off new test runs that are running now. [j8|https://app.circleci.com/pipelines/github/jrwest/cassandra/160/workflows/5d438432-7ca7-4819-ba7b-c787caa1e2a7] [j11|https://app.circleci.com/pipelines/github/jrwest/cassandra/160/workflows/3d3e02fc-7ffc-4a7f-adcc-053a1d29f100] > Add Option to Timebox write timestamps > -- > > Key: CASSANDRA-18352 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18352 > Project: Cassandra > Issue Type: New Feature > Components: CQL/Semantics >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > In several cases it is desirable to have client provided timestamps generated > at the application-level. This can be error prone, however. In particular, > applications can choose timestamps that may be nonsensical for a given > application. One dangerous manifestation of this is the "doomstone" (a > tombstone far in the future of any realistic write). This feature would allow > either operators or users to specify a minimum and maximum timebound of > "reasonable" timestamps. The default would be negative infinity, positive > infinity to maintain backwards compatibility. Writes that are USING TIMESTAMP > with a timestamp outside of the timebox will see an exception. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18352) Add Option to Timebox write timestamps
[ https://issues.apache.org/jira/browse/CASSANDRA-18352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17707398#comment-17707398 ] Jordan West commented on CASSANDRA-18352: - I will change to that naming > Add Option to Timebox write timestamps > -- > > Key: CASSANDRA-18352 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18352 > Project: Cassandra > Issue Type: New Feature > Components: CQL/Semantics >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > In several cases it is desirable to have client provided timestamps generated > at the application-level. This can be error prone, however. In particular, > applications can choose timestamps that may be nonsensical for a given > application. One dangerous manifestation of this is the "doomstone" (a > tombstone far in the future of any realistic write). This feature would allow > either operators or users to specify a minimum and maximum timebound of > "reasonable" timestamps. The default would be negative infinity, positive > infinity to maintain backwards compatibility. Writes that are USING TIMESTAMP > with a timestamp outside of the timebox will see an exception. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18352) Add Option to Timebox write timestamps
[ https://issues.apache.org/jira/browse/CASSANDRA-18352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17707398#comment-17707398 ] Jordan West edited comment on CASSANDRA-18352 at 3/31/23 7:08 PM: -- I will change to that naming (and will share another test run) was (Author: jrwest): I will change to that naming > Add Option to Timebox write timestamps > -- > > Key: CASSANDRA-18352 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18352 > Project: Cassandra > Issue Type: New Feature > Components: CQL/Semantics >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > In several cases it is desirable to have client provided timestamps generated > at the application-level. This can be error prone, however. In particular, > applications can choose timestamps that may be nonsensical for a given > application. One dangerous manifestation of this is the "doomstone" (a > tombstone far in the future of any realistic write). This feature would allow > either operators or users to specify a minimum and maximum timebound of > "reasonable" timestamps. The default would be negative infinity, positive > infinity to maintain backwards compatibility. Writes that are USING TIMESTAMP > with a timestamp outside of the timebox will see an exception. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18352) Add Option to Timebox write timestamps
[ https://issues.apache.org/jira/browse/CASSANDRA-18352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17707392#comment-17707392 ] Jordan West commented on CASSANDRA-18352: - Planning to add that before commit, yes. Had only rebased and kicked off a new test run when I had a moment. > Add Option to Timebox write timestamps > -- > > Key: CASSANDRA-18352 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18352 > Project: Cassandra > Issue Type: New Feature > Components: CQL/Semantics >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > In several cases it is desirable to have client provided timestamps generated > at the application-level. This can be error prone, however. In particular, > applications can choose timestamps that may be nonsensical for a given > application. One dangerous manifestation of this is the "doomstone" (a > tombstone far in the future of any realistic write). This feature would allow > either operators or users to specify a minimum and maximum timebound of > "reasonable" timestamps. The default would be negative infinity, positive > infinity to maintain backwards compatibility. Writes that are USING TIMESTAMP > with a timestamp outside of the timebox will see an exception. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18352) Add Option to Timebox write timestamps
[ https://issues.apache.org/jira/browse/CASSANDRA-18352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17707325#comment-17707325 ] Jordan West commented on CASSANDRA-18352: - The newest run is much more green: [j8|https://app.circleci.com/pipelines/github/jrwest/cassandra/158/workflows/c10f7c56-3821-4f27-98e1-0664b1e9c73f] [j11 |https://app.circleci.com/pipelines/github/jrwest/cassandra/158/workflows/cf28afe3-b9f7-4f63-8f07-3c35af8398e2] > Add Option to Timebox write timestamps > -- > > Key: CASSANDRA-18352 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18352 > Project: Cassandra > Issue Type: New Feature > Components: CQL/Semantics >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > In several cases it is desirable to have client provided timestamps generated > at the application-level. This can be error prone, however. In particular, > applications can choose timestamps that may be nonsensical for a given > application. One dangerous manifestation of this is the "doomstone" (a > tombstone far in the future of any realistic write). This feature would allow > either operators or users to specify a minimum and maximum timebound of > "reasonable" timestamps. The default would be negative infinity, positive > infinity to maintain backwards compatibility. Writes that are USING TIMESTAMP > with a timestamp outside of the timebox will see an exception. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18352) Add Option to Timebox write timestamps
[ https://issues.apache.org/jira/browse/CASSANDRA-18352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17706187#comment-17706187 ] Jordan West commented on CASSANDRA-18352: - I’ll rebase and do a new run. I would prefer to see it more green as well. > Add Option to Timebox write timestamps > -- > > Key: CASSANDRA-18352 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18352 > Project: Cassandra > Issue Type: New Feature > Components: CQL/Semantics >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > In several cases it is desirable to have client provided timestamps generated > at the application-level. This can be error prone, however. In particular, > applications can choose timestamps that may be nonsensical for a given > application. One dangerous manifestation of this is the "doomstone" (a > tombstone far in the future of any realistic write). This feature would allow > either operators or users to specify a minimum and maximum timebound of > "reasonable" timestamps. The default would be negative infinity, positive > infinity to maintain backwards compatibility. Writes that are USING TIMESTAMP > with a timestamp outside of the timebox will see an exception. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18352) Add Option to Timebox write timestamps
[ https://issues.apache.org/jira/browse/CASSANDRA-18352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17706186#comment-17706186 ] Jordan West commented on CASSANDRA-18352: - will make sure to add those. I am not necessarily opposed to having a resolution option as long as it defaults to micros. I will say that we commonly see folks accidentally use millisecond resolution (when they had implicitly been using microseconds but need to write some data USING TIMESTAMP or other reason). That is the one of the main scenarios we want to prevent. > Add Option to Timebox write timestamps > -- > > Key: CASSANDRA-18352 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18352 > Project: Cassandra > Issue Type: New Feature > Components: CQL/Semantics >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > In several cases it is desirable to have client provided timestamps generated > at the application-level. This can be error prone, however. In particular, > applications can choose timestamps that may be nonsensical for a given > application. One dangerous manifestation of this is the "doomstone" (a > tombstone far in the future of any realistic write). This feature would allow > either operators or users to specify a minimum and maximum timebound of > "reasonable" timestamps. The default would be negative infinity, positive > infinity to maintain backwards compatibility. Writes that are USING TIMESTAMP > with a timestamp outside of the timebox will see an exception. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18352) Add Option to Timebox write timestamps
[ https://issues.apache.org/jira/browse/CASSANDRA-18352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-18352: Test and Documentation Plan: Added 2 new tests Status: Patch Available (was: Open) PR: [https://github.com/apache/cassandra/pull/2246] Test: [j8|https://app.circleci.com/pipelines/github/jrwest/cassandra/156/workflows/aa68c379-040a-4a6e-ad8f-5f71aaeecf34] [j11|https://app.circleci.com/pipelines/github/jrwest/cassandra/156/workflows/06e61722-30c7-4ef9-b31b-ce2a11550ac0] (I checked the test failures also match trunk but can re-run if they get fixed) > Add Option to Timebox write timestamps > -- > > Key: CASSANDRA-18352 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18352 > Project: Cassandra > Issue Type: New Feature > Components: CQL/Semantics >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > In several cases it is desirable to have client provided timestamps generated > at the application-level. This can be error prone, however. In particular, > applications can choose timestamps that may be nonsensical for a given > application. One dangerous manifestation of this is the "doomstone" (a > tombstone far in the future of any realistic write). This feature would allow > either operators or users to specify a minimum and maximum timebound of > "reasonable" timestamps. The default would be negative infinity, positive > infinity to maintain backwards compatibility. Writes that are USING TIMESTAMP > with a timestamp outside of the timebox will see an exception. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18352) Add Option to Timebox write timestamps
[ https://issues.apache.org/jira/browse/CASSANDRA-18352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-18352: Change Category: Semantic Complexity: Normal Component/s: CQL/Semantics Status: Open (was: Triage Needed) > Add Option to Timebox write timestamps > -- > > Key: CASSANDRA-18352 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18352 > Project: Cassandra > Issue Type: New Feature > Components: CQL/Semantics >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > In several cases it is desirable to have client provided timestamps generated > at the application-level. This can be error prone, however. In particular, > applications can choose timestamps that may be nonsensical for a given > application. One dangerous manifestation of this is the "doomstone" (a > tombstone far in the future of any realistic write). This feature would allow > either operators or users to specify a minimum and maximum timebound of > "reasonable" timestamps. The default would be negative infinity, positive > infinity to maintain backwards compatibility. Writes that are USING TIMESTAMP > with a timestamp outside of the timebox will see an exception. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18352) Add Option to Timebox write timestamps
[
https://issues.apache.org/jira/browse/CASSANDRA-18352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17703067#comment-17703067
]
Jordan West commented on CASSANDRA-18352:
-
would likely implement this as a new guardrail that is checked after
{{Guardrails.userTimestampsEnabled}} in {{ModificationStatement#validate}}
> Add Option to Timebox write timestamps
> --
>
> Key: CASSANDRA-18352
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18352
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
>
> In several cases it is desirable to have client provided timestamps generated
> at the application-level. This can be error prone, however. In particular,
> applications can choose timestamps that may be nonsensical for a given
> application. One dangerous manifestation of this is the "doomstone" (a
> tombstone far in the future of any realistic write). This feature would allow
> either operators or users to specify a minimum and maximum timebound of
> "reasonable" timestamps. The default would be negative infinity, positive
> infinity to maintain backwards compatibility. Writes that are USING TIMESTAMP
> with a timestamp outside of the timebox will see an exception.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18352) Add Option to Timebox write timestamps
Jordan West created CASSANDRA-18352: --- Summary: Add Option to Timebox write timestamps Key: CASSANDRA-18352 URL: https://issues.apache.org/jira/browse/CASSANDRA-18352 Project: Cassandra Issue Type: New Feature Reporter: Jordan West Assignee: Jordan West In several cases it is desirable to have client provided timestamps generated at the application-level. This can be error prone, however. In particular, applications can choose timestamps that may be nonsensical for a given application. One dangerous manifestation of this is the "doomstone" (a tombstone far in the future of any realistic write). This feature would allow either operators or users to specify a minimum and maximum timebound of "reasonable" timestamps. The default would be negative infinity, positive infinity to maintain backwards compatibility. Writes that are USING TIMESTAMP with a timestamp outside of the timebox will see an exception. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18022) Extend profileload to track by additional metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-18022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-18022: Fix Version/s: 4.2 Source Control Link: https://github.com/apache/cassandra/commit/2d323cb56572e867b13b6d102a61aaff8bd66c86 Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed to trunk as 2d323cb56572e867b13b6d102a61aaff8bd66c86 > Extend profileload to track by additional metrics > - > > Key: CASSANDRA-18022 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18022 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Fix For: 4.2 > > > Add option to toppartitions to track by: > * Row Count > * Tombstone Count > * SSTable Count > * Latency -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18022) Extend profileload to track by additional metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-18022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-18022: Status: Ready to Commit (was: Review In Progress) > Extend profileload to track by additional metrics > - > > Key: CASSANDRA-18022 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18022 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > Add option to toppartitions to track by: > * Row Count > * Tombstone Count > * SSTable Count > * Latency -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18022) Extend profileload to track by additional metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-18022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-18022: Reviewers: David Capwell Status: Review In Progress (was: Patch Available) > Extend profileload to track by additional metrics > - > > Key: CASSANDRA-18022 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18022 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > Add option to toppartitions to track by: > * Row Count > * Tombstone Count > * SSTable Count > * Latency -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18022) Extend profileload to track by additional metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-18022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17675786#comment-17675786 ] Jordan West commented on CASSANDRA-18022: - New test runs here: [j11|https://app.circleci.com/pipelines/github/jrwest/cassandra/142/workflows/99bb5ff6-2704-443b-8c26-396fffa2dd2d] [j8|https://app.circleci.com/pipelines/github/jrwest/cassandra/142/workflows/88ad9267-90ed-433d-b4f0-347d2687dab6] Pretty sure those failures were a race condition between the sampler threads and the test thread. While I don't love the solution I used, adding sleeps, it does accurately simulate how profileload works in prod (sleeping between begin/finishSampling calls). > Extend profileload to track by additional metrics > - > > Key: CASSANDRA-18022 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18022 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > Add option to toppartitions to track by: > * Row Count > * Tombstone Count > * SSTable Count > * Latency -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18022) Extend profileload to track by additional metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-18022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17675782#comment-17675782 ] Jordan West commented on CASSANDRA-18022: - hmm wasn't expecting the new tests to fail. will take a closer look. > Extend profileload to track by additional metrics > - > > Key: CASSANDRA-18022 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18022 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > Add option to toppartitions to track by: > * Row Count > * Tombstone Count > * SSTable Count > * Latency -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18022) Extend profileload to track by additional metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-18022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17675776#comment-17675776 ] Jordan West commented on CASSANDRA-18022: - You are correct about the bug. I had conflated onClose with onPartitionClose. Fixed and added a test. New test runs: [j11|https://app.circleci.com/pipelines/github/jrwest/cassandra/140/workflows/b67f4535-e9f5-4874-a3cc-3d3812400e51] [j8|https://app.circleci.com/pipelines/github/jrwest/cassandra/140/workflows/8fa018e1-1237-4f83-94b4-7b2693e26d3e] > Extend profileload to track by additional metrics > - > > Key: CASSANDRA-18022 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18022 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > Add option to toppartitions to track by: > * Row Count > * Tombstone Count > * SSTable Count > * Latency -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18140) getsstables --show-levels JMX serialization error
[
https://issues.apache.org/jira/browse/CASSANDRA-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jordan West updated CASSANDRA-18140:
Fix Version/s: NA
Since Version: 4.2
Source Control Link:
https://github.com/apache/cassandra/commit/e936b2cc1ba7f525c636de5f9fb1780ca70f1762
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Commited as
https://github.com/apache/cassandra/commit/e936b2cc1ba7f525c636de5f9fb1780ca70f1762
> getsstables --show-levels JMX serialization error
> -
>
> Key: CASSANDRA-18140
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18140
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
> Fix For: NA
>
>
> While the interface is compliant and tested by JMXStandardsTest the
> implementation is not actually serializable:
> {{java.io.NotSerializableException:
> com.google.common.collect.AbstractMapBasedMultimap$AsMap}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18140) getsstables --show-levels JMX serialization error
[
https://issues.apache.org/jira/browse/CASSANDRA-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jordan West updated CASSANDRA-18140:
Reviewers: Brandon Williams, Cheng Wang (was: Brandon Williams)
> getsstables --show-levels JMX serialization error
> -
>
> Key: CASSANDRA-18140
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18140
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
>
> While the interface is compliant and tested by JMXStandardsTest the
> implementation is not actually serializable:
> {{java.io.NotSerializableException:
> com.google.common.collect.AbstractMapBasedMultimap$AsMap}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18140) getsstables --show-levels JMX serialization error
[
https://issues.apache.org/jira/browse/CASSANDRA-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jordan West updated CASSANDRA-18140:
Status: Ready to Commit (was: Review In Progress)
> getsstables --show-levels JMX serialization error
> -
>
> Key: CASSANDRA-18140
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18140
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
>
> While the interface is compliant and tested by JMXStandardsTest the
> implementation is not actually serializable:
> {{java.io.NotSerializableException:
> com.google.common.collect.AbstractMapBasedMultimap$AsMap}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18140) getsstables --show-levels JMX serialization error
[
https://issues.apache.org/jira/browse/CASSANDRA-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17656769#comment-17656769
]
Jordan West commented on CASSANDRA-18140:
-
Thanks for the quick review. I could have sworn I did but it mustve been on an
older copy of the code on accident :/.
> getsstables --show-levels JMX serialization error
> -
>
> Key: CASSANDRA-18140
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18140
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
>
> While the interface is compliant and tested by JMXStandardsTest the
> implementation is not actually serializable:
> {{java.io.NotSerializableException:
> com.google.common.collect.AbstractMapBasedMultimap$AsMap}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18140) getsstables --show-levels JMX serialization error
[
https://issues.apache.org/jira/browse/CASSANDRA-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jordan West updated CASSANDRA-18140:
Test and Documentation Plan: manual verification
Status: Patch Available (was: Open)
[https://github.com/jrwest/cassandra/tree/jwest/18140]
Tests:
[j11|https://app.circleci.com/pipelines/github/jrwest/cassandra/136/workflows/43420c29-1030-4629-adca-784492e481b1]
[j8|https://app.circleci.com/pipelines/github/jrwest/cassandra/136/workflows/3e7bc7a1-8761-4cea-a5ea-c4afa372872c]
> getsstables --show-levels JMX serialization error
> -
>
> Key: CASSANDRA-18140
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18140
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
>
> While the interface is compliant and tested by JMXStandardsTest the
> implementation is not actually serializable:
> {{java.io.NotSerializableException:
> com.google.common.collect.AbstractMapBasedMultimap$AsMap}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18140) getsstables --show-levels JMX serialization error
[
https://issues.apache.org/jira/browse/CASSANDRA-18140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jordan West updated CASSANDRA-18140:
Bug Category: Parent values: Code(13163)Level 1 values: Bug - Unclear
Impact(13164)
Complexity: Normal
Component/s: Tool/nodetool
Discovered By: Adhoc Test
Severity: Normal
Status: Open (was: Triage Needed)
> getsstables --show-levels JMX serialization error
> -
>
> Key: CASSANDRA-18140
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18140
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
>
> While the interface is compliant and tested by JMXStandardsTest the
> implementation is not actually serializable:
> {{java.io.NotSerializableException:
> com.google.common.collect.AbstractMapBasedMultimap$AsMap}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18140) getsstables --show-levels JMX serialization error
Jordan West created CASSANDRA-18140:
---
Summary: getsstables --show-levels JMX serialization error
Key: CASSANDRA-18140
URL: https://issues.apache.org/jira/browse/CASSANDRA-18140
Project: Cassandra
Issue Type: Bug
Reporter: Jordan West
Assignee: Jordan West
While the interface is compliant and tested by JMXStandardsTest the
implementation is not actually serializable:
{{java.io.NotSerializableException:
com.google.common.collect.AbstractMapBasedMultimap$AsMap}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18116) Denylist can load perpetually and too frequently if it fails persistently
[
https://issues.apache.org/jira/browse/CASSANDRA-18116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jordan West updated CASSANDRA-18116:
Since Version: 4.1.1
Source Control Link:
https://github.com/apache/cassandra/commit/073f7c36fa20e8d9410f306e57e7c7734ce74d1e
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Second round of tests was all green. Had to do a trunk specific patch because
it didn't merge using {{-s ours}}
> Denylist can load perpetually and too frequently if it fails persistently
> --
>
> Key: CASSANDRA-18116
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18116
> Project: Cassandra
> Issue Type: Bug
> Components: Feature/Rate Limiting
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
> Fix For: 4.1.x
>
>
> When the denylist fails its initial load it can [return a value of
> null|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/schema/PartitionDenylist.java#L453]
> for a key in the cache. The [BoundedLoadingCache
> implementation|https://github.com/ben-manes/caffeine/blob/master/caffeine/src/main/java/com/github/benmanes/caffeine/cache/BoundedLocalCache.java#L2591-L2615]
> that is used treats null as missing and will end up trying to load the key
> on reads. Besides the performance impact this can also lead to significant
> log spam leading to disk or CPU usage issues.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18116) Denylist can load perpetually and too frequently if it fails persistently
[ https://issues.apache.org/jira/browse/CASSANDRA-18116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-18116: Fix Version/s: (was: 4.0.x) > Denylist can load perpetually and too frequently if it fails persistently > -- > > Key: CASSANDRA-18116 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18116 > Project: Cassandra > Issue Type: Bug > Components: Feature/Rate Limiting >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Fix For: 4.1.x > > > When the denylist fails its initial load it can [return a value of > null|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/schema/PartitionDenylist.java#L453] > for a key in the cache. The [BoundedLoadingCache > implementation|https://github.com/ben-manes/caffeine/blob/master/caffeine/src/main/java/com/github/benmanes/caffeine/cache/BoundedLocalCache.java#L2591-L2615] > that is used treats null as missing and will end up trying to load the key > on reads. Besides the performance impact this can also lead to significant > log spam leading to disk or CPU usage issues. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18116) Denylist can load perpetually and too frequently if it fails persistently
[ https://issues.apache.org/jira/browse/CASSANDRA-18116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17655172#comment-17655172 ] Jordan West commented on CASSANDRA-18116: - Changed javadoc to "@return empty denylist if we do not or cannot find the data, preserving the old value, otherwise the new value". Second run of tests here: [j11|https://app.circleci.com/pipelines/github/jrwest/cassandra/129/workflows/d8db359f-3d98-4ba5-84ab-14083e387a6e] [j8|https://app.circleci.com/pipelines/github/jrwest/cassandra/129/workflows/94cf9c40-bd35-4983-a787-15b466fbd6df] > Denylist can load perpetually and too frequently if it fails persistently > -- > > Key: CASSANDRA-18116 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18116 > Project: Cassandra > Issue Type: Bug > Components: Feature/Rate Limiting >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Fix For: 4.0.x, 4.1.x > > > When the denylist fails its initial load it can [return a value of > null|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/schema/PartitionDenylist.java#L453] > for a key in the cache. The [BoundedLoadingCache > implementation|https://github.com/ben-manes/caffeine/blob/master/caffeine/src/main/java/com/github/benmanes/caffeine/cache/BoundedLocalCache.java#L2591-L2615] > that is used treats null as missing and will end up trying to load the key > on reads. Besides the performance impact this can also lead to significant > log spam leading to disk or CPU usage issues. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18116) Denylist can load perpetually and too frequently if it fails persistently
[ https://issues.apache.org/jira/browse/CASSANDRA-18116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-18116: Test and Documentation Plan: Existing tests (exploring if I can add one more to produce this case and show how reads trigger a load of the denylist) Status: Patch Available (was: Open) [https://github.com/jrwest/cassandra/tree/jwest/18116-4.1] Tests: [j8|https://app.circleci.com/pipelines/github/jrwest/cassandra/128/workflows/474f1780-f3e5-4c61-8c26-c0ddf3408017] [j11|https://app.circleci.com/pipelines/github/jrwest/cassandra/128/workflows/2fe470f3-6743-42e6-b4ab-5d8399a9e814] > Denylist can load perpetually and too frequently if it fails persistently > -- > > Key: CASSANDRA-18116 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18116 > Project: Cassandra > Issue Type: Bug > Components: Feature/Rate Limiting >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Fix For: 4.0.x, 4.1.x > > > When the denylist fails its initial load it can [return a value of > null|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/schema/PartitionDenylist.java#L453] > for a key in the cache. The [BoundedLoadingCache > implementation|https://github.com/ben-manes/caffeine/blob/master/caffeine/src/main/java/com/github/benmanes/caffeine/cache/BoundedLocalCache.java#L2591-L2615] > that is used treats null as missing and will end up trying to load the key > on reads. Besides the performance impact this can also lead to significant > log spam leading to disk or CPU usage issues. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18022) Extend profileload to track by additional metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-18022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17653161#comment-17653161 ] Jordan West commented on CASSANDRA-18022: - There were 2 test failures on Java 11 that don't seem related but can take a closer look > Extend profileload to track by additional metrics > - > > Key: CASSANDRA-18022 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18022 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > Add option to toppartitions to track by: > * Row Count > * Tombstone Count > * SSTable Count > * Latency -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18022) Extend profileload to track by additional metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-18022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-18022: Test and Documentation Plan: Existing tests and manual demonstration / reviewer manual test. Existing tests cover scheduled sampling only not what is sampled. Status: Patch Available (was: Open) Patch: [https://github.com/jrwest/cassandra/tree/jwest/18022] Running tests to see what I broke: [j11|https://app.circleci.com/pipelines/github/jrwest/cassandra/127/workflows/2ecef45a-7883-48f5-98e8-1fead328ed29] [j8|https://app.circleci.com/pipelines/github/jrwest/cassandra/127/workflows/92416dfd-13ee-4643-a21c-bb4ca30cd5e8] A few notes: * I didn't add latency since it exists already by query * I opted to keep these metrics by partition key instead of by query like latency but I am not strongly opposed to switching it. My initial implementation pre-4.x was by key and usually I've wanted to debug by key but for range queries I could see a case for by query. At least for tombstone and row count. * The currentKey check in ReadCommand instead of putting the code in SinglePartitionReadCommand could likely be factored out if someone feels strongly about it moving it to SinglePartitionReadCommand. > Extend profileload to track by additional metrics > - > > Key: CASSANDRA-18022 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18022 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > Add option to toppartitions to track by: > * Row Count > * Tombstone Count > * SSTable Count > * Latency -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18022) Extend profileload to track by additional metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-18022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-18022: Summary: Extend profileload to track by additional metrics (was: Extend toppartitions to track by additional metrics) > Extend profileload to track by additional metrics > - > > Key: CASSANDRA-18022 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18022 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > Add option to toppartitions to track by: > * Row Count > * Tombstone Count > * SSTable Count > * Latency -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18116) Denylist can load perpetually and too frequently if it fails persistently
[ https://issues.apache.org/jira/browse/CASSANDRA-18116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-18116: Bug Category: Parent values: Degradation(12984)Level 1 values: Other Exception(12998) Complexity: Low Hanging Fruit Component/s: Feature/Rate Limiting Discovered By: User Report Fix Version/s: 4.0.x 4.1.x Severity: Normal Status: Open (was: Triage Needed) > Denylist can load perpetually and too frequently if it fails persistently > -- > > Key: CASSANDRA-18116 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18116 > Project: Cassandra > Issue Type: Bug > Components: Feature/Rate Limiting >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Fix For: 4.0.x, 4.1.x > > > When the denylist fails its initial load it can [return a value of > null|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/schema/PartitionDenylist.java#L453] > for a key in the cache. The [BoundedLoadingCache > implementation|https://github.com/ben-manes/caffeine/blob/master/caffeine/src/main/java/com/github/benmanes/caffeine/cache/BoundedLocalCache.java#L2591-L2615] > that is used treats null as missing and will end up trying to load the key > on reads. Besides the performance impact this can also lead to significant > log spam leading to disk or CPU usage issues. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18116) Denylist can load perpetually and too frequently if it fails persistently
Jordan West created CASSANDRA-18116: --- Summary: Denylist can load perpetually and too frequently if it fails persistently Key: CASSANDRA-18116 URL: https://issues.apache.org/jira/browse/CASSANDRA-18116 Project: Cassandra Issue Type: Bug Reporter: Jordan West Assignee: Jordan West When the denylist fails its initial load it can [return a value of null|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/schema/PartitionDenylist.java#L453] for a key in the cache. The [BoundedLoadingCache implementation|https://github.com/ben-manes/caffeine/blob/master/caffeine/src/main/java/com/github/benmanes/caffeine/cache/BoundedLocalCache.java#L2591-L2615] that is used treats null as missing and will end up trying to load the key on reads. Besides the performance impact this can also lead to significant log spam leading to disk or CPU usage issues. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18023) Add option to print level with getsstables output
[ https://issues.apache.org/jira/browse/CASSANDRA-18023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-18023: Fix Version/s: 4.2 Source Control Link: https://github.com/apache/cassandra/commit/279f284da5cfe8b4766921d3b1b4d6e299dbe66d Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed in [https://github.com/apache/cassandra/commit/279f284da5cfe8b4766921d3b1b4d6e299dbe66d] Thanks for the review [~brandon.williams] [~superwangcheng] > Add option to print level with getsstables output > - > > Key: CASSANDRA-18023 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18023 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Fix For: 4.2 > > > A common operation is to pipe getsstables output to sstablemetadata to > determine the level. This adds friction to operations. Add an optional flag > to getsstables to print the level with data file path. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18023) Add option to print level with getsstables output
[ https://issues.apache.org/jira/browse/CASSANDRA-18023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-18023: Status: Ready to Commit (was: Review In Progress) > Add option to print level with getsstables output > - > > Key: CASSANDRA-18023 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18023 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > A common operation is to pipe getsstables output to sstablemetadata to > determine the level. This adds friction to operations. Add an optional flag > to getsstables to print the level with data file path. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18023) Add option to print level with getsstables output
[ https://issues.apache.org/jira/browse/CASSANDRA-18023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17641621#comment-17641621 ] Jordan West commented on CASSANDRA-18023: - Tests on the most recent trunk: [j8|https://app.circleci.com/pipelines/github/jrwest/cassandra/122/workflows/b510c3f0-3d15-40a5-b377-dd4671fbc534] [j11|https://app.circleci.com/pipelines/github/jrwest/cassandra/122/workflows/ad1cab37-a8f5-48b7-8c30-099e9a33dad6] Still a few failures that I don't believe are related / due to flaky tests. > Add option to print level with getsstables output > - > > Key: CASSANDRA-18023 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18023 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > A common operation is to pipe getsstables output to sstablemetadata to > determine the level. This adds friction to operations. Add an optional flag > to getsstables to print the level with data file path. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17254) nodetool toppartitions can fail because ByteBuffer.array() returns more bytes than would be considered valid by UTF8Serializer.validate
[
https://issues.apache.org/jira/browse/CASSANDRA-17254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jordan West updated CASSANDRA-17254:
Since Version: 3.0.29
Source Control Link:
https://github.com/apache/cassandra/commit/13d495aa7d5b7a7c121fcc9e105f79107c5c2a1c
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Commited as
https://github.com/apache/cassandra/commit/13d495aa7d5b7a7c121fcc9e105f79107c5c2a1c
> nodetool toppartitions can fail because ByteBuffer.array() returns more bytes
> than would be considered valid by UTF8Serializer.validate
> ---
>
> Key: CASSANDRA-17254
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17254
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
> Fix For: 3.0.x
>
>
> The error below is caused by the use of
> [{{ByteBuffer.array()}}|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java#L1628].
> Doing so not only makes the hex key potentially incorrect but causes invalid
> data to be passed to {{AbstractType.getString}} and ultimately
> {{UTF8Validator.validate}}.
> {code}
> error: String didn't validate.
> -- StackTrace --
> org.apache.cassandra.serializers.MarshalException: String didn't validate.
> at
> org.apache.cassandra.serializers.UTF8Serializer.validate(UTF8Serializer.java:35)
> at
> org.apache.cassandra.db.marshal.AbstractType.getString(AbstractType.java:129)
> at
> org.apache.cassandra.db.ColumnFamilyStore.finishLocalSampling(ColumnFamilyStore.java:1633)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17254) nodetool toppartitions can fail because ByteBuffer.array() returns more bytes than would be considered valid by UTF8Serializer.validate
[
https://issues.apache.org/jira/browse/CASSANDRA-17254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jordan West updated CASSANDRA-17254:
Status: Ready to Commit (was: Review In Progress)
> nodetool toppartitions can fail because ByteBuffer.array() returns more bytes
> than would be considered valid by UTF8Serializer.validate
> ---
>
> Key: CASSANDRA-17254
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17254
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
> Fix For: 3.0.x
>
>
> The error below is caused by the use of
> [{{ByteBuffer.array()}}|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java#L1628].
> Doing so not only makes the hex key potentially incorrect but causes invalid
> data to be passed to {{AbstractType.getString}} and ultimately
> {{UTF8Validator.validate}}.
> {code}
> error: String didn't validate.
> -- StackTrace --
> org.apache.cassandra.serializers.MarshalException: String didn't validate.
> at
> org.apache.cassandra.serializers.UTF8Serializer.validate(UTF8Serializer.java:35)
> at
> org.apache.cassandra.db.marshal.AbstractType.getString(AbstractType.java:129)
> at
> org.apache.cassandra.db.ColumnFamilyStore.finishLocalSampling(ColumnFamilyStore.java:1633)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18023) Add option to print level with getsstables output
[ https://issues.apache.org/jira/browse/CASSANDRA-18023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17640967#comment-17640967 ] Jordan West commented on CASSANDRA-18023: - Pushed a fix. Test runs linked below. The failing tests are related to describe which I don't think would be affected by this patch. New test runs: [j8|https://app.circleci.com/pipelines/github/jrwest/cassandra/115/workflows/603d5ca4-8f6a-49af-9dea-f95759d31f1c] [j11|https://app.circleci.com/pipelines/github/jrwest/cassandra/115/workflows/ba9ed183-1ff0-4630-a5ef-87636f3d6b2b] > Add option to print level with getsstables output > - > > Key: CASSANDRA-18023 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18023 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > A common operation is to pipe getsstables output to sstablemetadata to > determine the level. This adds friction to operations. Add an optional flag > to getsstables to print the level with data file path. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18023) Add option to print level with getsstables output
[ https://issues.apache.org/jira/browse/CASSANDRA-18023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17631979#comment-17631979 ] Jordan West commented on CASSANDRA-18023: - Talked to David and this patch needs to move off of Guava MultiMap to something in the standard library. I'm headed out of town for a week. Will pick this up and address when I return. > Add option to print level with getsstables output > - > > Key: CASSANDRA-18023 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18023 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > A common operation is to pipe getsstables output to sstablemetadata to > determine the level. This adds friction to operations. Add an optional flag > to getsstables to print the level with data file path. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-18023) Add option to print level with getsstables output
[ https://issues.apache.org/jira/browse/CASSANDRA-18023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17631385#comment-17631385 ] Jordan West edited comment on CASSANDRA-18023 at 11/10/22 1:54 AM: --- Its mad about the use of the guava MultiMap. From the comment it looks like I need to find an alternative in java/javax. I saw a lot more red than expected so planning to look at that before moving further. was (Author: jrwest): Its mad about the use of the guava MultiMap. From the comment it looks like I need to find an alternative in java/javax. > Add option to print level with getsstables output > - > > Key: CASSANDRA-18023 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18023 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > A common operation is to pipe getsstables output to sstablemetadata to > determine the level. This adds friction to operations. Add an optional flag > to getsstables to print the level with data file path. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18023) Add option to print level with getsstables output
[ https://issues.apache.org/jira/browse/CASSANDRA-18023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17631385#comment-17631385 ] Jordan West commented on CASSANDRA-18023: - Its mad about the use of the guava MultiMap. From the comment it looks like I need to find an alternative in java/javax. > Add option to print level with getsstables output > - > > Key: CASSANDRA-18023 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18023 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > A common operation is to pipe getsstables output to sstablemetadata to > determine the level. This adds friction to operations. Add an optional flag > to getsstables to print the level with data file path. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18023) Add option to print level with getsstables output
[ https://issues.apache.org/jira/browse/CASSANDRA-18023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-18023: Reviewers: Cheng Wang Status: Review In Progress (was: Patch Available) > Add option to print level with getsstables output > - > > Key: CASSANDRA-18023 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18023 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > A common operation is to pipe getsstables output to sstablemetadata to > determine the level. This adds friction to operations. Add an optional flag > to getsstables to print the level with data file path. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18023) Add option to print level with getsstables output
[
https://issues.apache.org/jira/browse/CASSANDRA-18023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17631380#comment-17631380
]
Jordan West commented on CASSANDRA-18023:
-
Good catch. Updated my branch w/ the recommended change. the {{synchronized}}
was a holdover from my original 3.0 implementation.
> Add option to print level with getsstables output
> -
>
> Key: CASSANDRA-18023
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18023
> Project: Cassandra
> Issue Type: Improvement
> Components: Tool/nodetool
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
>
> A common operation is to pipe getsstables output to sstablemetadata to
> determine the level. This adds friction to operations. Add an optional flag
> to getsstables to print the level with data file path.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17254) nodetool toppartitions can fail because ByteBuffer.array() returns more bytes than would be considered valid by UTF8Serializer.validate
[
https://issues.apache.org/jira/browse/CASSANDRA-17254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jordan West updated CASSANDRA-17254:
Fix Version/s: (was: 3.11.x)
> nodetool toppartitions can fail because ByteBuffer.array() returns more bytes
> than would be considered valid by UTF8Serializer.validate
> ---
>
> Key: CASSANDRA-17254
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17254
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
> Fix For: 3.0.x
>
>
> The error below is caused by the use of
> [{{ByteBuffer.array()}}|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java#L1628].
> Doing so not only makes the hex key potentially incorrect but causes invalid
> data to be passed to {{AbstractType.getString}} and ultimately
> {{UTF8Validator.validate}}.
> {code}
> error: String didn't validate.
> -- StackTrace --
> org.apache.cassandra.serializers.MarshalException: String didn't validate.
> at
> org.apache.cassandra.serializers.UTF8Serializer.validate(UTF8Serializer.java:35)
> at
> org.apache.cassandra.db.marshal.AbstractType.getString(AbstractType.java:129)
> at
> org.apache.cassandra.db.ColumnFamilyStore.finishLocalSampling(ColumnFamilyStore.java:1633)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17254) nodetool toppartitions can fail because ByteBuffer.array() returns more bytes than would be considered valid by UTF8Serializer.validate
[
https://issues.apache.org/jira/browse/CASSANDRA-17254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17631300#comment-17631300
]
Jordan West commented on CASSANDRA-17254:
-
Updated the branch linked above with the 3.11 code and kicked off tests
[here|https://app.circleci.com/pipelines/github/jrwest/cassandra/112/workflows/4c88ab60-cd11-4579-b4ee-d8f3c241d6fc]
> nodetool toppartitions can fail because ByteBuffer.array() returns more bytes
> than would be considered valid by UTF8Serializer.validate
> ---
>
> Key: CASSANDRA-17254
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17254
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
> Fix For: 3.0.x, 3.11.x
>
>
> The error below is caused by the use of
> [{{ByteBuffer.array()}}|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java#L1628].
> Doing so not only makes the hex key potentially incorrect but causes invalid
> data to be passed to {{AbstractType.getString}} and ultimately
> {{UTF8Validator.validate}}.
> {code}
> error: String didn't validate.
> -- StackTrace --
> org.apache.cassandra.serializers.MarshalException: String didn't validate.
> at
> org.apache.cassandra.serializers.UTF8Serializer.validate(UTF8Serializer.java:35)
> at
> org.apache.cassandra.db.marshal.AbstractType.getString(AbstractType.java:129)
> at
> org.apache.cassandra.db.ColumnFamilyStore.finishLocalSampling(ColumnFamilyStore.java:1633)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18023) Add option to print level with getsstables output
[
https://issues.apache.org/jira/browse/CASSANDRA-18023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jordan West updated CASSANDRA-18023:
Test and Documentation Plan: Existing test suite plus sample output
Status: Patch Available (was: Open)
Code: [https://github.com/apache/cassandra/compare/trunk...jrwest:jwest/18023]
Test Runs:
[j8|https://app.circleci.com/pipelines/github/jrwest/cassandra/110/workflows/b1b3c268-ebd7-468a-bd74-876ce176c7be]
[j11|https://app.circleci.com/pipelines/github/jrwest/cassandra/110/workflows/e19e4309-a831-4674-be28-f001b39d50de]
Example output:
{code:java}
$ bin/nodetool getsstables foo bar 4
/home/jordanw/sbx/cass/oss/cassandra/data/data/foo/bar-a58b5910607011ed910347124b0dc061/nb-1-big-Data.db
/home/jordanw/sbx/cass/oss/cassandra/data/data/foo/bar-a58b5910607011ed910347124b0dc061/nb-2-big-Data.db
$ bin/nodetool getsstables foo bar 4 -l
0:
/home/jordanw/sbx/cass/oss/cassandra/data/data/foo/bar-a58b5910607011ed910347124b0dc061/nb-1-big-Data.db
0:
/home/jordanw/sbx/cass/oss/cassandra/data/data/foo/bar-a58b5910607011ed910347124b0dc061/nb-2-big-Data.db
$ bin/nodetool compact foo bar
$ bin/nodetool getsstables foo bar 4 -l
1:
/home/jordanw/sbx/cass/oss/cassandra/data/data/foo/bar-a58b5910607011ed910347124b0dc061/nb-3-big-Data.db
{code}
> Add option to print level with getsstables output
> -
>
> Key: CASSANDRA-18023
> URL: https://issues.apache.org/jira/browse/CASSANDRA-18023
> Project: Cassandra
> Issue Type: Improvement
> Components: Tool/nodetool
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
>
> A common operation is to pipe getsstables output to sstablemetadata to
> determine the level. This adds friction to operations. Add an optional flag
> to getsstables to print the level with data file path.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17254) nodetool toppartitions can fail because ByteBuffer.array() returns more bytes than would be considered valid by UTF8Serializer.validate
[
https://issues.apache.org/jira/browse/CASSANDRA-17254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17630073#comment-17630073
]
Jordan West edited comment on CASSANDRA-17254 at 11/7/22 10:38 PM:
---
If I am reading the 3.11 code right (pasted below jic), it uses {{bytesToHex}}
for the key but just uses the {{ByteBuffer}} directly unlike my patch which
duplicates it for the value. I am fine with the 3.11 approach as well assuming
the comment is correct:
{code:java}
//Not duplicating the buffer for safety because AbstractSerializer and
ByteBufferUtil.bytesToHex
//don't modify position or limit
ByteBuffer key = counter.getItem();
result.put(new CompositeDataSupport(COUNTER_COMPOSITE_TYPE, COUNTER_NAMES, new
Object[] {
ByteBufferUtil.bytesToHex(key), // raw
counter.getCount(), // count
counter.getError(), // error
metadata.getKeyValidator().getString(key) })); // string {code}
was (Author: jrwest):
If I am reading the 3.11 code right (pasted below jic), it uses {{bytesToHex}}
for the key but just uses the {{ByteBuffer}} directly unlike my patch which
duplicates it. I am fine with the 3.11 approach as well assuming the comment is
correct:
{code:java}
//Not duplicating the buffer for safety because AbstractSerializer and
ByteBufferUtil.bytesToHex
//don't modify position or limit
ByteBuffer key = counter.getItem();
result.put(new CompositeDataSupport(COUNTER_COMPOSITE_TYPE, COUNTER_NAMES, new
Object[] {
ByteBufferUtil.bytesToHex(key), // raw
counter.getCount(), // count
counter.getError(), // error
metadata.getKeyValidator().getString(key) })); // string {code}
> nodetool toppartitions can fail because ByteBuffer.array() returns more bytes
> than would be considered valid by UTF8Serializer.validate
> ---
>
> Key: CASSANDRA-17254
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17254
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
> Fix For: 3.0.x, 3.11.x
>
>
> The error below is caused by the use of
> [{{ByteBuffer.array()}}|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java#L1628].
> Doing so not only makes the hex key potentially incorrect but causes invalid
> data to be passed to {{AbstractType.getString}} and ultimately
> {{UTF8Validator.validate}}.
> {code}
> error: String didn't validate.
> -- StackTrace --
> org.apache.cassandra.serializers.MarshalException: String didn't validate.
> at
> org.apache.cassandra.serializers.UTF8Serializer.validate(UTF8Serializer.java:35)
> at
> org.apache.cassandra.db.marshal.AbstractType.getString(AbstractType.java:129)
> at
> org.apache.cassandra.db.ColumnFamilyStore.finishLocalSampling(ColumnFamilyStore.java:1633)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17254) nodetool toppartitions can fail because ByteBuffer.array() returns more bytes than would be considered valid by UTF8Serializer.validate
[
https://issues.apache.org/jira/browse/CASSANDRA-17254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17630073#comment-17630073
]
Jordan West commented on CASSANDRA-17254:
-
If I am reading the 3.11 code right (pasted below jic), it uses {{bytesToHex}}
for the key but just uses the {{ByteBuffer}} directly unlike my patch which
duplicates it. I am fine with the 3.11 approach as well assuming the comment is
correct:
{code:java}
//Not duplicating the buffer for safety because AbstractSerializer and
ByteBufferUtil.bytesToHex
//don't modify position or limit
ByteBuffer key = counter.getItem();
result.put(new CompositeDataSupport(COUNTER_COMPOSITE_TYPE, COUNTER_NAMES, new
Object[] {
ByteBufferUtil.bytesToHex(key), // raw
counter.getCount(), // count
counter.getError(), // error
metadata.getKeyValidator().getString(key) })); // string {code}
> nodetool toppartitions can fail because ByteBuffer.array() returns more bytes
> than would be considered valid by UTF8Serializer.validate
> ---
>
> Key: CASSANDRA-17254
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17254
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
> Fix For: 3.0.x, 3.11.x
>
>
> The error below is caused by the use of
> [{{ByteBuffer.array()}}|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java#L1628].
> Doing so not only makes the hex key potentially incorrect but causes invalid
> data to be passed to {{AbstractType.getString}} and ultimately
> {{UTF8Validator.validate}}.
> {code}
> error: String didn't validate.
> -- StackTrace --
> org.apache.cassandra.serializers.MarshalException: String didn't validate.
> at
> org.apache.cassandra.serializers.UTF8Serializer.validate(UTF8Serializer.java:35)
> at
> org.apache.cassandra.db.marshal.AbstractType.getString(AbstractType.java:129)
> at
> org.apache.cassandra.db.ColumnFamilyStore.finishLocalSampling(ColumnFamilyStore.java:1633)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17711) Create a new node tool supporting force compaction of a partition
[
https://issues.apache.org/jira/browse/CASSANDRA-17711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jordan West updated CASSANDRA-17711:
Fix Version/s: 4.2
Source Control Link:
https://github.com/apache/cassandra/commit/873e024a32d37de08550c8106a8d7fd52bda588b
Resolution: Fixed
Status: Resolved (was: Ready to Commit)
Committed to trunk as 873e024a32d37de08550c8106a8d7fd52bda588b
> Create a new node tool supporting force compaction of a partition
> -
>
> Key: CASSANDRA-17711
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17711
> Project: Cassandra
> Issue Type: Improvement
> Components: Tool/nodetool
>Reporter: Cheng Wang
>Assignee: Cheng Wang
>Priority: Normal
> Fix For: 4.2
>
>
> Need to create new tool called {*}nodetool forcecompact{*}. The syntax of the
> command is
> *nodetool forcecompact keyspace table gc_grace_seconds>.*
> One of the use cases of the tool is to handle the bad partitions which may
> contain too many tombstones. The tool will compact all the sstables
> containing the partition keys. In addition, by ignoring the
> {*}gc_grace_seconds{*}, it will help purge the tombstones faster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17711) Create a new node tool supporting force compaction of a partition
[
https://issues.apache.org/jira/browse/CASSANDRA-17711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jordan West updated CASSANDRA-17711:
Status: Ready to Commit (was: Review In Progress)
Rebased and ready to merge. Final test run:
https://app.circleci.com/pipelines/github/jrwest/cassandra/108/workflows/e5c488ed-3ea2-469a-bcbc-5b2f08e05075
> Create a new node tool supporting force compaction of a partition
> -
>
> Key: CASSANDRA-17711
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17711
> Project: Cassandra
> Issue Type: Improvement
> Components: Tool/nodetool
>Reporter: Cheng Wang
>Assignee: Cheng Wang
>Priority: Normal
>
> Need to create new tool called {*}nodetool forcecompact{*}. The syntax of the
> command is
> *nodetool forcecompact keyspace table gc_grace_seconds>.*
> One of the use cases of the tool is to handle the bad partitions which may
> contain too many tombstones. The tool will compact all the sstables
> containing the partition keys. In addition, by ignoring the
> {*}gc_grace_seconds{*}, it will help purge the tombstones faster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18022) Extend toppartitions to track by additional metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-18022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-18022: Change Category: Operability Complexity: Normal Component/s: Tool/nodetool Status: Open (was: Triage Needed) > Extend toppartitions to track by additional metrics > --- > > Key: CASSANDRA-18022 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18022 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > Add option to toppartitions to track by: > * Row Count > * Tombstone Count > * SSTable Count > * Latency -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18023) Add option to print level with getsstables output
[ https://issues.apache.org/jira/browse/CASSANDRA-18023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-18023: Change Category: Operability Complexity: Normal Component/s: Tool/nodetool Status: Open (was: Triage Needed) > Add option to print level with getsstables output > - > > Key: CASSANDRA-18023 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18023 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > A common operation is to pipe getsstables output to sstablemetadata to > determine the level. This adds friction to operations. Add an optional flag > to getsstables to print the level with data file path. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17254) nodetool toppartitions can fail because ByteBuffer.array() returns more bytes than would be considered valid by UTF8Serializer.validate
[
https://issues.apache.org/jira/browse/CASSANDRA-17254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17629985#comment-17629985
]
Jordan West commented on CASSANDRA-17254:
-
I'm generally +1 on adding tests but it looks like it will be somewhat
cumbersome for this function. Any recommendations on approaches? I was thinking
maybe an in-JVM dtest but the key has to be a certain format to trigger the
issue. On the other hand, the usage of the ByteBuffer API here is definitely
wrong. Thoughts?
> nodetool toppartitions can fail because ByteBuffer.array() returns more bytes
> than would be considered valid by UTF8Serializer.validate
> ---
>
> Key: CASSANDRA-17254
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17254
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
> Fix For: 3.0.x, 3.11.x
>
>
> The error below is caused by the use of
> [{{ByteBuffer.array()}}|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java#L1628].
> Doing so not only makes the hex key potentially incorrect but causes invalid
> data to be passed to {{AbstractType.getString}} and ultimately
> {{UTF8Validator.validate}}.
> {code}
> error: String didn't validate.
> -- StackTrace --
> org.apache.cassandra.serializers.MarshalException: String didn't validate.
> at
> org.apache.cassandra.serializers.UTF8Serializer.validate(UTF8Serializer.java:35)
> at
> org.apache.cassandra.db.marshal.AbstractType.getString(AbstractType.java:129)
> at
> org.apache.cassandra.db.ColumnFamilyStore.finishLocalSampling(ColumnFamilyStore.java:1633)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17254) nodetool toppartitions can fail because ByteBuffer.array() returns more bytes than would be considered valid by UTF8Serializer.validate
[
https://issues.apache.org/jira/browse/CASSANDRA-17254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jordan West updated CASSANDRA-17254:
Test and Documentation Plan: manual / production
Status: Patch Available (was: Open)
> nodetool toppartitions can fail because ByteBuffer.array() returns more bytes
> than would be considered valid by UTF8Serializer.validate
> ---
>
> Key: CASSANDRA-17254
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17254
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
> Fix For: 3.0.x, 3.11.x
>
>
> The error below is caused by the use of
> [{{ByteBuffer.array()}}|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java#L1628].
> Doing so not only makes the hex key potentially incorrect but causes invalid
> data to be passed to {{AbstractType.getString}} and ultimately
> {{UTF8Validator.validate}}.
> {code}
> error: String didn't validate.
> -- StackTrace --
> org.apache.cassandra.serializers.MarshalException: String didn't validate.
> at
> org.apache.cassandra.serializers.UTF8Serializer.validate(UTF8Serializer.java:35)
> at
> org.apache.cassandra.db.marshal.AbstractType.getString(AbstractType.java:129)
> at
> org.apache.cassandra.db.ColumnFamilyStore.finishLocalSampling(ColumnFamilyStore.java:1633)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18023) Add option to print level with getsstables output
Jordan West created CASSANDRA-18023: --- Summary: Add option to print level with getsstables output Key: CASSANDRA-18023 URL: https://issues.apache.org/jira/browse/CASSANDRA-18023 Project: Cassandra Issue Type: Improvement Reporter: Jordan West Assignee: Jordan West A common operation is to pipe getsstables output to sstablemetadata to determine the level. This adds friction to operations. Add an optional flag to getsstables to print the level with data file path. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18022) Extend toppartitions to track by additional metrics
Jordan West created CASSANDRA-18022: --- Summary: Extend toppartitions to track by additional metrics Key: CASSANDRA-18022 URL: https://issues.apache.org/jira/browse/CASSANDRA-18022 Project: Cassandra Issue Type: Improvement Reporter: Jordan West Assignee: Jordan West Add option to toppartitions to track by: * Row Count * Tombstone Count * SSTable Count * Latency -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17928) Test Failure: org.apache.cassandra.db.commitlog.CommitLogInitWithExceptionTest.testCommitLogInitWithException-compression
[
https://issues.apache.org/jira/browse/CASSANDRA-17928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17629967#comment-17629967
]
Jordan West commented on CASSANDRA-17928:
-
[~benedict] unfortunately its been long enough I've lost the context. Giving it
a quick read it seems like the goal is the clean up a resource the test created
(I suppose it could also do this by marking the thread as a daemon instead)
> Test Failure:
> org.apache.cassandra.db.commitlog.CommitLogInitWithExceptionTest.testCommitLogInitWithException-compression
> -
>
> Key: CASSANDRA-17928
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17928
> Project: Cassandra
> Issue Type: Bug
> Components: Test/unit
>Reporter: Josh McKenzie
>Assignee: Brandon Williams
>Priority: Normal
> Fix For: 4.1-rc
>
>
> [Link|https://ci-cassandra.apache.org/job/Cassandra-4.1/169/testReport/org.apache.cassandra.db.commitlog/CommitLogInitWithExceptionTest/testCommitLogInitWithException_compression/]
> Failed 1 times in the last 14 runs. Flakiness: 7%, Stability: 92%
> Stacktrace
> {code:java}
> java.lang.NullPointerException
> at
> org.apache.cassandra.db.commitlog.CommitLogInitWithExceptionTest.testCommitLogInitWithException(CommitLogInitWithExceptionTest.java:93)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
> at java.base/java.lang.Thread.run(Thread.java:829)
> {code}
> {code:java}
> Standard Output
> INFO [main] 2022-09-25 11:43:16,512 Reflections.java:219 - Reflections took
> 1221 ms to scan 8 urls, producing 1756 keys and 6922 values
> INFO [main] 2022-09-25 11:43:17,480 Reflections.java:219 - Reflections took
> 907 ms to scan 8 urls, producing 1756 keys and 6922 values
> INFO [main] 2022-09-25 11:43:17,573 YamlConfigurationLoader.java:104 -
> Configuration location:
> file:home/cassandra/cassandra/build/test/cassandra.compressed.yaml
> DEBUG [main] 2022-09-25 11:43:17,574 YamlConfigurationLoader
> ...[truncated 35568 chars]...
> .apache.cassandra.db.commitlog.CommitLogInitWithExceptionTest$MockCommitLogSegmentMgr.createSegment(CommitLogInitWithExceptionTest.java:106)
> at
> org.apache.cassandra.db.commitlog.AbstractCommitLogSegmentManager$AllocatorRunnable.run(AbstractCommitLogSegmentManager.java:155)
> at
> org.apache.cassandra.concurrent.InfiniteLoopExecutor.loop(InfiniteLoopExecutor.java:121)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17711) Create a new node tool supporting force compaction of a partition
[
https://issues.apache.org/jira/browse/CASSANDRA-17711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17611973#comment-17611973
]
Jordan West commented on CASSANDRA-17711:
-
+1
> Create a new node tool supporting force compaction of a partition
> -
>
> Key: CASSANDRA-17711
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17711
> Project: Cassandra
> Issue Type: Improvement
> Components: Tool/nodetool
>Reporter: Cheng Wang
>Assignee: Cheng Wang
>Priority: Normal
>
> Need to create new tool called {*}nodetool forcecompact{*}. The syntax of the
> command is
> *nodetool forcecompact keyspace table gc_grace_seconds>.*
> One of the use cases of the tool is to handle the bad partitions which may
> contain too many tombstones. The tool will compact all the sstables
> containing the partition keys. In addition, by ignoring the
> {*}gc_grace_seconds{*}, it will help purge the tombstones faster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17711) Create a new node tool supporting force compaction of a partition
[
https://issues.apache.org/jira/browse/CASSANDRA-17711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jordan West updated CASSANDRA-17711:
Status: Review In Progress (was: Patch Available)
Test runs on rebased trunk:
https://app.circleci.com/pipelines/github/jrwest/cassandra/105/workflows/eb474db8-d834-4de0-b121-a1c983e9ec65
> Create a new node tool supporting force compaction of a partition
> -
>
> Key: CASSANDRA-17711
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17711
> Project: Cassandra
> Issue Type: Improvement
> Components: Tool/nodetool
>Reporter: Cheng Wang
>Assignee: Cheng Wang
>Priority: Normal
>
> Need to create new tool called {*}nodetool forcecompact{*}. The syntax of the
> command is
> *nodetool forcecompact keyspace table gc_grace_seconds>.*
> One of the use cases of the tool is to handle the bad partitions which may
> contain too many tombstones. The tool will compact all the sstables
> containing the partition keys. In addition, by ignoring the
> {*}gc_grace_seconds{*}, it will help purge the tombstones faster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17711) Create a new node tool supporting force compaction of a partition
[
https://issues.apache.org/jira/browse/CASSANDRA-17711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jordan West updated CASSANDRA-17711:
Test and Documentation Plan: Added tests
Status: Patch Available (was: Open)
https://github.com/apache/cassandra/pull/1700
> Create a new node tool supporting force compaction of a partition
> -
>
> Key: CASSANDRA-17711
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17711
> Project: Cassandra
> Issue Type: Improvement
> Components: Tool/nodetool
>Reporter: Cheng Wang
>Assignee: Cheng Wang
>Priority: Normal
>
> Need to create new tool called {*}nodetool forcecompact{*}. The syntax of the
> command is
> *nodetool forcecompact keyspace table gc_grace_seconds>.*
> One of the use cases of the tool is to handle the bad partitions which may
> contain too many tombstones. The tool will compact all the sstables
> containing the partition keys. In addition, by ignoring the
> {*}gc_grace_seconds{*}, it will help purge the tombstones faster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15737) Add StorageServiceMBean#setDynamicBadnessThreshold to expose dynamic badness threshold
[ https://issues.apache.org/jira/browse/CASSANDRA-15737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17531463#comment-17531463 ] Jordan West commented on CASSANDRA-15737: - This had fallen off my radar but I can pick it back up if there is interest [~blerer] [~dcapwell] > Add StorageServiceMBean#setDynamicBadnessThreshold to expose dynamic badness > threshold > -- > > Key: CASSANDRA-15737 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15737 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Internode >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > https://issues.apache.org/jira/browse/CASSANDRA-12179 made the > DynamicEndpointSnitch properties dynamic at runtime but didn’t expose a > method to modify badness threshold. This can be useful in operations that > also modify severity manually or if operators want to temporarily disable > dynamic snitch behavior. -- This message was sent by Atlassian Jira (v8.20.7#820007) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17547) Documentation from Partition Denylist Lost in Document Migration + Minor Fixes
[ https://issues.apache.org/jira/browse/CASSANDRA-17547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17522898#comment-17522898 ] Jordan West commented on CASSANDRA-17547: - I may have linked the wrong ticket. I thought that was the parent one for all new doc stuff and I assume this got caught up in the doc move (per the cassandra-dev thread it was expected to a degree) > Documentation from Partition Denylist Lost in Document Migration + Minor Fixes > -- > > Key: CASSANDRA-17547 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17547 > Project: Cassandra > Issue Type: Bug > Components: Documentation/Website >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > The documentation added in > https://issues.apache.org/jira/browse/CASSANDRA-12106 went missing when the > documents were migrated to the new format. We just need to bring the doc > back. Along with this fix there are a couple minor edits to make to the > document itself to correct the examples. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17547) Documentation from Partition Denylist Lost in Document Migration + Minor Fixes
[ https://issues.apache.org/jira/browse/CASSANDRA-17547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-17547: Bug Category: Parent values: Documentation(13562) Complexity: Low Hanging Fruit Component/s: Documentation/Website Discovered By: Code Inspection Severity: Normal Status: Open (was: Triage Needed) > Documentation from Partition Denylist Lost in Document Migration + Minor Fixes > -- > > Key: CASSANDRA-17547 > URL: https://issues.apache.org/jira/browse/CASSANDRA-17547 > Project: Cassandra > Issue Type: Bug > Components: Documentation/Website >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > > The documentation added in > https://issues.apache.org/jira/browse/CASSANDRA-12106 went missing when the > documents were migrated to the new format. We just need to bring the doc > back. Along with this fix there are a couple minor edits to make to the > document itself to correct the examples. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17547) Documentation from Partition Denylist Lost in Document Migration + Minor Fixes
Jordan West created CASSANDRA-17547: --- Summary: Documentation from Partition Denylist Lost in Document Migration + Minor Fixes Key: CASSANDRA-17547 URL: https://issues.apache.org/jira/browse/CASSANDRA-17547 Project: Cassandra Issue Type: Bug Reporter: Jordan West Assignee: Jordan West The documentation added in https://issues.apache.org/jira/browse/CASSANDRA-12106 went missing when the documents were migrated to the new format. We just need to bring the doc back. Along with this fix there are a couple minor edits to make to the document itself to correct the examples. -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17254) nodetool toppartitions can fail because ByteBuffer.array() returns more bytes than would be considered valid by UTF8Serializer.validate
[
https://issues.apache.org/jira/browse/CASSANDRA-17254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17472381#comment-17472381
]
Jordan West commented on CASSANDRA-17254:
-
https://github.com/apache/cassandra/compare/cassandra-3.0...jrwest:jwest/17254-3.0
> nodetool toppartitions can fail because ByteBuffer.array() returns more bytes
> than would be considered valid by UTF8Serializer.validate
> ---
>
> Key: CASSANDRA-17254
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17254
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
> Fix For: 3.0.x, 3.11.x
>
>
> The error below is caused by the use of
> [{{ByteBuffer.array()}}|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java#L1628].
> Doing so not only makes the hex key potentially incorrect but causes invalid
> data to be passed to {{AbstractType.getString}} and ultimately
> {{UTF8Validator.validate}}.
> {code}
> error: String didn't validate.
> -- StackTrace --
> org.apache.cassandra.serializers.MarshalException: String didn't validate.
> at
> org.apache.cassandra.serializers.UTF8Serializer.validate(UTF8Serializer.java:35)
> at
> org.apache.cassandra.db.marshal.AbstractType.getString(AbstractType.java:129)
> at
> org.apache.cassandra.db.ColumnFamilyStore.finishLocalSampling(ColumnFamilyStore.java:1633)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-17251) USING writetime + ttl is non-idempotent leading to non-deterministic merge iteration results
[
https://issues.apache.org/jira/browse/CASSANDRA-17251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17472374#comment-17472374
]
Jordan West edited comment on CASSANDRA-17251 at 1/11/22, 12:43 AM:
https://github.com/apache/cassandra/compare/cassandra-3.0...jrwest:jwest/17251-3.0
was (Author: jrwest):
https://github.com/apache/cassandra/compare/trunk...jrwest:jwest/17251-3.0
> USING writetime + ttl is non-idempotent leading to non-deterministic merge
> iteration results
>
>
> Key: CASSANDRA-17251
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17251
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Other
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> The combination of {{USING writetime = timestamp and ttl = ttl}} can result
> in non-deterministic MergeIterator results causing DigestMismatchExceptions
> and increased latencies. The increased latencies are caused by additional
> round trips due to the digest mismatch as well as read repair rewriting the
> data. The additional writes lead to an increase in the number of sstables the
> key is stored in and must be scanned on read.
> The order of events is:
> 1. for a given partition a write is performed with {{USING timestamp =
> sometime and ttl = ttl1}}.
> 2. Cassandra records this write with timestamp = sometime, ttl = ttl1,
> expires_at = now + ttl1
> 3. after N seconds, for the same partition, another write is performed with
> {{USING timestamp = sometime and ttl = ttl2 where ttl2 = ttl1 - N}}. This
> write only makes it to a subset of replicas* for some reason (e.g. partial
> write, node down, etc).
> 4. Cassandra records this write with timestamp = sometime, ttl = ttl2,
> expires_at = now + ttl2. Its important to note that at this point, expires_at
> in 2 above is equal to expires at here. This is because it is calculated
> relative to the current write time not the provided timestamp and the ttl has
> been adjusted by the time passed. This write also makes it to a subset of
> replicas*.
> 5. A read of the data is performed.
> 5a. The MergeIterator resolves conflicts locally (accross sstables) using
> {{Conflicts.resolveRegular}} or {{Cells.resolveRegular}}. The resolution
> takes into account the write timestamp , the liveness of the cell, the values
> themselves, and how much time is left to live via the expires_at field. In
> this scenario, all of these fields are equal, leading to Cassandra picking
> the sstable "on the right" – this is non-deterministic. The only item that
> differs is the ttl itself.
> 5b. One node returns the non-deterministically chosen value for the row, the
> other two calculate and send a digest to the coordinator. The digest includes
> the relative ttl field which may not match. This results in a
> DigestMismatchException at the coordinator.
> 6. Read repair is triggered
> *NOTE: its not strictly necessary for the write to make it to a subset of
> replicas. sstables can also be ordered in random orders for reasons like
> compaction or repair when returned from the live set which can lead to the
> same behavior. This also affects repair from what we can tell.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17254) nodetool toppartitions can fail because ByteBuffer.array() returns more bytes than would be considered valid by UTF8Serializer.validate
[
https://issues.apache.org/jira/browse/CASSANDRA-17254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jordan West updated CASSANDRA-17254:
Bug Category: Parent values: Degradation(12984)Level 1 values: Other
Exception(12998)
Complexity: Normal
Component/s: Tool/nodetool
Discovered By: User Report
Fix Version/s: 3.0.x
3.11.x
Severity: Normal
Status: Open (was: Triage Needed)
> nodetool toppartitions can fail because ByteBuffer.array() returns more bytes
> than would be considered valid by UTF8Serializer.validate
> ---
>
> Key: CASSANDRA-17254
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17254
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/nodetool
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
> Fix For: 3.0.x, 3.11.x
>
>
> The error below is caused by the use of
> [{{ByteBuffer.array()}}|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java#L1628].
> Doing so not only makes the hex key potentially incorrect but causes invalid
> data to be passed to {{AbstractType.getString}} and ultimately
> {{UTF8Validator.validate}}.
> {code}
> error: String didn't validate.
> -- StackTrace --
> org.apache.cassandra.serializers.MarshalException: String didn't validate.
> at
> org.apache.cassandra.serializers.UTF8Serializer.validate(UTF8Serializer.java:35)
> at
> org.apache.cassandra.db.marshal.AbstractType.getString(AbstractType.java:129)
> at
> org.apache.cassandra.db.ColumnFamilyStore.finishLocalSampling(ColumnFamilyStore.java:1633)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17254) nodetool toppartitions can fail because ByteBuffer.array() returns more bytes than would be considered valid by UTF8Serializer.validate
Jordan West created CASSANDRA-17254:
---
Summary: nodetool toppartitions can fail because
ByteBuffer.array() returns more bytes than would be considered valid by
UTF8Serializer.validate
Key: CASSANDRA-17254
URL: https://issues.apache.org/jira/browse/CASSANDRA-17254
Project: Cassandra
Issue Type: Bug
Reporter: Jordan West
Assignee: Jordan West
The error below is caused by the use of
[{{ByteBuffer.array()}}|https://github.com/apache/cassandra/blob/cassandra-3.0/src/java/org/apache/cassandra/db/ColumnFamilyStore.java#L1628].
Doing so not only makes the hex key potentially incorrect but causes invalid
data to be passed to {{AbstractType.getString}} and ultimately
{{UTF8Validator.validate}}.
{code}
error: String didn't validate.
-- StackTrace --
org.apache.cassandra.serializers.MarshalException: String didn't validate.
at
org.apache.cassandra.serializers.UTF8Serializer.validate(UTF8Serializer.java:35)
at
org.apache.cassandra.db.marshal.AbstractType.getString(AbstractType.java:129)
at
org.apache.cassandra.db.ColumnFamilyStore.finishLocalSampling(ColumnFamilyStore.java:1633)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
{code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17251) USING writetime + ttl is non-idempotent leading to non-deterministic merge iteration results
[
https://issues.apache.org/jira/browse/CASSANDRA-17251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jordan West updated CASSANDRA-17251:
Test and Documentation Plan: Added {{ConflictsTest}}
Status: Patch Available (was: Open)
https://github.com/apache/cassandra/compare/trunk...jrwest:jwest/17251-3.0
> USING writetime + ttl is non-idempotent leading to non-deterministic merge
> iteration results
>
>
> Key: CASSANDRA-17251
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17251
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Other
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> The combination of {{USING writetime = timestamp and ttl = ttl}} can result
> in non-deterministic MergeIterator results causing DigestMismatchExceptions
> and increased latencies. The increased latencies are caused by additional
> round trips due to the digest mismatch as well as read repair rewriting the
> data. The additional writes lead to an increase in the number of sstables the
> key is stored in and must be scanned on read.
> The order of events is:
> 1. for a given partition a write is performed with {{USING timestamp =
> sometime and ttl = ttl1}}.
> 2. Cassandra records this write with timestamp = sometime, ttl = ttl1,
> expires_at = now + ttl1
> 3. after N seconds, for the same partition, another write is performed with
> {{USING timestamp = sometime and ttl = ttl2 where ttl2 = ttl1 - N}}. This
> write only makes it to a subset of replicas* for some reason (e.g. partial
> write, node down, etc).
> 4. Cassandra records this write with timestamp = sometime, ttl = ttl2,
> expires_at = now + ttl2. Its important to note that at this point, expires_at
> in 2 above is equal to expires at here. This is because it is calculated
> relative to the current write time not the provided timestamp and the ttl has
> been adjusted by the time passed. This write also makes it to a subset of
> replicas*.
> 5. A read of the data is performed.
> 5a. The MergeIterator resolves conflicts locally (accross sstables) using
> {{Conflicts.resolveRegular}} or {{Cells.resolveRegular}}. The resolution
> takes into account the write timestamp , the liveness of the cell, the values
> themselves, and how much time is left to live via the expires_at field. In
> this scenario, all of these fields are equal, leading to Cassandra picking
> the sstable "on the right" – this is non-deterministic. The only item that
> differs is the ttl itself.
> 5b. One node returns the non-deterministically chosen value for the row, the
> other two calculate and send a digest to the coordinator. The digest includes
> the relative ttl field which may not match. This results in a
> DigestMismatchException at the coordinator.
> 6. Read repair is triggered
> *NOTE: its not strictly necessary for the write to make it to a subset of
> replicas. sstables can also be ordered in random orders for reasons like
> compaction or repair when returned from the live set which can lead to the
> same behavior. This also affects repair from what we can tell.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17251) USING writetime + ttl is non-idempotent leading to non-deterministic merge iteration results
[
https://issues.apache.org/jira/browse/CASSANDRA-17251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jordan West updated CASSANDRA-17251:
Bug Category: Parent values: Correctness(12982)Level 1 values:
Consistency(12989)
Complexity: Normal
Component/s: Local/Other
Discovered By: User Report
Severity: Normal
Status: Open (was: Triage Needed)
> USING writetime + ttl is non-idempotent leading to non-deterministic merge
> iteration results
>
>
> Key: CASSANDRA-17251
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17251
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Other
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> The combination of {{USING writetime = timestamp and ttl = ttl}} can result
> in non-deterministic MergeIterator results causing DigestMismatchExceptions
> and increased latencies. The increased latencies are caused by additional
> round trips due to the digest mismatch as well as read repair rewriting the
> data. The additional writes lead to an increase in the number of sstables the
> key is stored in and must be scanned on read.
> The order of events is:
> 1. for a given partition a write is performed with {{USING timestamp =
> sometime and ttl = ttl1}}.
> 2. Cassandra records this write with timestamp = sometime, ttl = ttl1,
> expires_at = now + ttl1
> 3. after N seconds, for the same partition, another write is performed with
> {{USING timestamp = sometime and ttl = ttl2 where ttl2 = ttl1 - N}}. This
> write only makes it to a subset of replicas* for some reason (e.g. partial
> write, node down, etc).
> 4. Cassandra records this write with timestamp = sometime, ttl = ttl2,
> expires_at = now + ttl2. Its important to note that at this point, expires_at
> in 2 above is equal to expires at here. This is because it is calculated
> relative to the current write time not the provided timestamp and the ttl has
> been adjusted by the time passed. This write also makes it to a subset of
> replicas*.
> 5. A read of the data is performed.
> 5a. The MergeIterator resolves conflicts locally (accross sstables) using
> {{Conflicts.resolveRegular}} or {{Cells.resolveRegular}}. The resolution
> takes into account the write timestamp , the liveness of the cell, the values
> themselves, and how much time is left to live via the expires_at field. In
> this scenario, all of these fields are equal, leading to Cassandra picking
> the sstable "on the right" – this is non-deterministic. The only item that
> differs is the ttl itself.
> 5b. One node returns the non-deterministically chosen value for the row, the
> other two calculate and send a digest to the coordinator. The digest includes
> the relative ttl field which may not match. This results in a
> DigestMismatchException at the coordinator.
> 6. Read repair is triggered
> *NOTE: its not strictly necessary for the write to make it to a subset of
> replicas. sstables can also be ordered in random orders for reasons like
> compaction or repair when returned from the live set which can lead to the
> same behavior. This also affects repair from what we can tell.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-17251) USING writetime + ttl is non-idempotent leading to non-deterministic merge iteration results
Jordan West created CASSANDRA-17251:
---
Summary: USING writetime + ttl is non-idempotent leading to
non-deterministic merge iteration results
Key: CASSANDRA-17251
URL: https://issues.apache.org/jira/browse/CASSANDRA-17251
Project: Cassandra
Issue Type: Bug
Reporter: Jordan West
Assignee: Jordan West
The combination of {{USING writetime = timestamp and ttl = ttl}} can result in
non-deterministic MergeIterator results causing DigestMismatchExceptions and
increased latencies. The increased latencies are caused by additional round
trips due to the digest mismatch as well as read repair rewriting the data. The
additional writes lead to an increase in the number of sstables the key is
stored in and must be scanned on read.
The order of events is:
1. for a given partition a write is performed with {{USING timestamp = sometime
and ttl = ttl1}}.
2. Cassandra records this write with timestamp = sometime, ttl = ttl1,
expires_at = now + ttl1
3. after N seconds, for the same partition, another write is performed with
{{USING timestamp = sometime and ttl = ttl2 where ttl2 = ttl1 - N}}. This write
only makes it to a subset of replicas* for some reason (e.g. partial write,
node down, etc).
4. Cassandra records this write with timestamp = sometime, ttl = ttl2,
expires_at = now + ttl2. Its important to note that at this point, expires_at
in 2 above is equal to expires at here. This is because it is calculated
relative to the current write time not the provided timestamp and the ttl has
been adjusted by the time passed. This write also makes it to a subset of
replicas*.
5. A read of the data is performed.
5a. The MergeIterator resolves conflicts locally (accross sstables) using
{{Conflicts.resolveRegular}} or {{Cells.resolveRegular}}. The resolution takes
into account the write timestamp , the liveness of the cell, the values
themselves, and how much time is left to live via the expires_at field. In this
scenario, all of these fields are equal, leading to Cassandra picking the
sstable "on the right" – this is non-deterministic. The only item that differs
is the ttl itself.
5b. One node returns the non-deterministically chosen value for the row, the
other two calculate and send a digest to the coordinator. The digest includes
the relative ttl field which may not match. This results in a
DigestMismatchException at the coordinator.
6. Read repair is triggered
*NOTE: its not strictly necessary for the write to make it to a subset of
replicas. sstables can also be ordered in random orders for reasons like
compaction or repair when returned from the live set which can lead to the same
behavior. This also affects repair from what we can tell.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-17251) USING writetime + ttl is non-idempotent leading to non-deterministic merge iteration results
[
https://issues.apache.org/jira/browse/CASSANDRA-17251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jordan West updated CASSANDRA-17251:
Fix Version/s: 3.0.x
3.11.x
4.0.x
> USING writetime + ttl is non-idempotent leading to non-deterministic merge
> iteration results
>
>
> Key: CASSANDRA-17251
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17251
> Project: Cassandra
> Issue Type: Bug
>Reporter: Jordan West
>Assignee: Jordan West
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> The combination of {{USING writetime = timestamp and ttl = ttl}} can result
> in non-deterministic MergeIterator results causing DigestMismatchExceptions
> and increased latencies. The increased latencies are caused by additional
> round trips due to the digest mismatch as well as read repair rewriting the
> data. The additional writes lead to an increase in the number of sstables the
> key is stored in and must be scanned on read.
> The order of events is:
> 1. for a given partition a write is performed with {{USING timestamp =
> sometime and ttl = ttl1}}.
> 2. Cassandra records this write with timestamp = sometime, ttl = ttl1,
> expires_at = now + ttl1
> 3. after N seconds, for the same partition, another write is performed with
> {{USING timestamp = sometime and ttl = ttl2 where ttl2 = ttl1 - N}}. This
> write only makes it to a subset of replicas* for some reason (e.g. partial
> write, node down, etc).
> 4. Cassandra records this write with timestamp = sometime, ttl = ttl2,
> expires_at = now + ttl2. Its important to note that at this point, expires_at
> in 2 above is equal to expires at here. This is because it is calculated
> relative to the current write time not the provided timestamp and the ttl has
> been adjusted by the time passed. This write also makes it to a subset of
> replicas*.
> 5. A read of the data is performed.
> 5a. The MergeIterator resolves conflicts locally (accross sstables) using
> {{Conflicts.resolveRegular}} or {{Cells.resolveRegular}}. The resolution
> takes into account the write timestamp , the liveness of the cell, the values
> themselves, and how much time is left to live via the expires_at field. In
> this scenario, all of these fields are equal, leading to Cassandra picking
> the sstable "on the right" – this is non-deterministic. The only item that
> differs is the ttl itself.
> 5b. One node returns the non-deterministically chosen value for the row, the
> other two calculate and send a digest to the coordinator. The digest includes
> the relative ttl field which may not match. This results in a
> DigestMismatchException at the coordinator.
> 6. Read repair is triggered
> *NOTE: its not strictly necessary for the write to make it to a subset of
> replicas. sstables can also be ordered in random orders for reasons like
> compaction or repair when returned from the live set which can lead to the
> same behavior. This also affects repair from what we can tell.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16095) When a table attempts to clean up metrics, it was cleaning up all global table metrics
[
https://issues.apache.org/jira/browse/CASSANDRA-16095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242679#comment-17242679
]
Jordan West commented on CASSANDRA-16095:
-
+1 LGTM
> When a table attempts to clean up metrics, it was cleaning up all global
> table metrics
> --
>
> Key: CASSANDRA-16095
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16095
> Project: Cassandra
> Issue Type: Bug
> Components: Observability/JMX
>Reporter: Altan Özlü
>Assignee: David Capwell
>Priority: Urgent
> Fix For: 4.0-beta
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> when i use
> {code:java}
> bin/nodetool tablestats {code}
> i get this error
> {code:java}
> error:
> org.apache.cassandra.metrics:type=Table,keyspace=,scope=feed,name=WriteLatency
> -- StackTrace --
> javax.management.InstanceNotFoundException:
> org.apache.cassandra.metrics:type=Table,keyspace=,scope=feed,name=WriteLatency
> {code}
> if i restart the node and check it works but if i write something then i get
> the error again
> * Cassandra 4.0-beta1
> * Ubuntu 20
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15214) OOMs caught and not rethrown
[ https://issues.apache.org/jira/browse/CASSANDRA-15214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-15214: Status: Ready to Commit (was: Changes Suggested) > OOMs caught and not rethrown > > > Key: CASSANDRA-15214 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15214 > Project: Cassandra > Issue Type: Bug > Components: Messaging/Client, Messaging/Internode >Reporter: Benedict Elliott Smith >Assignee: Yifan Cai >Priority: Normal > Fix For: 4.0, 4.0-rc > > Attachments: oom-experiments.zip > > Time Spent: 1h 40m > Remaining Estimate: 0h > > Netty (at least, and perhaps elsewhere in Executors) catches all exceptions, > so presently there is no way to ensure that an OOM reaches the JVM handler to > trigger a crash/heapdump. > It may be that the simplest most consistent way to do this would be to have a > single thread spawned at startup that waits for any exceptions we must > propagate to the Runtime. > We could probably submit a patch upstream to Netty, but for a guaranteed > future proof approach, it may be worth paying the cost of a single thread. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16285) Change Dynamic Snitch Default Badness Threshold to 1.0
[ https://issues.apache.org/jira/browse/CASSANDRA-16285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-16285: Fix Version/s: 4.0-beta4 Source Control Link: https://github.com/apache/cassandra/commit/fae1f883541f329f7575d6ff4117b230e371293b Resolution: Fixed Status: Resolved (was: Ready to Commit) Committed as https://github.com/apache/cassandra/commit/fae1f883541f329f7575d6ff4117b230e371293b > Change Dynamic Snitch Default Badness Threshold to 1.0 > -- > > Key: CASSANDRA-16285 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16285 > Project: Cassandra > Issue Type: Improvement > Components: Consistency/Coordination >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Fix For: 4.0-beta4 > > Attachments: readcount-0.1.png, readcount-1.0.png, > readlatency-0.1.png, readlatency-1.0.png > > > With the removal of compaction and IO from the DynamicEndpointSnitch score > calculation, the default badness threshold of 10% (0.1) is too small of a > margin from experience with production clusters. When compaction and IO > values were included, the resulting scores were dominated by them and 10% was > a much more noticeable difference. When relying solely on latency, the > DynamicEndpointSnitch can rely on nodes that are performing only marginally > better than their peers. This results in a lopsided request distribution > among the replicas despite similar performance. > Some graphs are attached from a production cluster showing the read count and > latency among the nodes with the default of 0.1 and with the badness > threshold set to 1. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16285) Change Dynamic Snitch Default Badness Threshold to 1.0
[ https://issues.apache.org/jira/browse/CASSANDRA-16285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jordan West updated CASSANDRA-16285: Status: Ready to Commit (was: Review In Progress) > Change Dynamic Snitch Default Badness Threshold to 1.0 > -- > > Key: CASSANDRA-16285 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16285 > Project: Cassandra > Issue Type: Improvement > Components: Consistency/Coordination >Reporter: Jordan West >Assignee: Jordan West >Priority: Normal > Attachments: readcount-0.1.png, readcount-1.0.png, > readlatency-0.1.png, readlatency-1.0.png > > > With the removal of compaction and IO from the DynamicEndpointSnitch score > calculation, the default badness threshold of 10% (0.1) is too small of a > margin from experience with production clusters. When compaction and IO > values were included, the resulting scores were dominated by them and 10% was > a much more noticeable difference. When relying solely on latency, the > DynamicEndpointSnitch can rely on nodes that are performing only marginally > better than their peers. This results in a lopsided request distribution > among the replicas despite similar performance. > Some graphs are attached from a production cluster showing the read count and > latency among the nodes with the default of 0.1 and with the badness > threshold set to 1. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
