[jira] [Assigned] (CASSANDRA-18016) BLOG - Update author for availability blog and front door backpressure blog
[ https://issues.apache.org/jira/browse/CASSANDRA-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumanth Pasupuleti reassigned CASSANDRA-18016: -- Assignee: Erick Ramirez (was: Sumanth Pasupuleti) > BLOG - Update author for availability blog and front door backpressure blog > --- > > Key: CASSANDRA-18016 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18016 > Project: Cassandra > Issue Type: Task > Components: Documentation/Blog >Reporter: Sumanth Pasupuleti >Assignee: Erick Ramirez >Priority: Normal > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-18016) BLOG - Update author for availability blog and front door backpressure blog
[ https://issues.apache.org/jira/browse/CASSANDRA-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumanth Pasupuleti reassigned CASSANDRA-18016: -- Assignee: Sumanth Pasupuleti > BLOG - Update author for availability blog and front door backpressure blog > --- > > Key: CASSANDRA-18016 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18016 > Project: Cassandra > Issue Type: Task > Components: Documentation/Blog >Reporter: Sumanth Pasupuleti >Assignee: Sumanth Pasupuleti >Priority: Normal > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-18016) BLOG - Update author for availability blog and front door backpressure blog
[ https://issues.apache.org/jira/browse/CASSANDRA-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17629058#comment-17629058 ] Sumanth Pasupuleti commented on CASSANDRA-18016: https://github.com/apache/cassandra-website/pull/192 > BLOG - Update author for availability blog and front door backpressure blog > --- > > Key: CASSANDRA-18016 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18016 > Project: Cassandra > Issue Type: Task > Components: Documentation/Blog >Reporter: Sumanth Pasupuleti >Assignee: Sumanth Pasupuleti >Priority: Normal > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-18016) BLOG - Update author for availability blog and front door backpressure blog
[ https://issues.apache.org/jira/browse/CASSANDRA-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumanth Pasupuleti updated CASSANDRA-18016: --- Summary: BLOG - Update author for availability blog and front door backpressure blog (was: BLOG - Update author for Zero copy streaming blog and front door backpressure blog) > BLOG - Update author for availability blog and front door backpressure blog > --- > > Key: CASSANDRA-18016 > URL: https://issues.apache.org/jira/browse/CASSANDRA-18016 > Project: Cassandra > Issue Type: Task > Components: Documentation/Blog >Reporter: Sumanth Pasupuleti >Priority: Normal > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-18016) BLOG - Update author for Zero copy streaming blog and front door backpressure blog
Sumanth Pasupuleti created CASSANDRA-18016: -- Summary: BLOG - Update author for Zero copy streaming blog and front door backpressure blog Key: CASSANDRA-18016 URL: https://issues.apache.org/jira/browse/CASSANDRA-18016 Project: Cassandra Issue Type: Task Components: Documentation/Blog Reporter: Sumanth Pasupuleti -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-17711) Create a new node tool supporting force compaction of a partition
[
https://issues.apache.org/jira/browse/CASSANDRA-17711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17611923#comment-17611923
]
Sumanth Pasupuleti commented on CASSANDRA-17711:
+1
> Create a new node tool supporting force compaction of a partition
> -
>
> Key: CASSANDRA-17711
> URL: https://issues.apache.org/jira/browse/CASSANDRA-17711
> Project: Cassandra
> Issue Type: Improvement
> Components: Tool/nodetool
>Reporter: Cheng Wang
>Assignee: Cheng Wang
>Priority: Normal
>
> Need to create new tool called {*}nodetool forcecompact{*}. The syntax of the
> command is
> *nodetool forcecompact keyspace table gc_grace_seconds>.*
> One of the use cases of the tool is to handle the bad partitions which may
> contain too many tombstones. The tool will compact all the sstables
> containing the partition keys. In addition, by ignoring the
> {*}gc_grace_seconds{*}, it will help purge the tombstones faster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14557) Add default and required keyspace replication options
[
https://issues.apache.org/jira/browse/CASSANDRA-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17428078#comment-17428078
]
Sumanth Pasupuleti commented on CASSANDRA-14557:
[~stefan.miklosovic] Made a follow up commit to fix the failing UTs.
Specifically this change
https://github.com/sumanth-pasupuleti/cassandra/commit/f10e681b60436b2c587747f4e48f0635d348280c#diff-8e56094eaea78601c862aca6d18a2b2e583ceecd933f01ebb85ea7019dfb54daR754-R755
> Add default and required keyspace replication options
> -
>
> Key: CASSANDRA-14557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14557
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
> Labels: 4.0-feature-freeze-review-requested
> Fix For: 4.1
>
> Attachments: 14557-4.0.txt, 14557-trunk.patch
>
>
> Ending up with a keyspace of RF=1 is unfortunately pretty easy in C* right
> now - the system_auth table for example is created with RF=1 (to take into
> account single node setups afaict from CASSANDRA-5112), and a user can
> further create a keyspace with RF=1 posing availability and streaming risks
> (e.g. rebuild).
> I propose we add two configuration options in cassandra.yaml:
> # {{default_keyspace_rf}} (default: 1) - If replication factors are not
> specified, use this number.
> # {{required_minimum_keyspace_rf}} (default: unset) - Prevent users from
> creating a keyspace with an RF less than what is configured
> These settings could further be re-used to:
> * Provide defaults for new keyspaces created with SimpleStrategy or
> NetworkTopologyStrategy (CASSANDRA-14303)
> * Make the automatic token [allocation
> algorithm|https://issues.apache.org/jira/browse/CASSANDRA-13701?focusedCommentId=16095662=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16095662]
> interface more intuitive allowing easy use of the new token allocation
> algorithm.
> At the end of the day, if someone really wants to allow RF=1, they simply
> don’t set the setting. For backwards compatibility the default remains 1 and
> C* would create with RF=1, and would default to current behavior of allowing
> any RF on keyspaces.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14557) Add default and required keyspace replication options
[
https://issues.apache.org/jira/browse/CASSANDRA-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17427963#comment-17427963
]
Sumanth Pasupuleti commented on CASSANDRA-14557:
I've squashed the changes, ready to be committed
https://github.com/sumanth-pasupuleti/cassandra/tree/14557-trunk
> Add default and required keyspace replication options
> -
>
> Key: CASSANDRA-14557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14557
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
> Labels: 4.0-feature-freeze-review-requested
> Fix For: 4.1
>
> Attachments: 14557-4.0.txt, 14557-trunk.patch
>
>
> Ending up with a keyspace of RF=1 is unfortunately pretty easy in C* right
> now - the system_auth table for example is created with RF=1 (to take into
> account single node setups afaict from CASSANDRA-5112), and a user can
> further create a keyspace with RF=1 posing availability and streaming risks
> (e.g. rebuild).
> I propose we add two configuration options in cassandra.yaml:
> # {{default_keyspace_rf}} (default: 1) - If replication factors are not
> specified, use this number.
> # {{required_minimum_keyspace_rf}} (default: unset) - Prevent users from
> creating a keyspace with an RF less than what is configured
> These settings could further be re-used to:
> * Provide defaults for new keyspaces created with SimpleStrategy or
> NetworkTopologyStrategy (CASSANDRA-14303)
> * Make the automatic token [allocation
> algorithm|https://issues.apache.org/jira/browse/CASSANDRA-13701?focusedCommentId=16095662=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16095662]
> interface more intuitive allowing easy use of the new token allocation
> algorithm.
> At the end of the day, if someone really wants to allow RF=1, they simply
> don’t set the setting. For backwards compatibility the default remains 1 and
> C* would create with RF=1, and would default to current behavior of allowing
> any RF on keyspaces.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15433) Pending ranges are not recalculated on keyspace creation
[
https://issues.apache.org/jira/browse/CASSANDRA-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17427960#comment-17427960
]
Sumanth Pasupuleti commented on CASSANDRA-15433:
[~ifesdjeen] curious if you will be able to commit this patch
> Pending ranges are not recalculated on keyspace creation
>
>
> Key: CASSANDRA-15433
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15433
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Membership
>Reporter: Josh Snyder
>Assignee: Sumanth Pasupuleti
>Priority: Normal
>
> When a node begins bootstrapping, Cassandra recalculates pending tokens for
> each keyspace that exists when the state change is observed (in
> StorageService:handleState*). When new keyspaces are created, we do not
> recalculate pending ranges (around Schema:merge). As a result, writes for new
> keyspaces are not received by nodes in BOOT or BOOT_REPLACE modes. When
> bootstrapping finishes, the node which just bootstrapped will not have data
> for the newly created keyspace.
> Consider a ring with bootstrapped nodes A, B, and C. Node D is pending, and
> when it finishes bootstrapping, C will cede ownership of some ranges to D. A
> quorum write is acknowledged by C and A. B missed the write, and the
> coordinator didn't send it to D at all. When D finishes bootstrapping, the
> quorum B+D will not contain the mutation.
> Steps to reproduce:
> # Join a node in BOOT mode
> # Create a keyspace
> # Send writes to that keyspace
> # On the joining node, observe that {{nodetool cfstats}} records zero writes
> to the new keyspace
> I have observed this directly in Cassandra 3.0, and based on my reading the
> code, I believe it affects up through trunk.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16203) Rack Aware Topology Strategy
[ https://issues.apache.org/jira/browse/CASSANDRA-16203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumanth Pasupuleti reassigned CASSANDRA-16203: -- Assignee: Sumanth Pasupuleti (was: Stefan Miklosovic) > Rack Aware Topology Strategy > > > Key: CASSANDRA-16203 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16203 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Membership >Reporter: Cameron Zemek >Assignee: Sumanth Pasupuleti >Priority: Normal > Fix For: 4.x > > Attachments: rackaware.patch > > > If replication factor > racks then NetworkTopologyStrategy assigns the extras > in ring order. This means losing a rack can result in the loss of quorum. I > have implemented a similar topology strategy that evenly distributes replicas > across racks. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14557) Add default and required keyspace replication options
[
https://issues.apache.org/jira/browse/CASSANDRA-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17427487#comment-17427487
]
Sumanth Pasupuleti commented on CASSANDRA-14557:
[~stefan.miklosovic] fyi, updated javadoc
> Add default and required keyspace replication options
> -
>
> Key: CASSANDRA-14557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14557
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
> Labels: 4.0-feature-freeze-review-requested
> Fix For: 4.x
>
> Attachments: 14557-4.0.txt, 14557-trunk.patch
>
>
> Ending up with a keyspace of RF=1 is unfortunately pretty easy in C* right
> now - the system_auth table for example is created with RF=1 (to take into
> account single node setups afaict from CASSANDRA-5112), and a user can
> further create a keyspace with RF=1 posing availability and streaming risks
> (e.g. rebuild).
> I propose we add two configuration options in cassandra.yaml:
> # {{default_keyspace_rf}} (default: 1) - If replication factors are not
> specified, use this number.
> # {{required_minimum_keyspace_rf}} (default: unset) - Prevent users from
> creating a keyspace with an RF less than what is configured
> These settings could further be re-used to:
> * Provide defaults for new keyspaces created with SimpleStrategy or
> NetworkTopologyStrategy (CASSANDRA-14303)
> * Make the automatic token [allocation
> algorithm|https://issues.apache.org/jira/browse/CASSANDRA-13701?focusedCommentId=16095662=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16095662]
> interface more intuitive allowing easy use of the new token allocation
> algorithm.
> At the end of the day, if someone really wants to allow RF=1, they simply
> don’t set the setting. For backwards compatibility the default remains 1 and
> C* would create with RF=1, and would default to current behavior of allowing
> any RF on keyspaces.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14557) Add default and required keyspace replication options
[
https://issues.apache.org/jira/browse/CASSANDRA-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17427149#comment-17427149
]
Sumanth Pasupuleti commented on CASSANDRA-14557:
[~azotcsit] [~stefan.miklosovic] made another commit to reflect further
feedback. Will squash once we are good.
> Add default and required keyspace replication options
> -
>
> Key: CASSANDRA-14557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14557
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
> Labels: 4.0-feature-freeze-review-requested
> Fix For: 4.x
>
> Attachments: 14557-4.0.txt, 14557-trunk.patch
>
>
> Ending up with a keyspace of RF=1 is unfortunately pretty easy in C* right
> now - the system_auth table for example is created with RF=1 (to take into
> account single node setups afaict from CASSANDRA-5112), and a user can
> further create a keyspace with RF=1 posing availability and streaming risks
> (e.g. rebuild).
> I propose we add two configuration options in cassandra.yaml:
> # {{default_keyspace_rf}} (default: 1) - If replication factors are not
> specified, use this number.
> # {{required_minimum_keyspace_rf}} (default: unset) - Prevent users from
> creating a keyspace with an RF less than what is configured
> These settings could further be re-used to:
> * Provide defaults for new keyspaces created with SimpleStrategy or
> NetworkTopologyStrategy (CASSANDRA-14303)
> * Make the automatic token [allocation
> algorithm|https://issues.apache.org/jira/browse/CASSANDRA-13701?focusedCommentId=16095662=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16095662]
> interface more intuitive allowing easy use of the new token allocation
> algorithm.
> At the end of the day, if someone really wants to allow RF=1, they simply
> don’t set the setting. For backwards compatibility the default remains 1 and
> C* would create with RF=1, and would default to current behavior of allowing
> any RF on keyspaces.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14557) Consider adding default and required keyspace replication options
[
https://issues.apache.org/jira/browse/CASSANDRA-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17426824#comment-17426824
]
Sumanth Pasupuleti commented on CASSANDRA-14557:
[~azotcsit] Thanks for the further feedback. Incorporated the recommended
changes. Please let me know if you have further feedback. I plan to squash the
changes once we are happy with the code.
> Consider adding default and required keyspace replication options
> -
>
> Key: CASSANDRA-14557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14557
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
> Labels: 4.0-feature-freeze-review-requested
> Fix For: 4.x
>
> Attachments: 14557-4.0.txt, 14557-trunk.patch
>
>
> Ending up with a keyspace of RF=1 is unfortunately pretty easy in C* right
> now - the system_auth table for example is created with RF=1 (to take into
> account single node setups afaict from CASSANDRA-5112), and a user can
> further create a keyspace with RF=1 posing availability and streaming risks
> (e.g. rebuild).
> I propose we add two configuration options in cassandra.yaml:
> # {{default_keyspace_rf}} (default: 1) - If replication factors are not
> specified, use this number.
> # {{required_minimum_keyspace_rf}} (default: unset) - Prevent users from
> creating a keyspace with an RF less than what is configured
> These settings could further be re-used to:
> * Provide defaults for new keyspaces created with SimpleStrategy or
> NetworkTopologyStrategy (CASSANDRA-14303)
> * Make the automatic token [allocation
> algorithm|https://issues.apache.org/jira/browse/CASSANDRA-13701?focusedCommentId=16095662=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16095662]
> interface more intuitive allowing easy use of the new token allocation
> algorithm.
> At the end of the day, if someone really wants to allow RF=1, they simply
> don’t set the setting. For backwards compatibility the default remains 1 and
> C* would create with RF=1, and would default to current behavior of allowing
> any RF on keyspaces.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14557) Consider adding default and required keyspace replication options
[
https://issues.apache.org/jira/browse/CASSANDRA-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17426824#comment-17426824
]
Sumanth Pasupuleti edited comment on CASSANDRA-14557 at 10/10/21, 4:29 PM:
---
[~azotcsit] Thanks for the feedback. Incorporated the recommended changes.
Please let me know if you have further feedback. I plan to squash the changes
once we are happy with the code.
was (Author: sumanth.pasupuleti):
[~azotcsit] Thanks for the further feedback. Incorporated the recommended
changes. Please let me know if you have further feedback. I plan to squash the
changes once we are happy with the code.
> Consider adding default and required keyspace replication options
> -
>
> Key: CASSANDRA-14557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14557
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
> Labels: 4.0-feature-freeze-review-requested
> Fix For: 4.x
>
> Attachments: 14557-4.0.txt, 14557-trunk.patch
>
>
> Ending up with a keyspace of RF=1 is unfortunately pretty easy in C* right
> now - the system_auth table for example is created with RF=1 (to take into
> account single node setups afaict from CASSANDRA-5112), and a user can
> further create a keyspace with RF=1 posing availability and streaming risks
> (e.g. rebuild).
> I propose we add two configuration options in cassandra.yaml:
> # {{default_keyspace_rf}} (default: 1) - If replication factors are not
> specified, use this number.
> # {{required_minimum_keyspace_rf}} (default: unset) - Prevent users from
> creating a keyspace with an RF less than what is configured
> These settings could further be re-used to:
> * Provide defaults for new keyspaces created with SimpleStrategy or
> NetworkTopologyStrategy (CASSANDRA-14303)
> * Make the automatic token [allocation
> algorithm|https://issues.apache.org/jira/browse/CASSANDRA-13701?focusedCommentId=16095662=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16095662]
> interface more intuitive allowing easy use of the new token allocation
> algorithm.
> At the end of the day, if someone really wants to allow RF=1, they simply
> don’t set the setting. For backwards compatibility the default remains 1 and
> C* would create with RF=1, and would default to current behavior of allowing
> any RF on keyspaces.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12106) Add ability to blocklist / denylist a CQL partition so all requests are ignored
[
https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17423540#comment-17423540
]
Sumanth Pasupuleti commented on CASSANDRA-12106:
{quote}I tend to dislike risk (even the minimal risk of differing data types in
memory vs. table and conversion on load); we can document this (including the
CQL to query and convert to human readable) to work around that little
usability friction. wdyt?{quote}
I agree. +1 to documenting the CQL for easy reference. Also, nodetool commands
maybe able to make this easy - we can defer the nodetool discussion to a follow
up ticket like you mentioned.
{quote}the code-paths warn if you're above either the global or per-table
level{quote}
+1 to the improved logging. I still think we can do better in avoiding corner
case cache inconsistency
https://github.com/apache/cassandra/pull/1213/commits/0fb6ae95149a1a6601208cdae4b165835ea10070#r720686050
> Add ability to blocklist / denylist a CQL partition so all requests are
> ignored
> ---
>
> Key: CASSANDRA-12106
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12106
> Project: Cassandra
> Issue Type: New Feature
> Components: Legacy/Local Write-Read Paths, Local/Config
>Reporter: Geoffrey Yu
>Assignee: Josh McKenzie
>Priority: Low
> Fix For: 4.x
>
> Attachments: 12106-trunk.txt
>
>
> Sometimes reads/writes to a given partition may cause problems due to the
> data present. It would be useful to have a manual way to blocklist / denylist
> such partitions so all read and write requests to them are rejected.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-12106) Add ability to blocklist / denylist a CQL partition so all requests are ignored
[ https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumanth Pasupuleti updated CASSANDRA-12106: --- Reviewers: Aleksei Zotov, Dinesh Joshi, Sumanth Pasupuleti (was: Aleksei Zotov, Dinesh Joshi) > Add ability to blocklist / denylist a CQL partition so all requests are > ignored > --- > > Key: CASSANDRA-12106 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12106 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Local Write-Read Paths, Local/Config >Reporter: Geoffrey Yu >Assignee: Josh McKenzie >Priority: Low > Fix For: 4.x > > Attachments: 12106-trunk.txt > > > Sometimes reads/writes to a given partition may cause problems due to the > data present. It would be useful to have a manual way to blocklist / denylist > such partitions so all read and write requests to them are rejected. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14557) Consider adding default and required keyspace replication options
[
https://issues.apache.org/jira/browse/CASSANDRA-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17420273#comment-17420273
]
Sumanth Pasupuleti commented on CASSANDRA-14557:
[~azotcsit] [~stefan.miklosovic] Thanks for further feedback. Incorporated the
feedback. https://github.com/sumanth-pasupuleti/cassandra/tree/14557-trunk
> Consider adding default and required keyspace replication options
> -
>
> Key: CASSANDRA-14557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14557
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
> Labels: 4.0-feature-freeze-review-requested
> Fix For: 4.x
>
> Attachments: 14557-4.0.txt, 14557-trunk.patch
>
>
> Ending up with a keyspace of RF=1 is unfortunately pretty easy in C* right
> now - the system_auth table for example is created with RF=1 (to take into
> account single node setups afaict from CASSANDRA-5112), and a user can
> further create a keyspace with RF=1 posing availability and streaming risks
> (e.g. rebuild).
> I propose we add two configuration options in cassandra.yaml:
> # {{default_keyspace_rf}} (default: 1) - If replication factors are not
> specified, use this number.
> # {{required_minimum_keyspace_rf}} (default: unset) - Prevent users from
> creating a keyspace with an RF less than what is configured
> These settings could further be re-used to:
> * Provide defaults for new keyspaces created with SimpleStrategy or
> NetworkTopologyStrategy (CASSANDRA-14303)
> * Make the automatic token [allocation
> algorithm|https://issues.apache.org/jira/browse/CASSANDRA-13701?focusedCommentId=16095662=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16095662]
> interface more intuitive allowing easy use of the new token allocation
> algorithm.
> At the end of the day, if someone really wants to allow RF=1, they simply
> don’t set the setting. For backwards compatibility the default remains 1 and
> C* would create with RF=1, and would default to current behavior of allowing
> any RF on keyspaces.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-12106) Add ability to blocklist / denylist a CQL partition so all requests are ignored
[ https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419811#comment-17419811 ] Sumanth Pasupuleti edited comment on CASSANDRA-12106 at 9/24/21, 4:47 PM: -- [~jmckenzie] did a deeper pass, and I have following questions/suggestions 1. It maybe useful to add a documentation file around denylisting partitions (maybe along the lines of https://github.com/apache/cassandra/compare/trunk...sumanth-pasupuleti:feature/12106#diff-4264f2ac97017f73e4f6fff8f1cbef3082212b2edc531b57f6e8f064d3ab3e49) 2. -With regards to limiting the amount of memory we would allow the denylisted partitions to take, we could potentially limit/throw warning based on memory as an alternate approach to limiting based on number of partition keys. Not a strong opinion here since we expect operators (and not end users) to use this feature and it should be safe either ways, given we expect operators to be informed.- [EDIT] As I think through further, and as mentioned in CEP, it is probably best to log a warning if denylisted partitions exceed in either number or memory usage vs "limiting" them, in order to maintain consistency across all the nodes' caches in terms of what partitions are denylisted. 3. We could as an alternative, allow configuring denylisting specific operations (read vs write) per node per partition vs per node - I feel the way it is currently implemented (per node and not per node per partition) is less complex and easy to use and we should probably stick to that. We can revisit in the future if needed. 4. Curious on the choice of exposing consistency level for querying denylisting partitions. What do you think of a simpler approach doing "local execution" by each node when querying partitions_denylisted table. The benefit would be it can make it less complicated in terms of number of options we expose and not needing to check if we have enough nodes for the chosen consistency level. 5. Curious on why we disable denylisting range reads by default 6. Regarding the datatype for "key" in "system_distributed.partition_denylist", what do you think of choosing text (vs blob). Advantage would be readability of the key, and on cache refresh, we would convert it into ByteBuffer 7. What do you think about adding nodetool commands for a) refreshing denylist b) adding a partition to denylist c) removing a partition from denylist Happy to contribute to any of these, if we decide to make any related changes was (Author: sumanth.pasupuleti): [~jmckenzie] did a deeper pass, and I have following questions/suggestions 1. It maybe useful to add a documentation file around denylisting partitions (maybe along the lines of https://github.com/apache/cassandra/compare/trunk...sumanth-pasupuleti:feature/12106#diff-4264f2ac97017f73e4f6fff8f1cbef3082212b2edc531b57f6e8f064d3ab3e49) 2. -With regards to limiting the amount of memory we would allow the denylisted partitions to take, we could potentially limit/throw warning based on memory as an alternate approach to limiting based on number of partition keys. Not a strong opinion here since we expect operators (and not end users) to use this feature and it should be safe either ways, given we expect operators to be informed.- [EDIT] As I think through further, and as mentioned in CEP, it is probably best to log a warning if denylisted partitions exceed in either number or memory usage vs "limiting" them, in order to maintain consistency across all the nodes' caches in terms of what partitions are denylisted. 3. We could as an alternative, allow configuring denylisting specific operations (read vs write) per node per partition vs per node - I feel the way it is currently implemented (per node and not per node per partition) is less complex and easy to use and we should probably stick to that. We can revisit in the future if needed. 4. Curious on the choice of exposing consistency level for querying denylisting partitions. What do you think of a simpler approach doing "local execution" by each node when querying partitions_denylisted table. The benefit would be it can make it less complicated in terms of number of options we expose and not needing to check if we have enough nodes for the chosen consistency level. 5. Curious on why we disable denylisting range reads by default 6. Regarding the datatype for "key" in "system_distributed.partition_denylist", what do you think of choosing text (vs blob). Advantage would be readability of the key, and on cache refresh, we would convert it into ByteBuffer 7. What do you think about adding nodetool commands for a) refreshing denylist b) adding a partition to denylist c) removing a partition from denylist Happy to contribute to any of these, if decide to make any related changes > Add ability to blocklist / denylist a CQL partition so all
[jira] [Comment Edited] (CASSANDRA-12106) Add ability to blocklist / denylist a CQL partition so all requests are ignored
[ https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419811#comment-17419811 ] Sumanth Pasupuleti edited comment on CASSANDRA-12106 at 9/24/21, 2:36 PM: -- [~jmckenzie] did a deeper pass, and I have following questions/suggestions 1. It maybe useful to add a documentation file around denylisting partitions (maybe along the lines of https://github.com/apache/cassandra/compare/trunk...sumanth-pasupuleti:feature/12106#diff-4264f2ac97017f73e4f6fff8f1cbef3082212b2edc531b57f6e8f064d3ab3e49) 2. With regards to limiting the amount of memory we would allow the denylisted partitions to take, we could potentially limit/throw warning based on memory as an alternate approach to limiting based on number of partition keys. Not a strong opinion here since we expect operators (and not end users) to use this feature and it should be safe either ways, given we expect operators to be informed. [EDIT] As I think through further, and as mentioned in CEP, it is probably best to log a warning if denylisted partitions exceed in either number of memory usage vs "limiting" them, in order to maintain consistency across all the nodes' caches in terms of what partitions are denylisted. 3. We could as an alternative, allow configuring denylisting specific operations (read vs write) per node per partition vs per node - I feel the way it is currently implemented (per node and not per node per partition) is less complex and easy to use and we should probably stick to that. We can revisit in the future if needed. 4. Curious on the choice of exposing consistency level for querying denylisting partitions. What do you think of a simpler approach doing "local execution" by each node when querying partitions_denylisted table. The benefit would be it can make it less complicated in terms of number of options we expose and not needing to check if we have enough nodes for the chosen consistency level. 5. Curious on why we disable denylisting range reads by default 6. Regarding the datatype for "key" in "system_distributed.partition_denylist", what do you think of choosing text (vs blob). Advantage would be readability of the key, and on cache refresh, we would convert it into ByteBuffer 7. What do you think about adding nodetool commands for a) refreshing denylist b) adding a partition to denylist c) removing a partition from denylist Happy to contribute to any of these, if decide to make any related changes was (Author: sumanth.pasupuleti): [~jmckenzie] did a deeper pass, and I have following questions/suggestions 1. It maybe useful to add a documentation file around denylisting partitions (maybe along the lines of https://github.com/apache/cassandra/compare/trunk...sumanth-pasupuleti:feature/12106#diff-4264f2ac97017f73e4f6fff8f1cbef3082212b2edc531b57f6e8f064d3ab3e49) 2. With regards to limiting the amount of memory we would allow the denylisted partitions to take, we could potentially limit/throw warning based on memory as an alternate approach to limiting based on number of partition keys. Not a strong opinion here since we expect operators (and not end users) to use this feature and it should be safe either ways, given we expect operators to be informed. 3. We could as an alternative, allow configuring denylisting specific operations (read vs write) per node per partition vs per node - I feel the way it is currently implemented (per node and not per node per partition) is less complex and easy to use and we should probably stick to that. We can revisit in the future if needed. 4. Curious on the choice of exposing consistency level for querying denylisting partitions. What do you think of a simpler approach doing "local execution" by each node when querying partitions_denylisted table. The benefit would be it can make it less complicated in terms of number of options we expose and not needing to check if we have enough nodes for the chosen consistency level. 5. Curious on why we disable denylisting range reads by default 6. Regarding the datatype for "key" in "system_distributed.partition_denylist", what do you think of choosing text (vs blob). Advantage would be readability of the key, and on cache refresh, we would convert it into ByteBuffer 7. What do you think about adding nodetool commands for a) refreshing denylist b) adding a partition to denylist c) removing a partition from denylist Happy to contribute to any of these, if decide to make any related changes > Add ability to blocklist / denylist a CQL partition so all requests are > ignored > --- > > Key: CASSANDRA-12106 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12106 > Project: Cassandra > Issue Type: New Feature >
[jira] [Comment Edited] (CASSANDRA-12106) Add ability to blocklist / denylist a CQL partition so all requests are ignored
[ https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419811#comment-17419811 ] Sumanth Pasupuleti edited comment on CASSANDRA-12106 at 9/24/21, 2:36 PM: -- [~jmckenzie] did a deeper pass, and I have following questions/suggestions 1. It maybe useful to add a documentation file around denylisting partitions (maybe along the lines of https://github.com/apache/cassandra/compare/trunk...sumanth-pasupuleti:feature/12106#diff-4264f2ac97017f73e4f6fff8f1cbef3082212b2edc531b57f6e8f064d3ab3e49) 2. -With regards to limiting the amount of memory we would allow the denylisted partitions to take, we could potentially limit/throw warning based on memory as an alternate approach to limiting based on number of partition keys. Not a strong opinion here since we expect operators (and not end users) to use this feature and it should be safe either ways, given we expect operators to be informed.- [EDIT] As I think through further, and as mentioned in CEP, it is probably best to log a warning if denylisted partitions exceed in either number or memory usage vs "limiting" them, in order to maintain consistency across all the nodes' caches in terms of what partitions are denylisted. 3. We could as an alternative, allow configuring denylisting specific operations (read vs write) per node per partition vs per node - I feel the way it is currently implemented (per node and not per node per partition) is less complex and easy to use and we should probably stick to that. We can revisit in the future if needed. 4. Curious on the choice of exposing consistency level for querying denylisting partitions. What do you think of a simpler approach doing "local execution" by each node when querying partitions_denylisted table. The benefit would be it can make it less complicated in terms of number of options we expose and not needing to check if we have enough nodes for the chosen consistency level. 5. Curious on why we disable denylisting range reads by default 6. Regarding the datatype for "key" in "system_distributed.partition_denylist", what do you think of choosing text (vs blob). Advantage would be readability of the key, and on cache refresh, we would convert it into ByteBuffer 7. What do you think about adding nodetool commands for a) refreshing denylist b) adding a partition to denylist c) removing a partition from denylist Happy to contribute to any of these, if decide to make any related changes was (Author: sumanth.pasupuleti): [~jmckenzie] did a deeper pass, and I have following questions/suggestions 1. It maybe useful to add a documentation file around denylisting partitions (maybe along the lines of https://github.com/apache/cassandra/compare/trunk...sumanth-pasupuleti:feature/12106#diff-4264f2ac97017f73e4f6fff8f1cbef3082212b2edc531b57f6e8f064d3ab3e49) 2. With regards to limiting the amount of memory we would allow the denylisted partitions to take, we could potentially limit/throw warning based on memory as an alternate approach to limiting based on number of partition keys. Not a strong opinion here since we expect operators (and not end users) to use this feature and it should be safe either ways, given we expect operators to be informed. [EDIT] As I think through further, and as mentioned in CEP, it is probably best to log a warning if denylisted partitions exceed in either number of memory usage vs "limiting" them, in order to maintain consistency across all the nodes' caches in terms of what partitions are denylisted. 3. We could as an alternative, allow configuring denylisting specific operations (read vs write) per node per partition vs per node - I feel the way it is currently implemented (per node and not per node per partition) is less complex and easy to use and we should probably stick to that. We can revisit in the future if needed. 4. Curious on the choice of exposing consistency level for querying denylisting partitions. What do you think of a simpler approach doing "local execution" by each node when querying partitions_denylisted table. The benefit would be it can make it less complicated in terms of number of options we expose and not needing to check if we have enough nodes for the chosen consistency level. 5. Curious on why we disable denylisting range reads by default 6. Regarding the datatype for "key" in "system_distributed.partition_denylist", what do you think of choosing text (vs blob). Advantage would be readability of the key, and on cache refresh, we would convert it into ByteBuffer 7. What do you think about adding nodetool commands for a) refreshing denylist b) adding a partition to denylist c) removing a partition from denylist Happy to contribute to any of these, if decide to make any related changes > Add ability to blocklist / denylist a CQL partition so all requests
[jira] [Commented] (CASSANDRA-12106) Add ability to blocklist / denylist a CQL partition so all requests are ignored
[ https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419811#comment-17419811 ] Sumanth Pasupuleti commented on CASSANDRA-12106: [~jmckenzie] did a deeper pass, and I have following questions/suggestions 1. It maybe useful to add a documentation file around denylisting partitions (maybe along the lines of https://github.com/apache/cassandra/compare/trunk...sumanth-pasupuleti:feature/12106#diff-4264f2ac97017f73e4f6fff8f1cbef3082212b2edc531b57f6e8f064d3ab3e49) 2. With regards to limiting the amount of memory we would allow the denylisted partitions to take, we could potentially limit/throw warning based on memory as an alternate approach to limiting based on number of partition keys. Not a strong opinion here since we expect operators (and not end users) to use this feature and it should be safe either ways, given we expect operators to be informed. 3. We could as an alternative, allow configuring denylisting specific operations (read vs write) per node per partition vs per node - I feel the way it is currently implemented (per node and not per node per partition) is less complex and easy to use and we should probably stick to that. We can revisit in the future if needed. 4. Curious on the choice of exposing consistency level for querying denylisting partitions. What do you think of a simpler approach doing "local execution" by each node when querying partitions_denylisted table. The benefit would be it can make it less complicated in terms of number of options we expose and not needing to check if we have enough nodes for the chosen consistency level. 5. Curious on why we disable denylisting range reads by default 6. Regarding the datatype for "key" in "system_distributed.partition_denylist", what do you think of choosing text (vs blob). Advantage would be readability of the key, and on cache refresh, we would convert it into ByteBuffer 7. What do you think about adding nodetool commands for a) refreshing denylist b) adding a partition to denylist c) removing a partition from denylist Happy to contribute to any of these, if decide to make any related changes > Add ability to blocklist / denylist a CQL partition so all requests are > ignored > --- > > Key: CASSANDRA-12106 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12106 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Local Write-Read Paths, Local/Config >Reporter: Geoffrey Yu >Assignee: Sumanth Pasupuleti >Priority: Low > Fix For: 4.x > > Attachments: 12106-trunk.txt > > > Sometimes reads/writes to a given partition may cause problems due to the > data present. It would be useful to have a manual way to blocklist / denylist > such partitions so all read and write requests to them are rejected. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12106) Add ability to blocklist / denylist a CQL partition so all requests are ignored
[ https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17419455#comment-17419455 ] Sumanth Pasupuleti commented on CASSANDRA-12106: Thanks [~jmckenzie] for the PR. Did a quick pass and this PR is far more comprehensive than the patch I put out. +1 to the direction of moving ahead with your PR vs the patch I put out. Also, +1 to moving system_distributed from repair to schema package (didn't realize it is part of repair package!) and to refactoring system_distributed. I plan to do another pass of detailed review this week; will keep the ticket posted. > Add ability to blocklist / denylist a CQL partition so all requests are > ignored > --- > > Key: CASSANDRA-12106 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12106 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Local Write-Read Paths, Local/Config >Reporter: Geoffrey Yu >Assignee: Sumanth Pasupuleti >Priority: Low > Fix For: 4.x > > Attachments: 12106-trunk.txt > > > Sometimes reads/writes to a given partition may cause problems due to the > data present. It would be useful to have a manual way to blocklist / denylist > such partitions so all read and write requests to them are rejected. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14557) Consider adding default and required keyspace replication options
[
https://issues.apache.org/jira/browse/CASSANDRA-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17416166#comment-17416166
]
Sumanth Pasupuleti commented on CASSANDRA-14557:
[~stefan.miklosovic] Sorry for the temporary stall. Got pre-empted by a couple
of higher priority tasks. I am hoping to move this forward within a week or so.
> Consider adding default and required keyspace replication options
> -
>
> Key: CASSANDRA-14557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14557
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
> Labels: 4.0-feature-freeze-review-requested
> Fix For: 4.x
>
> Attachments: 14557-4.0.txt, 14557-trunk.patch
>
>
> Ending up with a keyspace of RF=1 is unfortunately pretty easy in C* right
> now - the system_auth table for example is created with RF=1 (to take into
> account single node setups afaict from CASSANDRA-5112), and a user can
> further create a keyspace with RF=1 posing availability and streaming risks
> (e.g. rebuild).
> I propose we add two configuration options in cassandra.yaml:
> # {{default_keyspace_rf}} (default: 1) - If replication factors are not
> specified, use this number.
> # {{required_minimum_keyspace_rf}} (default: unset) - Prevent users from
> creating a keyspace with an RF less than what is configured
> These settings could further be re-used to:
> * Provide defaults for new keyspaces created with SimpleStrategy or
> NetworkTopologyStrategy (CASSANDRA-14303)
> * Make the automatic token [allocation
> algorithm|https://issues.apache.org/jira/browse/CASSANDRA-13701?focusedCommentId=16095662=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16095662]
> interface more intuitive allowing easy use of the new token allocation
> algorithm.
> At the end of the day, if someone really wants to allow RF=1, they simply
> don’t set the setting. For backwards compatibility the default remains 1 and
> C* would create with RF=1, and would default to current behavior of allowing
> any RF on keyspaces.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16917) Create property to hold local maven repository location
[
https://issues.apache.org/jira/browse/CASSANDRA-16917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumanth Pasupuleti updated CASSANDRA-16917:
---
Fix Version/s: 4.0.x
3.11.x
3.0.x
> Create property to hold local maven repository location
> ---
>
> Key: CASSANDRA-16917
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16917
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> In their current state, both build.xml and .build/build-resolver.xml refer to
> the local maven repository by "${user.home}/.m2/repository" in multiple
> places.
> It would be nice instead to have a property store this location, and refer
> the property vs the actual location itself.
> If one needs to change this maven repository location for their environmental
> purposes, this change makes it easy since it will need to be changed only at
> one place (the value of the property) vs changing all the direct references
> to the repository location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16917) Create property to hold local maven repository location
[
https://issues.apache.org/jira/browse/CASSANDRA-16917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumanth Pasupuleti updated CASSANDRA-16917:
---
Component/s: Build
> Create property to hold local maven repository location
> ---
>
> Key: CASSANDRA-16917
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16917
> Project: Cassandra
> Issue Type: Improvement
> Components: Build
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Normal
>
> In their current state, both build.xml and .build/build-resolver.xml refer to
> the local maven repository by "${user.home}/.m2/repository" in multiple
> places.
> It would be nice instead to have a property store this location, and refer
> the property vs the actual location itself.
> If one needs to change this maven repository location for their environmental
> purposes, this change makes it easy since it will need to be changed only at
> one place (the value of the property) vs changing all the direct references
> to the repository location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16917) Create property to hold local maven repository location
[
https://issues.apache.org/jira/browse/CASSANDRA-16917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17409985#comment-17409985
]
Sumanth Pasupuleti commented on CASSANDRA-16917:
Patches
[Trunk|https://github.com/apache/cassandra/compare/trunk...sumanth-pasupuleti:feature/trunk_property_for_local_repository]
[4.0|https://github.com/apache/cassandra/compare/cassandra-4.0...sumanth-pasupuleti:feature/40_property_for_local_repository]
[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...sumanth-pasupuleti:feature/311_property_for_local_repository]
[3.0|https://github.com/apache/cassandra/compare/cassandra-3.11...sumanth-pasupuleti:feature/30_property_for_local_repository]
> Create property to hold local maven repository location
> ---
>
> Key: CASSANDRA-16917
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16917
> Project: Cassandra
> Issue Type: Improvement
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Normal
>
> In their current state, both build.xml and .build/build-resolver.xml refer to
> the local maven repository by "${user.home}/.m2/repository" in multiple
> places.
> It would be nice instead to have a property store this location, and refer
> the property vs the actual location itself.
> If one needs to change this maven repository location for their environmental
> purposes, this change makes it easy since it will need to be changed only at
> one place (the value of the property) vs changing all the direct references
> to the repository location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16917) Create property to hold local maven repository location
Sumanth Pasupuleti created CASSANDRA-16917:
--
Summary: Create property to hold local maven repository location
Key: CASSANDRA-16917
URL: https://issues.apache.org/jira/browse/CASSANDRA-16917
Project: Cassandra
Issue Type: Improvement
Reporter: Sumanth Pasupuleti
Assignee: Sumanth Pasupuleti
In their current state, both build.xml and .build/build-resolver.xml refer to
the local maven repository by "${user.home}/.m2/repository" in multiple places.
It would be nice instead to have a property store this location, and refer the
property vs the actual location itself.
If one needs to change this maven repository location for their environmental
purposes, this change makes it easy since it will need to be changed only at
one place (the value of the property) vs changing all the direct references to
the repository location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14557) Consider adding default and required keyspace replication options
[
https://issues.apache.org/jira/browse/CASSANDRA-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17406552#comment-17406552
]
Sumanth Pasupuleti commented on CASSANDRA-14557:
[~azotcsit] Thank you, responded to your comments and incorporated the
feedback. https://github.com/sumanth-pasupuleti/cassandra/tree/14557-trunk
> Consider adding default and required keyspace replication options
> -
>
> Key: CASSANDRA-14557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14557
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
> Labels: 4.0-feature-freeze-review-requested
> Fix For: 4.x
>
> Attachments: 14557-4.0.txt, 14557-trunk.patch
>
>
> Ending up with a keyspace of RF=1 is unfortunately pretty easy in C* right
> now - the system_auth table for example is created with RF=1 (to take into
> account single node setups afaict from CASSANDRA-5112), and a user can
> further create a keyspace with RF=1 posing availability and streaming risks
> (e.g. rebuild).
> I propose we add two configuration options in cassandra.yaml:
> # {{default_keyspace_rf}} (default: 1) - If replication factors are not
> specified, use this number.
> # {{required_minimum_keyspace_rf}} (default: unset) - Prevent users from
> creating a keyspace with an RF less than what is configured
> These settings could further be re-used to:
> * Provide defaults for new keyspaces created with SimpleStrategy or
> NetworkTopologyStrategy (CASSANDRA-14303)
> * Make the automatic token [allocation
> algorithm|https://issues.apache.org/jira/browse/CASSANDRA-13701?focusedCommentId=16095662=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16095662]
> interface more intuitive allowing easy use of the new token allocation
> algorithm.
> At the end of the day, if someone really wants to allow RF=1, they simply
> don’t set the setting. For backwards compatibility the default remains 1 and
> C* would create with RF=1, and would default to current behavior of allowing
> any RF on keyspaces.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14557) Consider adding default and required keyspace replication options
[
https://issues.apache.org/jira/browse/CASSANDRA-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404307#comment-17404307
]
Sumanth Pasupuleti edited comment on CASSANDRA-14557 at 8/25/21, 9:25 AM:
--
[~azotcsit] latest branch
https://github.com/sumanth-pasupuleti/cassandra/tree/14557-trunk reflects the
changes from your latest round of review, and you can find responses to your
comments at
https://github.com/sumanth-pasupuleti/cassandra/commit/139a01531f1b51f6b3b7dc005a7df929ec9409a5.
[~stefan.miklosovic] Good call out on CASSANDRA-16203. This does fit
philosophically and logically, however,
[fromMapWithDefaults|https://github.com/sumanth-pasupuleti/cassandra/blob/14557-trunk/src/java/org/apache/cassandra/schema/ReplicationParams.java#L92]
will have to accommodate to this new class. Will add a comment to
CASSANDRA-16203 if this gets committed before CASSANDRA-16203.
Also, added nodetool commands to override default rf and minimum rf. This is
still cumbersome to do it for each node, but yes it serves the purpose of
avoiding node restart. We may ideally want to put an endpoint in the
[sidecar|https://github.com/apache/cassandra-sidecar], that could potentially
change these configurations across the cluster.
Latest code at https://github.com/sumanth-pasupuleti/cassandra/tree/14557-trunk
was (Author: sumanth.pasupuleti):
[~azotcsit] latest branch
https://github.com/sumanth-pasupuleti/cassandra/tree/14557-trunk reflects the
changes from your latest round of review, and you can find responses to your
comments at
https://github.com/sumanth-pasupuleti/cassandra/commit/139a01531f1b51f6b3b7dc005a7df929ec9409a5.
[~stefan.miklosovic] Good call out on CASSANDRA-16203. This does fit
philosophically and logically, however,
[fromMapWithDefaults|https://github.com/sumanth-pasupuleti/cassandra/blob/14557-trunk/src/java/org/apache/cassandra/schema/ReplicationParams.java#L92]
will have to accommodate to this new class. Will add a comment to
CASSANDRA-16203 if this gets committed before CASSANDRA-16203.
Also, added nodetool commands to override default rf and minimum rf. This is
still cumbersome to do it for each node, but yes it serves the purpose of
avoiding node restart. We may ideally want to put an endpoint in the
[sidecar|https://github.com/apache/cassandra-sidecar], that could potentially
change these configurations across the cluster.
> Consider adding default and required keyspace replication options
> -
>
> Key: CASSANDRA-14557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14557
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
> Labels: 4.0-feature-freeze-review-requested
> Fix For: 4.x
>
> Attachments: 14557-4.0.txt, 14557-trunk.patch
>
>
> Ending up with a keyspace of RF=1 is unfortunately pretty easy in C* right
> now - the system_auth table for example is created with RF=1 (to take into
> account single node setups afaict from CASSANDRA-5112), and a user can
> further create a keyspace with RF=1 posing availability and streaming risks
> (e.g. rebuild).
> I propose we add two configuration options in cassandra.yaml:
> # {{default_keyspace_rf}} (default: 1) - If replication factors are not
> specified, use this number.
> # {{required_minimum_keyspace_rf}} (default: unset) - Prevent users from
> creating a keyspace with an RF less than what is configured
> These settings could further be re-used to:
> * Provide defaults for new keyspaces created with SimpleStrategy or
> NetworkTopologyStrategy (CASSANDRA-14303)
> * Make the automatic token [allocation
> algorithm|https://issues.apache.org/jira/browse/CASSANDRA-13701?focusedCommentId=16095662=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16095662]
> interface more intuitive allowing easy use of the new token allocation
> algorithm.
> At the end of the day, if someone really wants to allow RF=1, they simply
> don’t set the setting. For backwards compatibility the default remains 1 and
> C* would create with RF=1, and would default to current behavior of allowing
> any RF on keyspaces.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-14557) Consider adding default and required keyspace replication options
[
https://issues.apache.org/jira/browse/CASSANDRA-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404307#comment-17404307
]
Sumanth Pasupuleti edited comment on CASSANDRA-14557 at 8/25/21, 9:23 AM:
--
[~azotcsit] latest branch
https://github.com/sumanth-pasupuleti/cassandra/tree/14557-trunk reflects the
changes from your latest round of review, and you can find responses to your
comments at
https://github.com/sumanth-pasupuleti/cassandra/commit/139a01531f1b51f6b3b7dc005a7df929ec9409a5.
[~stefan.miklosovic] Good call out on CASSANDRA-16203. This does fit
philosophically and logically, however,
[fromMapWithDefaults|https://github.com/sumanth-pasupuleti/cassandra/blob/14557-trunk/src/java/org/apache/cassandra/schema/ReplicationParams.java#L92]
will have to accommodate to this new class. Will add a comment to
CASSANDRA-16203 if this gets committed before CASSANDRA-16203.
Also, added nodetool commands to override default rf and minimum rf. This is
still cumbersome to do it for each node, but yes it serves the purpose of
avoiding node restart. We may ideally want to put an endpoint in the
[sidecar|https://github.com/apache/cassandra-sidecar], that could potentially
change these configurations across the cluster.
was (Author: sumanth.pasupuleti):
[~azotcsit] latest branch
https://github.com/sumanth-pasupuleti/cassandra/tree/14557-trunk reflects the
changes from your latest round of review, and you can find responses to your
comments at
https://github.com/sumanth-pasupuleti/cassandra/commit/139a01531f1b51f6b3b7dc005a7df929ec9409a5.
[~stefan.miklosovic] Good call out on CASSANDRA-16203. This does fit
philosophically and logically, however, fromMapWithDefaults will have to
accommodate to this new class. Will add a comment to CASSANDRA-16203 if this
gets committed before CASSANDRA-16203.
Also, added nodetool commands to override default rf and minimum rf. This is
still cumbersome to do it for each node, but yes it serves the purpose of
avoiding node restart. We may ideally want to put an endpoint in the
[sidecar|https://github.com/apache/cassandra-sidecar], that could potentially
change these configurations across the cluster.
> Consider adding default and required keyspace replication options
> -
>
> Key: CASSANDRA-14557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14557
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
> Labels: 4.0-feature-freeze-review-requested
> Fix For: 4.x
>
> Attachments: 14557-4.0.txt, 14557-trunk.patch
>
>
> Ending up with a keyspace of RF=1 is unfortunately pretty easy in C* right
> now - the system_auth table for example is created with RF=1 (to take into
> account single node setups afaict from CASSANDRA-5112), and a user can
> further create a keyspace with RF=1 posing availability and streaming risks
> (e.g. rebuild).
> I propose we add two configuration options in cassandra.yaml:
> # {{default_keyspace_rf}} (default: 1) - If replication factors are not
> specified, use this number.
> # {{required_minimum_keyspace_rf}} (default: unset) - Prevent users from
> creating a keyspace with an RF less than what is configured
> These settings could further be re-used to:
> * Provide defaults for new keyspaces created with SimpleStrategy or
> NetworkTopologyStrategy (CASSANDRA-14303)
> * Make the automatic token [allocation
> algorithm|https://issues.apache.org/jira/browse/CASSANDRA-13701?focusedCommentId=16095662=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16095662]
> interface more intuitive allowing easy use of the new token allocation
> algorithm.
> At the end of the day, if someone really wants to allow RF=1, they simply
> don’t set the setting. For backwards compatibility the default remains 1 and
> C* would create with RF=1, and would default to current behavior of allowing
> any RF on keyspaces.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14557) Consider adding default and required keyspace replication options
[
https://issues.apache.org/jira/browse/CASSANDRA-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17404307#comment-17404307
]
Sumanth Pasupuleti commented on CASSANDRA-14557:
[~azotcsit] latest branch
https://github.com/sumanth-pasupuleti/cassandra/tree/14557-trunk reflects the
changes from your latest round of review, and you can find responses to your
comments at
https://github.com/sumanth-pasupuleti/cassandra/commit/139a01531f1b51f6b3b7dc005a7df929ec9409a5.
[~stefan.miklosovic] Good call out on CASSANDRA-16203. This does fit
philosophically and logically, however, fromMapWithDefaults will have to
accommodate to this new class. Will add a comment to CASSANDRA-16203 if this
gets committed before CASSANDRA-16203.
Also, added nodetool commands to override default rf and minimum rf. This is
still cumbersome to do it for each node, but yes it serves the purpose of
avoiding node restart. We may ideally want to put an endpoint in the
[sidecar|https://github.com/apache/cassandra-sidecar], that could potentially
change these configurations across the cluster.
> Consider adding default and required keyspace replication options
> -
>
> Key: CASSANDRA-14557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14557
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
> Labels: 4.0-feature-freeze-review-requested
> Fix For: 4.x
>
> Attachments: 14557-4.0.txt, 14557-trunk.patch
>
>
> Ending up with a keyspace of RF=1 is unfortunately pretty easy in C* right
> now - the system_auth table for example is created with RF=1 (to take into
> account single node setups afaict from CASSANDRA-5112), and a user can
> further create a keyspace with RF=1 posing availability and streaming risks
> (e.g. rebuild).
> I propose we add two configuration options in cassandra.yaml:
> # {{default_keyspace_rf}} (default: 1) - If replication factors are not
> specified, use this number.
> # {{required_minimum_keyspace_rf}} (default: unset) - Prevent users from
> creating a keyspace with an RF less than what is configured
> These settings could further be re-used to:
> * Provide defaults for new keyspaces created with SimpleStrategy or
> NetworkTopologyStrategy (CASSANDRA-14303)
> * Make the automatic token [allocation
> algorithm|https://issues.apache.org/jira/browse/CASSANDRA-13701?focusedCommentId=16095662=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16095662]
> interface more intuitive allowing easy use of the new token allocation
> algorithm.
> At the end of the day, if someone really wants to allow RF=1, they simply
> don’t set the setting. For backwards compatibility the default remains 1 and
> C* would create with RF=1, and would default to current behavior of allowing
> any RF on keyspaces.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16862) Fix flaky test DatabaseDescriptorRefTest
[
https://issues.apache.org/jira/browse/CASSANDRA-16862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17403078#comment-17403078
]
Sumanth Pasupuleti commented on CASSANDRA-16862:
fwiw, the test passes successfully for me, when I try it both from my IDE
(idea) and from command line.
> Fix flaky test DatabaseDescriptorRefTest
>
>
> Key: CASSANDRA-16862
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16862
> Project: Cassandra
> Issue Type: Bug
> Components: CI
>Reporter: Ekaterina Dimitrova
>Priority: Normal
> Fix For: 3.11.x, 4.0.x, 4.x
>
>
> While working on another ticket I found out that DatabaseDescriptorRefTest is
> failing consistently locally for me and one more community member on 3.11,
> 4.0 and trunk.
> {code:java}
> java.lang.AssertionError: thread started in clientInitialization
> Expected :5
> Actual :8
>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:834)
> at org.junit.Assert.assertEquals(Assert.java:645)
> at
> org.apache.cassandra.config.DatabaseDescriptorRefTest.testDatabaseDescriptorRef(DatabaseDescriptorRefTest.java:285)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> at
> com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
> at
> com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
> at
> com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:221)
> at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15433) Pending ranges are not recalculated on keyspace creation
[
https://issues.apache.org/jira/browse/CASSANDRA-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumanth Pasupuleti updated CASSANDRA-15433:
---
Status: Patch Available (was: In Progress)
> Pending ranges are not recalculated on keyspace creation
>
>
> Key: CASSANDRA-15433
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15433
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Membership
>Reporter: Josh Snyder
>Assignee: Sumanth Pasupuleti
>Priority: Normal
>
> When a node begins bootstrapping, Cassandra recalculates pending tokens for
> each keyspace that exists when the state change is observed (in
> StorageService:handleState*). When new keyspaces are created, we do not
> recalculate pending ranges (around Schema:merge). As a result, writes for new
> keyspaces are not received by nodes in BOOT or BOOT_REPLACE modes. When
> bootstrapping finishes, the node which just bootstrapped will not have data
> for the newly created keyspace.
> Consider a ring with bootstrapped nodes A, B, and C. Node D is pending, and
> when it finishes bootstrapping, C will cede ownership of some ranges to D. A
> quorum write is acknowledged by C and A. B missed the write, and the
> coordinator didn't send it to D at all. When D finishes bootstrapping, the
> quorum B+D will not contain the mutation.
> Steps to reproduce:
> # Join a node in BOOT mode
> # Create a keyspace
> # Send writes to that keyspace
> # On the joining node, observe that {{nodetool cfstats}} records zero writes
> to the new keyspace
> I have observed this directly in Cassandra 3.0, and based on my reading the
> code, I believe it affects up through trunk.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15433) Pending ranges are not recalculated on keyspace creation
[
https://issues.apache.org/jira/browse/CASSANDRA-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17403070#comment-17403070
]
Sumanth Pasupuleti commented on CASSANDRA-15433:
[~ifesdjeen] I was finally able to get back to making progress on this ticket.
Please find the updated patches below that include in-jvm tests for this
scenario.
[Trunk
changes|https://github.com/apache/cassandra/compare/trunk...sumanth-pasupuleti:bugfix/trunk_15433?expand=1]
[3.0
changes|https://github.com/apache/cassandra/compare/cassandra-3.0...sumanth-pasupuleti:bugfix/30_15433?expand=1]
> Pending ranges are not recalculated on keyspace creation
>
>
> Key: CASSANDRA-15433
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15433
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Membership
>Reporter: Josh Snyder
>Assignee: Sumanth Pasupuleti
>Priority: Normal
>
> When a node begins bootstrapping, Cassandra recalculates pending tokens for
> each keyspace that exists when the state change is observed (in
> StorageService:handleState*). When new keyspaces are created, we do not
> recalculate pending ranges (around Schema:merge). As a result, writes for new
> keyspaces are not received by nodes in BOOT or BOOT_REPLACE modes. When
> bootstrapping finishes, the node which just bootstrapped will not have data
> for the newly created keyspace.
> Consider a ring with bootstrapped nodes A, B, and C. Node D is pending, and
> when it finishes bootstrapping, C will cede ownership of some ranges to D. A
> quorum write is acknowledged by C and A. B missed the write, and the
> coordinator didn't send it to D at all. When D finishes bootstrapping, the
> quorum B+D will not contain the mutation.
> Steps to reproduce:
> # Join a node in BOOT mode
> # Create a keyspace
> # Send writes to that keyspace
> # On the joining node, observe that {{nodetool cfstats}} records zero writes
> to the new keyspace
> I have observed this directly in Cassandra 3.0, and based on my reading the
> code, I believe it affects up through trunk.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15433) Pending ranges are not recalculated on keyspace creation
[
https://issues.apache.org/jira/browse/CASSANDRA-15433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumanth Pasupuleti updated CASSANDRA-15433:
---
Status: In Progress (was: Changes Suggested)
> Pending ranges are not recalculated on keyspace creation
>
>
> Key: CASSANDRA-15433
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15433
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Membership
>Reporter: Josh Snyder
>Assignee: Sumanth Pasupuleti
>Priority: Normal
>
> When a node begins bootstrapping, Cassandra recalculates pending tokens for
> each keyspace that exists when the state change is observed (in
> StorageService:handleState*). When new keyspaces are created, we do not
> recalculate pending ranges (around Schema:merge). As a result, writes for new
> keyspaces are not received by nodes in BOOT or BOOT_REPLACE modes. When
> bootstrapping finishes, the node which just bootstrapped will not have data
> for the newly created keyspace.
> Consider a ring with bootstrapped nodes A, B, and C. Node D is pending, and
> when it finishes bootstrapping, C will cede ownership of some ranges to D. A
> quorum write is acknowledged by C and A. B missed the write, and the
> coordinator didn't send it to D at all. When D finishes bootstrapping, the
> quorum B+D will not contain the mutation.
> Steps to reproduce:
> # Join a node in BOOT mode
> # Create a keyspace
> # Send writes to that keyspace
> # On the joining node, observe that {{nodetool cfstats}} records zero writes
> to the new keyspace
> I have observed this directly in Cassandra 3.0, and based on my reading the
> code, I believe it affects up through trunk.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16203) Rack Aware Topology Strategy
[ https://issues.apache.org/jira/browse/CASSANDRA-16203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17402059#comment-17402059 ] Sumanth Pasupuleti commented on CASSANDRA-16203: +1 on the nice by-product of architectural documentation through CEPs :) [~cam1982] Below are my review comments 1. It maybe useful to declare a String constant to store the "replication_factor" value, given it is used 4 times. 2. In the ConfigurationException string, did you mean to indicate "RackAwareTopologyStrategy" instead of "NetworkTopologyStrategy" 3. It can be useful to add comments explaining the parameters for constructor of DatacenterEndpoints 4. I believe there is a redundant "return done()" statement at the end of the if block "if (rackReplicaCount < replicasPerRack)" inside the method "boolean addEndpointAndCheckIfDone" 5. Rebase is needed since AbstractReplicationStrategy requires implementing calculateNaturalReplicas() vs calculateNaturalEndpoints(),and the latest code uses InetAddressAndPort vs InetAddress 6. It would also be nice to add description to RackAwareTopologyStrategy in terms of how it varies from NetworkTopologyStrategy 7. You may want to add testcases against RackAwareTopologyStrategy > Rack Aware Topology Strategy > > > Key: CASSANDRA-16203 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16203 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Membership >Reporter: Cameron Zemek >Assignee: Cameron Zemek >Priority: Normal > Fix For: 4.x > > Attachments: rackaware.patch > > > If replication factor > racks then NetworkTopologyStrategy assigns the extras > in ring order. This means losing a rack can result in the loss of quorum. I > have implemented a similar topology strategy that evenly distributes replicas > across racks. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16203) Rack Aware Topology Strategy
[ https://issues.apache.org/jira/browse/CASSANDRA-16203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17401880#comment-17401880 ] Sumanth Pasupuleti commented on CASSANDRA-16203: I was just curious if we needed a CEP, but I do agree, given that the strategy is pluggable, we should be fine without a CEP. > Rack Aware Topology Strategy > > > Key: CASSANDRA-16203 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16203 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Membership >Reporter: Cameron Zemek >Assignee: Cameron Zemek >Priority: Normal > Fix For: 4.x > > Attachments: rackaware.patch > > > If replication factor > racks then NetworkTopologyStrategy assigns the extras > in ring order. This means losing a rack can result in the loss of quorum. I > have implemented a similar topology strategy that evenly distributes replicas > across racks. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12106) Add ability to blacklist a CQL partition so all requests are ignored
[ https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17399912#comment-17399912 ] Sumanth Pasupuleti commented on CASSANDRA-12106: Rebased and reworded. Updated patch: https://github.com/apache/cassandra/compare/trunk...sumanth-pasupuleti:feature/12106 > Add ability to blacklist a CQL partition so all requests are ignored > > > Key: CASSANDRA-12106 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12106 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Local Write-Read Paths, Local/Config >Reporter: Geoffrey Yu >Assignee: Sumanth Pasupuleti >Priority: Low > Fix For: 4.x > > Attachments: 12106-trunk.txt > > > Sometimes reads/writes to a given partition may cause problems due to the > data present. It would be useful to have a manual way to blacklist such > partitions so all read and write requests to them are rejected. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14557) Consider adding default and required keyspace replication options
[
https://issues.apache.org/jira/browse/CASSANDRA-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17397530#comment-17397530
]
Sumanth Pasupuleti commented on CASSANDRA-14557:
[~azotcsit] Thanks for the review. I put my responses to the comments and
incorporated some of the feedback and updated the branch
https://github.com/sumanth-pasupuleti/cassandra/tree/14557-trunk
> Consider adding default and required keyspace replication options
> -
>
> Key: CASSANDRA-14557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14557
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
> Labels: 4.0-feature-freeze-review-requested
> Fix For: 4.x
>
> Attachments: 14557-4.0.txt, 14557-trunk.patch
>
>
> Ending up with a keyspace of RF=1 is unfortunately pretty easy in C* right
> now - the system_auth table for example is created with RF=1 (to take into
> account single node setups afaict from CASSANDRA-5112), and a user can
> further create a keyspace with RF=1 posing availability and streaming risks
> (e.g. rebuild).
> I propose we add two configuration options in cassandra.yaml:
> # {{default_keyspace_rf}} (default: 1) - If replication factors are not
> specified, use this number.
> # {{required_minimum_keyspace_rf}} (default: unset) - Prevent users from
> creating a keyspace with an RF less than what is configured
> These settings could further be re-used to:
> * Provide defaults for new keyspaces created with SimpleStrategy or
> NetworkTopologyStrategy (CASSANDRA-14303)
> * Make the automatic token [allocation
> algorithm|https://issues.apache.org/jira/browse/CASSANDRA-13701?focusedCommentId=16095662=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16095662]
> interface more intuitive allowing easy use of the new token allocation
> algorithm.
> At the end of the day, if someone really wants to allow RF=1, they simply
> don’t set the setting. For backwards compatibility the default remains 1 and
> C* would create with RF=1, and would default to current behavior of allowing
> any RF on keyspaces.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16203) Rack Aware Topology Strategy
[ https://issues.apache.org/jira/browse/CASSANDRA-16203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17397413#comment-17397413 ] Sumanth Pasupuleti commented on CASSANDRA-16203: [~cam1982] I think this may qualify for getting a CEP done. [~paulo] thoughts? > Rack Aware Topology Strategy > > > Key: CASSANDRA-16203 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16203 > Project: Cassandra > Issue Type: Improvement > Components: Cluster/Membership >Reporter: Cameron Zemek >Assignee: Cameron Zemek >Priority: Normal > Fix For: 5.x > > Attachments: rackaware.patch > > > If replication factor > racks then NetworkTopologyStrategy assigns the extras > in ring order. This means losing a rack can result in the loss of quorum. I > have implemented a similar topology strategy that evenly distributes replicas > across racks. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16467) speculative retry should allow more friendly params, allowing upgrade from 2.x not to break
[ https://issues.apache.org/jira/browse/CASSANDRA-16467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380869#comment-17380869 ] Sumanth Pasupuleti commented on CASSANDRA-16467: Updated PRs (after rebase) *3.0.x* [Patch|https://github.com/apache/cassandra/compare/cassandra-3.0...sumanth-pasupuleti:bugfix/30_speculative_retry_params_case] UTs and Dtests passing except for three DTests that seem unrelated to me (test_sstableverify, test_simple_rebuild and test_closing_connections) [Circle CI run|https://app.circleci.com/pipelines/github/sumanth-pasupuleti/cassandra/79/workflows/b6f2cd11-d08d-46ef-acf8-6fd0ccf0d61e] 3.11.x [Patch|https://github.com/apache/cassandra/compare/cassandra-3.11...sumanth-pasupuleti:bugfix/311_speculative_retry_params_case] UTs and Dtests passing except for three DTests that seem unrelated to me (test_sstableverify, test_view_metadata_cleanup and test_closing_connections) [Circle CI run|https://app.circleci.com/pipelines/github/sumanth-pasupuleti/cassandra/81/workflows/6c45519e-1120-4fa5-8a72-3e465a5467cb] > speculative retry should allow more friendly params, allowing upgrade from > 2.x not to break > --- > > Key: CASSANDRA-16467 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16467 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Sumanth Pasupuleti >Assignee: Sumanth Pasupuleti >Priority: Normal > Fix For: 3.0.x, 3.11.x > > > 2.x speculative retry params are case insensitive, while 3.0 and 3.11 have > added case sensitivity. As as result of this, one of our internal > applications suffered an issue during > C* upgrade from 2.x to 3.0. > This ticket is to propose making 3.0 and 3.11 speculative_retry params case > insensitive as well (essentially a slightly modified backport of > CASSANDRA-13876, but not to allow something like "99p" which 4.0 allows) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16467) speculative retry should allow more friendly params, allowing upgrade from 2.x not to break
[ https://issues.apache.org/jira/browse/CASSANDRA-16467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380869#comment-17380869 ] Sumanth Pasupuleti edited comment on CASSANDRA-16467 at 7/14/21, 9:18 PM: -- Updated PRs (after rebase) 3.0.x [Patch|https://github.com/apache/cassandra/compare/cassandra-3.0...sumanth-pasupuleti:bugfix/30_speculative_retry_params_case] UTs and Dtests passing except for three DTests that seem unrelated to me (test_sstableverify, test_simple_rebuild and test_closing_connections) [Circle CI run|https://app.circleci.com/pipelines/github/sumanth-pasupuleti/cassandra/79/workflows/b6f2cd11-d08d-46ef-acf8-6fd0ccf0d61e] 3.11.x [Patch|https://github.com/apache/cassandra/compare/cassandra-3.11...sumanth-pasupuleti:bugfix/311_speculative_retry_params_case] UTs and Dtests passing except for three DTests that seem unrelated to me (test_sstableverify, test_view_metadata_cleanup and test_closing_connections) [Circle CI run|https://app.circleci.com/pipelines/github/sumanth-pasupuleti/cassandra/81/workflows/6c45519e-1120-4fa5-8a72-3e465a5467cb] was (Author: sumanth.pasupuleti): Updated PRs (after rebase) *3.0.x* [Patch|https://github.com/apache/cassandra/compare/cassandra-3.0...sumanth-pasupuleti:bugfix/30_speculative_retry_params_case] UTs and Dtests passing except for three DTests that seem unrelated to me (test_sstableverify, test_simple_rebuild and test_closing_connections) [Circle CI run|https://app.circleci.com/pipelines/github/sumanth-pasupuleti/cassandra/79/workflows/b6f2cd11-d08d-46ef-acf8-6fd0ccf0d61e] 3.11.x [Patch|https://github.com/apache/cassandra/compare/cassandra-3.11...sumanth-pasupuleti:bugfix/311_speculative_retry_params_case] UTs and Dtests passing except for three DTests that seem unrelated to me (test_sstableverify, test_view_metadata_cleanup and test_closing_connections) [Circle CI run|https://app.circleci.com/pipelines/github/sumanth-pasupuleti/cassandra/81/workflows/6c45519e-1120-4fa5-8a72-3e465a5467cb] > speculative retry should allow more friendly params, allowing upgrade from > 2.x not to break > --- > > Key: CASSANDRA-16467 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16467 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Sumanth Pasupuleti >Assignee: Sumanth Pasupuleti >Priority: Normal > Fix For: 3.0.x, 3.11.x > > > 2.x speculative retry params are case insensitive, while 3.0 and 3.11 have > added case sensitivity. As as result of this, one of our internal > applications suffered an issue during > C* upgrade from 2.x to 3.0. > This ticket is to propose making 3.0 and 3.11 speculative_retry params case > insensitive as well (essentially a slightly modified backport of > CASSANDRA-13876, but not to allow something like "99p" which 4.0 allows) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16467) speculative retry should allow more friendly params, allowing upgrade from 2.x not to break
[ https://issues.apache.org/jira/browse/CASSANDRA-16467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17380855#comment-17380855 ] Sumanth Pasupuleti commented on CASSANDRA-16467: Thanks for the review [~azotcsit], [~maedhroz] and [~e.dimitrova]. I have rebased; CI runs are in progress - will update the ticket once the runs complete. > speculative retry should allow more friendly params, allowing upgrade from > 2.x not to break > --- > > Key: CASSANDRA-16467 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16467 > Project: Cassandra > Issue Type: Bug > Components: Local/Config >Reporter: Sumanth Pasupuleti >Assignee: Sumanth Pasupuleti >Priority: Normal > Fix For: 3.0.x, 3.11.x > > > 2.x speculative retry params are case insensitive, while 3.0 and 3.11 have > added case sensitivity. As as result of this, one of our internal > applications suffered an issue during > C* upgrade from 2.x to 3.0. > This ticket is to propose making 3.0 and 3.11 speculative_retry params case > insensitive as well (essentially a slightly modified backport of > CASSANDRA-13876, but not to allow something like "99p" which 4.0 allows) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12106) Add ability to blacklist a CQL partition so all requests are ignored
[ https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17377605#comment-17377605 ] Sumanth Pasupuleti commented on CASSANDRA-12106: Thanks for the reference to CASSANDRA-15862. Will work on rebasing and rewording. > Add ability to blacklist a CQL partition so all requests are ignored > > > Key: CASSANDRA-12106 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12106 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Local Write-Read Paths, Local/Config >Reporter: Geoffrey Yu >Assignee: Sumanth Pasupuleti >Priority: Low > Fix For: 4.x > > Attachments: 12106-trunk.txt > > > Sometimes reads/writes to a given partition may cause problems due to the > data present. It would be useful to have a manual way to blacklist such > partitions so all read and write requests to them are rejected. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16669) Password obfuscation for DCL audit log statements
[
https://issues.apache.org/jira/browse/CASSANDRA-16669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361958#comment-17361958
]
Sumanth Pasupuleti commented on CASSANDRA-16669:
I am out due to emergency. [~vinaykumarcse] / [~stefan.miklosovic] /
[~e.dimitrova] please take the patch forward, thank you!
> Password obfuscation for DCL audit log statements
> -
>
> Key: CASSANDRA-16669
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16669
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/auditlogging
>Reporter: Vinay Chella
>Assignee: Sumanth Pasupuleti
>Priority: Normal
> Labels: audit, security
> Fix For: 4.0-rc, 4.0.x, 4.x
>
> Time Spent: 5h 10m
> Remaining Estimate: 0h
>
> The goal of this JIRA is to obfuscate passwords or any sensitive information
> from DCL audit log statements.
> Currently, (Cassandra version 4.0-rc1) logs query statements for any DCL
> ([ROLE|https://cassandra.apache.org/doc/latest/cql/security.html#database-roles]
> and [USER|https://cassandra.apache.org/doc/latest/cql/security.html#users] )
> queries with passwords in plaintext format in audit log files.
> The current workaround to avoid plain text passwords from being logged in
> audit log files is either by
> [excluding|https://cassandra.apache.org/doc/latest/operating/audit_logging.html#options]
> DCL statements from auditing or by excluding the user who is creating these
> roles from auditing.
> It would be ideal for Cassandra to provide an option or default to obfuscate
> passwords or any sensitive information from DCL audit log statements.
> Sample audit logs with DCL queries
> {code:sh}
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190499676|type:CREATE_ROLE|category:DCL|operation:CREATE
> ROLE new_role;
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190505313|type:CREATE_ROLE|category:DCL|operation:CREATE
> ROLE alice WITH PASSWORD = 'password_a' AND LOGIN = true;
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190519521|type:REQUEST_FAILURE|category:ERROR|operation:ALTER
> ROLE bob WITH PASSWORD = 'PASSWORD_B' AND SUPERUSER = false;; bob doesn't
> exist
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190525376|type:CREATE_ROLE|category:DCL|operation:CREATE
> ROLE bob WITH PASSWORD = 'password_b' AND LOGIN = true AND SUPERUSER = true;
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190532462|type:ALTER_ROLE|category:DCL|operation:ALTER
> ROLE bob WITH PASSWORD = 'PASSWORD_B' AND SUPERUSER = false;
> {code}
> It is also ideal to document this workaround or assumption in Cassandra audit
> log documentation until we close this JIRA
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16732) Unable to replace node if cluster is in mixed major version
[
https://issues.apache.org/jira/browse/CASSANDRA-16732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361957#comment-17361957
]
Sumanth Pasupuleti edited comment on CASSANDRA-16732 at 6/11/21, 6:35 PM:
--
Yeah, I also realized I was wrong when I mentioned C* on this cluster includes
fix for CASSANDRA-16692. My bad. We can close this.
was (Author: sumanth.pasupuleti):
Yeah, I also realized I was wrong when I mentioned C* on this cluster includes
fix for CASSANDRA-16692. My bad. Closing this.
> Unable to replace node if cluster is in mixed major version
> ---
>
> Key: CASSANDRA-16732
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16732
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sumanth Pasupuleti
>Priority: Normal
> Fix For: 3.0.x
>
>
> This should be independent of cluster size, but in my example repro, I have a
> 6 node cluster where 5 nodes are on 3.0, one node is on 2.1, and replacing
> any of the 3.0 nodes (such that new node bootstraps from neighbors) fails
> during the bootstrap phase of the new node with the below exception.
> This version of C* includes fix for CASSANDRA-16692.
> {code:java}
> ERROR [main] 2021-06-11 07:47:36,500 CassandraDaemon.java:822 - Exception
> encountered during startup
> java.lang.RuntimeException: Didn't receive schemas for all known versions
> within the timeout. Use -Dcassandra.skip_schema_check=true to skip this check.
> at
> org.apache.cassandra.service.StorageService.waitForSchema(StorageService.java:909)
> ~[nf-cassandra-3.0.25.1.jar:3.0.25.1]
> at
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:960)
> ~[nf-cassandra-3.0.25.1.jar:3.0.25.1]
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:751)
> ~[nf-cassandra-3.0.25.1.jar:3.0.25.1]
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:659)
> ~[nf-cassandra-3.0.25.1.jar:3.0.25.1]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:374)
> [nf-cassandra-3.0.25.1.jar:3.0.25.1]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:616)
> [nf-cassandra-3.0.25.1.jar:3.0.25.1]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:809)
> [nf-cassandra-3.0.25.1.jar:3.0.25.1]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16732) Unable to replace node if cluster is in mixed major version
[
https://issues.apache.org/jira/browse/CASSANDRA-16732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17361957#comment-17361957
]
Sumanth Pasupuleti commented on CASSANDRA-16732:
Yeah, I also realized I was wrong when I mentioned C* on this cluster includes
fix for CASSANDRA-16692. My bad. Closing this.
> Unable to replace node if cluster is in mixed major version
> ---
>
> Key: CASSANDRA-16732
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16732
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sumanth Pasupuleti
>Priority: Normal
> Fix For: 3.0.x
>
>
> This should be independent of cluster size, but in my example repro, I have a
> 6 node cluster where 5 nodes are on 3.0, one node is on 2.1, and replacing
> any of the 3.0 nodes (such that new node bootstraps from neighbors) fails
> during the bootstrap phase of the new node with the below exception.
> This version of C* includes fix for CASSANDRA-16692.
> {code:java}
> ERROR [main] 2021-06-11 07:47:36,500 CassandraDaemon.java:822 - Exception
> encountered during startup
> java.lang.RuntimeException: Didn't receive schemas for all known versions
> within the timeout. Use -Dcassandra.skip_schema_check=true to skip this check.
> at
> org.apache.cassandra.service.StorageService.waitForSchema(StorageService.java:909)
> ~[nf-cassandra-3.0.25.1.jar:3.0.25.1]
> at
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:960)
> ~[nf-cassandra-3.0.25.1.jar:3.0.25.1]
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:751)
> ~[nf-cassandra-3.0.25.1.jar:3.0.25.1]
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:659)
> ~[nf-cassandra-3.0.25.1.jar:3.0.25.1]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:374)
> [nf-cassandra-3.0.25.1.jar:3.0.25.1]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:616)
> [nf-cassandra-3.0.25.1.jar:3.0.25.1]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:809)
> [nf-cassandra-3.0.25.1.jar:3.0.25.1]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16732) Unable to replace node if cluster is in mixed major version
Sumanth Pasupuleti created CASSANDRA-16732:
--
Summary: Unable to replace node if cluster is in mixed major
version
Key: CASSANDRA-16732
URL: https://issues.apache.org/jira/browse/CASSANDRA-16732
Project: Cassandra
Issue Type: Bug
Reporter: Sumanth Pasupuleti
This should be independent of cluster size, but in my example repro, I have a 6
node cluster where 5 nodes are on 3.0, one node is on 2.1, and replacing any of
the 3.0 nodes (such that new node bootstraps from neighbors) fails during the
bootstrap phase of the new node with the below exception.
This version of C* includes fix for CASSANDRA-16692.
{code:java}
ERROR [main] 2021-06-11 07:47:36,500 CassandraDaemon.java:822 - Exception
encountered during startup
java.lang.RuntimeException: Didn't receive schemas for all known versions
within the timeout. Use -Dcassandra.skip_schema_check=true to skip this check.
at
org.apache.cassandra.service.StorageService.waitForSchema(StorageService.java:909)
~[nf-cassandra-3.0.25.1.jar:3.0.25.1]
at
org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:960)
~[nf-cassandra-3.0.25.1.jar:3.0.25.1]
at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:751)
~[nf-cassandra-3.0.25.1.jar:3.0.25.1]
at
org.apache.cassandra.service.StorageService.initServer(StorageService.java:659)
~[nf-cassandra-3.0.25.1.jar:3.0.25.1]
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:374)
[nf-cassandra-3.0.25.1.jar:3.0.25.1]
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:616)
[nf-cassandra-3.0.25.1.jar:3.0.25.1]
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:809)
[nf-cassandra-3.0.25.1.jar:3.0.25.1]
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16732) Unable to replace node if cluster is in mixed major version
[
https://issues.apache.org/jira/browse/CASSANDRA-16732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumanth Pasupuleti updated CASSANDRA-16732:
---
Fix Version/s: 3.0.x
> Unable to replace node if cluster is in mixed major version
> ---
>
> Key: CASSANDRA-16732
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16732
> Project: Cassandra
> Issue Type: Bug
>Reporter: Sumanth Pasupuleti
>Priority: Normal
> Fix For: 3.0.x
>
>
> This should be independent of cluster size, but in my example repro, I have a
> 6 node cluster where 5 nodes are on 3.0, one node is on 2.1, and replacing
> any of the 3.0 nodes (such that new node bootstraps from neighbors) fails
> during the bootstrap phase of the new node with the below exception.
> This version of C* includes fix for CASSANDRA-16692.
> {code:java}
> ERROR [main] 2021-06-11 07:47:36,500 CassandraDaemon.java:822 - Exception
> encountered during startup
> java.lang.RuntimeException: Didn't receive schemas for all known versions
> within the timeout. Use -Dcassandra.skip_schema_check=true to skip this check.
> at
> org.apache.cassandra.service.StorageService.waitForSchema(StorageService.java:909)
> ~[nf-cassandra-3.0.25.1.jar:3.0.25.1]
> at
> org.apache.cassandra.service.StorageService.joinTokenRing(StorageService.java:960)
> ~[nf-cassandra-3.0.25.1.jar:3.0.25.1]
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:751)
> ~[nf-cassandra-3.0.25.1.jar:3.0.25.1]
> at
> org.apache.cassandra.service.StorageService.initServer(StorageService.java:659)
> ~[nf-cassandra-3.0.25.1.jar:3.0.25.1]
> at
> org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:374)
> [nf-cassandra-3.0.25.1.jar:3.0.25.1]
> at
> org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:616)
> [nf-cassandra-3.0.25.1.jar:3.0.25.1]
> at
> org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:809)
> [nf-cassandra-3.0.25.1.jar:3.0.25.1]
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16719) Allow for transitioning from using one SSLContextFactory implementation to another
[ https://issues.apache.org/jira/browse/CASSANDRA-16719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumanth Pasupuleti updated CASSANDRA-16719: --- Summary: Allow for transitioning from using one SSLContextFactory implementation to another (was: Allow for transitioning from using one SSLFactory implementation to another) > Allow for transitioning from using one SSLContextFactory implementation to > another > -- > > Key: CASSANDRA-16719 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16719 > Project: Cassandra > Issue Type: New Feature >Reporter: Sumanth Pasupuleti >Assignee: Sumanth Pasupuleti >Priority: Normal > > With CASSANDRA-1 providing pluggable SSLContext, this ticket is to > provide mechanics for being able to transparently transition from using one > SSLContext implementation to another. > As indicated in > https://issues.apache.org/jira/browse/CASSANDRA-1?focusedCommentId=17358655=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17358655, > one could do the following on their cluster for moving from say > Implementation1 to Implementation2 > Stage #1: Current state of being only Implementation1 aware. Use keystore and > trustmanager of implementation1 > Stage #2: Start trusting both implementation1 and implementation2. Use > keystore of implementation1, but use trustmanager of both implementation1 and > implementation2 (using MultiTrustManagerFactory) (and perform a rolling > restart of the cluster) > Stage #3: Start using implementation2 for keystore, and perform a rolling > restart of the cluster > Stage #4: At this point, all nodes of the cluster are using implementation2 > for keystore, but trust both implementation1 and implementation2, and we can > now remove implementation1 from trustmanagers, and do a rolling restart. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16719) Allow for transitioning from using one SSLFactory implementation to another
[ https://issues.apache.org/jira/browse/CASSANDRA-16719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumanth Pasupuleti updated CASSANDRA-16719: --- Description: With CASSANDRA-1 providing pluggable SSLContext, this ticket is to provide mechanics for being able to transparently transition from using one SSLContext implementation to another. As indicated in https://issues.apache.org/jira/browse/CASSANDRA-1?focusedCommentId=17358655=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17358655, one could do the following on their cluster for moving from say Implementation1 to Implementation2 Stage #1: Current state of being only Implementation1 aware. Use keystore and trustmanager of implementation1 Stage #2: Start trusting both implementation1 and implementation2. Use keystore of implementation1, but use trustmanager of both implementation1 and implementation2 (using MultiTrustManagerFactory) (and perform a rolling restart of the cluster) Stage #3: Start using implementation2 for keystore, and perform a rolling restart of the cluster Stage #4: At this point, all nodes of the cluster are using implementation2 for keystore, but trust both implementation1 and implementation2, and we can now remove implementation1 from trustmanagers, and do a rolling restart. was:With CASSANDRA-1 providing pluggable SSLContext, this ticket is to provide mechanics for being able to transparently transition from using one SSLContext implementation to another. > Allow for transitioning from using one SSLFactory implementation to another > --- > > Key: CASSANDRA-16719 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16719 > Project: Cassandra > Issue Type: New Feature >Reporter: Sumanth Pasupuleti >Assignee: Sumanth Pasupuleti >Priority: Normal > > With CASSANDRA-1 providing pluggable SSLContext, this ticket is to > provide mechanics for being able to transparently transition from using one > SSLContext implementation to another. > As indicated in > https://issues.apache.org/jira/browse/CASSANDRA-1?focusedCommentId=17358655=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17358655, > one could do the following on their cluster for moving from say > Implementation1 to Implementation2 > Stage #1: Current state of being only Implementation1 aware. Use keystore and > trustmanager of implementation1 > Stage #2: Start trusting both implementation1 and implementation2. Use > keystore of implementation1, but use trustmanager of both implementation1 and > implementation2 (using MultiTrustManagerFactory) (and perform a rolling > restart of the cluster) > Stage #3: Start using implementation2 for keystore, and perform a rolling > restart of the cluster > Stage #4: At this point, all nodes of the cluster are using implementation2 > for keystore, but trust both implementation1 and implementation2, and we can > now remove implementation1 from trustmanagers, and do a rolling restart. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16666) Make SSLContext creation pluggable/extensible
[ https://issues.apache.org/jira/browse/CASSANDRA-1?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17358655#comment-17358655 ] Sumanth Pasupuleti commented on CASSANDRA-1: [~maulin.vasavada] left a couple of review comments, but lgtm overall. One of the things I was hoping we can also achieve is to be able to provide mechanics to transparently transition from using one SSLFactory implementation to another, and using those mechanics, one could do the following on their cluster for moving from say Implementation1 to Implementation2 Stage #1: Current state of being only Implementation1 aware. Use keystore and trustmanager of implementation1 Stage #2: Start trusting both implementation1 and implementation2. Use keystore of implementation1, but use trustmanager of both implementation1 and implementation2 (using MultiTrustManagerFactory) (and perform a rolling restart of the cluster) Stage #3: Start using implementation2 for keystore, and perform a rolling restart of the cluster Stage #4: At this point, all nodes of the cluster are using implementation2 for keystore, but trust both implementation1 and implementation2, and we can now remove implementation1 from trustmanagers, and do a rolling restart. Since this ticket is about making SSLContext pluggable, I believe this is out of scope; cut a separate ticket CASSANDRA-16719 to track that work (I did an internal 3.0 patch for this transition work, and I can try porting it to 4.x once this ticket is committed) > Make SSLContext creation pluggable/extensible > - > > Key: CASSANDRA-1 > URL: https://issues.apache.org/jira/browse/CASSANDRA-1 > Project: Cassandra > Issue Type: Improvement > Components: Messaging/Internode >Reporter: Maulin Vasavada >Assignee: Maulin Vasavada >Priority: Normal > Fix For: 4.x > > > Currently Cassandra creates the SSLContext via SSLFactory.java. SSLFactory is > a final class with static methods and not overridable. The SSLFactory loads > the keys and certs from the file based artifacts for the same. While this > works for many, in the industry where security is stricter and contextual, > this approach falls short. Many big organizations need flexibility to load > the SSL artifacts from a custom resource (like custom Key Management > Solution, HashiCorp Vault, Amazon KMS etc). While JSSE SecurityProvider > architecture allows us flexibility to build our custom mechanisms to validate > and process security artifacts, many times all we need is to build upon > Java's existing extensibility that Trust/Key Manager interfaces provide to > load keystores from various resources in the absence of any customized > requirements on the Keys/Certificate formats. > My proposal here is to make the SSLContext creation pluggable/extensible and > have the current SSLFactory.java implement an extensible interface. > I contributed a similar change that is live now in Apache Kafka (2.6.0) - > https://issues.apache.org/jira/browse/KAFKA-8890 > I can spare some time writing the pluggable interface and run by the required > reviewers. > > Created [CEP-9: Make SSLContext creation > pluggable|https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-9%3A+Make+SSLContext+creation+pluggable] > > > cc: [~dcapwell] [~djoshi] -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16719) Allow for transitioning from using one SSLFactory implementation to another
Sumanth Pasupuleti created CASSANDRA-16719: -- Summary: Allow for transitioning from using one SSLFactory implementation to another Key: CASSANDRA-16719 URL: https://issues.apache.org/jira/browse/CASSANDRA-16719 Project: Cassandra Issue Type: New Feature Reporter: Sumanth Pasupuleti Assignee: Sumanth Pasupuleti With CASSANDRA-1 providing pluggable SSLContext, this ticket is to provide mechanics for being able to transparently transition from using one SSLContext implementation to another. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16669) Password obfuscation for DCL audit log statements
[
https://issues.apache.org/jira/browse/CASSANDRA-16669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17358357#comment-17358357
]
Sumanth Pasupuleti edited comment on CASSANDRA-16669 at 6/7/21, 5:15 AM:
-
[~stefan.miklosovic] Thank you. Meanwhile, here is the patch against
QueryEvents that would apply obfuscation to all the listeners like AuditLogger
and FullQueryLogger - https://github.com/apache/cassandra/pull/1043/files
All UTs and JVMDtests pass except for one that seems unrelated
(incompletePropose jvm dtest failed with timeout)
https://app.circleci.com/pipelines/github/sumanth-pasupuleti/cassandra?branch=feature%2FCASSANDRA-16669-queryevents-ut
JVMDtest re-run succeeded
(https://app.circleci.com/pipelines/github/sumanth-pasupuleti/cassandra/67/workflows/fa067000-181f-4ceb-8998-78e39f26e00e/jobs/1343/tests)
was (Author: sumanth.pasupuleti):
[~stefan.miklosovic] Thank you. Meanwhile, here is the patch against
QueryEvents that would apply obfuscation to all the listeners like AuditLogger
and FullQueryLogger - https://github.com/apache/cassandra/pull/1043/files
All UTs and JVMDtests pass except for one that seems unrelated
(incompletePropose jvm dtest failed with timeout)
https://app.circleci.com/pipelines/github/sumanth-pasupuleti/cassandra?branch=feature%2FCASSANDRA-16669-queryevents-ut
> Password obfuscation for DCL audit log statements
> -
>
> Key: CASSANDRA-16669
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16669
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/auditlogging
>Reporter: Vinay Chella
>Assignee: Sumanth Pasupuleti
>Priority: Normal
> Labels: audit, security
> Fix For: 4.0-rc, 4.0.x, 4.x
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> The goal of this JIRA is to obfuscate passwords or any sensitive information
> from DCL audit log statements.
> Currently, (Cassandra version 4.0-rc1) logs query statements for any DCL
> ([ROLE|https://cassandra.apache.org/doc/latest/cql/security.html#database-roles]
> and [USER|https://cassandra.apache.org/doc/latest/cql/security.html#users] )
> queries with passwords in plaintext format in audit log files.
> The current workaround to avoid plain text passwords from being logged in
> audit log files is either by
> [excluding|https://cassandra.apache.org/doc/latest/operating/audit_logging.html#options]
> DCL statements from auditing or by excluding the user who is creating these
> roles from auditing.
> It would be ideal for Cassandra to provide an option or default to obfuscate
> passwords or any sensitive information from DCL audit log statements.
> Sample audit logs with DCL queries
> {code:sh}
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190499676|type:CREATE_ROLE|category:DCL|operation:CREATE
> ROLE new_role;
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190505313|type:CREATE_ROLE|category:DCL|operation:CREATE
> ROLE alice WITH PASSWORD = 'password_a' AND LOGIN = true;
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190519521|type:REQUEST_FAILURE|category:ERROR|operation:ALTER
> ROLE bob WITH PASSWORD = 'PASSWORD_B' AND SUPERUSER = false;; bob doesn't
> exist
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190525376|type:CREATE_ROLE|category:DCL|operation:CREATE
> ROLE bob WITH PASSWORD = 'password_b' AND LOGIN = true AND SUPERUSER = true;
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190532462|type:ALTER_ROLE|category:DCL|operation:ALTER
> ROLE bob WITH PASSWORD = 'PASSWORD_B' AND SUPERUSER = false;
> {code}
> It is also ideal to document this workaround or assumption in Cassandra audit
> log documentation until we close this JIRA
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16669) Password obfuscation for DCL audit log statements
[
https://issues.apache.org/jira/browse/CASSANDRA-16669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17358357#comment-17358357
]
Sumanth Pasupuleti commented on CASSANDRA-16669:
[~stefan.miklosovic] Thank you. Meanwhile, here is the patch against
QueryEvents that would apply obfuscation to all the listeners like AuditLogger
and FullQueryLogger.
All UTs and JVMDtests pass except for one that seems unrelated
(incompletePropose jvm dtest failed with timeout)
https://app.circleci.com/pipelines/github/sumanth-pasupuleti/cassandra?branch=feature%2FCASSANDRA-16669-queryevents-ut
> Password obfuscation for DCL audit log statements
> -
>
> Key: CASSANDRA-16669
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16669
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/auditlogging
>Reporter: Vinay Chella
>Assignee: Sumanth Pasupuleti
>Priority: Normal
> Labels: audit, security
> Fix For: 4.0-rc, 4.0.x, 4.x
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> The goal of this JIRA is to obfuscate passwords or any sensitive information
> from DCL audit log statements.
> Currently, (Cassandra version 4.0-rc1) logs query statements for any DCL
> ([ROLE|https://cassandra.apache.org/doc/latest/cql/security.html#database-roles]
> and [USER|https://cassandra.apache.org/doc/latest/cql/security.html#users] )
> queries with passwords in plaintext format in audit log files.
> The current workaround to avoid plain text passwords from being logged in
> audit log files is either by
> [excluding|https://cassandra.apache.org/doc/latest/operating/audit_logging.html#options]
> DCL statements from auditing or by excluding the user who is creating these
> roles from auditing.
> It would be ideal for Cassandra to provide an option or default to obfuscate
> passwords or any sensitive information from DCL audit log statements.
> Sample audit logs with DCL queries
> {code:sh}
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190499676|type:CREATE_ROLE|category:DCL|operation:CREATE
> ROLE new_role;
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190505313|type:CREATE_ROLE|category:DCL|operation:CREATE
> ROLE alice WITH PASSWORD = 'password_a' AND LOGIN = true;
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190519521|type:REQUEST_FAILURE|category:ERROR|operation:ALTER
> ROLE bob WITH PASSWORD = 'PASSWORD_B' AND SUPERUSER = false;; bob doesn't
> exist
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190525376|type:CREATE_ROLE|category:DCL|operation:CREATE
> ROLE bob WITH PASSWORD = 'password_b' AND LOGIN = true AND SUPERUSER = true;
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190532462|type:ALTER_ROLE|category:DCL|operation:ALTER
> ROLE bob WITH PASSWORD = 'PASSWORD_B' AND SUPERUSER = false;
> {code}
> It is also ideal to document this workaround or assumption in Cassandra audit
> log documentation until we close this JIRA
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16669) Password obfuscation for DCL audit log statements
[
https://issues.apache.org/jira/browse/CASSANDRA-16669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17358357#comment-17358357
]
Sumanth Pasupuleti edited comment on CASSANDRA-16669 at 6/7/21, 5:01 AM:
-
[~stefan.miklosovic] Thank you. Meanwhile, here is the patch against
QueryEvents that would apply obfuscation to all the listeners like AuditLogger
and FullQueryLogger - https://github.com/apache/cassandra/pull/1043/files
All UTs and JVMDtests pass except for one that seems unrelated
(incompletePropose jvm dtest failed with timeout)
https://app.circleci.com/pipelines/github/sumanth-pasupuleti/cassandra?branch=feature%2FCASSANDRA-16669-queryevents-ut
was (Author: sumanth.pasupuleti):
[~stefan.miklosovic] Thank you. Meanwhile, here is the patch against
QueryEvents that would apply obfuscation to all the listeners like AuditLogger
and FullQueryLogger.
All UTs and JVMDtests pass except for one that seems unrelated
(incompletePropose jvm dtest failed with timeout)
https://app.circleci.com/pipelines/github/sumanth-pasupuleti/cassandra?branch=feature%2FCASSANDRA-16669-queryevents-ut
> Password obfuscation for DCL audit log statements
> -
>
> Key: CASSANDRA-16669
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16669
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/auditlogging
>Reporter: Vinay Chella
>Assignee: Sumanth Pasupuleti
>Priority: Normal
> Labels: audit, security
> Fix For: 4.0-rc, 4.0.x, 4.x
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> The goal of this JIRA is to obfuscate passwords or any sensitive information
> from DCL audit log statements.
> Currently, (Cassandra version 4.0-rc1) logs query statements for any DCL
> ([ROLE|https://cassandra.apache.org/doc/latest/cql/security.html#database-roles]
> and [USER|https://cassandra.apache.org/doc/latest/cql/security.html#users] )
> queries with passwords in plaintext format in audit log files.
> The current workaround to avoid plain text passwords from being logged in
> audit log files is either by
> [excluding|https://cassandra.apache.org/doc/latest/operating/audit_logging.html#options]
> DCL statements from auditing or by excluding the user who is creating these
> roles from auditing.
> It would be ideal for Cassandra to provide an option or default to obfuscate
> passwords or any sensitive information from DCL audit log statements.
> Sample audit logs with DCL queries
> {code:sh}
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190499676|type:CREATE_ROLE|category:DCL|operation:CREATE
> ROLE new_role;
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190505313|type:CREATE_ROLE|category:DCL|operation:CREATE
> ROLE alice WITH PASSWORD = 'password_a' AND LOGIN = true;
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190519521|type:REQUEST_FAILURE|category:ERROR|operation:ALTER
> ROLE bob WITH PASSWORD = 'PASSWORD_B' AND SUPERUSER = false;; bob doesn't
> exist
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190525376|type:CREATE_ROLE|category:DCL|operation:CREATE
> ROLE bob WITH PASSWORD = 'password_b' AND LOGIN = true AND SUPERUSER = true;
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190532462|type:ALTER_ROLE|category:DCL|operation:ALTER
> ROLE bob WITH PASSWORD = 'PASSWORD_B' AND SUPERUSER = false;
> {code}
> It is also ideal to document this workaround or assumption in Cassandra audit
> log documentation until we close this JIRA
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16669) Password obfuscation for DCL audit log statements
[
https://issues.apache.org/jira/browse/CASSANDRA-16669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17357568#comment-17357568
]
Sumanth Pasupuleti edited comment on CASSANDRA-16669 at 6/4/21, 7:31 PM:
-
[~stefan.miklosovic] [~vinaykumarcse] Incorporated your review comments. Please
verify and let me know if you have further feedback.
https://github.com/apache/cassandra/pull/1028
>I hope you are not finding this offensive
Absolutely not :), one of the biggest advantage working as a community is to
improve and provide quality patches, so please keep the feedback coming.
[~stefan.miklosovic] jfyi, I merged your PR but made slight modifications to
comments in IObfuscator to keep it generic from PasswordObfuscator. I added
specific comments in PasswordObfuscator.
was (Author: sumanth.pasupuleti):
[~stefan.miklosovic] [~vinaykumarcse] Incorporated your review comments. Please
verify and let me know if you have further feedback.
>I hope you are not finding this offensive
Absolutely not :), one of the biggest advantage working as a community is to
improve and provide quality patches, so please keep the feedback coming.
[~stefan.miklosovic] jfyi, I merged your PR but made slight modifications to
comments in IObfuscator to keep it generic from PasswordObfuscator. I added
specific comments in PasswordObfuscator.
> Password obfuscation for DCL audit log statements
> -
>
> Key: CASSANDRA-16669
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16669
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/auditlogging
>Reporter: Vinay Chella
>Assignee: Sumanth Pasupuleti
>Priority: Normal
> Labels: audit, security
> Fix For: 4.0-rc, 4.0.x, 4.x
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> The goal of this JIRA is to obfuscate passwords or any sensitive information
> from DCL audit log statements.
> Currently, (Cassandra version 4.0-rc1) logs query statements for any DCL
> ([ROLE|https://cassandra.apache.org/doc/latest/cql/security.html#database-roles]
> and [USER|https://cassandra.apache.org/doc/latest/cql/security.html#users] )
> queries with passwords in plaintext format in audit log files.
> The current workaround to avoid plain text passwords from being logged in
> audit log files is either by
> [excluding|https://cassandra.apache.org/doc/latest/operating/audit_logging.html#options]
> DCL statements from auditing or by excluding the user who is creating these
> roles from auditing.
> It would be ideal for Cassandra to provide an option or default to obfuscate
> passwords or any sensitive information from DCL audit log statements.
> Sample audit logs with DCL queries
> {code:sh}
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190499676|type:CREATE_ROLE|category:DCL|operation:CREATE
> ROLE new_role;
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190505313|type:CREATE_ROLE|category:DCL|operation:CREATE
> ROLE alice WITH PASSWORD = 'password_a' AND LOGIN = true;
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190519521|type:REQUEST_FAILURE|category:ERROR|operation:ALTER
> ROLE bob WITH PASSWORD = 'PASSWORD_B' AND SUPERUSER = false;; bob doesn't
> exist
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190525376|type:CREATE_ROLE|category:DCL|operation:CREATE
> ROLE bob WITH PASSWORD = 'password_b' AND LOGIN = true AND SUPERUSER = true;
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190532462|type:ALTER_ROLE|category:DCL|operation:ALTER
> ROLE bob WITH PASSWORD = 'PASSWORD_B' AND SUPERUSER = false;
> {code}
> It is also ideal to document this workaround or assumption in Cassandra audit
> log documentation until we close this JIRA
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16669) Password obfuscation for DCL audit log statements
[
https://issues.apache.org/jira/browse/CASSANDRA-16669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17357568#comment-17357568
]
Sumanth Pasupuleti commented on CASSANDRA-16669:
[~stefan.miklosovic] [~vinaykumarcse] Incorporated your review comments. Please
verify and let me know if you have further feedback.
>I hope you are not finding this offensive
Absolutely not :), one of the biggest advantage working as a community is to
improve and provide quality patches, so please keep the feedback coming.
[~stefan.miklosovic] jfyi, I merged your PR but made slight modifications to
comments in IObfuscator to keep it generic from PasswordObfuscator. I added
specific comments in PasswordObfuscator.
> Password obfuscation for DCL audit log statements
> -
>
> Key: CASSANDRA-16669
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16669
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/auditlogging
>Reporter: Vinay Chella
>Assignee: Sumanth Pasupuleti
>Priority: Normal
> Labels: audit, security
> Fix For: 4.0-rc, 4.0.x, 4.x
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> The goal of this JIRA is to obfuscate passwords or any sensitive information
> from DCL audit log statements.
> Currently, (Cassandra version 4.0-rc1) logs query statements for any DCL
> ([ROLE|https://cassandra.apache.org/doc/latest/cql/security.html#database-roles]
> and [USER|https://cassandra.apache.org/doc/latest/cql/security.html#users] )
> queries with passwords in plaintext format in audit log files.
> The current workaround to avoid plain text passwords from being logged in
> audit log files is either by
> [excluding|https://cassandra.apache.org/doc/latest/operating/audit_logging.html#options]
> DCL statements from auditing or by excluding the user who is creating these
> roles from auditing.
> It would be ideal for Cassandra to provide an option or default to obfuscate
> passwords or any sensitive information from DCL audit log statements.
> Sample audit logs with DCL queries
> {code:sh}
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190499676|type:CREATE_ROLE|category:DCL|operation:CREATE
> ROLE new_role;
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190505313|type:CREATE_ROLE|category:DCL|operation:CREATE
> ROLE alice WITH PASSWORD = 'password_a' AND LOGIN = true;
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190519521|type:REQUEST_FAILURE|category:ERROR|operation:ALTER
> ROLE bob WITH PASSWORD = 'PASSWORD_B' AND SUPERUSER = false;; bob doesn't
> exist
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190525376|type:CREATE_ROLE|category:DCL|operation:CREATE
> ROLE bob WITH PASSWORD = 'password_b' AND LOGIN = true AND SUPERUSER = true;
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190532462|type:ALTER_ROLE|category:DCL|operation:ALTER
> ROLE bob WITH PASSWORD = 'PASSWORD_B' AND SUPERUSER = false;
> {code}
> It is also ideal to document this workaround or assumption in Cassandra audit
> log documentation until we close this JIRA
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16669) Password obfuscation for DCL audit log statements
[
https://issues.apache.org/jira/browse/CASSANDRA-16669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356230#comment-17356230
]
Sumanth Pasupuleti edited comment on CASSANDRA-16669 at 6/3/21, 6:43 AM:
-
Thanks [~stefan.miklosovic] and [~vinaykumarcse] for the feedback.
I've incorporated following updates and the
[PR|https://github.com/apache/cassandra/pull/1028] reflects the changes.
1. I changed obfuscation logic to just replace the password with ** and
retain rest of the operation string.
2. Having thought about it again, I agree we may not need a toggle to turn off
password obfuscation for audit logging. I've now removed the configurable, and
we now always obfuscate password for DCL statements
3. Extracted obfuscation logic into its own (singleton) class that implements
an interface.
4. Added test cases specific to password obfuscation logic.
5. Audit logging documentation has been updated to reflect this change about
obfuscating passwords
With regards to "Why did you choose to obfuscate passwords in AuditLogging vs
QueryEvents? What is your stance on password being visible in FQL or other
subscribers of QueryEvents?",
1. Given that QueryEvents is a centralized common emitter of events to all
registered listeners, choosing to obfuscate password in QueryEvents would force
the obfuscation behavior to all the registered listeners vs leaving that
decision to individual listeners. This is the reason why I chose to keep this
obfuscation change localized to AuditLogging.
2. My stance on password visibility in FQL is that, given FQL is meant to
replay traffic to achieve identical results, I would vote for password staying
visible in FQL.
Please let me know if you have further feedback on the updated PR.
was (Author: sumanth.pasupuleti):
Thanks [~stefan.miklosovic] and [~vinaykumarcse] for the feedback.
I've incorporated following updates and the
[PR|https://github.com/apache/cassandra/pull/1028] reflects the changes.
1. I changed obfuscation logic to just replace the password with ** and
retain rest of the operation string.
2. Having thought about it again, I agree we may not need a toggle to turn off
password obfuscation for audit logging. I've now removed the configurable, and
we now always obfuscate password for DCL statements
3. Extracted obfuscation logic into its own (singleton) class that implements
an interface.
4. Added test cases specific to password obfuscation logic.
5. Audit logging documentation has been updated to reflect this change about
obfuscating passwords
With regards to "Why did you choose to obfuscate passwords in AuditLogging vs
QueryEvents? What is your stance on password being visible in FQL or other
subscribers of QueryEvents?",
1. Given that QueryEvents is a centralized common emitter of events to all
registered listeners, choosing to obfuscate password in QueryEvents would force
the obfuscation behavior to all the registered listeners vs leaving that
decision to individual listeners. This is the reason why I chose to keep this
obfuscation change localized to AuditLogging.
2. My stance on password visibility in FQL is that, given FQL is meant to
replay traffic to achieve identical results, I would vote for password staying
visible in FQL.
Please let me know if you have further feedback on the PR.
> Password obfuscation for DCL audit log statements
> -
>
> Key: CASSANDRA-16669
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16669
> Project: Cassandra
> Issue Type: Improvement
> Components: Tool/auditlogging
>Reporter: Vinay Chella
>Assignee: Sumanth Pasupuleti
>Priority: Normal
> Labels: audit, security
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The goal of this JIRA is to obfuscate passwords or any sensitive information
> from DCL audit log statements.
> Currently, (Cassandra version 4.0-rc1) logs query statements for any DCL
> ([ROLE|https://cassandra.apache.org/doc/latest/cql/security.html#database-roles]
> and [USER|https://cassandra.apache.org/doc/latest/cql/security.html#users] )
> queries with passwords in plaintext format in audit log files.
> The current workaround to avoid plain text passwords from being logged in
> audit log files is either by
> [excluding|https://cassandra.apache.org/doc/latest/operating/audit_logging.html#options]
> DCL statements from auditing or by excluding the user who is creating these
> roles from auditing.
> It would be ideal for Cassandra to provide an option or default to obfuscate
> passwords or any sensitive information from DCL audit log statements.
> Sample audit logs with DCL queries
> {code:sh}
> Type: audit
> LogMessage:
>
[jira] [Comment Edited] (CASSANDRA-16669) Password obfuscation for DCL audit log statements
[
https://issues.apache.org/jira/browse/CASSANDRA-16669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356230#comment-17356230
]
Sumanth Pasupuleti edited comment on CASSANDRA-16669 at 6/3/21, 6:43 AM:
-
Thanks [~stefan.miklosovic] and [~vinaykumarcse] for the feedback.
I've incorporated following updates and the
[PR|https://github.com/apache/cassandra/pull/1028] reflects the changes.
1. I changed obfuscation logic to just replace the password with ** and
retain rest of the operation string.
2. Having thought about it again, I agree we may not need a toggle to turn off
password obfuscation for audit logging. I've now removed the configurable, and
we now always obfuscate password for DCL statements
3. Extracted obfuscation logic into its own (singleton) class that implements
an interface.
4. Added test cases specific to password obfuscation logic.
5. Audit logging documentation has been updated to reflect this change about
obfuscating passwords
With regards to "Why did you choose to obfuscate passwords in AuditLogging vs
QueryEvents? What is your stance on password being visible in FQL or other
subscribers of QueryEvents?",
1. Given that QueryEvents is a centralized common emitter of events to all
registered listeners, choosing to obfuscate password in QueryEvents would force
the obfuscation behavior to all the registered listeners vs leaving that
decision to individual listeners. This is the reason why I chose to keep this
obfuscation change localized to AuditLogging.
2. My stance on password visibility in FQL is that, given FQL is meant to
replay traffic to achieve identical results, I would vote for password staying
visible in FQL.
Please let me know if you have further feedback on the PR.
was (Author: sumanth.pasupuleti):
Thanks [~stefan.miklosovic] and [~vinaykumarcse] for the feedback.
I've incorporated following updates and the
[PR|https://github.com/apache/cassandra/pull/1028] reflects the changes.
1. I changed obfuscation logic to just replace the password with ** and
retain rest of the operation string.
2. Having thought about it again, I agree we may not need a toggle to turn off
password obfuscation for audit logging. I've now removed the configurable, and
we now always obfuscate password for DCL statements
3. Extracted obfuscation logic into its own (singleton) class that implements
an interface.
4. Added test cases specific to password obfuscation logic.
5. Audit logging documentation has been updated to reflect this change about
obfuscating passwords
With regards to "Why did you choose to obfuscate passwords in AuditLogging vs
QueryEvents? What is your stance on password being visible in FQL or other
subscribers of QueryEvents?",
1. Given that QueryEvents is a centralized common emitter of events to all
registered listeners, choosing to obfuscate password in QueryEvents would force
the obfuscation behavior to all the registered listeners vs leaving that
decision to individual listeners. This is the reason why I chose to keep this
obfuscation change localized to AuditLogging.
2. My stance on password visibility in FQL is that, given FQL is meant to
replay traffic to achieve identical results, I would vote for password staying
visible in FQL.
> Password obfuscation for DCL audit log statements
> -
>
> Key: CASSANDRA-16669
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16669
> Project: Cassandra
> Issue Type: Improvement
> Components: Tool/auditlogging
>Reporter: Vinay Chella
>Assignee: Sumanth Pasupuleti
>Priority: Normal
> Labels: audit, security
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The goal of this JIRA is to obfuscate passwords or any sensitive information
> from DCL audit log statements.
> Currently, (Cassandra version 4.0-rc1) logs query statements for any DCL
> ([ROLE|https://cassandra.apache.org/doc/latest/cql/security.html#database-roles]
> and [USER|https://cassandra.apache.org/doc/latest/cql/security.html#users] )
> queries with passwords in plaintext format in audit log files.
> The current workaround to avoid plain text passwords from being logged in
> audit log files is either by
> [excluding|https://cassandra.apache.org/doc/latest/operating/audit_logging.html#options]
> DCL statements from auditing or by excluding the user who is creating these
> roles from auditing.
> It would be ideal for Cassandra to provide an option or default to obfuscate
> passwords or any sensitive information from DCL audit log statements.
> Sample audit logs with DCL queries
> {code:sh}
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190499676|type:CREATE_ROLE|category:DCL|operation:CREATE
>
[jira] [Commented] (CASSANDRA-16669) Password obfuscation for DCL audit log statements
[
https://issues.apache.org/jira/browse/CASSANDRA-16669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17356230#comment-17356230
]
Sumanth Pasupuleti commented on CASSANDRA-16669:
Thanks [~stefan.miklosovic] and [~vinaykumarcse] for the feedback.
I've incorporated following updates and the
[PR|https://github.com/apache/cassandra/pull/1028] reflects the changes.
1. I changed obfuscation logic to just replace the password with ** and
retain rest of the operation string.
2. Having thought about it again, I agree we may not need a toggle to turn off
password obfuscation for audit logging. I've now removed the configurable, and
we now always obfuscate password for DCL statements
3. Extracted obfuscation logic into its own (singleton) class that implements
an interface.
4. Added test cases specific to password obfuscation logic.
5. Audit logging documentation has been updated to reflect this change about
obfuscating passwords
With regards to "Why did you choose to obfuscate passwords in AuditLogging vs
QueryEvents? What is your stance on password being visible in FQL or other
subscribers of QueryEvents?",
1. Given that QueryEvents is a centralized common emitter of events to all
registered listeners, choosing to obfuscate password in QueryEvents would force
the obfuscation behavior to all the registered listeners vs leaving that
decision to individual listeners. This is the reason why I chose to keep this
obfuscation change localized to AuditLogging.
2. My stance on password visibility in FQL is that, given FQL is meant to
replay traffic to achieve identical results, I would vote for password staying
visible in FQL.
> Password obfuscation for DCL audit log statements
> -
>
> Key: CASSANDRA-16669
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16669
> Project: Cassandra
> Issue Type: Improvement
> Components: Tool/auditlogging
>Reporter: Vinay Chella
>Assignee: Sumanth Pasupuleti
>Priority: Normal
> Labels: audit, security
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The goal of this JIRA is to obfuscate passwords or any sensitive information
> from DCL audit log statements.
> Currently, (Cassandra version 4.0-rc1) logs query statements for any DCL
> ([ROLE|https://cassandra.apache.org/doc/latest/cql/security.html#database-roles]
> and [USER|https://cassandra.apache.org/doc/latest/cql/security.html#users] )
> queries with passwords in plaintext format in audit log files.
> The current workaround to avoid plain text passwords from being logged in
> audit log files is either by
> [excluding|https://cassandra.apache.org/doc/latest/operating/audit_logging.html#options]
> DCL statements from auditing or by excluding the user who is creating these
> roles from auditing.
> It would be ideal for Cassandra to provide an option or default to obfuscate
> passwords or any sensitive information from DCL audit log statements.
> Sample audit logs with DCL queries
> {code:sh}
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190499676|type:CREATE_ROLE|category:DCL|operation:CREATE
> ROLE new_role;
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190505313|type:CREATE_ROLE|category:DCL|operation:CREATE
> ROLE alice WITH PASSWORD = 'password_a' AND LOGIN = true;
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190519521|type:REQUEST_FAILURE|category:ERROR|operation:ALTER
> ROLE bob WITH PASSWORD = 'PASSWORD_B' AND SUPERUSER = false;; bob doesn't
> exist
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190525376|type:CREATE_ROLE|category:DCL|operation:CREATE
> ROLE bob WITH PASSWORD = 'password_b' AND LOGIN = true AND SUPERUSER = true;
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190532462|type:ALTER_ROLE|category:DCL|operation:ALTER
> ROLE bob WITH PASSWORD = 'PASSWORD_B' AND SUPERUSER = false;
> {code}
> It is also ideal to document this workaround or assumption in Cassandra audit
> log documentation until we close this JIRA
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16669) Password obfuscation for DCL audit log statements
[
https://issues.apache.org/jira/browse/CASSANDRA-16669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17353577#comment-17353577
]
Sumanth Pasupuleti commented on CASSANDRA-16669:
Here is the first version of the patch
https://github.com/apache/cassandra/pull/1028.
This follows a rather less complicated approach of obfuscating anything that
follows "password" token in the DCL query, inspired from what the [PostgreSQL
Audit Extension
(pgAudit)|https://github.com/pgaudit/pgaudit/blob/master/pgaudit.c#L489-L535]
does.
> Password obfuscation for DCL audit log statements
> -
>
> Key: CASSANDRA-16669
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16669
> Project: Cassandra
> Issue Type: Improvement
> Components: Tool/auditlogging
>Reporter: Vinay Chella
>Assignee: Sumanth Pasupuleti
>Priority: Normal
> Labels: audit, security
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The goal of this JIRA is to obfuscate passwords or any sensitive information
> from DCL audit log statements.
> Currently, (Cassandra version 4.0-rc1) logs query statements for any DCL
> ([ROLE|https://cassandra.apache.org/doc/latest/cql/security.html#database-roles]
> and [USER|https://cassandra.apache.org/doc/latest/cql/security.html#users] )
> queries with passwords in plaintext format in audit log files.
> The current workaround to avoid plain text passwords from being logged in
> audit log files is either by
> [excluding|https://cassandra.apache.org/doc/latest/operating/audit_logging.html#options]
> DCL statements from auditing or by excluding the user who is creating these
> roles from auditing.
> It would be ideal for Cassandra to provide an option or default to obfuscate
> passwords or any sensitive information from DCL audit log statements.
> Sample audit logs with DCL queries
> {code:sh}
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190499676|type:CREATE_ROLE|category:DCL|operation:CREATE
> ROLE new_role;
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190505313|type:CREATE_ROLE|category:DCL|operation:CREATE
> ROLE alice WITH PASSWORD = 'password_a' AND LOGIN = true;
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190519521|type:REQUEST_FAILURE|category:ERROR|operation:ALTER
> ROLE bob WITH PASSWORD = 'PASSWORD_B' AND SUPERUSER = false;; bob doesn't
> exist
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190525376|type:CREATE_ROLE|category:DCL|operation:CREATE
> ROLE bob WITH PASSWORD = 'password_b' AND LOGIN = true AND SUPERUSER = true;
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190532462|type:ALTER_ROLE|category:DCL|operation:ALTER
> ROLE bob WITH PASSWORD = 'PASSWORD_B' AND SUPERUSER = false;
> {code}
> It is also ideal to document this workaround or assumption in Cassandra audit
> log documentation until we close this JIRA
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16669) Password obfuscation for DCL audit log statements
[
https://issues.apache.org/jira/browse/CASSANDRA-16669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17345829#comment-17345829
]
Sumanth Pasupuleti commented on CASSANDRA-16669:
Thanks [~stefan.miklosovic]. I should be ready with a patch in a day or two.
> Password obfuscation for DCL audit log statements
> -
>
> Key: CASSANDRA-16669
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16669
> Project: Cassandra
> Issue Type: Improvement
> Components: Tool/auditlogging
>Reporter: Vinay Chella
>Assignee: Sumanth Pasupuleti
>Priority: Normal
> Labels: audit, security
>
> The goal of this JIRA is to obfuscate passwords or any sensitive information
> from DCL audit log statements.
> Currently, (Cassandra version 4.0-rc1) logs query statements for any DCL
> ([ROLE|https://cassandra.apache.org/doc/latest/cql/security.html#database-roles]
> and [USER|https://cassandra.apache.org/doc/latest/cql/security.html#users] )
> queries with passwords in plaintext format in audit log files.
> The current workaround to avoid plain text passwords from being logged in
> audit log files is either by
> [excluding|https://cassandra.apache.org/doc/latest/operating/audit_logging.html#options]
> DCL statements from auditing or by excluding the user who is creating these
> roles from auditing.
> It would be ideal for Cassandra to provide an option or default to obfuscate
> passwords or any sensitive information from DCL audit log statements.
> Sample audit logs with DCL queries
> {code:sh}
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190499676|type:CREATE_ROLE|category:DCL|operation:CREATE
> ROLE new_role;
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190505313|type:CREATE_ROLE|category:DCL|operation:CREATE
> ROLE alice WITH PASSWORD = 'password_a' AND LOGIN = true;
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190519521|type:REQUEST_FAILURE|category:ERROR|operation:ALTER
> ROLE bob WITH PASSWORD = 'PASSWORD_B' AND SUPERUSER = false;; bob doesn't
> exist
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190525376|type:CREATE_ROLE|category:DCL|operation:CREATE
> ROLE bob WITH PASSWORD = 'password_b' AND LOGIN = true AND SUPERUSER = true;
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190532462|type:ALTER_ROLE|category:DCL|operation:ALTER
> ROLE bob WITH PASSWORD = 'PASSWORD_B' AND SUPERUSER = false;
> {code}
> It is also ideal to document this workaround or assumption in Cassandra audit
> log documentation until we close this JIRA
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16669) Password obfuscation for DCL audit log statements
[
https://issues.apache.org/jira/browse/CASSANDRA-16669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumanth Pasupuleti reassigned CASSANDRA-16669:
--
Assignee: Sumanth Pasupuleti
> Password obfuscation for DCL audit log statements
> -
>
> Key: CASSANDRA-16669
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16669
> Project: Cassandra
> Issue Type: Improvement
> Components: Tool/auditlogging
>Reporter: Vinay Chella
>Assignee: Sumanth Pasupuleti
>Priority: Normal
> Labels: audit, security
>
> The goal of this JIRA is to obfuscate passwords or any sensitive information
> from DCL audit log statements.
> Currently, (Cassandra version 4.0-rc1) logs query statements for any DCL
> ([ROLE|https://cassandra.apache.org/doc/latest/cql/security.html#database-roles]
> and [USER|https://cassandra.apache.org/doc/latest/cql/security.html#users] )
> queries with passwords in plaintext format in audit log files.
> The current workaround to avoid plain text passwords from being logged in
> audit log files is either by
> [excluding|https://cassandra.apache.org/doc/latest/operating/audit_logging.html#options]
> DCL statements from auditing or by excluding the user who is creating these
> roles from auditing.
> It would be ideal for Cassandra to provide an option or default to obfuscate
> passwords or any sensitive information from DCL audit log statements.
> Sample audit logs with DCL queries
> {code:sh}
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190499676|type:CREATE_ROLE|category:DCL|operation:CREATE
> ROLE new_role;
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190505313|type:CREATE_ROLE|category:DCL|operation:CREATE
> ROLE alice WITH PASSWORD = 'password_a' AND LOGIN = true;
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190519521|type:REQUEST_FAILURE|category:ERROR|operation:ALTER
> ROLE bob WITH PASSWORD = 'PASSWORD_B' AND SUPERUSER = false;; bob doesn't
> exist
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190525376|type:CREATE_ROLE|category:DCL|operation:CREATE
> ROLE bob WITH PASSWORD = 'password_b' AND LOGIN = true AND SUPERUSER = true;
> Type: audit
> LogMessage:
> user:cassandra|host:localhost/127.0.0.1:7000|source:/127.0.0.1|port:51908|timestamp:1620190532462|type:ALTER_ROLE|category:DCL|operation:ALTER
> ROLE bob WITH PASSWORD = 'PASSWORD_B' AND SUPERUSER = false;
> {code}
> It is also ideal to document this workaround or assumption in Cassandra audit
> log documentation until we close this JIRA
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16404) Provide a nodetool way of invalidating auth caches
[ https://issues.apache.org/jira/browse/CASSANDRA-16404?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320549#comment-17320549 ] Sumanth Pasupuleti commented on CASSANDRA-16404: Thanks [~blerer] for keeping me in loop for assignee change. I was waiting for the ticket to be triaged before implementing :), but thanks much to [~azotcsit] for taking this ticket forward with discussions and patch! [~azotcsit] I had a chance to review your patch and here are a few comments I have from the review. # In addition to roles having hierarchy that you already mention, even resources have hierarchy. It maybe worth considering clearing entries for all resources underneath a resource in context. # This [commit|https://github.com/apache/cassandra/pull/950/commits/e5660717deeb86872e4c868264f79f96c6b87d78] touches a lot more tests than just cache tests; I would recommend tackling that change as part of a separate ticket and change. # Code style - You may want to run the * intellij code styler; especially for same line vs next style braces styling. > Provide a nodetool way of invalidating auth caches > -- > > Key: CASSANDRA-16404 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16404 > Project: Cassandra > Issue Type: Improvement > Components: Feature/Authorization >Reporter: Sumanth Pasupuleti >Assignee: Alexey Zotov >Priority: Normal > Fix For: 4.x > > Time Spent: 20m > Remaining Estimate: 0h > > We currently have nodetool commands to invalidate certain caches like > KeyCache, RowCache and CounterCache. > Being able to invalidate auth caches as well can come in handy in situations > where, critical backend auth changes may need to be in effect right away for > all the connections, especially in configurations where cache validity is > chosen to be for a longer duration. An example can be that an authenticated > user "User1" is no longer authorized to access a table resource "table1" and > it is vital that this change is reflected right away, without having to wait > for cache expiry/refresh to trigger. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16467) speculative retry should allow more friendly params, allowing upgrade from 2.x not to break
[ https://issues.apache.org/jira/browse/CASSANDRA-16467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17289304#comment-17289304 ] Sumanth Pasupuleti commented on CASSANDRA-16467: *3.0.x* [Patch|https://github.com/apache/cassandra/compare/cassandra-3.0...sumanth-pasupuleti:bugfix/30_speculative_retry_params_case] [Passing UTs|https://app.circleci.com/pipelines/github/sumanth-pasupuleti/cassandra/53/workflows/3a0b4cdd-32c8-4b8a-83fa-34e6a78679d5] *3.11.x* [Patch|https://github.com/apache/cassandra/compare/cassandra-3.11...sumanth-pasupuleti:bugfix/311_speculative_retry_params_case] [UTs|https://app.circleci.com/pipelines/github/sumanth-pasupuleti/cassandra/54/workflows/6241a82b-3dde-4599-9c72-7d11e39c7206] (one failing that seems unrelated) > speculative retry should allow more friendly params, allowing upgrade from > 2.x not to break > --- > > Key: CASSANDRA-16467 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16467 > Project: Cassandra > Issue Type: Bug >Reporter: Sumanth Pasupuleti >Assignee: Sumanth Pasupuleti >Priority: Normal > Fix For: 3.0.x, 3.11.x > > > 2.x speculative retry params are case insensitive, while 3.0 and 3.11 have > added case sensitivity. As as result of this, one of our internal > applications suffered an issue during > C* upgrade from 2.x to 3.0. > This ticket is to propose making 3.0 and 3.11 speculative_retry params case > insensitive as well (essentially a slightly modified backport of > CASSANDRA-13876, but not to allow something like "99p" which 4.0 allows) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16467) speculative retry should allow more friendly params, allowing upgrade from 2.x not to break
[ https://issues.apache.org/jira/browse/CASSANDRA-16467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumanth Pasupuleti updated CASSANDRA-16467: --- Fix Version/s: 3.11.x 3.0.x > speculative retry should allow more friendly params, allowing upgrade from > 2.x not to break > --- > > Key: CASSANDRA-16467 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16467 > Project: Cassandra > Issue Type: Bug >Reporter: Sumanth Pasupuleti >Assignee: Sumanth Pasupuleti >Priority: Normal > Fix For: 3.0.x, 3.11.x > > > 2.x speculative retry params are case insensitive, while 3.0 and 3.11 have > added case sensitivity. As as result of this, one of our internal > applications suffered an issue during > C* upgrade from 2.x to 3.0. > This ticket is to propose making 3.0 and 3.11 speculative_retry params case > insensitive as well (essentially a slightly modified backport of > CASSANDRA-13876, but not to allow something like "99p" which 4.0 allows) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16467) speculative retry should allow more friendly params, allowing upgrade from 2.x not to break
Sumanth Pasupuleti created CASSANDRA-16467: -- Summary: speculative retry should allow more friendly params, allowing upgrade from 2.x not to break Key: CASSANDRA-16467 URL: https://issues.apache.org/jira/browse/CASSANDRA-16467 Project: Cassandra Issue Type: Bug Reporter: Sumanth Pasupuleti Assignee: Sumanth Pasupuleti 2.x speculative retry params are case insensitive, while 3.0 and 3.11 have added case sensitivity. As as result of this, one of our internal applications suffered an issue during C* upgrade from 2.x to 3.0. This ticket is to propose making 3.0 and 3.11 speculative_retry params case insensitive as well (essentially a slightly modified backport of CASSANDRA-13876, but not to allow something like "99p" which 4.0 allows) -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16456) Add Plugin Support for CQLSH
[ https://issues.apache.org/jira/browse/CASSANDRA-16456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17289288#comment-17289288 ] Sumanth Pasupuleti commented on CASSANDRA-16456: +1 to the idea of adding plugin support that would allow using both an AuthProvider that is provided within the python C* driver, as well as a custom AuthProvider one may potentially write. We currently have an internal minor patch to cqlsh to accommodate our internal AuthProvider. This ticket would certainly help us not have that minor fork and rather plugin our custom AuthProvider. > Add Plugin Support for CQLSH > > > Key: CASSANDRA-16456 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16456 > Project: Cassandra > Issue Type: New Feature >Reporter: Brian Houser >Priority: Normal > Labels: gsoc2021, mentor > > Currently the Cassandra drivers offer a plugin authenticator architecture for > the support of different authentication methods. This has been leveraged to > provide support for LDAP, Kerberos, and Sigv4 authentication. Unfortunately, > cqlsh, the included CLI tool, does not offer such support. Switching to a new > enhanced authentication scheme thus means being cut off from using cqlsh in > normal operation. > We should have a means of using the same plugins and authentication providers > as the Python Cassandra driver. > Here's a link to an initial draft of > [CEP|https://docs.google.com/document/d/1_G-OZCAEmDyuQuAN2wQUYUtZBEJpMkHWnkYELLhqvKc/edit?usp=sharing]. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15472) Read failure due to exception from metrics-core dependency
[
https://issues.apache.org/jira/browse/CASSANDRA-15472?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17271932#comment-17271932
]
Sumanth Pasupuleti commented on CASSANDRA-15472:
+1 on not being a release blocker. Sorry, been busy with a few other things and
haven't been able to push this through. [~aholmber] thanks for offering help, I
will reach out to you offline as I resume work on this.
> Read failure due to exception from metrics-core dependency
> --
>
> Key: CASSANDRA-15472
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15472
> Project: Cassandra
> Issue Type: Bug
> Components: Dependencies
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0.x
>
>
> Stacktrace
> {code:java}
> Uncaught exception on thread Thread[SharedPool-Worker-27,5,main]: {}
> java.util.NoSuchElementException: null
> at
> java.util.concurrent.ConcurrentSkipListMap.firstKey(ConcurrentSkipListMap.java:2053)
> ~[na:1.8.0_222]
> at
> com.yammer.metrics.stats.ExponentiallyDecayingSample.update(ExponentiallyDecayingSample.java:102)
> ~[metrics-core-2.2.0.jar:na]
> at
> com.yammer.metrics.stats.ExponentiallyDecayingSample.update(ExponentiallyDecayingSample.java:81)
> ~[metrics-core-2.2.0.jar:na]
> at com.yammer.metrics.core.Histogram.update(Histogram.java:110)
> ~[metrics-core-2.2.0.jar:na]
> at com.yammer.metrics.core.Timer.update(Timer.java:198)
> ~[metrics-core-2.2.0.jar:na]
> at com.yammer.metrics.core.Timer.update(Timer.java:76)
> ~[metrics-core-2.2.0.jar:na]
> at
> org.apache.cassandra.metrics.LatencyMetrics.addNano(LatencyMetrics.java:108)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> org.apache.cassandra.metrics.LatencyMetrics.addNano(LatencyMetrics.java:114)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1897)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:353)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:85)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:47)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:64)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_222]
> at
> org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
> [nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_222]
> {code}
> This [issue|https://github.com/dropwizard/metrics/issues/1278] has been
> [fixed|https://github.com/dropwizard/metrics/pull/1436] in
> [v4.0.6|https://github.com/dropwizard/metrics/releases/tag/v4.0.6].
> This is observed on a 2.1.19 cluster, but this would impact pretty much any
> version of C* since we depend on lower versions of metrics-core that do not
> have the fix.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16404) Provide a nodetool way of invalidating auth caches
[ https://issues.apache.org/jira/browse/CASSANDRA-16404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumanth Pasupuleti updated CASSANDRA-16404: --- Fix Version/s: 4.x > Provide a nodetool way of invalidating auth caches > -- > > Key: CASSANDRA-16404 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16404 > Project: Cassandra > Issue Type: Improvement >Reporter: Sumanth Pasupuleti >Assignee: Sumanth Pasupuleti >Priority: Normal > Fix For: 4.x > > > We currently have nodetool commands to invalidate certain caches like > KeyCache, RowCache and CounterCache. > Being able to invalidate auth caches as well can come in handy in situations > where, critical backend auth changes may need to be in effect right away for > all the connections, especially in configurations where cache validity is > chosen to be for a longer duration. An example can be that an authenticated > user "User1" is no longer authorized to access a table resource "table1" and > it is vital that this change is reflected right away, without having to wait > for cache expiry/refresh to trigger. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16404) Provide a nodetool way of invalidating auth caches
Sumanth Pasupuleti created CASSANDRA-16404: -- Summary: Provide a nodetool way of invalidating auth caches Key: CASSANDRA-16404 URL: https://issues.apache.org/jira/browse/CASSANDRA-16404 Project: Cassandra Issue Type: Improvement Reporter: Sumanth Pasupuleti Assignee: Sumanth Pasupuleti We currently have nodetool commands to invalidate certain caches like KeyCache, RowCache and CounterCache. Being able to invalidate auth caches as well can come in handy in situations where, critical backend auth changes may need to be in effect right away for all the connections, especially in configurations where cache validity is chosen to be for a longer duration. An example can be that an authenticated user "User1" is no longer authorized to access a table resource "table1" and it is vital that this change is reflected right away, without having to wait for cache expiry/refresh to trigger. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16216) Add tests to cover ClientMetrics metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17259870#comment-17259870 ] Sumanth Pasupuleti commented on CASSANDRA-16216: Working branch: https://github.com/apache/cassandra/compare/trunk...sumanth-pasupuleti:16216_40?expand=1 > Add tests to cover ClientMetrics metrics > > > Key: CASSANDRA-16216 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16216 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java, Test/unit >Reporter: Sumanth Pasupuleti >Assignee: Sumanth Pasupuleti >Priority: Normal > Fix For: 4.0-rc > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16188) Add more tests to cover Keyspace and Table metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17259868#comment-17259868 ] Sumanth Pasupuleti commented on CASSANDRA-16188: Working branch: https://github.com/apache/cassandra/compare/trunk...sumanth-pasupuleti:16188_40?expand=1 > Add more tests to cover Keyspace and Table metrics > --- > > Key: CASSANDRA-16188 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16188 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java, Test/unit >Reporter: Benjamin Lerer >Assignee: Sumanth Pasupuleti >Priority: Normal > Fix For: 4.0-rc > > > Several Keyspace and Table related metrics are currently not tested. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16333) Document provide_overlapping_tombstones compaction option
[ https://issues.apache.org/jira/browse/CASSANDRA-16333?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumanth Pasupuleti reassigned CASSANDRA-16333: -- Assignee: Sumanth Pasupuleti > Document provide_overlapping_tombstones compaction option > - > > Key: CASSANDRA-16333 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16333 > Project: Cassandra > Issue Type: Improvement > Components: Documentation/Website >Reporter: Paulo Motta >Assignee: Sumanth Pasupuleti >Priority: Normal > > This option was added on CASSANDRA-7019 but it's not documented. We should > add it to > https://cassandra.apache.org/doc/latest/operating/compaction/index.html. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16216) Add tests to cover ClientMetrics metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumanth Pasupuleti updated CASSANDRA-16216: --- Change Category: Quality Assurance Complexity: Normal > Add tests to cover ClientMetrics metrics > > > Key: CASSANDRA-16216 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16216 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java, Test/unit >Reporter: Sumanth Pasupuleti >Assignee: Sumanth Pasupuleti >Priority: Normal > Fix For: 4.0-beta > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Created] (CASSANDRA-16216) Add tests to cover ClientMetrics metrics
Sumanth Pasupuleti created CASSANDRA-16216: -- Summary: Add tests to cover ClientMetrics metrics Key: CASSANDRA-16216 URL: https://issues.apache.org/jira/browse/CASSANDRA-16216 Project: Cassandra Issue Type: Improvement Components: Test/dtest/java, Test/unit Reporter: Sumanth Pasupuleti Assignee: Sumanth Pasupuleti -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16216) Add tests to cover ClientMetrics metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumanth Pasupuleti updated CASSANDRA-16216: --- Fix Version/s: 4.0-beta > Add tests to cover ClientMetrics metrics > > > Key: CASSANDRA-16216 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16216 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java, Test/unit >Reporter: Sumanth Pasupuleti >Assignee: Sumanth Pasupuleti >Priority: Normal > Fix For: 4.0-beta > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Assigned] (CASSANDRA-16188) Add more tests to cover Keyspace and Table metrics
[ https://issues.apache.org/jira/browse/CASSANDRA-16188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sumanth Pasupuleti reassigned CASSANDRA-16188: -- Assignee: Sumanth Pasupuleti > Add more tests to cover Keyspace and Table metrics > --- > > Key: CASSANDRA-16188 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16188 > Project: Cassandra > Issue Type: Improvement > Components: Test/dtest/java, Test/unit >Reporter: Benjamin Lerer >Assignee: Sumanth Pasupuleti >Priority: Normal > Fix For: 4.0-beta > > > Several Keyspace and Table related metrics are currently not tested. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16182) A replacement node, although completed bootstrap and joined ring according to itself, stuck in Joining state as per the peers
[
https://issues.apache.org/jira/browse/CASSANDRA-16182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209772#comment-17209772
]
Sumanth Pasupuleti commented on CASSANDRA-16182:
{quote}C' would not complete replacement if it detects C presence before
then.{quote}
I agree it will be nice if C' would halt its replacement, however, this depends
on the timing/chance that C' should hear about C before it completes it
replacement.
{quote}nodes should apply the C' state once C is taken offline, after
confirming it is still valid{quote}
There would be a window of time between C' announcing itself to be available
and returning inconsistent results for the LOCAL_ONE requests coming directly
to it and when C is marked DOWN by other peers. But +1 to this approach since
this would address the issue deterministically imo.
{quote}But that's the contract accepted at ONE anyway{quote}
I agree with this w.r.t. legit data inconsistency that would eventually be
fixed either through hints / repairs. However, in this scenario, until the
operator takes an action, there is no bound on the potential inconsistency.
> A replacement node, although completed bootstrap and joined ring according to
> itself, stuck in Joining state as per the peers
> -
>
> Key: CASSANDRA-16182
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16182
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Normal
> Fix For: 3.0.x
>
>
> This issue occurred in a production 3.0.21 cluster.
> Here is what happened
> # We had, say, a three node Cassandra cluster with nodes A, B and C
> # C got "terminated by cloud provider" due to health check failure and a
> replacement node C' got launched.
> # C' started bootstrapping data from its neighbors
> # Network flaw: Nodes A,B were still able to communicate with terminated node
> C and consequently still have C as alive.
> # The replacement node C' learnt about C through gossip but was unable to
> communicate with C and marked C as DOWN.
> # C' completed bootstrapping successfully and itself and its peers logged
> this statement "Node C' will complete replacement of C for tokens
> [-7686143363672898397]"
> # C' logged the statement "Nodes C' and C have the same token
> -7686143363672898397. C' is the new owner"
> # C' started listening for thrift and cql clients
> # Peer nodes A and B logged "Node C' cannot complete replacement of alive
> node C "
> # A few seconds later, A and B marked C as DOWN
> C' continued to log below lines in an endless fashion
> {code:java}
> Node C is now part of the cluster
> Nodes () and C' have the same token C. Ignoring -7686143363672898397 (Needs
> a log statement fix)
> FatClient C has been silent for 3ms, removing from gossip
> {code}
> My reasoning of what happened:
> By the time replacement node (C') finished bootstrapping and announced it's
> state to Normal, A and B were still able to communicate with the replacing
> node C (while C' was not able to with C), and hence rejected C' replacing C.
> C' does not know this and does not attempt to recommunicate its "Normal"
> state to rest of the cluster. (Worth noting that A and B marked C as down
> soon after)
> Gossip keeps telling C' to add C to its metadata, and C' keeps kicking C out
> eventually based on FailureDetector.
> Proposed fix:
> When C' is notified through gossip about C, and given both own the same token
> and given C' has finished bootstrapping, C' can emit its Normal state again
> which should fix this in my opinion (so long as A and B have marked C as
> DOWN, which they did eventually)
> I ended up manually fixing this by restarting Cassandra on C', which forced
> it to announce its "Normal" state via
> StorageService.initServer --> joinTokenRing() --> finishJoiningRing() -->
> setTokens() --> setGossipTokens()
> Alternately, I could have possibly achieved the same behavior if I disabled
> and enabled gossip via jmx/nodetool.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-15249) Add documentation on release lifecycle
[ https://issues.apache.org/jira/browse/CASSANDRA-15249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209267#comment-17209267 ] Sumanth Pasupuleti edited comment on CASSANDRA-15249 at 10/7/20, 3:10 AM: -- I do not think there is anything left to do here, given that the content has been moved to the wiki was (Author: sumanth.pasupuleti): I do not think there is anything left to here, given that the content has been moved to the wiki > Add documentation on release lifecycle > -- > > Key: CASSANDRA-15249 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15249 > Project: Cassandra > Issue Type: Improvement > Components: Documentation/Website >Reporter: Sumanth Pasupuleti >Assignee: Sumanth Pasupuleti >Priority: Normal > Fix For: 4.0, 4.0-triage > > Attachments: release_lifecycle.patch > > > Relevant dev list mail thread: > https://lists.apache.org/thread.html/1a768d057d1af5a0f373c4c399a23e65cb04c61bbfff612634b9437c@%3Cdev.cassandra.apache.org%3E > Cassandra wiki on release lifecycle - > https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle > [Legacy - this google doc content is now moved to above cwiki; keeping google > doc link here for preserving comments history] Google doc with community > collaboration on documenting release lifecycle > https://docs.google.com/document/d/1bS6sr-HSrHFjZb0welife6Qx7u3ZDgRiAoENMLYlfz8/edit. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15249) Add documentation on release lifecycle
[ https://issues.apache.org/jira/browse/CASSANDRA-15249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17209267#comment-17209267 ] Sumanth Pasupuleti commented on CASSANDRA-15249: I do not think there is anything left to here, given that the content has been moved to the wiki > Add documentation on release lifecycle > -- > > Key: CASSANDRA-15249 > URL: https://issues.apache.org/jira/browse/CASSANDRA-15249 > Project: Cassandra > Issue Type: Improvement > Components: Documentation/Website >Reporter: Sumanth Pasupuleti >Assignee: Sumanth Pasupuleti >Priority: Normal > Fix For: 4.0, 4.0-triage > > Attachments: release_lifecycle.patch > > > Relevant dev list mail thread: > https://lists.apache.org/thread.html/1a768d057d1af5a0f373c4c399a23e65cb04c61bbfff612634b9437c@%3Cdev.cassandra.apache.org%3E > Cassandra wiki on release lifecycle - > https://cwiki.apache.org/confluence/display/CASSANDRA/Release+Lifecycle > [Legacy - this google doc content is now moved to above cwiki; keeping google > doc link here for preserving comments history] Google doc with community > collaboration on documenting release lifecycle > https://docs.google.com/document/d/1bS6sr-HSrHFjZb0welife6Qx7u3ZDgRiAoENMLYlfz8/edit. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-15472) Read failure due to exception from metrics-core dependency
[
https://issues.apache.org/jira/browse/CASSANDRA-15472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumanth Pasupuleti updated CASSANDRA-15472:
---
Fix Version/s: (was: 4.0-triage)
> Read failure due to exception from metrics-core dependency
> --
>
> Key: CASSANDRA-15472
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15472
> Project: Cassandra
> Issue Type: Bug
> Components: Dependencies
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Normal
> Fix For: 3.0.x, 3.11.x, 4.0-beta
>
>
> Stacktrace
> {code:java}
> Uncaught exception on thread Thread[SharedPool-Worker-27,5,main]: {}
> java.util.NoSuchElementException: null
> at
> java.util.concurrent.ConcurrentSkipListMap.firstKey(ConcurrentSkipListMap.java:2053)
> ~[na:1.8.0_222]
> at
> com.yammer.metrics.stats.ExponentiallyDecayingSample.update(ExponentiallyDecayingSample.java:102)
> ~[metrics-core-2.2.0.jar:na]
> at
> com.yammer.metrics.stats.ExponentiallyDecayingSample.update(ExponentiallyDecayingSample.java:81)
> ~[metrics-core-2.2.0.jar:na]
> at com.yammer.metrics.core.Histogram.update(Histogram.java:110)
> ~[metrics-core-2.2.0.jar:na]
> at com.yammer.metrics.core.Timer.update(Timer.java:198)
> ~[metrics-core-2.2.0.jar:na]
> at com.yammer.metrics.core.Timer.update(Timer.java:76)
> ~[metrics-core-2.2.0.jar:na]
> at
> org.apache.cassandra.metrics.LatencyMetrics.addNano(LatencyMetrics.java:108)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> org.apache.cassandra.metrics.LatencyMetrics.addNano(LatencyMetrics.java:114)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1897)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:353)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:85)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:47)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:64)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> ~[na:1.8.0_222]
> at
> org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164)
> ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
> [nf-cassandra-2.1.19.10.jar:2.1.19.10]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_222]
> {code}
> This [issue|https://github.com/dropwizard/metrics/issues/1278] has been
> [fixed|https://github.com/dropwizard/metrics/pull/1436] in
> [v4.0.6|https://github.com/dropwizard/metrics/releases/tag/v4.0.6].
> This is observed on a 2.1.19 cluster, but this would impact pretty much any
> version of C* since we depend on lower versions of metrics-core that do not
> have the fix.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Comment Edited] (CASSANDRA-16182) A replacement node, although completed bootstrap and joined ring according to itself, stuck in Joining state as per the peers
[
https://issues.apache.org/jira/browse/CASSANDRA-16182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17208370#comment-17208370
]
Sumanth Pasupuleti edited comment on CASSANDRA-16182 at 10/5/20, 10:17 PM:
---
Thanks for the clarification [~brandon.williams]. I should have been clearer; I
meant increasing the generation # to "force update" the already communicated
state. I kind of agree that this seems hacky to increment generation # for this
purpose, but I was also thinking this is a potentially rare/isolated scenario
and given that this scenario can be detected deterministically (by hearing
about a node that owns the same token through gossip as yourself, and a node
that you cannot reach).
Given that this node C' makes itself available for reads worries me of the
consequences, and makes me think we should attempt to make the cluster self
heal from this situation, so long as there are no dire consequences of
incrementing generation # (other than forcing peers to update their in memory
state)
was (Author: sumanth.pasupuleti):
Thanks for the clarification [~brandon.williams]. I should have been clearer; I
meant increasing the generation # to "force update" the already communicated
state. I kind of agree that this seems hacky to increment generation # for this
purpose, but I was also thinking this is a potentially rare/isolated scenario
and given that this scenario can be detected deterministically (by hearing
about a node that owns the same token through gossip as yourself, and a node
that you cannot reach).
Given that this node C' makes itself available for reads worries me of the
consequences, and makes me think we should attempt to make the cluster self
heal from this situation.
> A replacement node, although completed bootstrap and joined ring according to
> itself, stuck in Joining state as per the peers
> -
>
> Key: CASSANDRA-16182
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16182
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Normal
> Fix For: 3.0.x
>
>
> This issue occurred in a production 3.0.21 cluster.
> Here is what happened
> # We had, say, a three node Cassandra cluster with nodes A, B and C
> # C got "terminated by cloud provider" due to health check failure and a
> replacement node C' got launched.
> # C' started bootstrapping data from its neighbors
> # Network flaw: Nodes A,B were still able to communicate with terminated node
> C and consequently still have C as alive.
> # The replacement node C' learnt about C through gossip but was unable to
> communicate with C and marked C as DOWN.
> # C' completed bootstrapping successfully and itself and its peers logged
> this statement "Node C' will complete replacement of C for tokens
> [-7686143363672898397]"
> # C' logged the statement "Nodes C' and C have the same token
> -7686143363672898397. C' is the new owner"
> # C' started listening for thrift and cql clients
> # Peer nodes A and B logged "Node C' cannot complete replacement of alive
> node C "
> # A few seconds later, A and B marked C as DOWN
> C' continued to log below lines in an endless fashion
> {code:java}
> Node C is now part of the cluster
> Nodes () and C' have the same token C. Ignoring -7686143363672898397 (Needs
> a log statement fix)
> FatClient C has been silent for 3ms, removing from gossip
> {code}
> My reasoning of what happened:
> By the time replacement node (C') finished bootstrapping and announced it's
> state to Normal, A and B were still able to communicate with the replacing
> node C (while C' was not able to with C), and hence rejected C' replacing C.
> C' does not know this and does not attempt to recommunicate its "Normal"
> state to rest of the cluster. (Worth noting that A and B marked C as down
> soon after)
> Gossip keeps telling C' to add C to its metadata, and C' keeps kicking C out
> eventually based on FailureDetector.
> Proposed fix:
> When C' is notified through gossip about C, and given both own the same token
> and given C' has finished bootstrapping, C' can emit its Normal state again
> which should fix this in my opinion (so long as A and B have marked C as
> DOWN, which they did eventually)
> I ended up manually fixing this by restarting Cassandra on C', which forced
> it to announce its "Normal" state via
> StorageService.initServer --> joinTokenRing() --> finishJoiningRing() -->
> setTokens() --> setGossipTokens()
> Alternately, I could have possibly achieved the same behavior if I disabled
> and enabled gossip via jmx/nodetool.
--
This message was
[jira] [Commented] (CASSANDRA-16182) A replacement node, although completed bootstrap and joined ring according to itself, stuck in Joining state as per the peers
[
https://issues.apache.org/jira/browse/CASSANDRA-16182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17208370#comment-17208370
]
Sumanth Pasupuleti commented on CASSANDRA-16182:
Thanks for the clarification [~brandon.williams]. I should have been clearer; I
meant increasing the generation # to "force update" the already communicated
state. I kind of agree that this seems hacky to increment generation # for this
purpose, but I was also thinking this is a potentially rare/isolated scenario
and given that this scenario can be detected deterministically (by hearing
about a node that owns the same token through gossip as yourself, and a node
that you cannot reach).
Given that this node C' makes itself available for reads worries me of the
consequences, and makes me think we should attempt to make the cluster self
heal from this situation.
> A replacement node, although completed bootstrap and joined ring according to
> itself, stuck in Joining state as per the peers
> -
>
> Key: CASSANDRA-16182
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16182
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Normal
> Fix For: 3.0.x
>
>
> This issue occurred in a production 3.0.21 cluster.
> Here is what happened
> # We had, say, a three node Cassandra cluster with nodes A, B and C
> # C got "terminated by cloud provider" due to health check failure and a
> replacement node C' got launched.
> # C' started bootstrapping data from its neighbors
> # Network flaw: Nodes A,B were still able to communicate with terminated node
> C and consequently still have C as alive.
> # The replacement node C' learnt about C through gossip but was unable to
> communicate with C and marked C as DOWN.
> # C' completed bootstrapping successfully and itself and its peers logged
> this statement "Node C' will complete replacement of C for tokens
> [-7686143363672898397]"
> # C' logged the statement "Nodes C' and C have the same token
> -7686143363672898397. C' is the new owner"
> # C' started listening for thrift and cql clients
> # Peer nodes A and B logged "Node C' cannot complete replacement of alive
> node C "
> # A few seconds later, A and B marked C as DOWN
> C' continued to log below lines in an endless fashion
> {code:java}
> Node C is now part of the cluster
> Nodes () and C' have the same token C. Ignoring -7686143363672898397 (Needs
> a log statement fix)
> FatClient C has been silent for 3ms, removing from gossip
> {code}
> My reasoning of what happened:
> By the time replacement node (C') finished bootstrapping and announced it's
> state to Normal, A and B were still able to communicate with the replacing
> node C (while C' was not able to with C), and hence rejected C' replacing C.
> C' does not know this and does not attempt to recommunicate its "Normal"
> state to rest of the cluster. (Worth noting that A and B marked C as down
> soon after)
> Gossip keeps telling C' to add C to its metadata, and C' keeps kicking C out
> eventually based on FailureDetector.
> Proposed fix:
> When C' is notified through gossip about C, and given both own the same token
> and given C' has finished bootstrapping, C' can emit its Normal state again
> which should fix this in my opinion (so long as A and B have marked C as
> DOWN, which they did eventually)
> I ended up manually fixing this by restarting Cassandra on C', which forced
> it to announce its "Normal" state via
> StorageService.initServer --> joinTokenRing() --> finishJoiningRing() -->
> setTokens() --> setGossipTokens()
> Alternately, I could have possibly achieved the same behavior if I disabled
> and enabled gossip via jmx/nodetool.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16182) A replacement node, although completed bootstrap and joined ring according to itself, stuck in Joining state as per the peers
[
https://issues.apache.org/jira/browse/CASSANDRA-16182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumanth Pasupuleti updated CASSANDRA-16182:
---
Description:
This issue occurred in a production 3.0.21 cluster.
Here is what happened
# We had, say, a three node Cassandra cluster with nodes A, B and C
# C got "terminated by cloud provider" due to health check failure and a
replacement node C' got launched.
# C' started bootstrapping data from its neighbors
# Network flaw: Nodes A,B were still able to communicate with terminated node C
and consequently still have C as alive.
# The replacement node C' learnt about C through gossip but was unable to
communicate with C and marked C as DOWN.
# C' completed bootstrapping successfully and itself and its peers logged this
statement "Node C' will complete replacement of C for tokens
[-7686143363672898397]"
# C' logged the statement "Nodes C' and C have the same token
-7686143363672898397. C' is the new owner"
# C' started listening for thrift and cql clients
# Peer nodes A and B logged "Node C' cannot complete replacement of alive node
C "
# A few seconds later, A and B marked C as DOWN
C' continued to log below lines in an endless fashion
{code:java}
Node C is now part of the cluster
Nodes () and C' have the same token C. Ignoring -7686143363672898397 (Needs a
log statement fix)
FatClient C has been silent for 3ms, removing from gossip
{code}
My reasoning of what happened:
By the time replacement node (C') finished bootstrapping and announced it's
state to Normal, A and B were still able to communicate with the replacing node
C (while C' was not able to with C), and hence rejected C' replacing C. C' does
not know this and does not attempt to recommunicate its "Normal" state to rest
of the cluster. (Worth noting that A and B marked C as down soon after)
Gossip keeps telling C' to add C to its metadata, and C' keeps kicking C out
eventually based on FailureDetector.
Proposed fix:
When C' is notified through gossip about C, and given both own the same token
and given C' has finished bootstrapping, C' can emit its Normal state again
which should fix this in my opinion (so long as A and B have marked C as DOWN,
which they did eventually)
I ended up manually fixing this by restarting Cassandra on C', which forced it
to announce its "Normal" state via
StorageService.initServer --> joinTokenRing() --> finishJoiningRing() -->
setTokens() --> setGossipTokens()
Alternately, I could have possibly achieved the same behavior if I disabled and
enabled gossip via jmx/nodetool.
was:
This issue occurred in a production 3.0.21 cluster.
Here is what happened
# We had, say, a three node Cassandra cluster with nodes A, B and C
# C got "terminated" due to health check failure and a replacement node C' got
launched.
# C' started bootstrapping data from its neighbors
# Network flaw: Nodes A,B were still able to communicate with terminated node C
and consequently still have C as alive.
# The replacement node C' learnt about C through gossip but was unable to
communicate with C and marked C as DOWN.
# C' completed bootstrapping successfully and itself and its peers logged this
statement "Node C' will complete replacement of C for tokens
[-7686143363672898397]"
# C' logged the statement "Nodes C' and C have the same token
-7686143363672898397. C' is the new owner"
# C' started listening for thrift and cql clients
# Peer nodes A and B logged "Node C' cannot complete replacement of alive node
C "
# A few seconds later, A and B marked C as DOWN
C' continued to log below lines in an endless fashion
{code:java}
Node C is now part of the cluster
Nodes () and C' have the same token C. Ignoring -7686143363672898397 (Needs a
log statement fix)
FatClient C has been silent for 3ms, removing from gossip
{code}
My reasoning of what happened:
By the time replacement node (C') finished bootstrapping and announced it's
state to Normal, A and B were still able to communicate with the replacing node
C (while C' was not able to with C), and hence rejected C' replacing C. C' does
not know this and does not attempt to recommunicate its "Normal" state to rest
of the cluster. (Worth noting that A and B marked C as down soon after)
Gossip keeps telling C' to add C to its metadata, and C' keeps kicking C out
eventually based on FailureDetector.
Proposed fix:
When C' is notified through gossip about C, and given both own the same token
and given C' has finished bootstrapping, C' can emit its Normal state again
which should fix this in my opinion (so long as A and B have marked C as DOWN,
which they did eventually)
I ended up manually fixing this by restarting Cassandra on C', which forced it
to announce its "Normal" state via
StorageService.initServer --> joinTokenRing() --> finishJoiningRing() -->
setTokens() --> setGossipTokens()
Alternately, I could have possibly
[jira] [Commented] (CASSANDRA-16182) A replacement node, although completed bootstrap and joined ring according to itself, stuck in Joining state as per the peers
[
https://issues.apache.org/jira/browse/CASSANDRA-16182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17208249#comment-17208249
]
Sumanth Pasupuleti commented on CASSANDRA-16182:
{quote}Then it was dead to C', or the replace would've failed on that
node{quote}
+1
{quote} That's interesting since it should have seen C alive via A or B since
it could talk to them {quote}
Aliveness would have to be determined by C' itself, vs via A or B isn't it (my
understanding is, Gossip would help discover cluster members but Aliveness will
be determined by each individual nodes' FailureDetector). My hypothesis is, C
was good enough to still hold the connections it had (to A and B), but was bad
enough not to be able to establish new connections from newer nodes like C'.
{quote} I'm not sure what else we can do {quote}
Curious to know your thoughts on the proposed fix
> A replacement node, although completed bootstrap and joined ring according to
> itself, stuck in Joining state as per the peers
> -
>
> Key: CASSANDRA-16182
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16182
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Normal
> Fix For: 3.0.x
>
>
> This issue occurred in a production 3.0.21 cluster.
> Here is what happened
> # We had, say, a three node Cassandra cluster with nodes A, B and C
> # C got "terminated" due to health check failure and a replacement node C'
> got launched.
> # C' started bootstrapping data from its neighbors
> # Network flaw: Nodes A,B were still able to communicate with terminated node
> C and consequently still have C as alive.
> # The replacement node C' learnt about C through gossip but was unable to
> communicate with C and marked C as DOWN.
> # C' completed bootstrapping successfully and itself and its peers logged
> this statement "Node C' will complete replacement of C for tokens
> [-7686143363672898397]"
> # C' logged the statement "Nodes C' and C have the same token
> -7686143363672898397. C' is the new owner"
> # C' started listening for thrift and cql clients
> # Peer nodes A and B logged "Node C' cannot complete replacement of alive
> node C "
> # A few seconds later, A and B marked C as DOWN
> C' continued to log below lines in an endless fashion
> {code:java}
> Node C is now part of the cluster
> Nodes () and C' have the same token C. Ignoring -7686143363672898397 (Needs
> a log statement fix)
> FatClient C has been silent for 3ms, removing from gossip
> {code}
> My reasoning of what happened:
> By the time replacement node (C') finished bootstrapping and announced it's
> state to Normal, A and B were still able to communicate with the replacing
> node C (while C' was not able to with C), and hence rejected C' replacing C.
> C' does not know this and does not attempt to recommunicate its "Normal"
> state to rest of the cluster. (Worth noting that A and B marked C as down
> soon after)
> Gossip keeps telling C' to add C to its metadata, and C' keeps kicking C out
> eventually based on FailureDetector.
> Proposed fix:
> When C' is notified through gossip about C, and given both own the same token
> and given C' has finished bootstrapping, C' can emit its Normal state again
> which should fix this in my opinion (so long as A and B have marked C as
> DOWN, which they did eventually)
> I ended up manually fixing this by restarting Cassandra on C', which forced
> it to announce its "Normal" state via
> StorageService.initServer --> joinTokenRing() --> finishJoiningRing() -->
> setTokens() --> setGossipTokens()
> Alternately, I could have possibly achieved the same behavior if I disabled
> and enabled gossip via jmx/nodetool.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16182) A replacement node, although completed bootstrap and joined ring according to itself, stuck in Joining state as per the peers
[
https://issues.apache.org/jira/browse/CASSANDRA-16182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17208233#comment-17208233
]
Sumanth Pasupuleti commented on CASSANDRA-16182:
Yes Brandon. I believe the same (that C was still alive). C was alive long
enough to surpass C' bootstrap completion w.r.t. timeline. As mentioned, a few
seconds later, C could not communicate anymore and A and B failure detector
marked it DOWN, after which, had C' (re)announced its NORMAL state, C would
have been accepted to join the ring by the peers.
> A replacement node, although completed bootstrap and joined ring according to
> itself, stuck in Joining state as per the peers
> -
>
> Key: CASSANDRA-16182
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16182
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Normal
> Fix For: 3.0.x
>
>
> This issue occurred in a production 3.0.21 cluster.
> Here is what happened
> # We had, say, a three node Cassandra cluster with nodes A, B and C
> # C got "terminated" due to health check failure and a replacement node C'
> got launched.
> # C' started bootstrapping data from its neighbors
> # Network flaw: Nodes A,B were still able to communicate with terminated node
> C and consequently still have C as alive.
> # The replacement node C' learnt about C through gossip but was unable to
> communicate with C and marked C as DOWN.
> # C' completed bootstrapping successfully and itself and its peers logged
> this statement "Node C' will complete replacement of C for tokens
> [-7686143363672898397]"
> # C' logged the statement "Nodes C' and C have the same token
> -7686143363672898397. C' is the new owner"
> # C' started listening for thrift and cql clients
> # Peer nodes A and B logged "Node C' cannot complete replacement of alive
> node C "
> # A few seconds later, A and B marked C as DOWN
> C' continued to log below lines in an endless fashion
> {code:java}
> Node C is now part of the cluster
> Nodes () and C' have the same token C. Ignoring -7686143363672898397 (Needs
> a log statement fix)
> FatClient C has been silent for 3ms, removing from gossip
> {code}
> My reasoning of what happened:
> By the time replacement node (C') finished bootstrapping and announced it's
> state to Normal, A and B were still able to communicate with the replacing
> node C (while C' was not able to with C), and hence rejected C' replacing C.
> C' does not know this and does not attempt to recommunicate its "Normal"
> state to rest of the cluster. (Worth noting that A and B marked C as down
> soon after)
> Gossip keeps telling C' to add C to its metadata, and C' keeps kicking C out
> eventually based on FailureDetector.
> Proposed fix:
> When C' is notified through gossip about C, and given both own the same token
> and given C' has finished bootstrapping, C' can emit its Normal state again
> which should fix this in my opinion (so long as A and B have marked C as
> DOWN, which they did eventually)
> I ended up manually fixing this by restarting Cassandra on C', which forced
> it to announce its "Normal" state via
> StorageService.initServer --> joinTokenRing() --> finishJoiningRing() -->
> setTokens() --> setGossipTokens()
> Alternately, I could have possibly achieved the same behavior if I disabled
> and enabled gossip via jmx/nodetool.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16182) A replacement node, although completed bootstrap and joined ring according to itself, stuck in Joining state as per the peers
[
https://issues.apache.org/jira/browse/CASSANDRA-16182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumanth Pasupuleti updated CASSANDRA-16182:
---
Description:
This issue occurred in a production 3.0.21 cluster.
Here is what happened
# We had, say, a three node Cassandra cluster with nodes A, B and C
# C got "terminated" due to health check failure and a replacement node C' got
launched.
# C' started bootstrapping data from its neighbors
# Network flaw: Nodes A,B were still able to communicate with terminated node C
and consequently still have C as alive.
# The replacement node C' learnt about C through gossip but was unable to
communicate with C and marked C as DOWN.
# C' completed bootstrapping successfully and itself and its peers logged this
statement "Node C' will complete replacement of C for tokens
[-7686143363672898397]"
# C' logged the statement "Nodes C' and C have the same token
-7686143363672898397. C' is the new owner"
# C' started listening for thrift and cql clients
# Peer nodes A and B logged "Node C' cannot complete replacement of alive node
C "
# A few seconds later, A and B marked C as DOWN
C' continued to log below lines in an endless fashion
{code:java}
Node C is now part of the cluster
Nodes () and C' have the same token C. Ignoring -7686143363672898397 (Needs a
log statement fix)
FatClient C has been silent for 3ms, removing from gossip
{code}
My reasoning of what happened:
By the time replacement node (C') finished bootstrapping and announced it's
state to Normal, A and B were still able to communicate with the replacing node
C (while C' was not able to with C), and hence rejected C' replacing C. C' does
not know this and does not attempt to recommunicate its "Normal" state to rest
of the cluster. (Worth noting that A and B marked C as down soon after)
Gossip keeps telling C' to add C to its metadata, and C' keeps kicking C out
eventually based on FailureDetector.
Proposed fix:
When C' is notified through gossip about C, and given both own the same token
and given C' has finished bootstrapping, C' can emit its Normal state again
which should fix this in my opinion (so long as A and B have marked C as DOWN,
which they did eventually)
I ended up manually fixing this by restarting Cassandra on C', which forced it
to announce its "Normal" state via
StorageService.initServer --> joinTokenRing() --> finishJoiningRing() -->
setTokens() --> setGossipTokens()
Alternately, I could have possibly achieved the same behavior if I disabled and
enabled gossip via jmx/nodetool.
was:
This issue occurred in a production 3.0.21 cluster.
Here is what happened
# We had, say, a three node Cassandra cluster with nodes A, B and C
# C got "terminated" due to health check failure and a replacement node C' got
launched.
# C' started bootstrapping data from its neighbors
# Network flaw: Nodes A,B were still able to communicate with terminated node C
and consequently still have C as alive.
# The replacement node C' learnt about C through gossip but was unable to
communicate with C and marked C as DOWN.
# C' completed bootstrapping successfully and itself and its peers logged this
statement "Node C' will complete replacement of C for tokens
[-7686143363672898397]"
# C' logged the statement "Nodes C' and C have the same token
-7686143363672898397. C' is the new owner"
# C' started listening for thrift and cql clients
# Peer nodes A and B logged "Node C' cannot complete replacement of alive node
C "
# A few seconds later, A and B marked C as DOWN
C' continued to log below lines in an endless fashion
{code:java}
Node C is now part of the cluster
Nodes () and C' have the same token C. Ignoring -7686143363672898397 (Needs a
log statement fix)
FatClient C has been silent for 3ms, removing from gossip
{code}
My reasoning of what happened: By the time replacement node (C') finished
bootstrapping and announced it's state to Normal, A and B were still able to
communicate with the replacing node C (while C' was not able to with C), and
hence rejected C' replacing C. C' does not know this and does not attempt to
recommunicate its "Normal" state to rest of the cluster. (Worth noting that A
and B marked C as down soon after)
Gossip keeps telling C' to add C to its metadata, and C' keeps kicking C out
eventually based on FailureDetector.
Proposed fix:
When C' is notified through gossip about C, and given both own the same token
and given C' has finished bootstrapping, C' can emit its Normal state again
which should fix this in my opinion (so long as A and B have marked C as DOWN,
which they did eventually)
I ended up manually fixing this by restarting Cassandra on C', which forced it
to announce its "Normal" state via
StorageService.initServer --> joinTokenRing() --> finishJoiningRing() -->
setTokens() --> setGossipTokens()
Alternately, I could have possibly achieved the same
[jira] [Updated] (CASSANDRA-16182) A replacement node, although completed bootstrap and joined ring according to itself, stuck in Joining state as per the peers
[
https://issues.apache.org/jira/browse/CASSANDRA-16182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumanth Pasupuleti updated CASSANDRA-16182:
---
Summary: A replacement node, although completed bootstrap and joined ring
according to itself, stuck in Joining state as per the peers (was: A
replacement node, although completed bootstrap and joined ring according to
itself, maybe stuck in Joining state as per the peers)
> A replacement node, although completed bootstrap and joined ring according to
> itself, stuck in Joining state as per the peers
> -
>
> Key: CASSANDRA-16182
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16182
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Normal
> Fix For: 3.0.x
>
>
> This issue occurred in a production 3.0.21 cluster.
> Here is what happened
> # We had, say, a three node Cassandra cluster with nodes A, B and C
> # C got "terminated" due to health check failure and a replacement node C'
> got launched.
> # C' started bootstrapping data from its neighbors
> # Network flaw: Nodes A,B were still able to communicate with terminated node
> C and consequently still have C as alive.
> # The replacement node C' learnt about C through gossip but was unable to
> communicate with C and marked C as DOWN.
> # C' completed bootstrapping successfully and itself and its peers logged
> this statement "Node C' will complete replacement of C for tokens
> [-7686143363672898397]"
> # C' logged the statement "Nodes C' and C have the same token
> -7686143363672898397. C' is the new owner"
> # C' started listening for thrift and cql clients
> # Peer nodes A and B logged "Node C' cannot complete replacement of alive
> node C "
> # A few seconds later, A and B marked C as DOWN
> C' continued to log below lines in an endless fashion
> {code:java}
> Node C is now part of the cluster
> Nodes () and C' have the same token C. Ignoring -7686143363672898397 (Needs
> a log statement fix)
> FatClient C has been silent for 3ms, removing from gossip
> {code}
> My reasoning of what happened: By the time replacement node (C') finished
> bootstrapping and announced it's state to Normal, A and B were still able to
> communicate with the replacing node C (while C' was not able to with C), and
> hence rejected C' replacing C. C' does not know this and does not attempt to
> recommunicate its "Normal" state to rest of the cluster. (Worth noting that A
> and B marked C as down soon after)
> Gossip keeps telling C' to add C to its metadata, and C' keeps kicking C out
> eventually based on FailureDetector.
> Proposed fix:
> When C' is notified through gossip about C, and given both own the same token
> and given C' has finished bootstrapping, C' can emit its Normal state again
> which should fix this in my opinion (so long as A and B have marked C as
> DOWN, which they did eventually)
> I ended up manually fixing this by restarting Cassandra on C', which forced
> it to announce its "Normal" state via
> StorageService.initServer --> joinTokenRing() --> finishJoiningRing() -->
> setTokens() --> setGossipTokens()
> Alternately, I could have possibly achieved the same behavior if I disabled
> and enabled gossip via jmx/nodetool.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16182) A replacement node, although completed bootstrap and joined ring according to itself, maybe stuck in Joining state as per the peers
[
https://issues.apache.org/jira/browse/CASSANDRA-16182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumanth Pasupuleti updated CASSANDRA-16182:
---
Description:
This issue occurred in a production 3.0.21 cluster.
Here is what happened
# We had, say, a three node Cassandra cluster with nodes A, B and C
# C got "terminated" due to health check failure and a replacement node C' got
launched.
# C' started bootstrapping data from its neighbors
# Network flaw: Nodes A,B were still able to communicate with terminated node C
and consequently still have C as alive.
# The replacement node C' learnt about C through gossip but was unable to
communicate with C and marked C as DOWN.
# C' completed bootstrapping successfully and itself and its peers logged this
statement "Node C' will complete replacement of C for tokens
[-7686143363672898397]"
# C' logged the statement "Nodes C' and C have the same token
-7686143363672898397. C' is the new owner"
# C' started listening for thrift and cql clients
# Peer nodes A and B logged "Node C' cannot complete replacement of alive node
C "
# A few seconds later, A and B marked C as DOWN
C' continued to log below lines in an endless fashion
{code:java}
Node C is now part of the cluster
Nodes () and C' have the same token C. Ignoring -7686143363672898397 (Needs a
log statement fix)
FatClient C has been silent for 3ms, removing from gossip
{code}
My reasoning of what happened: By the time replacement node (C') finished
bootstrapping and announced it's state to Normal, A and B were still able to
communicate with the replacing node C (while C' was not able to with C), and
hence rejected C' replacing C. C' does not know this and does not attempt to
recommunicate its "Normal" state to rest of the cluster. (Worth noting that A
and B marked C as down soon after)
Gossip keeps telling C' to add C to its metadata, and C' keeps kicking C out
eventually based on FailureDetector.
Proposed fix:
When C' is notified through gossip about C, and given both own the same token
and given C' has finished bootstrapping, C' can emit its Normal state again
which should fix this in my opinion (so long as A and B have marked C as DOWN,
which they did eventually)
I ended up manually fixing this by restarting Cassandra on C', which forced it
to announce its "Normal" state via
StorageService.initServer --> joinTokenRing() --> finishJoiningRing() -->
setTokens() --> setGossipTokens()
Alternately, I could have possibly achieved the same behavior if I disabled and
enabled gossip via jmx/nodetool.
was:
This issue occurred in a production 3.0.21 cluster.
Here is what happened
# We had, say, a three node Cassandra cluster with nodes A, B and C
# C got "terminated" due to health check failure and a replacement node C' got
launched.
# C' started bootstrapping data from its neighbors
# Network flaw: Nodes A,B were still able to communicate with terminated node C
and consequently still have C as alive.
# The replacement node C' learnt about C through gossip but was unable to
communicate with C and marked C as DOWN.
# C' completed bootstrapping successfully and itself and its peers logged this
statement "Node C' will complete replacement of C for tokens
[-7686143363672898397]"
# C' logged the statement "Nodes C' and C have the same token
-7686143363672898397. C' is the new owner"
# C' started listening for thrift and cql clients
# Peer nodes A and B logged "Node C' cannot complete replacement of alive node
C "
# A few seconds later, A and B marked C as DOWN
C' continued to log below lines in an endless fashion
{code:java}
Node C is now part of the cluster
Nodes () and C' have the same token C. Ignoring -7686143363672898397 (Needs a
log statement fix)
FatClient C has been silent for 3ms, removing from gossip
{code}
My reasoning of what happened: By the time replacement node (C') finished
bootstrapping and announced it's state to Normal, A and B were still able to
communicate with the replacing node C (while C' was not able to with C), and
hence rejected C' replacing C. C' does not know this and does not attempt to
recommunicate its "Normal" state to rest of the cluster. (Worth noting that A
and B marked C as down soon after)
Gossip keeps telling C' to add C to its metadata, and C' keeps kicking C out
eventually based on FailureDetector.
Proposed fix:
When C' is notified through gossip about C, and given both own the same token
and given C' has finished bootstrapping, C' can emit its Normal state again
which should fix this in my opinion (so long as A and B have marked C as DOWN,
which they did eventually)
> A replacement node, although completed bootstrap and joined ring according to
> itself, maybe stuck in Joining state as per the peers
> ---
>
>
[jira] [Assigned] (CASSANDRA-16182) A replacement node, although completed bootstrap and joined ring according to itself, maybe stuck in Joining state as per the peers
[
https://issues.apache.org/jira/browse/CASSANDRA-16182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumanth Pasupuleti reassigned CASSANDRA-16182:
--
Assignee: Sumanth Pasupuleti
> A replacement node, although completed bootstrap and joined ring according to
> itself, maybe stuck in Joining state as per the peers
> ---
>
> Key: CASSANDRA-16182
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16182
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Normal
> Fix For: 3.0.x
>
>
> This issue occurred in a production 3.0.21 cluster.
> Here is what happened
> # We had, say, a three node Cassandra cluster with nodes A, B and C
> # C got "terminated" due to health check failure and a replacement node C'
> got launched.
> # C' started bootstrapping data from its neighbors
> # Network flaw: Nodes A,B were still able to communicate with terminated node
> C and consequently still have C as alive.
> # The replacement node C' learnt about C through gossip but was unable to
> communicate with C and marked C as DOWN.
> # C' completed bootstrapping successfully and itself and its peers logged
> this statement "Node C' will complete replacement of C for tokens
> [-7686143363672898397]"
> # C' logged the statement "Nodes C' and C have the same token
> -7686143363672898397. C' is the new owner"
> # C' started listening for thrift and cql clients
> # Peer nodes A and B logged "Node C' cannot complete replacement of alive
> node C "
> # A few seconds later, A and B marked C as DOWN
> C' continued to log below lines in an endless fashion
> {code:java}
> Node C is now part of the cluster
> Nodes () and C' have the same token C. Ignoring -7686143363672898397 (Needs
> a log statement fix)
> FatClient C has been silent for 3ms, removing from gossip
> {code}
> My reasoning of what happened: By the time replacement node (C') finished
> bootstrapping and announced it's state to Normal, A and B were still able to
> communicate with the replacing node C (while C' was not able to with C), and
> hence rejected C' replacing C. C' does not know this and does not attempt to
> recommunicate its "Normal" state to rest of the cluster. (Worth noting that A
> and B marked C as down soon after)
> Gossip keeps telling C' to add C to its metadata, and C' keeps kicking C out
> eventually based on FailureDetector.
> Proposed fix:
> When C' is notified through gossip about C, and given both own the same token
> and given C' has finished bootstrapping, C' can emit its Normal state again
> which should fix this in my opinion (so long as A and B have marked C as
> DOWN, which they did eventually)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16182) A replacement node, although completed bootstrap and joined ring according to itself, maybe stuck in Joining state as per the peers
[
https://issues.apache.org/jira/browse/CASSANDRA-16182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumanth Pasupuleti updated CASSANDRA-16182:
---
Description:
This issue occurred in a production 3.0.21 cluster.
Here is what happened
# We had, say, a three node Cassandra cluster with nodes A, B and C
# C got "terminated" due to health check failure and a replacement node C' got
launched.
# C' started bootstrapping data from its neighbors
# Network flaw: Nodes A,B were still able to communicate with terminated node C
and consequently still have C as alive.
# The replacement node C' learnt about C through gossip but was unable to
communicate with C and marked C as DOWN.
# C' completed bootstrapping successfully and itself and its peers logged this
statement "Node C' will complete replacement of C for tokens
[-7686143363672898397]"
# C' logged the statement "Nodes C' and C have the same token
-7686143363672898397. C' is the new owner"
# C' started listening for thrift and cql clients
# Peer nodes A and B logged "Node C' cannot complete replacement of alive node
C "
# A few seconds later, A and B marked C as DOWN
C' continued to log below lines in an endless fashion
{code:java}
Node C is now part of the cluster
Nodes () and C' have the same token C. Ignoring -7686143363672898397 (Needs a
log statement fix)
FatClient C has been silent for 3ms, removing from gossip
{code}
My reasoning of what happened: By the time replacement node (C') finished
bootstrapping and announced it's state to Normal, A and B were still able to
communicate with the replacing node C (while C' was not able to with C), and
hence rejected C' replacing C. C' does not know this and does not attempt to
recommunicate its "Normal" state to rest of the cluster. (Worth noting that A
and B marked C as down soon after)
Gossip keeps telling C' to add C to its metadata, and C' keeps kicking C out
eventually based on FailureDetector.
Proposed fix:
When C' is notified through gossip about C, and given both own the same token
and given C' has finished bootstrapping, C' can emit its Normal state again
which should fix this in my opinion (so long as A and B have marked C as DOWN,
which they did eventually)
was:
This issue occurred in a production 3.0.21 cluster.
Here is what happened
# We had, say, a three node Cassandra cluster with nodes A, B and C
# C got "terminated" due to health check failure and a replacement node C' got
launched.
# C' started bootstrapping data from its neighbors
# Network flaw: Nodes A,B were still able to communicate with terminated node C
and consequently still have C as alive.
# The replacement node C' learnt about C through gossip but was unable to
communicate with C and marked C as DOWN.
# C' completed bootstrapping successfully and itself and its peers logged this
statement "Node C' will complete replacement of C for tokens
[-7686143363672898397]"
# C' logged the statement "Nodes C' and C have the same token
-7686143363672898397. C' is the new owner"
# C' started listening for thrift and cql clients
# Peer nodes A and B logged "Node C' cannot complete replacement of alive node
C "
# A few seconds later, A and B marked C as DOWN
C' continued to log below lines
{code:java}
Node C is now part of the cluster
Nodes () and C' have the same token C. Ignoring -7686143363672898397 (Needs a
log statement fix)
FatClient C has been silent for 3ms, removing from gossip
{code}
My reasoning of what happened: By the time replacement node (C') finished
bootstrapping and announced it's state to Normal, A and B were still able to
communicate with the replacing node C (while C' was not able to with C), and
hence rejected C' replacing C. C' does not know this and does not attempt to
recommunicate its "Normal" state to rest of the cluster. (Worth noting that A
and B marked C as down soon after)
Gossip keeps telling C' to add C to its metadata, and C' keeps kicking C out
eventually based on FailureDetector.
Proposed fix:
When C' is notified through gossip about C, and given both own the same token
and given C' has finished bootstrapping, C' can emit its Normal state again
which should fix this in my opinion (so long as A and B have marked C as DOWN,
which they did eventually)
> A replacement node, although completed bootstrap and joined ring according to
> itself, maybe stuck in Joining state as per the peers
> ---
>
> Key: CASSANDRA-16182
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16182
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip
>Reporter: Sumanth Pasupuleti
>Priority: Normal
> Fix For: 3.0.x
>
>
> This issue occurred in a production 3.0.21
[jira] [Updated] (CASSANDRA-16182) A replacement node, although completed bootstrap and joined ring according to itself, maybe stuck in Joining state as per the peers
[
https://issues.apache.org/jira/browse/CASSANDRA-16182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumanth Pasupuleti updated CASSANDRA-16182:
---
Description:
This issue occurred in a production 3.0.21 cluster.
Here is what happened
# We had, say, a three node Cassandra cluster with nodes A, B and C
# C got "terminated" due to health check failure and a replacement node C' got
launched.
# C' started bootstrapping data from its neighbors
# Network flaw: Nodes A,B were still able to communicate with terminated node C
and consequently still have C as alive.
# The replacement node C' learnt about C through gossip but was unable to
communicate with C and marked C as DOWN.
# C' completed bootstrapping successfully and itself and its peers logged this
statement "Node C' will complete replacement of C for tokens
[-7686143363672898397]"
# C' logged the statement "Nodes C' and C have the same token
-7686143363672898397. C' is the new owner"
# C' started listening for thrift and cql clients
# Peer nodes A and B logged 'Node C' cannot complete replacement of alive node
C'
# A few seconds later, A and B marked C as DOWN
C' continued to log below lines
{code:java}
Node C is now part of the cluster
Nodes () and C' have the same token C. Ignoring -7686143363672898397 (Needs a
log statement fix)
FatClient C has been silent for 3ms, removing from gossip
{code}
My reasoning of what happened: By the time replacement node (C') finished
bootstrapping and announced it's state to Normal, A and B were still able to
communicate with the replacing node C (while C' was not able to with C), and
hence rejected C' replacing C. C' does not know this and does not attempt to
recommunicate its "Normal" state to rest of the cluster. (Worth noting that A
and B marked C as down soon after)
Gossip keeps telling C' to add C to its metadata, and C' keeps kicking C out
eventually based on FailureDetector.
Proposed fix:
When C' is notified through gossip about C, and given both own the same token
and given C' has finished bootstrapping, C' can emit its Normal state again
which should fix this in my opinion (so long as A and B have marked C as DOWN,
which they did eventually)
was:
This issue occurred in a production 3.0.21 cluster.
Here is what happened
# We had, say, a three node Cassandra cluster with nodes A, B and C
# C got "terminated" due to health check failure and a replacement node C' got
launched.
# C' started bootstrapping data from its neighbors
# Network flaw: Nodes A,B were still able to communicate with terminated node C
and consequently still have C as alive.
# The replacement node C' learnt about C through gossip but was unable to
communicate with C and marked C as DOWN.
# C' completed bootstrapping successfully and itself and its peers logged this
statement "Node C' will complete replacement of C for tokens
[-7686143363672898397]"
# C' logged the statement "Nodes C' and C have the same token
-7686143363672898397. C' is the new owner"
# C' started listening for thrift and cql clients
# Peer nodes A and B logged 'Node C' cannot complete replacement of alive node
C'
# A few seconds later, A and B marked C' as DOWN
C' continued to log below lines
{code:java}
Node C is now part of the cluster
Nodes () and C' have the same token C. Ignoring -7686143363672898397 (Needs a
log statement fix)
FatClient C has been silent for 3ms, removing from gossip
{code}
My reasoning of what happened: By the time replacement node (C') finished
bootstrapping and announced it's state to Normal, A and B were still able to
communicate with the replacing node C (while C' was not able to with C), and
hence rejected C' replacing C. C' does not know this and does not attempt to
recommunicate its "Normal" state to rest of the cluster. (Worth noting that A
and B marked C as down soon after)
Gossip keeps telling C' to add C to its metadata, and C' keeps kicking C out
eventually based on FailureDetector.
Proposed fix:
When C' is notified through gossip about C, and given both own the same token
and given C' has finished bootstrapping, C' can emit its Normal state again
which should fix this in my opinion (so long as A and B have marked C as DOWN,
which they did eventually)
> A replacement node, although completed bootstrap and joined ring according to
> itself, maybe stuck in Joining state as per the peers
> ---
>
> Key: CASSANDRA-16182
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16182
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip
>Reporter: Sumanth Pasupuleti
>Priority: Normal
> Fix For: 3.0.x
>
>
> This issue occurred in a production 3.0.21 cluster.
> Here is what
[jira] [Updated] (CASSANDRA-16182) A replacement node, although completed bootstrap and joined ring according to itself, maybe stuck in Joining state as per the peers
[
https://issues.apache.org/jira/browse/CASSANDRA-16182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumanth Pasupuleti updated CASSANDRA-16182:
---
Description:
This issue occurred in a production 3.0.21 cluster.
Here is what happened
# We had, say, a three node Cassandra cluster with nodes A, B and C
# C got "terminated" due to health check failure and a replacement node C' got
launched.
# C' started bootstrapping data from its neighbors
# Network flaw: Nodes A,B were still able to communicate with terminated node C
and consequently still have C as alive.
# The replacement node C' learnt about C through gossip but was unable to
communicate with C and marked C as DOWN.
# C' completed bootstrapping successfully and itself and its peers logged this
statement "Node C' will complete replacement of C for tokens
[-7686143363672898397]"
# C' logged the statement "Nodes C' and C have the same token
-7686143363672898397. C' is the new owner"
# C' started listening for thrift and cql clients
# Peer nodes A and B logged "Node C' cannot complete replacement of alive node
C "
# A few seconds later, A and B marked C as DOWN
C' continued to log below lines
{code:java}
Node C is now part of the cluster
Nodes () and C' have the same token C. Ignoring -7686143363672898397 (Needs a
log statement fix)
FatClient C has been silent for 3ms, removing from gossip
{code}
My reasoning of what happened: By the time replacement node (C') finished
bootstrapping and announced it's state to Normal, A and B were still able to
communicate with the replacing node C (while C' was not able to with C), and
hence rejected C' replacing C. C' does not know this and does not attempt to
recommunicate its "Normal" state to rest of the cluster. (Worth noting that A
and B marked C as down soon after)
Gossip keeps telling C' to add C to its metadata, and C' keeps kicking C out
eventually based on FailureDetector.
Proposed fix:
When C' is notified through gossip about C, and given both own the same token
and given C' has finished bootstrapping, C' can emit its Normal state again
which should fix this in my opinion (so long as A and B have marked C as DOWN,
which they did eventually)
was:
This issue occurred in a production 3.0.21 cluster.
Here is what happened
# We had, say, a three node Cassandra cluster with nodes A, B and C
# C got "terminated" due to health check failure and a replacement node C' got
launched.
# C' started bootstrapping data from its neighbors
# Network flaw: Nodes A,B were still able to communicate with terminated node C
and consequently still have C as alive.
# The replacement node C' learnt about C through gossip but was unable to
communicate with C and marked C as DOWN.
# C' completed bootstrapping successfully and itself and its peers logged this
statement "Node C' will complete replacement of C for tokens
[-7686143363672898397]"
# C' logged the statement "Nodes C' and C have the same token
-7686143363672898397. C' is the new owner"
# C' started listening for thrift and cql clients
# Peer nodes A and B logged 'Node C' cannot complete replacement of alive node
C'
# A few seconds later, A and B marked C as DOWN
C' continued to log below lines
{code:java}
Node C is now part of the cluster
Nodes () and C' have the same token C. Ignoring -7686143363672898397 (Needs a
log statement fix)
FatClient C has been silent for 3ms, removing from gossip
{code}
My reasoning of what happened: By the time replacement node (C') finished
bootstrapping and announced it's state to Normal, A and B were still able to
communicate with the replacing node C (while C' was not able to with C), and
hence rejected C' replacing C. C' does not know this and does not attempt to
recommunicate its "Normal" state to rest of the cluster. (Worth noting that A
and B marked C as down soon after)
Gossip keeps telling C' to add C to its metadata, and C' keeps kicking C out
eventually based on FailureDetector.
Proposed fix:
When C' is notified through gossip about C, and given both own the same token
and given C' has finished bootstrapping, C' can emit its Normal state again
which should fix this in my opinion (so long as A and B have marked C as DOWN,
which they did eventually)
> A replacement node, although completed bootstrap and joined ring according to
> itself, maybe stuck in Joining state as per the peers
> ---
>
> Key: CASSANDRA-16182
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16182
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip
>Reporter: Sumanth Pasupuleti
>Priority: Normal
> Fix For: 3.0.x
>
>
> This issue occurred in a production 3.0.21 cluster.
> Here is what
[jira] [Created] (CASSANDRA-16182) A replacement node, although completed bootstrap and joined ring according to itself, maybe stuck in Joining state as per the peers
Sumanth Pasupuleti created CASSANDRA-16182:
--
Summary: A replacement node, although completed bootstrap and
joined ring according to itself, maybe stuck in Joining state as per the peers
Key: CASSANDRA-16182
URL: https://issues.apache.org/jira/browse/CASSANDRA-16182
Project: Cassandra
Issue Type: Bug
Components: Cluster/Gossip
Reporter: Sumanth Pasupuleti
This issue occurred in a production 3.0.21 cluster.
Here is what happened
# We had, say, a three node Cassandra cluster with nodes A, B and C
# C got "terminated" due to health check failure and a replacement node C' got
launched.
# C' started bootstrapping data from its neighbors
# Network flaw: Nodes A,B were still able to communicate with terminated node C
and consequently still have C as alive.
# The replacement node C' learnt about C through gossip but was unable to
communicate with C and marked C as DOWN.
# C' completed bootstrapping successfully and itself and its peers logged this
statement "Node C' will complete replacement of C for tokens
[-7686143363672898397]"
# C' logged the statement "Nodes C' and C have the same token
-7686143363672898397. C' is the new owner"
# C' started listening for thrift and cql clients
# Peer nodes A and B logged 'Node C' cannot complete replacement of alive node
C'
# A few seconds later, A and B marked C' as DOWN
C' continued to log below lines
{code:java}
Node C is now part of the cluster
Nodes () and C' have the same token C. Ignoring -7686143363672898397 (Needs a
log statement fix)
FatClient C has been silent for 3ms, removing from gossip
{code}
My reasoning of what happened: By the time replacement node (C') finished
bootstrapping and announced it's state to Normal, A and B were still able to
communicate with the replacing node C (while C' was not able to with C), and
hence rejected C' replacing C. C' does not know this and does not attempt to
recommunicate its "Normal" state to rest of the cluster. (Worth noting that A
and B marked C as down soon after)
Gossip keeps telling C' to add C to its metadata, and C' keeps kicking C out
eventually based on FailureDetector.
Proposed fix:
When C' is notified through gossip about C, and given both own the same token
and given C' has finished bootstrapping, C' can emit its Normal state again
which should fix this in my opinion (so long as A and B have marked C as DOWN,
which they did eventually)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Updated] (CASSANDRA-16182) A replacement node, although completed bootstrap and joined ring according to itself, maybe stuck in Joining state as per the peers
[
https://issues.apache.org/jira/browse/CASSANDRA-16182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sumanth Pasupuleti updated CASSANDRA-16182:
---
Fix Version/s: 3.0.x
> A replacement node, although completed bootstrap and joined ring according to
> itself, maybe stuck in Joining state as per the peers
> ---
>
> Key: CASSANDRA-16182
> URL: https://issues.apache.org/jira/browse/CASSANDRA-16182
> Project: Cassandra
> Issue Type: Bug
> Components: Cluster/Gossip
>Reporter: Sumanth Pasupuleti
>Priority: Normal
> Fix For: 3.0.x
>
>
> This issue occurred in a production 3.0.21 cluster.
> Here is what happened
> # We had, say, a three node Cassandra cluster with nodes A, B and C
> # C got "terminated" due to health check failure and a replacement node C'
> got launched.
> # C' started bootstrapping data from its neighbors
> # Network flaw: Nodes A,B were still able to communicate with terminated node
> C and consequently still have C as alive.
> # The replacement node C' learnt about C through gossip but was unable to
> communicate with C and marked C as DOWN.
> # C' completed bootstrapping successfully and itself and its peers logged
> this statement "Node C' will complete replacement of C for tokens
> [-7686143363672898397]"
> # C' logged the statement "Nodes C' and C have the same token
> -7686143363672898397. C' is the new owner"
> # C' started listening for thrift and cql clients
> # Peer nodes A and B logged 'Node C' cannot complete replacement of alive
> node C'
> # A few seconds later, A and B marked C' as DOWN
> C' continued to log below lines
> {code:java}
> Node C is now part of the cluster
> Nodes () and C' have the same token C. Ignoring -7686143363672898397 (Needs
> a log statement fix)
> FatClient C has been silent for 3ms, removing from gossip
> {code}
> My reasoning of what happened: By the time replacement node (C') finished
> bootstrapping and announced it's state to Normal, A and B were still able to
> communicate with the replacing node C (while C' was not able to with C), and
> hence rejected C' replacing C. C' does not know this and does not attempt to
> recommunicate its "Normal" state to rest of the cluster. (Worth noting that A
> and B marked C as down soon after)
> Gossip keeps telling C' to add C to its metadata, and C' keeps kicking C out
> eventually based on FailureDetector.
> Proposed fix:
> When C' is notified through gossip about C, and given both own the same token
> and given C' has finished bootstrapping, C' can emit its Normal state again
> which should fix this in my opinion (so long as A and B have marked C as
> DOWN, which they did eventually)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-15778) CorruptSSTableException after a 2.1 SSTable is upgraded to 3.0, failing reads
[
https://issues.apache.org/jira/browse/CASSANDRA-15778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17123353#comment-17123353
]
Sumanth Pasupuleti commented on CASSANDRA-15778:
Thanks for the patch [~ifesdjeen]. I was able to validate both the flows of the
patch on the actual data that I was able to repro the issue with.
In either of the flows, the resulting table ended up with the following schema,
and doing a select * on the table includes the third column.
{code:java}
cqlsh:ks> desc "cf";
CREATE TABLE ks."cf" (
key text,
value text,
value1 blob,
PRIMARY KEY (key, value)
) WITH COMPACT STORAGE
AND CLUSTERING ORDER BY (value ASC)
...
{code}
A couple of comments on the patch:
1. It maybe helpful to add the following tests to {{testAlterTableAlterType()}}
method.
{code:java}
createTable("CREATE TABLE %s (a int, value int, PRIMARY KEY (a,value))
WITH COMPACT STORAGE");
assertInvalidMessage("Altering of types is not allowed",
"ALTER TABLE %s ALTER value TYPE
'org.apache.cassandra.db.marshal.IntegerType'");
execute("ALTER TABLE %s ALTER value1 TYPE
'org.apache.cassandra.db.marshal.BytesType'");
{code}
2. For the system property
{{cassandra.init_dense_table_compact_value_as_bytes}}, I am not sure if folks
may find it useful to control what keyspace/cf this needs to be applied to. I
personally feel it may be rare enough to ignore this, but wanted to bring it up
nevertheless.
> CorruptSSTableException after a 2.1 SSTable is upgraded to 3.0, failing reads
> -
>
> Key: CASSANDRA-15778
> URL: https://issues.apache.org/jira/browse/CASSANDRA-15778
> Project: Cassandra
> Issue Type: Bug
> Components: Local/Compaction, Local/SSTable
>Reporter: Sumanth Pasupuleti
>Assignee: Alex Petrov
>Priority: Normal
> Fix For: 3.0.x
>
>
> Below is the exception with stack trace. This issue is consistently
> reproduce-able.
> {code:java}
> ERROR [SharedPool-Worker-1] 2020-05-01 14:57:57,661
> AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread
> Thread[SharedPool-Worker-1,5,main]ERROR [SharedPool-Worker-1] 2020-05-01
> 14:57:57,661 AbstractLocalAwareExecutorService.java:169 - Uncaught exception
> on thread
> Thread[SharedPool-Worker-1,5,main]org.apache.cassandra.io.sstable.CorruptSSTableException:
> Corrupted:
> /mnt/data/cassandra/data// at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator$Reader.hasNext(AbstractSSTableIterator.java:349)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.columniterator.AbstractSSTableIterator.hasNext(AbstractSSTableIterator.java:220)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.columniterator.SSTableIterator.hasNext(SSTableIterator.java:33)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:95)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:129)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:95)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.rows.LazilyInitializedUnfilteredRowIterator.computeNext(LazilyInitializedUnfilteredRowIterator.java:32)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:129)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.transform.BaseRows.hasNext(BaseRows.java:129)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:131)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:87)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
> org.apache.cassandra.db.rows.UnfilteredRowIteratorSerializer.serialize(UnfilteredRowIteratorSerializer.java:77)
> ~[nf-cassandra-3.0.19.8.jar:3.0.19.8] at
>
[jira] [Comment Edited] (CASSANDRA-14557) Consider adding default and required keyspace replication options
[
https://issues.apache.org/jira/browse/CASSANDRA-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110462#comment-17110462
]
Sumanth Pasupuleti edited comment on CASSANDRA-14557 at 5/18/20, 6:57 PM:
--
Based on the suggestion in the dev list, I've updated the patch to include a
note about impact on system keyspaces in cassandra.yaml.
Updated github branch:
https://github.com/sumanth-pasupuleti/cassandra/tree/14557-trunk
Added patch file: [^14557-4.0.txt]
was (Author: sumanth.pasupuleti):
Based on the suggestion in the dev list, I've updated the patch to include a
note about impact on system keyspaces in cassandra.yaml.
Updated github branch:
https://github.com/sumanth-pasupuleti/cassandra/tree/14557-trunk
Updated patch file: [^14557-4.0.txt]
> Consider adding default and required keyspace replication options
> -
>
> Key: CASSANDRA-14557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14557
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
> Labels: 4.0-feature-freeze-review-requested
> Fix For: 4.x
>
> Attachments: 14557-4.0.txt, 14557-trunk.patch
>
>
> Ending up with a keyspace of RF=1 is unfortunately pretty easy in C* right
> now - the system_auth table for example is created with RF=1 (to take into
> account single node setups afaict from CASSANDRA-5112), and a user can
> further create a keyspace with RF=1 posing availability and streaming risks
> (e.g. rebuild).
> I propose we add two configuration options in cassandra.yaml:
> # {{default_keyspace_rf}} (default: 1) - If replication factors are not
> specified, use this number.
> # {{required_minimum_keyspace_rf}} (default: unset) - Prevent users from
> creating a keyspace with an RF less than what is configured
> These settings could further be re-used to:
> * Provide defaults for new keyspaces created with SimpleStrategy or
> NetworkTopologyStrategy (CASSANDRA-14303)
> * Make the automatic token [allocation
> algorithm|https://issues.apache.org/jira/browse/CASSANDRA-13701?focusedCommentId=16095662=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16095662]
> interface more intuitive allowing easy use of the new token allocation
> algorithm.
> At the end of the day, if someone really wants to allow RF=1, they simply
> don’t set the setting. For backwards compatibility the default remains 1 and
> C* would create with RF=1, and would default to current behavior of allowing
> any RF on keyspaces.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-14557) Consider adding default and required keyspace replication options
[
https://issues.apache.org/jira/browse/CASSANDRA-14557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17110462#comment-17110462
]
Sumanth Pasupuleti commented on CASSANDRA-14557:
Based on the suggestion in the dev list, I've updated the patch to include a
note about impact on system keyspaces in cassandra.yaml.
Updated github branch:
https://github.com/sumanth-pasupuleti/cassandra/tree/14557-trunk
Updated patch file: [^14557-4.0.txt]
> Consider adding default and required keyspace replication options
> -
>
> Key: CASSANDRA-14557
> URL: https://issues.apache.org/jira/browse/CASSANDRA-14557
> Project: Cassandra
> Issue Type: Improvement
> Components: Local/Config
>Reporter: Sumanth Pasupuleti
>Assignee: Sumanth Pasupuleti
>Priority: Low
> Labels: 4.0-feature-freeze-review-requested
> Fix For: 4.x
>
> Attachments: 14557-4.0.txt, 14557-trunk.patch
>
>
> Ending up with a keyspace of RF=1 is unfortunately pretty easy in C* right
> now - the system_auth table for example is created with RF=1 (to take into
> account single node setups afaict from CASSANDRA-5112), and a user can
> further create a keyspace with RF=1 posing availability and streaming risks
> (e.g. rebuild).
> I propose we add two configuration options in cassandra.yaml:
> # {{default_keyspace_rf}} (default: 1) - If replication factors are not
> specified, use this number.
> # {{required_minimum_keyspace_rf}} (default: unset) - Prevent users from
> creating a keyspace with an RF less than what is configured
> These settings could further be re-used to:
> * Provide defaults for new keyspaces created with SimpleStrategy or
> NetworkTopologyStrategy (CASSANDRA-14303)
> * Make the automatic token [allocation
> algorithm|https://issues.apache.org/jira/browse/CASSANDRA-13701?focusedCommentId=16095662=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16095662]
> interface more intuitive allowing easy use of the new token allocation
> algorithm.
> At the end of the day, if someone really wants to allow RF=1, they simply
> don’t set the setting. For backwards compatibility the default remains 1 and
> C* would create with RF=1, and would default to current behavior of allowing
> any RF on keyspaces.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org
For additional commands, e-mail: commits-h...@cassandra.apache.org
