[jira] [Commented] (CASSANDRA-12106) Add ability to blacklist a CQL partition so all requests are ignored
[ https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17399912#comment-17399912 ] Sumanth Pasupuleti commented on CASSANDRA-12106: Rebased and reworded. Updated patch: https://github.com/apache/cassandra/compare/trunk...sumanth-pasupuleti:feature/12106 > Add ability to blacklist a CQL partition so all requests are ignored > > > Key: CASSANDRA-12106 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12106 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Local Write-Read Paths, Local/Config >Reporter: Geoffrey Yu >Assignee: Sumanth Pasupuleti >Priority: Low > Fix For: 4.x > > Attachments: 12106-trunk.txt > > > Sometimes reads/writes to a given partition may cause problems due to the > data present. It would be useful to have a manual way to blacklist such > partitions so all read and write requests to them are rejected. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12106) Add ability to blacklist a CQL partition so all requests are ignored
[ https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17377605#comment-17377605 ] Sumanth Pasupuleti commented on CASSANDRA-12106: Thanks for the reference to CASSANDRA-15862. Will work on rebasing and rewording. > Add ability to blacklist a CQL partition so all requests are ignored > > > Key: CASSANDRA-12106 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12106 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Local Write-Read Paths, Local/Config >Reporter: Geoffrey Yu >Assignee: Sumanth Pasupuleti >Priority: Low > Fix For: 4.x > > Attachments: 12106-trunk.txt > > > Sometimes reads/writes to a given partition may cause problems due to the > data present. It would be useful to have a manual way to blacklist such > partitions so all read and write requests to them are rejected. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12106) Add ability to blacklist a CQL partition so all requests are ignored
[ https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17372921#comment-17372921 ] Brandon Williams commented on CASSANDRA-12106: -- Needs a rebase. Also terms like 'blacklist' should be substituted to match CASSANDRA-15862. > Add ability to blacklist a CQL partition so all requests are ignored > > > Key: CASSANDRA-12106 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12106 > Project: Cassandra > Issue Type: New Feature > Components: Legacy/Local Write-Read Paths, Local/Config >Reporter: Geoffrey Yu >Assignee: Sumanth Pasupuleti >Priority: Low > Fix For: 4.x > > Attachments: 12106-trunk.txt > > > Sometimes reads/writes to a given partition may cause problems due to the > data present. It would be useful to have a manual way to blacklist such > partitions so all read and write requests to them are rejected. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12106) Add ability to blacklist a CQL partition so all requests are ignored
[ https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691269#comment-16691269 ] Sumanth Pasupuleti commented on CASSANDRA-12106: Based on the feedback, here is the patch for blacklisting partitions for point reads. https://github.com/apache/cassandra/pull/277 > Add ability to blacklist a CQL partition so all requests are ignored > > > Key: CASSANDRA-12106 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12106 > Project: Cassandra > Issue Type: New Feature > Components: Configuration, Local Write-Read Paths >Reporter: Geoffrey Yu >Assignee: Sumanth Pasupuleti >Priority: Minor > Fix For: 4.x > > Attachments: 12106-trunk.txt > > > Sometimes reads/writes to a given partition may cause problems due to the > data present. It would be useful to have a manual way to blacklist such > partitions so all read and write requests to them are rejected. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12106) Add ability to blacklist a CQL partition so all requests are ignored
[ https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16608620#comment-16608620 ] Sumanth Pasupuleti commented on CASSANDRA-12106: Thanks [~jjirsa] and [~vinaykumarcse] for reviewing the design. Others interested in this ticket, please review the design and provide any feedback. Planning to start development soon. > Add ability to blacklist a CQL partition so all requests are ignored > > > Key: CASSANDRA-12106 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12106 > Project: Cassandra > Issue Type: New Feature >Reporter: Geoffrey Yu >Assignee: Sumanth Pasupuleti >Priority: Minor > Fix For: 4.x > > Attachments: 12106-trunk.txt > > > Sometimes reads/writes to a given partition may cause problems due to the > data present. It would be useful to have a manual way to blacklist such > partitions so all read and write requests to them are rejected. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12106) Add ability to blacklist a CQL partition so all requests are ignored
[ https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16606495#comment-16606495 ] Sumanth Pasupuleti commented on CASSANDRA-12106: After further analysis, proposing system_distributed based coordinator level rejection of bad partitions. [Design document|https://docs.google.com/document/d/1obg2IFL-UBU1KErvItxVBQL--oURvaHX-eDjq-Jz9Ys/edit?usp=sharing] Please provide feedback on the document. > Add ability to blacklist a CQL partition so all requests are ignored > > > Key: CASSANDRA-12106 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12106 > Project: Cassandra > Issue Type: New Feature >Reporter: Geoffrey Yu >Assignee: Sumanth Pasupuleti >Priority: Minor > Fix For: 4.x > > Attachments: 12106-trunk.txt > > > Sometimes reads/writes to a given partition may cause problems due to the > data present. It would be useful to have a manual way to blacklist such > partitions so all read and write requests to them are rejected. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12106) Add ability to blacklist a CQL partition so all requests are ignored
[ https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16422742#comment-16422742 ] Ben Bromhead commented on CASSANDRA-12106: -- +1 to just setting it at the single node level. > Add ability to blacklist a CQL partition so all requests are ignored > > > Key: CASSANDRA-12106 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12106 > Project: Cassandra > Issue Type: New Feature >Reporter: Geoffrey Yu >Assignee: Sumanth Pasupuleti >Priority: Minor > Fix For: 4.x > > Attachments: 12106-trunk.txt > > > Sometimes reads/writes to a given partition may cause problems due to the > data present. It would be useful to have a manual way to blacklist such > partitions so all read and write requests to them are rejected. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12106) Add ability to blacklist a CQL partition so all requests are ignored
[ https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418456#comment-16418456 ] Sumanth Pasupuleti commented on CASSANDRA-12106: Taking an incremental approach, let's implement this at a single node level (like through JMX), and then handle this in a distributed manner at the cluster level. I shall start on working on a patch for this. > Add ability to blacklist a CQL partition so all requests are ignored > > > Key: CASSANDRA-12106 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12106 > Project: Cassandra > Issue Type: New Feature >Reporter: Geoffrey Yu >Assignee: Geoffrey Yu >Priority: Minor > Fix For: 4.x > > Attachments: 12106-trunk.txt > > > Sometimes reads/writes to a given partition may cause problems due to the > data present. It would be useful to have a manual way to blacklist such > partitions so all read and write requests to them are rejected. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12106) Add ability to blacklist a CQL partition so all requests are ignored
[ https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16182138#comment-16182138 ] Vinay Chella commented on CASSANDRA-12106: -- +1 on this, I agree with Jeff on making it a `system_distributed` table instead of waiting on CASSANDRA-12345 if that is doable/ possible. > Add ability to blacklist a CQL partition so all requests are ignored > > > Key: CASSANDRA-12106 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12106 > Project: Cassandra > Issue Type: New Feature >Reporter: Geoffrey Yu >Assignee: Geoffrey Yu >Priority: Minor > Fix For: 4.x > > Attachments: 12106-trunk.txt > > > Sometimes reads/writes to a given partition may cause problems due to the > data present. It would be useful to have a manual way to blacklist such > partitions so all read and write requests to them are rejected. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12106) Add ability to blacklist a CQL partition so all requests are ignored
[ https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181655#comment-16181655 ] Jeff Jirsa commented on CASSANDRA-12106: Came up at NGCC that a number of different users are implementing something along these lines in an ad hoc sense. May be time to revisit this ticket. > Add ability to blacklist a CQL partition so all requests are ignored > > > Key: CASSANDRA-12106 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12106 > Project: Cassandra > Issue Type: New Feature >Reporter: Geoffrey Yu >Assignee: Geoffrey Yu >Priority: Minor > Fix For: 4.x > > Attachments: 12106-trunk.txt > > > Sometimes reads/writes to a given partition may cause problems due to the > data present. It would be useful to have a manual way to blacklist such > partitions so all read and write requests to them are rejected. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-12106) Add ability to blacklist a CQL partition so all requests are ignored
[ https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15405854#comment-15405854 ] Jason Brown commented on CASSANDRA-12106: - The solution we've discussed internally is to use CASSANDRA-12347 as a generic transport mechanism to replicate the blacklisted partitions to each, which eliminates all the special message/verb handling aspects of the initial version of this patch. What would remain, then, is a cache of the blacklisted partitions and then the code on the read and write paths to look up values in the cache. > Add ability to blacklist a CQL partition so all requests are ignored > > > Key: CASSANDRA-12106 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12106 > Project: Cassandra > Issue Type: New Feature >Reporter: Geoffrey Yu >Assignee: Geoffrey Yu >Priority: Minor > Fix For: 4.x > > Attachments: 12106-trunk.txt > > > Sometimes reads/writes to a given partition may cause problems due to the > data present. It would be useful to have a manual way to blacklist such > partitions so all read and write requests to them are rejected. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12106) Add ability to blacklist a CQL partition so all requests are ignored
[ https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15403647#comment-15403647 ] Benjamin Lerer commented on CASSANDRA-12106: bq. Regarding keeping this cache b/w co-ordinator and replicas, we chose replica since it is more efficient to keep it there since this list can grow and we want to reduce the things in memory. In my opinion, we should probably look at the pros and cons of each approach. Implementing the functionnalty at the coordinator level will result in a much smaller change. The logic will be easier to understand/maintain and there will less chances to let a problem slip through. I agree, that amount of memory used could be a problem at some point. How much memory do you think you will need in the worst case scenario that you have seen so far? Is it really unrealistic to have it at the coordinator level? > Add ability to blacklist a CQL partition so all requests are ignored > > > Key: CASSANDRA-12106 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12106 > Project: Cassandra > Issue Type: New Feature >Reporter: Geoffrey Yu >Assignee: Geoffrey Yu >Priority: Minor > Fix For: 4.x > > Attachments: 12106-trunk.txt > > > Sometimes reads/writes to a given partition may cause problems due to the > data present. It would be useful to have a manual way to blacklist such > partitions so all read and write requests to them are rejected. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12106) Add ability to blacklist a CQL partition so all requests are ignored
[ https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387760#comment-15387760 ] Jason Brown commented on CASSANDRA-12106: - [~slebresne]: The [thundering herd problem|https://en.wikipedia.org/wiki/Thundering_herd_problem] is one of the main concerns, as it will suck up all resources just processing that partition > Add ability to blacklist a CQL partition so all requests are ignored > > > Key: CASSANDRA-12106 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12106 > Project: Cassandra > Issue Type: New Feature >Reporter: Geoffrey Yu >Assignee: Geoffrey Yu >Priority: Minor > Fix For: 4.x > > Attachments: 12106-trunk.txt > > > Sometimes reads/writes to a given partition may cause problems due to the > data present. It would be useful to have a manual way to blacklist such > partitions so all read and write requests to them are rejected. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12106) Add ability to blacklist a CQL partition so all requests are ignored
[ https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387746#comment-15387746 ] sankalp kohli commented on CASSANDRA-12106: --- The patch which we will contribute for IndexInfo makes it better but we will still need a limit on partition sizes. I agree we cannot keep adding features to Cassandra but this is the top requested feature we get from ops teams. I guess thats why we have features like large batch rejection, CASSANDRA-8303, CASSANDRA-9318, etc. We can see how we can simplify the implementation. Since we need to reject requests, we will need to keep it in memory and hence the cache. Regarding keeping this cache b/w co-ordinator and replicas, we chose replica since it is more efficient to keep it there since this list can grow and we want to reduce the things in memory. > Add ability to blacklist a CQL partition so all requests are ignored > > > Key: CASSANDRA-12106 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12106 > Project: Cassandra > Issue Type: New Feature >Reporter: Geoffrey Yu >Assignee: Geoffrey Yu >Priority: Minor > Fix For: 4.x > > Attachments: 12106-trunk.txt > > > Sometimes reads/writes to a given partition may cause problems due to the > data present. It would be useful to have a manual way to blacklist such > partitions so all read and write requests to them are rejected. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12106) Add ability to blacklist a CQL partition so all requests are ignored
[
https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387687#comment-15387687
]
Sylvain Lebresne commented on CASSANDRA-12106:
--
bq. but neither is it entirely rare that a 'bad' partition gets a cluster into
a bad state (not exclusively the due to IndexInfo GC-pressure, either).
I'll admit that what bothers me a bit here is that you guys keeps referring to
some mysterious partition related problem(s) that brings down entire cluster
and that isn't what we already know ({{IndexInfo}} problems, which we are
fixing), implying some underlying problem(s) we're not aware of, but you don't
seem willing to give any more detail than that.
> Add ability to blacklist a CQL partition so all requests are ignored
>
>
> Key: CASSANDRA-12106
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12106
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Geoffrey Yu
>Assignee: Geoffrey Yu
>Priority: Minor
> Fix For: 4.x
>
> Attachments: 12106-trunk.txt
>
>
> Sometimes reads/writes to a given partition may cause problems due to the
> data present. It would be useful to have a manual way to blacklist such
> partitions so all read and write requests to them are rejected.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12106) Add ability to blacklist a CQL partition so all requests are ignored
[
https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387642#comment-15387642
]
Jason Brown commented on CASSANDRA-12106:
-
bq. it is not Cassandra's job to protect against malicious clients
True enough, but it would be great if we could prevent an entire cluster from
falling over under due to (some degree of) adverse conditions.
bq. bit too hand-wavy to my taste
We absolutely have these problems, in production, at our scale. It is not an
everyday occurrence, but neither is it entirely rare that a 'bad' partition
gets a cluster into a bad state (not exclusively the due to {{IndexInfo}}
GC-pressure, either). Thus, I think the desire to add additional resilience to
the database is a reasonable idea.
I worked with [~geoffxy] on some of the initial ideas for this patch, and
perhaps the implementation ended up a bit large, yet I would argue thorough.
That being said, I can work with Geoff on modifying/simplifying his patch to
perhaps something like [~slebresne]'s suggestion assuming it is a) workable and
b) meets the goals for this ticket and our production needs.
> Add ability to blacklist a CQL partition so all requests are ignored
>
>
> Key: CASSANDRA-12106
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12106
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Geoffrey Yu
>Assignee: Geoffrey Yu
>Priority: Minor
> Fix For: 4.x
>
> Attachments: 12106-trunk.txt
>
>
> Sometimes reads/writes to a given partition may cause problems due to the
> data present. It would be useful to have a manual way to blacklist such
> partitions so all read and write requests to them are rejected.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12106) Add ability to blacklist a CQL partition so all requests are ignored
[
https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387421#comment-15387421
]
Sylvain Lebresne commented on CASSANDRA-12106:
--
bq. There are cases in which there are large partitions due to spammers, etc
We've always conscientiously say that it is not Cassandra's job to protect
against malicious clients. You should not expose C* to the wild and spammers is
something you should deal with at the application level.
More generally though, and to repeat myself a bit, we shouldn't feature creep
C* so any new feature should have decently general usefulness and
justification. And it's also a pretty big patch (lots of code to maintain)
which multiply the requirement on general usefulness. Personally, I'm not
convinced of that general usefulness and arguments like "Sometimes reads/writes
to a given partition may cause problems due to the data present", "There are
cases" or "this could be very common based on the app" are a bit too hand-wavy
to my taste.
I welcome other devs and user input on this, and I could revise my judgment if
it becomes clear that getting this feature is important to a large enough
amount of users that the maintenance cost becomes justified, but I'm currently
-1 on this particular patch.
I want to insist that I'm not so much against the idea of the ticket in general
(though I do am not entirely fan of it as I'd rather deal with the source of
problems leading to wanting to blacklist of partition) than I am put off by my
perceived mismatch between the relatively minor usefulness and the large
complexity involved by the patch (things like having to deal with cache
consistency on topology changes for instance).
However, if we were to consider the much (much) simpler approach of, say, add a
{{blacklisted_partitions}} (schema) property to tables and simply reject
queries that cover any of the key in that set on the coordinator, then I'd be
much more receptive to that.
> Add ability to blacklist a CQL partition so all requests are ignored
>
>
> Key: CASSANDRA-12106
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12106
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Geoffrey Yu
>Assignee: Geoffrey Yu
>Priority: Minor
> Fix For: 4.x
>
> Attachments: 12106-trunk.txt
>
>
> Sometimes reads/writes to a given partition may cause problems due to the
> data present. It would be useful to have a manual way to blacklist such
> partitions so all read and write requests to them are rejected.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12106) Add ability to blacklist a CQL partition so all requests are ignored
[ https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386515#comment-15386515 ] sankalp kohli commented on CASSANDRA-12106: --- Regarding fixing the partitions after it is blocked, many times we want to just delete the CQL partition by inserting a row tombstone but deciding that requires app teams to be involved which takes time. You cannot have an outage till it is decided what to you with this CQL partition. We can argue that it could be caught during testing of the app but in real world things happen more often in prod when you run at large scale. > Add ability to blacklist a CQL partition so all requests are ignored > > > Key: CASSANDRA-12106 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12106 > Project: Cassandra > Issue Type: New Feature >Reporter: Geoffrey Yu >Assignee: Geoffrey Yu >Priority: Minor > Fix For: 4.x > > Attachments: 12106-trunk.txt > > > Sometimes reads/writes to a given partition may cause problems due to the > data present. It would be useful to have a manual way to blacklist such > partitions so all read and write requests to them are rejected. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12106) Add ability to blacklist a CQL partition so all requests are ignored
[ https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15386505#comment-15386505 ] sankalp kohli commented on CASSANDRA-12106: --- I dont think you are negative here :) There are cases in which there are large partitions due to spammers, etc which we need to blacklist for the time being till it is fixed. We cannot have one partition take down the cluster while it is fixed. Uptime is important and hence this feature. The reason this patch is big is because we want to make the foundation so that we can automatically blacklist bad partitions in the future. Regarding occurrence of this, this could be very common based on the app using Cassandra. Also with CASSANDRA-9754, there is still a limit you want CQL partitions to grow. Regarding the unit test, Geoff is working on it and will also include a Dtest for this. > Add ability to blacklist a CQL partition so all requests are ignored > > > Key: CASSANDRA-12106 > URL: https://issues.apache.org/jira/browse/CASSANDRA-12106 > Project: Cassandra > Issue Type: New Feature >Reporter: Geoffrey Yu >Assignee: Geoffrey Yu >Priority: Minor > Fix For: 4.x > > Attachments: 12106-trunk.txt > > > Sometimes reads/writes to a given partition may cause problems due to the > data present. It would be useful to have a manual way to blacklist such > partitions so all read and write requests to them are rejected. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CASSANDRA-12106) Add ability to blacklist a CQL partition so all requests are ignored
[
https://issues.apache.org/jira/browse/CASSANDRA-12106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15385644#comment-15385644
]
Sylvain Lebresne commented on CASSANDRA-12106:
--
I'm going to play bad cop here (again?), but while I haven't looked at the
detail, it's not immediately clear to me that the usefulness justifies the
added complexity (it's a 109kb patch (with 0 tests), and it looks mostly
additive).
I get that you can have problems with a partition, but as far as I can say it's
mostly either:
# some data is corrupted for the partition. We already kind of handle that
through our failure policies/sstable blacklisting.
# the partition has grown too large. The main issue I know of in that case is
the big index to deserialize, but that's been partly addressed in
CASSANDRA-11206, and we have further ongoing improvement plans with
CASSANDRA-9754.
# the partition has too many tombstones. We already have tombstone thresholds
for that. Granted, blacklisting the partition would avoid the work that is
uselessly done before the {{TombstoneOverwhelmingException}} is thrown, but I'm
not sure it's such a meaningful difference that it justify a big patch, or even
the feature creeping in general.
A priori, I feel that, given the other mechanism we already have in place, the
feature is a bit niche (and there is such a thing as feature creeping).
I'm also not that sure it's the best way to handle a problem with a partition.
I mean, you have a problem with a partition and you decide to manually
blacklist it. And then what? Surely the next step should be to fix whatever
problem you have to restore service on that partition, but it feels harder to
do when you have prevented all access to that partition.
Anyway, even if we do decide it's sufficiently useful to add, I'm even less
convinced it warrants that level of complexity. Surely, having a partition get
so bad you have to blacklist is a pretty rare occurrence (otherwise, you're
doing something really wrong and that should have been caught in testing), and
a temporary one (one would hope you'd take action to fix whatever the issue is,
which as said above, is unclear to me how you do that). So that it should be
fine to have just a single list of blacklisted keys (per table) shared by all
nodes. Doing so should drastically simply this.
I'll note that I'm sorry to come so negative when you have obviously put a lot
of effort into this, but this is really why we should always at least sketch
some design on tickets and discuss them before implementing.
> Add ability to blacklist a CQL partition so all requests are ignored
>
>
> Key: CASSANDRA-12106
> URL: https://issues.apache.org/jira/browse/CASSANDRA-12106
> Project: Cassandra
> Issue Type: New Feature
>Reporter: Geoffrey Yu
>Assignee: Geoffrey Yu
>Priority: Minor
> Fix For: 4.x
>
> Attachments: 12106-trunk.txt
>
>
> Sometimes reads/writes to a given partition may cause problems due to the
> data present. It would be useful to have a manual way to blacklist such
> partitions so all read and write requests to them are rejected.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
