Re: Pluggable throttling of read and write queries

2017-02-22 Thread Abhishek Verma
On Wed, Feb 22, 2017 at 4:01 PM, Jay Zhuang wrote: > Here is the Scheduler interface: https://github.com/apache/cass > andra/blob/cassandra-3.11/conf/cassandra.yaml#L978 > Seems like it could be used for this case. The IRequestScheduler is only used in the thrift

Re: Pluggable throttling of read and write queries

2017-02-22 Thread Jay Zhuang
Here is the Scheduler interface: https://github.com/apache/cassandra/blob/cassandra-3.11/conf/cassandra.yaml#L978 Seems like it could be used for this case. It is removed in 4.x with thrift, not sure why: https://github.com/apache/cassandra/commit/4881d9c308ccd6b5ca70925bf6ebedb70e7705fc

Re: Pluggable throttling of read and write queries

2017-02-22 Thread Eric Stevens
> We’ve actually had several customers where we’ve done the opposite - split large clusters apart to separate uses cases We do something similar but for a single application. We're functionally sharding data to different clusters from a single application. We can have different server classes

Re: Pluggable throttling of read and write queries

2017-02-22 Thread Edward Capriolo
On Wed, Feb 22, 2017 at 1:20 PM, Abhishek Verma wrote: > We have lots of dedicated Cassandra clusters for large use cases, but we > have a long tail of (~100) of internal customers who want to store < 200GB > of data with < 5k qps and non-critical data. It does not make sense to

Re: Pluggable throttling of read and write queries

2017-02-22 Thread Abhishek Verma
We have lots of dedicated Cassandra clusters for large use cases, but we have a long tail of (~100) of internal customers who want to store < 200GB of data with < 5k qps and non-critical data. It does not make sense to create a 3 node dedicated cluster for each of these small use cases. So we have

Re: Pluggable throttling of read and write queries

2017-02-20 Thread vincent gromakowski
Aren't you using mesos Cassandra framework to manage your multiple clusters ? (Seen a presentation in cass summit) What's wrong with your current mesos approach ? I am also thinking it's better to split a large cluster into smallers except if you also manage client layer that query cass and you

Pluggable throttling of read and write queries

2017-02-20 Thread Edward Capriolo
Older versions had a request scheduler api. On Monday, February 20, 2017, Ben Slater > wrote: > We’ve actually had several customers where we’ve done the opposite - split > large clusters apart to separate

Re: Pluggable throttling of read and write queries

2017-02-20 Thread Ben Slater
We’ve actually had several customers where we’ve done the opposite - split large clusters apart to separate uses cases. We found that this allowed us to better align hardware with use case requirements (for example using AWS c3.2xlarge for very hot data at low latency, m4.xlarge for more general

Re: Pluggable throttling of read and write queries

2017-02-20 Thread Oleksandr Shulgin
On Sat, Feb 18, 2017 at 3:12 AM, Abhishek Verma wrote: > Cassandra is being used on a large scale at Uber. We usually create > dedicated clusters for each of our internal use cases, however that is > difficult to scale and manage. > > We are investigating the approach of using a

Pluggable throttling of read and write queries

2017-02-17 Thread Abhishek Verma
Cassandra is being used on a large scale at Uber. We usually create dedicated clusters for each of our internal use cases, however that is difficult to scale and manage. We are investigating the approach of using a single shared cluster with 100s of nodes and handle 10s to 100s of different use