Re: Evolving the client protocol

"Bullied"? Neither me nor anyone else made any demands or threats. I proposed cooperation, and acknowledged up front, in my first email, that cooperation might not be wanted by Cassandra. On 2018-04-28 20:50, Jeff Jirsa wrote: You're a committer Mick, if you think it belongs in the

Re: Evolving the client protocol

Jeff, the use of the terms "business concerns" and "technical merit" may be an unfortunate choice I made, they can be construed in a narrower way than I intended and i apologise Jeff for that confusion. Our community and how we look after it does matter. The hope I raised was only that we

Re: Evolving the client protocol

On Sat, Apr 28, 2018 at 4:49 AM, mck wrote: > We should, as open source contributors, put business concerns to the side > and welcome opportunities to work across company and product lines. > I resent the fact that you're calling this a business concern. This isn't a business

Re: Evolving the client protocol

Mick - reference Scylla's recent blog post where Dor speaks directly about the majority of their users migrating there from the Apache Cassandra ecosystem. This isn't about business concerns being first, this is about community concerns being first. On Sat, Apr 28, 2018 at 7:49 AM, mck

Re: Evolving the client protocol

> Let met just say that as an observer to this conversation -- and someone > who believes that compatibility, extensibility, and frankly competition > bring out the best in products -- I'm fairly surprised and disappointed > with the apparent hostility many community members have shown toward a >

Re: Evolving the client protocol

They aren't even remotely similar, they're VERY different. Here's a few starting points: 1) Most of Datastax's work for the first 5, 6, 8 years of existence focused on driving users to cassandra from other DBs (see all of the "Cassandra Summits" that eventually created trademark friction) ;

Re: Evolving the client protocol

The main point is that we decided to take a strategic decision to invest in the client side. We always wanted to get to the state but for natural reasons, it took us a while. The client side changes aren't just about a small feature here and there or stop at thread per core. Think about the

Re: Evolving the client protocol

On 2018-04-24 04:18, Nate McCall wrote: Folks, Before this goes much further, let's take a step back for a second. I am hearing the following: Folks are fine with CASSANDRA-14311 and CASSANDRA-2848 *BUT* they don't make much sense from the project's perspective without a reference

Re: Evolving the client protocol

Eric, You have to understand the poisonous GPL. It's very different from Apache licensing in the sense that, roughly speaking, you're welcome to contribute to Scylla, but legally barred from distributing it with or inside any product you base on it unless your product source code is also

Re: Evolving the client protocol

DataStax invested millions of dollars into Cassandra, tens of thousands of man hours, hosted hundreds of events and has been a major factor in the success of the project. ScyllaDB wants us to change the C* protocol in order to improve features in a competing database which contributes nothing

Re: Evolving the client protocol

Let met just say that as an observer to this conversation -- and someone who believes that compatibility, extensibility, and frankly competition bring out the best in products -- I'm fairly surprised and disappointed with the apparent hostility many community members have shown toward a sincere

Re: Evolving the client protocol

On 2018-04-23 17:59, Ben Bromhead wrote: >> This doesn't work without additional changes, for RF>1. The token ring could place two replicas of the same token range on the same physical server, even though those are two separate cores of the same server. You could add another

Re: Evolving the client protocol

I have not asked this list to do any work on the drivers. If Cassandra agrees to Scylla protocol changes (either proactively or retroactively) then the benefit to Cassandra is that if the drivers are changed (by the driver maintainers or by Scylla developers) then Cassandra developers need

Re: Evolving the client protocol

On Mon, Apr 23, 2018 at 9:13 PM, San Luoji wrote: > Dor, > > Setting the Thread Per Core code aside, will your developers commit to > contribute back both https://issues.apache.org/jira/browse/CASSANDRA-2848 > and https://issues.apache.org/jira/browse/CASSANDRA-14311? > >

Re: Evolving the client protocol

Dor, Setting the Thread Per Core code aside, will your developers commit to contribute back both https://issues.apache.org/jira/browse/CASSANDRA-2848 and https://issues.apache.org/jira/browse/CASSANDRA-14311? Looks like CASSANDRA-2848 has stalled even though some respectable work was done, and

Re: Evolving the client protocol

Respectfully, there’s pretty much already apparent consensus among those with a vote (unless I missed some dissenting opinion while I was on vacation). Its been expressed multiple times by committers and members of the PMC that it’s Cassandra native protocol, it belongs in the protocol when

Re: Evolving the client protocol

Apologies Nate - didn't realize I'd overlapped with you stepping in and trying to bring us all back to reason. I'll take my leave of the conversation at this point. :) On Mon, Apr 23, 2018 at 9:30 PM, Josh McKenzie wrote: > > Datastax, Apple, Instaclstr, > > thelastpickle

Re: Evolving the client protocol

> Datastax, Apple, Instaclstr, > thelastpickle and everyone else > receive different benefits You have mentioned a variety of vendors who received benefits while making major contributions back to the project. Comparing Scylla's relationship to the Cassandra ecosystem to this list is a false

Re: Evolving the client protocol

Folks, Before this goes much further, let's take a step back for a second. I am hearing the following: Folks are fine with CASSANDRA-14311 and CASSANDRA-2848 *BUT* they don't make much sense from the project's perspective without a reference implementation. I think the shard concept is too

Re: Evolving the client protocol

On Mon, Apr 23, 2018 at 5:28 PM, Josh McKenzie wrote: > > it's not > > reasonable to expect Scylla to contribute > > such a huge effort to the C* server > > But it's reasonable that a major portion of Scylla's business model is > profiting off those huge efforts other

Re: Evolving the client protocol

If you are so concerned about forking protocol, why did you fork the server? Please pick a side and not what is suitable for your Business. On Apr 23, 2018, at 17:28, Josh McKenzie wrote: >> it's not >> reasonable to expect Scylla to contribute >> such a huge effort

Re: Evolving the client protocol

> it's not > reasonable to expect Scylla to contribute > such a huge effort to the C* server But it's reasonable that a major portion of Scylla's business model is profiting off those huge efforts other companies have made? Seems a little hypocritical to me. On Mon, Apr 23, 2018, 8:18 PM Dor

Re: Evolving the client protocol

On Mon, Apr 23, 2018 at 5:03 PM, Sankalp Kohli wrote: > Is one of the “abuse” of Apache license is ScyllaDB which is using > Cassandra but not contributing back? > It's not that we have a private version of Cassandra and we don't release all of it or some of it back..

Re: Evolving the client protocol

Is one of the “abuse” of Apache license is ScyllaDB which is using Cassandra but not contributing back? Happy to be proved wrong as I am not a lawyer and don’t understand various licenses .. > On Apr 23, 2018, at 16:55, Dor Laor wrote: > >> On Mon, Apr 23, 2018 at 4:13 PM,

Re: Evolving the client protocol

On Mon, Apr 23, 2018 at 4:13 PM, Jonathan Haddad wrote: > From where I stand it looks like you've got only two options for any > feature that involves updating the protocol: > > 1. Don't built the feature > 2. Built it in Cassanda & scylladb, update the drivers accordingly >

Re: Evolving the client protocol

>From where I stand it looks like you've got only two options for any feature that involves updating the protocol: 1. Don't built the feature 2. Built it in Cassanda & scylladb, update the drivers accordingly I don't think you have a third option, which is built it only in ScyllaDB, because that

Re: Evolving the client protocol

> >> This doesn't work without additional changes, for RF>1. The token ring > could place two replicas of the same token range on the same physical > server, even though those are two separate cores of the same server. You > could add another element to the hierarchy (cluster -> datacenter -> rack

Re: Evolving the client protocol

On 2018-04-22 23:35, Josh McKenzie wrote: The drivers are not part of Cassandra, so what "the server" is for drivers is up to their maintainer. I'm pretty sure the driver communities don't spend a lot of time worrying about their Scylla compatibility. That's your cross to bear. To clarify,

Re: Evolving the client protocol

On 2018-04-22 18:00, Ariel Weisberg wrote: Hi, This doesn't work without additional changes, for RF>1. The token ring could place two replicas of the same token range on the same physical server, even though those are two separate cores of the same server. You could add another element to

Re: Evolving the client protocol

On Apr 20, 2018, at 5:03 AM, Sylvain Lebresne wrote: >> >> >> Those were just given as examples. Each would be discussed on its own, >> assuming we are able to find a way to cooperate. >> >> >> These are relatively simple and it wouldn't be hard for use to patch >>

Re: Evolving the client protocol

> The drivers are not part of Cassandra, so what "the server" is for drivers is > up to their maintainer. I'm pretty sure the driver communities don't spend a lot of time worrying about their Scylla compatibility. That's your cross to bear. On Sun, Apr 22, 2018 at 11:00 AM, Ariel Weisberg

Re: Evolving the client protocol

Hi, > This doesn't work without additional changes, for RF>1. The token ring could > place two replicas of the same token range on the same physical server, even > though those are two separate cores of the same server. You could add another > element to the hierarchy (cluster -> datacenter ->

Re: Evolving the client protocol

On 2018-04-19 21:15, Ben Bromhead wrote: Re #3: Yup I was thinking each shard/port would appear as a discrete server to the client. This doesn't work without additional changes, for RF>1. The token ring could place two replicas of the same token range on the same physical server, even

Re: Evolving the client protocol

You're right in principle, but in practice we haven't seen problems with the term. On 2018-04-19 20:31, Michael Shuler wrote: This is purely my own opinion, but I find the use of the term 'shard' quite unfortunate in the context of a distributed database. The historical usage of the term has

Re: Evolving the client protocol

On 2018-04-20 12:03, Sylvain Lebresne wrote: Those were just given as examples. Each would be discussed on its own, assuming we are able to find a way to cooperate. These are relatively simple and it wouldn't be hard for use to patch Cassandra. But I want to find a way to make more

Re: Evolving the client protocol

On 2018-04-19 20:43, Ariel Weisberg wrote: Hi, So at technical level I don't understand this yet. So you have a database consisting of single threaded shards and a socket for accept that is generating TCP connections and in advance you don't know which connection is going to send messages

Re: Evolving the client protocol

On 2018-04-19 20:33, Ariel Weisberg wrote: Hi, That basically means a fork in the protocol (perhaps a temporary fork if we go for mode 2 where Cassandra retroactively adopts our protocol changes, if they fit will). Implementing a protocol change may be easy for some simple changes, but in

Re: Evolving the client protocol

The protocol does already support optional/custom payloads to do such things. IIRC the zipkin tracing implementation https://github.com/thelastpickle/cassandra-zipkin-tracing for example uses this to pass the zipkin id to the server. > On Apr 20, 2018, at 1:02 PM, Max C.

Re: Evolving the client protocol

For things like #3, would it be a better idea to propose a generic enhancement for “optional vendor extensions” to the protocol? These extensions would be negotiated during connection formation and then the driver could (optionally) implement these additional features. These extensions would

Re: Evolving the client protocol

> > > Those were just given as examples. Each would be discussed on its own, > assuming we are able to find a way to cooperate. > > > These are relatively simple and it wouldn't be hard for use to patch > Cassandra. But I want to find a way to make more complicated protocol > changes where it

Re: Evolving the client protocol

Sounds interesting. Could 80% of what we gain with a “shard” approach be achieved via Mesos to wrap a stateful service? Technically it’s “Sharding” the whole machine and better utilizing resources. -- Rahul Singh rahul.si...@anant.us Anant Corporation On Apr 19, 2018, 1:23 PM -0500, sankalp

Re: Evolving the client protocol

If you donate Thread per core to C*, I am sure someone can help you review it and get it committed. On Thu, Apr 19, 2018 at 11:15 AM, Ben Bromhead wrote: > Re #3: > > Yup I was thinking each shard/port would appear as a discrete server to the > client. > > If the per port

Re: Evolving the client protocol

Re #3: Yup I was thinking each shard/port would appear as a discrete server to the client. If the per port suggestion is unacceptable due to hardware requirements, remembering that Cassandra is built with the concept scaling *commodity* hardware horizontally, you'll have to spend your time and

Re: Evolving the client protocol

Hi, So at technical level I don't understand this yet. So you have a database consisting of single threaded shards and a socket for accept that is generating TCP connections and in advance you don't know which connection is going to send messages to which shard. What is the mechanism by which

Re: Evolving the client protocol

Hi, >That basically means a fork in the protocol (perhaps a temporary fork if >we go for mode 2 where Cassandra retroactively adopts our protocol >changes, if they fit will). > >Implementing a protocol change may be easy for some simple changes, but >in the general case, it is not realistic to

Re: Evolving the client protocol

This is purely my own opinion, but I find the use of the term 'shard' quite unfortunate in the context of a distributed database. The historical usage of the term has been the notion of data partitions that reside on separate database servers. There is a learning curve with distributed databases,

Re: Evolving the client protocol

On 2018-04-19 10:19, kurt greaves wrote: 1. The protocol change is developed using the Cassandra process in a JIRA ticket, culminating in a patch to doc/native_protocol*.spec when consensus is achieved. I don't think forking would be desirable (for anyone) so this seems the most reasonable to

Re: Evolving the client protocol

On 2018-04-19 19:10, Ariel Weisberg wrote: Hi, I think that updating the protocol spec to Cassandra puts the onus on the party changing the protocol specification to have an implementation of the spec in Cassandra as well as the Java and Python driver (those are both used in the Cassandra

Re: Evolving the client protocol

Port-per-shard is likely the easiest option but it's too ugly to contemplate. We run on machines with 160 shards (IBM POWER 2s20c160t IIRC), it will be just horrible to have 160 open ports. It also doesn't fit will with the NICs ability to automatically distribute packets among cores using

Re: Evolving the client protocol

WRT to #3 To fit in the existing protocol, could you have each shard listen on a different port? Drivers are likely going to support this due to https://issues.apache.org/jira/browse/CASSANDRA-7544 ( https://issues.apache.org/jira/browse/CASSANDRA-11596). I'm not super familiar with the ticket so

Re: Evolving the client protocol

Hi, I think that updating the protocol spec to Cassandra puts the onus on the party changing the protocol specification to have an implementation of the spec in Cassandra as well as the Java and Python driver (those are both used in the Cassandra repo). Until it's implemented in Cassandra we

Re: Evolving the client protocol

On 2018/04/19 07:19:27, kurt greaves wrote: > > > > 1. The protocol change is developed using the Cassandra process in a JIRA > > ticket, culminating in a patch to doc/native_protocol*.spec when consensus > > is achieved. > > I don't think forking would be desirable (for

Re: Evolving the client protocol

> > 1. The protocol change is developed using the Cassandra process in a JIRA > ticket, culminating in a patch to doc/native_protocol*.spec when consensus > is achieved. I don't think forking would be desirable (for anyone) so this seems the most reasonable to me. For 1 and 2 it certainly makes

Re: Evolving the client protocol

Avi, if this is something that matters to you, you're more than welcome to submit a patch to both this project and to the different drivers. Getting the first 2 optimizations into 4.0 would be nice, I'm sure someone would be happy to work with you on it. The third, I can't see why we'd need that

Re: Evolving the client protocol

Do we have driver authors who wish to support both projects? On Wed, Apr 18, 2018 at 9:25 AM, Jeff Jirsa wrote: > Removed other lists (please don't cross post) > > > > > > On Wed, Apr 18, 2018 at 3:47 AM, Avi Kivity wrote: > > > Hello Cassandra developers,

Re: Evolving the client protocol

BTW, Folks from Cassandra apparently didn'tr eceive this message. On Wed, Apr 18, 2018 at 6:47 AM, Avi Kivity wrote: > Hello Cassandra developers, > > > We're starting to see client protocol limitations impact performance, and > so we'd like to evolve the protocol to remove

Re: Evolving the client protocol

Removed other lists (please don't cross post) On Wed, Apr 18, 2018 at 3:47 AM, Avi Kivity wrote: > Hello Cassandra developers, > > > We're starting to see client protocol limitations impact performance, and > so we'd like to evolve the protocol to remove the limitations.

Evolving the client protocol

Hello Cassandra developers, We're starting to see client protocol limitations impact performance, and so we'd like to evolve the protocol to remove the limitations. In order to avoid fragmenting the driver ecosystem and reduce work duplication for driver authors, we'd like to avoid forking