Re: Evolving the client protocol

Sounds interesting. Could 80% of what we gain with a “shard” approach be achieved via Mesos to wrap a stateful service? Technically it’s “Sharding” the whole machine and better utilizing resources. -- Rahul Singh rahul.si...@anant.us Anant Corporation On Apr 19, 2018, 1:23 PM -0500, sankalp

Re: Evolving the client protocol

If you donate Thread per core to C*, I am sure someone can help you review it and get it committed. On Thu, Apr 19, 2018 at 11:15 AM, Ben Bromhead wrote: > Re #3: > > Yup I was thinking each shard/port would appear as a discrete server to the > client. > > If the per port

Re: Evolving the client protocol

Re #3: Yup I was thinking each shard/port would appear as a discrete server to the client. If the per port suggestion is unacceptable due to hardware requirements, remembering that Cassandra is built with the concept scaling *commodity* hardware horizontally, you'll have to spend your time and

Re: Evolving the client protocol

Hi, So at technical level I don't understand this yet. So you have a database consisting of single threaded shards and a socket for accept that is generating TCP connections and in advance you don't know which connection is going to send messages to which shard. What is the mechanism by which

Re: Evolving the client protocol

Hi, >That basically means a fork in the protocol (perhaps a temporary fork if >we go for mode 2 where Cassandra retroactively adopts our protocol >changes, if they fit will). > >Implementing a protocol change may be easy for some simple changes, but >in the general case, it is not realistic to

Re: Evolving the client protocol

This is purely my own opinion, but I find the use of the term 'shard' quite unfortunate in the context of a distributed database. The historical usage of the term has been the notion of data partitions that reside on separate database servers. There is a learning curve with distributed databases,

Re: Evolving the client protocol

On 2018-04-19 10:19, kurt greaves wrote: 1. The protocol change is developed using the Cassandra process in a JIRA ticket, culminating in a patch to doc/native_protocol*.spec when consensus is achieved. I don't think forking would be desirable (for anyone) so this seems the most reasonable to

Re: Evolving the client protocol

On 2018-04-19 19:10, Ariel Weisberg wrote: Hi, I think that updating the protocol spec to Cassandra puts the onus on the party changing the protocol specification to have an implementation of the spec in Cassandra as well as the Java and Python driver (those are both used in the Cassandra

Re: Evolving the client protocol

Port-per-shard is likely the easiest option but it's too ugly to contemplate. We run on machines with 160 shards (IBM POWER 2s20c160t IIRC), it will be just horrible to have 160 open ports. It also doesn't fit will with the NICs ability to automatically distribute packets among cores using

Re: Evolving the client protocol

WRT to #3 To fit in the existing protocol, could you have each shard listen on a different port? Drivers are likely going to support this due to https://issues.apache.org/jira/browse/CASSANDRA-7544 ( https://issues.apache.org/jira/browse/CASSANDRA-11596). I'm not super familiar with the ticket so

Re: Evolving the client protocol

Hi, I think that updating the protocol spec to Cassandra puts the onus on the party changing the protocol specification to have an implementation of the spec in Cassandra as well as the Java and Python driver (those are both used in the Cassandra repo). Until it's implemented in Cassandra we

Re: Evolving the client protocol

On 2018/04/19 07:19:27, kurt greaves wrote: > > > > 1. The protocol change is developed using the Cassandra process in a JIRA > > ticket, culminating in a patch to doc/native_protocol*.spec when consensus > > is achieved. > > I don't think forking would be desirable (for

Re: Optimizing queries for partition keys

Hi Sam, Your finding is interesting. Effectively, if the number of bytes to skip is larger than the remaining bytes in the buffer + the buffer size it could be faster to use seek. Feel free to open a JIRA ticket and attach your patch. It will be great if you could add to the ticket your table

Re: Evolving the client protocol

> > 1. The protocol change is developed using the Cassandra process in a JIRA > ticket, culminating in a patch to doc/native_protocol*.spec when consensus > is achieved. I don't think forking would be desirable (for anyone) so this seems the most reasonable to me. For 1 and 2 it certainly makes