Re: Proposing an Apache Cassandra Management process

[Dinesh Joshi]

Thank you all for the feedback. Nate made a Google doc with the proposal in it 
and is linked off of the ticket. I will try to flesh it out as I gather your 
input.
Dinesh 

On Wednesday, April 18, 2018, 5:12:49 PM PDT, kurt greaves 
 wrote:  
 
 For anyone looking Dinesh made a ticket already: CASSANDRA-14395


On 18 April 2018 at 18:14, Vinay Chella  wrote:

> This is a good initiative. We have advocated for and run a sidecar for the
> past 5+ years, and we've learned and improved it a lot. We look forward to
> moving features from Priam (such as backup, HTTP -> JMX, etc) incrementally
> to this sidecar as they make sense.
>
>
> Thanks,
> Vinay Chella
>
> On Fri, Apr 13, 2018 at 7:01 AM, Eric Evans 
> wrote:
>
> > On Thu, Apr 12, 2018 at 4:41 PM, Dinesh Joshi
> >  wrote:
> > > Hey all -
> > > With the uptick in discussion around Cassandra operability and after
> > discussing potential solutions with various members of the community, we
> > would like to propose the addition of a management process/sub-project
> into
> > Apache Cassandra. The process would be responsible for common operational
> > tasks like bulk execution of nodetool commands, backup/restore, and
> health
> > checks, among others. We feel we have a proposal that will garner some
> > discussion and debate but is likely to reach consensus.
> > > While the community, in large part, agrees that these features should
> > exist “in the database”, there is debate on how they should be
> implemented.
> > Primarily, whether or not to use an external process or build on
> > CassandraDaemon. This is an important architectural decision but we feel
> > the most critical aspect is not where the code runs but that the operator
> > still interacts with the notion of a single database. Multi-process
> > databases are as old as Postgres and continue to be common in newer
> systems
> > like Druid. As such, we propose a separate management process for the
> > following reasons:
> > >
> > >    - Resource isolation & Safety: Features in the management process
> > will not affect C*'s read/write path which is critical for stability. An
> > isolated process has several technical advantages including preventing
> use
> > of unnecessary dependencies in CassandraDaemon, separation of JVM
> resources
> > like thread pools and heap, and preventing bugs from adversely affecting
> > the main process. In particular, GC tuning can be done separately for the
> > two processes, hopefully helping to improve, or at least not adversely
> > affect, tail latencies of the main process.
> > >
> > >    - Health Checks & Recovery: Currently users implement health checks
> > in their own sidecar process. Implementing them in the serving process
> does
> > not make sense because if the JVM running the CassandraDaemon goes south,
> > the healthchecks and potentially any recovery code may not be able to
> run.
> > Having a management process running in isolation opens up the possibility
> > to not only report the health of the C* process such as long GC pauses or
> > stuck JVM but also to recover from it. Having a list of basic health
> checks
> > that are tested with every C* release and officially supported will help
> > boost confidence in C* quality and make it easier to operate.
> > >
> > >    - Reduced Risk: By having a separate Daemon we open the possibility
> > to contribute features that otherwise would not have been considered
> before
> > eg. a UI. A library that started many background threads and is operated
> > completely differently would likely be considered too risky for
> > CassandraDaemon but is a good candidate for the management process.
> >
> > Makes sense IMO.
> >
> > > What can go into the management process?
> > >    - Features that are non-essential for serving reads & writes for eg.
> > Backup/Restore or Running Health Checks against the CassandraDaemon, etc.
> > >
> > >    - Features that do not make the management process critical for
> > functioning of the serving process. In other words, if someone does not
> > wish to use this management process, they are free to disable it.
> > >
> > > We would like to initially build minimal set of features such as health
> > checks and bulk commands into the first iteration of the management
> > process. We would use the same software stack that is used to build the
> > current CassandraDaemon binary. This would be critical for sharing code
> > between CassandraDaemon & management processes. The code should live
> > in-tree to make this easy.
> > > With regards to more in-depth features like repair scheduling and
> > discussions around compaction in or out of CassandraDaemon, while the
> > management process may be a suitable host, it is not our goal to decide
> > that at this time. The management process could be used in these cases,
> as
> > they meet 

Re: Proposing an Apache Cassandra Management process

[kurt greaves]

For anyone looking Dinesh made a ticket already: CASSANDRA-14395


On 18 April 2018 at 18:14, Vinay Chella  wrote:

> This is a good initiative. We have advocated for and run a sidecar for the
> past 5+ years, and we've learned and improved it a lot. We look forward to
> moving features from Priam (such as backup, HTTP -> JMX, etc) incrementally
> to this sidecar as they make sense.
>
>
> Thanks,
> Vinay Chella
>
> On Fri, Apr 13, 2018 at 7:01 AM, Eric Evans 
> wrote:
>
> > On Thu, Apr 12, 2018 at 4:41 PM, Dinesh Joshi
> >  wrote:
> > > Hey all -
> > > With the uptick in discussion around Cassandra operability and after
> > discussing potential solutions with various members of the community, we
> > would like to propose the addition of a management process/sub-project
> into
> > Apache Cassandra. The process would be responsible for common operational
> > tasks like bulk execution of nodetool commands, backup/restore, and
> health
> > checks, among others. We feel we have a proposal that will garner some
> > discussion and debate but is likely to reach consensus.
> > > While the community, in large part, agrees that these features should
> > exist “in the database”, there is debate on how they should be
> implemented.
> > Primarily, whether or not to use an external process or build on
> > CassandraDaemon. This is an important architectural decision but we feel
> > the most critical aspect is not where the code runs but that the operator
> > still interacts with the notion of a single database. Multi-process
> > databases are as old as Postgres and continue to be common in newer
> systems
> > like Druid. As such, we propose a separate management process for the
> > following reasons:
> > >
> > >- Resource isolation & Safety: Features in the management process
> > will not affect C*'s read/write path which is critical for stability. An
> > isolated process has several technical advantages including preventing
> use
> > of unnecessary dependencies in CassandraDaemon, separation of JVM
> resources
> > like thread pools and heap, and preventing bugs from adversely affecting
> > the main process. In particular, GC tuning can be done separately for the
> > two processes, hopefully helping to improve, or at least not adversely
> > affect, tail latencies of the main process.
> > >
> > >- Health Checks & Recovery: Currently users implement health checks
> > in their own sidecar process. Implementing them in the serving process
> does
> > not make sense because if the JVM running the CassandraDaemon goes south,
> > the healthchecks and potentially any recovery code may not be able to
> run.
> > Having a management process running in isolation opens up the possibility
> > to not only report the health of the C* process such as long GC pauses or
> > stuck JVM but also to recover from it. Having a list of basic health
> checks
> > that are tested with every C* release and officially supported will help
> > boost confidence in C* quality and make it easier to operate.
> > >
> > >- Reduced Risk: By having a separate Daemon we open the possibility
> > to contribute features that otherwise would not have been considered
> before
> > eg. a UI. A library that started many background threads and is operated
> > completely differently would likely be considered too risky for
> > CassandraDaemon but is a good candidate for the management process.
> >
> > Makes sense IMO.
> >
> > > What can go into the management process?
> > >- Features that are non-essential for serving reads & writes for eg.
> > Backup/Restore or Running Health Checks against the CassandraDaemon, etc.
> > >
> > >- Features that do not make the management process critical for
> > functioning of the serving process. In other words, if someone does not
> > wish to use this management process, they are free to disable it.
> > >
> > > We would like to initially build minimal set of features such as health
> > checks and bulk commands into the first iteration of the management
> > process. We would use the same software stack that is used to build the
> > current CassandraDaemon binary. This would be critical for sharing code
> > between CassandraDaemon & management processes. The code should live
> > in-tree to make this easy.
> > > With regards to more in-depth features like repair scheduling and
> > discussions around compaction in or out of CassandraDaemon, while the
> > management process may be a suitable host, it is not our goal to decide
> > that at this time. The management process could be used in these cases,
> as
> > they meet the criteria above, but other technical/architectural reasons
> may
> > exists for why it should not be.
> > > We are looking forward to your comments on our proposal,
> >
> > Sounds good to me.
> >
> > Personally, I'm a little less interested in things like
> > 

Re: Evolving the client protocol

[Jonathan Haddad]

Avi, if this is something that matters to you, you're more than welcome to
submit a patch to both this project and to the different drivers.  Getting
the first 2 optimizations into 4.0 would be nice, I'm sure someone would be
happy to work with you on it.

The third, I can't see why we'd need that right now.  It's going to be more
of an uphill battle, since we currently would have no notion of a shard in
Cassandra.  If you want to work on Thread Per Core for Cassandra 5.0 it
seems like it would be a reasonable addition to the protocol.

On Wed, Apr 18, 2018 at 1:24 PM sankalp kohli 
wrote:

> Do we have driver authors who wish to support both projects?
>
> On Wed, Apr 18, 2018 at 9:25 AM, Jeff Jirsa  wrote:
>
> > Removed other lists (please don't cross post)
> >
> >
> >
> >
> >
> > On Wed, Apr 18, 2018 at 3:47 AM, Avi Kivity  wrote:
> >
> > > Hello Cassandra developers,
> > >
> > >
> > > We're starting to see client protocol limitations impact performance,
> and
> > > so we'd like to evolve the protocol to remove the limitations. In order
> > to
> > > avoid fragmenting the driver ecosystem and reduce work duplication for
> > > driver authors, we'd like to avoid forking the protocol. Since these
> > issues
> > > affect Cassandra, either now or in the future, I'd like to cooperate on
> > > protocol development.
> > >
> > >
> > > Some issues that we'd like to work on near-term are:
> > >
> > >
> > > 1. Token-aware range queries
> > >
> > >
> > > When the server returns a page in a range query, it will also return a
> > > token to continue on. In case that token is on a different node, the
> > client
> > > selects a new coordinator based on the token. This eliminates a network
> > hop
> > > for range queries.
> > >
> > >
> > > For the first page, the PREPARE message returns information allowing
> the
> > > client to compute where the first page is held, given the query
> > parameters.
> > > This is just information identifying how to compute the token, given
> the
> > > query parameters (non-range queries already do this).
> > >
> > >
> > > https://issues.apache.org/jira/browse/CASSANDRA-14311
> > >
> > >
> > > 2. Per-request timeouts
> > >
> > >
> > > Allow each request to have its own timeout. This allows the user to set
> > > short timeouts on business-critical queries that are invalid if not
> > served
> > > within a short time, long timeouts for scanning or indexed queries, and
> > > even longer timeouts for administrative tasks like TRUNCATE and DROP.
> > >
> > >
> > > https://issues.apache.org/jira/browse/CASSANDRA-2848
> > >
> > >
> > > 3. Shard-aware driver
> > >
> > >
> > > This admittedly is a burning issue for ScyllaDB, but not so much for
> > > Cassandra at this time.
> > >
> > >
> > > In the same way that drivers are token-aware, they can be shard-aware -
> > > know how many shards each node has, and the sharding algorithm. They
> can
> > > then open a connection per shard and send cql requests directly to the
> > > shard that will serve them, instead of requiring cross-core
> communication
> > > to happen on the server.
> > >
> > >
> > > https://issues.apache.org/jira/browse/CASSANDRA-10989
> > >
> > >
> > > I see three possible modes of cooperation:
> > >
> > >
> > > 1. The protocol change is developed using the Cassandra process in a
> JIRA
> > > ticket, culminating in a patch to doc/native_protocol*.spec when
> > consensus
> > > is achieved.
> > >
> > >
> > > The advantage to this mode is that Cassandra developers can verify that
> > > the change is easily implementable; when they are ready to implement
> the
> > > feature, drivers that were already adapted to support it will just
> work.
> > >
> > >
> > > 2. The protocol change is developed outside the Cassandra process.
> > >
> > >
> > > In this mode, we develop the change in a forked version of
> > > native_protocol*.spec; Cassandra can still retroactively merge that
> > change
> > > when (and if) it is implemented, but the ability to influence the
> change
> > > during development is reduced.
> > >
> > >
> > > If we agree on this, I'd like to allocate a prefix for feature names in
> > > the SUPPORTED message for our use.
> > >
> > >
> > > 3. No cooperation.
> > >
> > >
> > > This requires the least amount of effort from Cassandra developers
> (just
> > > enough to reach this point in this email), but will cause duplication
> of
> > > effort for driver authors who wish to support both projects, and may
> > cause
> > > Cassandra developers to redo work that we already did.
> > >
> > >
> > > Looking forward to your views.
> > >
> > >
> > > Avi
> > >
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > > For additional commands, e-mail: dev-h...@cassandra.apache.org
> > >
> > >
> >
>

Re: Evolving the client protocol

[sankalp kohli]

Do we have driver authors who wish to support both projects?

On Wed, Apr 18, 2018 at 9:25 AM, Jeff Jirsa  wrote:

> Removed other lists (please don't cross post)
>
>
>
>
>
> On Wed, Apr 18, 2018 at 3:47 AM, Avi Kivity  wrote:
>
> > Hello Cassandra developers,
> >
> >
> > We're starting to see client protocol limitations impact performance, and
> > so we'd like to evolve the protocol to remove the limitations. In order
> to
> > avoid fragmenting the driver ecosystem and reduce work duplication for
> > driver authors, we'd like to avoid forking the protocol. Since these
> issues
> > affect Cassandra, either now or in the future, I'd like to cooperate on
> > protocol development.
> >
> >
> > Some issues that we'd like to work on near-term are:
> >
> >
> > 1. Token-aware range queries
> >
> >
> > When the server returns a page in a range query, it will also return a
> > token to continue on. In case that token is on a different node, the
> client
> > selects a new coordinator based on the token. This eliminates a network
> hop
> > for range queries.
> >
> >
> > For the first page, the PREPARE message returns information allowing the
> > client to compute where the first page is held, given the query
> parameters.
> > This is just information identifying how to compute the token, given the
> > query parameters (non-range queries already do this).
> >
> >
> > https://issues.apache.org/jira/browse/CASSANDRA-14311
> >
> >
> > 2. Per-request timeouts
> >
> >
> > Allow each request to have its own timeout. This allows the user to set
> > short timeouts on business-critical queries that are invalid if not
> served
> > within a short time, long timeouts for scanning or indexed queries, and
> > even longer timeouts for administrative tasks like TRUNCATE and DROP.
> >
> >
> > https://issues.apache.org/jira/browse/CASSANDRA-2848
> >
> >
> > 3. Shard-aware driver
> >
> >
> > This admittedly is a burning issue for ScyllaDB, but not so much for
> > Cassandra at this time.
> >
> >
> > In the same way that drivers are token-aware, they can be shard-aware -
> > know how many shards each node has, and the sharding algorithm. They can
> > then open a connection per shard and send cql requests directly to the
> > shard that will serve them, instead of requiring cross-core communication
> > to happen on the server.
> >
> >
> > https://issues.apache.org/jira/browse/CASSANDRA-10989
> >
> >
> > I see three possible modes of cooperation:
> >
> >
> > 1. The protocol change is developed using the Cassandra process in a JIRA
> > ticket, culminating in a patch to doc/native_protocol*.spec when
> consensus
> > is achieved.
> >
> >
> > The advantage to this mode is that Cassandra developers can verify that
> > the change is easily implementable; when they are ready to implement the
> > feature, drivers that were already adapted to support it will just work.
> >
> >
> > 2. The protocol change is developed outside the Cassandra process.
> >
> >
> > In this mode, we develop the change in a forked version of
> > native_protocol*.spec; Cassandra can still retroactively merge that
> change
> > when (and if) it is implemented, but the ability to influence the change
> > during development is reduced.
> >
> >
> > If we agree on this, I'd like to allocate a prefix for feature names in
> > the SUPPORTED message for our use.
> >
> >
> > 3. No cooperation.
> >
> >
> > This requires the least amount of effort from Cassandra developers (just
> > enough to reach this point in this email), but will cause duplication of
> > effort for driver authors who wish to support both projects, and may
> cause
> > Cassandra developers to redo work that we already did.
> >
> >
> > Looking forward to your views.
> >
> >
> > Avi
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
> >
>

Re: Proposing an Apache Cassandra Management process

[Vinay Chella]

This is a good initiative. We have advocated for and run a sidecar for the
past 5+ years, and we've learned and improved it a lot. We look forward to
moving features from Priam (such as backup, HTTP -> JMX, etc) incrementally
to this sidecar as they make sense.


Thanks,
Vinay Chella

On Fri, Apr 13, 2018 at 7:01 AM, Eric Evans 
wrote:

> On Thu, Apr 12, 2018 at 4:41 PM, Dinesh Joshi
>  wrote:
> > Hey all -
> > With the uptick in discussion around Cassandra operability and after
> discussing potential solutions with various members of the community, we
> would like to propose the addition of a management process/sub-project into
> Apache Cassandra. The process would be responsible for common operational
> tasks like bulk execution of nodetool commands, backup/restore, and health
> checks, among others. We feel we have a proposal that will garner some
> discussion and debate but is likely to reach consensus.
> > While the community, in large part, agrees that these features should
> exist “in the database”, there is debate on how they should be implemented.
> Primarily, whether or not to use an external process or build on
> CassandraDaemon. This is an important architectural decision but we feel
> the most critical aspect is not where the code runs but that the operator
> still interacts with the notion of a single database. Multi-process
> databases are as old as Postgres and continue to be common in newer systems
> like Druid. As such, we propose a separate management process for the
> following reasons:
> >
> >- Resource isolation & Safety: Features in the management process
> will not affect C*'s read/write path which is critical for stability. An
> isolated process has several technical advantages including preventing use
> of unnecessary dependencies in CassandraDaemon, separation of JVM resources
> like thread pools and heap, and preventing bugs from adversely affecting
> the main process. In particular, GC tuning can be done separately for the
> two processes, hopefully helping to improve, or at least not adversely
> affect, tail latencies of the main process.
> >
> >- Health Checks & Recovery: Currently users implement health checks
> in their own sidecar process. Implementing them in the serving process does
> not make sense because if the JVM running the CassandraDaemon goes south,
> the healthchecks and potentially any recovery code may not be able to run.
> Having a management process running in isolation opens up the possibility
> to not only report the health of the C* process such as long GC pauses or
> stuck JVM but also to recover from it. Having a list of basic health checks
> that are tested with every C* release and officially supported will help
> boost confidence in C* quality and make it easier to operate.
> >
> >- Reduced Risk: By having a separate Daemon we open the possibility
> to contribute features that otherwise would not have been considered before
> eg. a UI. A library that started many background threads and is operated
> completely differently would likely be considered too risky for
> CassandraDaemon but is a good candidate for the management process.
>
> Makes sense IMO.
>
> > What can go into the management process?
> >- Features that are non-essential for serving reads & writes for eg.
> Backup/Restore or Running Health Checks against the CassandraDaemon, etc.
> >
> >- Features that do not make the management process critical for
> functioning of the serving process. In other words, if someone does not
> wish to use this management process, they are free to disable it.
> >
> > We would like to initially build minimal set of features such as health
> checks and bulk commands into the first iteration of the management
> process. We would use the same software stack that is used to build the
> current CassandraDaemon binary. This would be critical for sharing code
> between CassandraDaemon & management processes. The code should live
> in-tree to make this easy.
> > With regards to more in-depth features like repair scheduling and
> discussions around compaction in or out of CassandraDaemon, while the
> management process may be a suitable host, it is not our goal to decide
> that at this time. The management process could be used in these cases, as
> they meet the criteria above, but other technical/architectural reasons may
> exists for why it should not be.
> > We are looking forward to your comments on our proposal,
>
> Sounds good to me.
>
> Personally, I'm a little less interested in things like
> health/availability checks and metrics collection, because there are
> already tools to solve this problem (and most places will already be
> using them).  I'm more interested in things like cluster status,
> streaming, repair, etc.  Something to automate/centralize
> database-specific command and control, and improve visibility.
>
> In-tree also makes sense (tools/ maybe?), but I would 

Re: Evolving the client protocol

[Glauber Costa]

BTW,

Folks from Cassandra apparently didn'tr eceive this message.

On Wed, Apr 18, 2018 at 6:47 AM, Avi Kivity  wrote:

> Hello Cassandra developers,
>
>
> We're starting to see client protocol limitations impact performance, and
> so we'd like to evolve the protocol to remove the limitations. In order to
> avoid fragmenting the driver ecosystem and reduce work duplication for
> driver authors, we'd like to avoid forking the protocol. Since these issues
> affect Cassandra, either now or in the future, I'd like to cooperate on
> protocol development.
>
>
> Some issues that we'd like to work on near-term are:
>
>
> 1. Token-aware range queries
>
>
> When the server returns a page in a range query, it will also return a
> token to continue on. In case that token is on a different node, the client
> selects a new coordinator based on the token. This eliminates a network hop
> for range queries.
>
>
> For the first page, the PREPARE message returns information allowing the
> client to compute where the first page is held, given the query parameters.
> This is just information identifying how to compute the token, given the
> query parameters (non-range queries already do this).
>
>
> https://issues.apache.org/jira/browse/CASSANDRA-14311
>
>
> 2. Per-request timeouts
>
>
> Allow each request to have its own timeout. This allows the user to set
> short timeouts on business-critical queries that are invalid if not served
> within a short time, long timeouts for scanning or indexed queries, and
> even longer timeouts for administrative tasks like TRUNCATE and DROP.
>
>
> https://issues.apache.org/jira/browse/CASSANDRA-2848
>
>
> 3. Shard-aware driver
>
>
> This admittedly is a burning issue for ScyllaDB, but not so much for
> Cassandra at this time.
>
>
> In the same way that drivers are token-aware, they can be shard-aware -
> know how many shards each node has, and the sharding algorithm. They can
> then open a connection per shard and send cql requests directly to the
> shard that will serve them, instead of requiring cross-core communication
> to happen on the server.
>
>
> https://issues.apache.org/jira/browse/CASSANDRA-10989
>
>
> I see three possible modes of cooperation:
>
>
> 1. The protocol change is developed using the Cassandra process in a JIRA
> ticket, culminating in a patch to doc/native_protocol*.spec when consensus
> is achieved.
>
>
> The advantage to this mode is that Cassandra developers can verify that
> the change is easily implementable; when they are ready to implement the
> feature, drivers that were already adapted to support it will just work.
>
>
> 2. The protocol change is developed outside the Cassandra process.
>
>
> In this mode, we develop the change in a forked version of
> native_protocol*.spec; Cassandra can still retroactively merge that change
> when (and if) it is implemented, but the ability to influence the change
> during development is reduced.
>
>
> If we agree on this, I'd like to allocate a prefix for feature names in
> the SUPPORTED message for our use.
>
>
> 3. No cooperation.
>
>
> This requires the least amount of effort from Cassandra developers (just
> enough to reach this point in this email), but will cause duplication of
> effort for driver authors who wish to support both projects, and may cause
> Cassandra developers to redo work that we already did.
>
>
> Looking forward to your views.
>
>
> Avi
>
> --
> You received this message because you are subscribed to the Google Groups
> "ScyllaDB development" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to scylladb-dev+unsubscr...@googlegroups.com.
> To post to this group, send email to scylladb-...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/ms
> gid/scylladb-dev/de6e33eb-b438-8524-ac99-a299e9ba0e72%40scylladb.com.
> For more options, visit https://groups.google.com/d/optout.
>

Re: Evolving the client protocol

[Jeff Jirsa]

Removed other lists (please don't cross post)





On Wed, Apr 18, 2018 at 3:47 AM, Avi Kivity  wrote:

> Hello Cassandra developers,
>
>
> We're starting to see client protocol limitations impact performance, and
> so we'd like to evolve the protocol to remove the limitations. In order to
> avoid fragmenting the driver ecosystem and reduce work duplication for
> driver authors, we'd like to avoid forking the protocol. Since these issues
> affect Cassandra, either now or in the future, I'd like to cooperate on
> protocol development.
>
>
> Some issues that we'd like to work on near-term are:
>
>
> 1. Token-aware range queries
>
>
> When the server returns a page in a range query, it will also return a
> token to continue on. In case that token is on a different node, the client
> selects a new coordinator based on the token. This eliminates a network hop
> for range queries.
>
>
> For the first page, the PREPARE message returns information allowing the
> client to compute where the first page is held, given the query parameters.
> This is just information identifying how to compute the token, given the
> query parameters (non-range queries already do this).
>
>
> https://issues.apache.org/jira/browse/CASSANDRA-14311
>
>
> 2. Per-request timeouts
>
>
> Allow each request to have its own timeout. This allows the user to set
> short timeouts on business-critical queries that are invalid if not served
> within a short time, long timeouts for scanning or indexed queries, and
> even longer timeouts for administrative tasks like TRUNCATE and DROP.
>
>
> https://issues.apache.org/jira/browse/CASSANDRA-2848
>
>
> 3. Shard-aware driver
>
>
> This admittedly is a burning issue for ScyllaDB, but not so much for
> Cassandra at this time.
>
>
> In the same way that drivers are token-aware, they can be shard-aware -
> know how many shards each node has, and the sharding algorithm. They can
> then open a connection per shard and send cql requests directly to the
> shard that will serve them, instead of requiring cross-core communication
> to happen on the server.
>
>
> https://issues.apache.org/jira/browse/CASSANDRA-10989
>
>
> I see three possible modes of cooperation:
>
>
> 1. The protocol change is developed using the Cassandra process in a JIRA
> ticket, culminating in a patch to doc/native_protocol*.spec when consensus
> is achieved.
>
>
> The advantage to this mode is that Cassandra developers can verify that
> the change is easily implementable; when they are ready to implement the
> feature, drivers that were already adapted to support it will just work.
>
>
> 2. The protocol change is developed outside the Cassandra process.
>
>
> In this mode, we develop the change in a forked version of
> native_protocol*.spec; Cassandra can still retroactively merge that change
> when (and if) it is implemented, but the ability to influence the change
> during development is reduced.
>
>
> If we agree on this, I'd like to allocate a prefix for feature names in
> the SUPPORTED message for our use.
>
>
> 3. No cooperation.
>
>
> This requires the least amount of effort from Cassandra developers (just
> enough to reach this point in this email), but will cause duplication of
> effort for driver authors who wish to support both projects, and may cause
> Cassandra developers to redo work that we already did.
>
>
> Looking forward to your views.
>
>
> Avi
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Evolving the client protocol

[Avi Kivity]

Hello Cassandra developers,


We're starting to see client protocol limitations impact performance, 
and so we'd like to evolve the protocol to remove the limitations. In 
order to avoid fragmenting the driver ecosystem and reduce work 
duplication for driver authors, we'd like to avoid forking the protocol. 
Since these issues affect Cassandra, either now or in the future, I'd 
like to cooperate on protocol development.



Some issues that we'd like to work on near-term are:


1. Token-aware range queries


When the server returns a page in a range query, it will also return a 
token to continue on. In case that token is on a different node, the 
client selects a new coordinator based on the token. This eliminates a 
network hop for range queries.



For the first page, the PREPARE message returns information allowing the 
client to compute where the first page is held, given the query 
parameters. This is just information identifying how to compute the 
token, given the query parameters (non-range queries already do this).



https://issues.apache.org/jira/browse/CASSANDRA-14311


2. Per-request timeouts


Allow each request to have its own timeout. This allows the user to set 
short timeouts on business-critical queries that are invalid if not 
served within a short time, long timeouts for scanning or indexed 
queries, and even longer timeouts for administrative tasks like TRUNCATE 
and DROP.



https://issues.apache.org/jira/browse/CASSANDRA-2848


3. Shard-aware driver


This admittedly is a burning issue for ScyllaDB, but not so much for 
Cassandra at this time.



In the same way that drivers are token-aware, they can be shard-aware - 
know how many shards each node has, and the sharding algorithm. They can 
then open a connection per shard and send cql requests directly to the 
shard that will serve them, instead of requiring cross-core 
communication to happen on the server.



https://issues.apache.org/jira/browse/CASSANDRA-10989


I see three possible modes of cooperation:


1. The protocol change is developed using the Cassandra process in a 
JIRA ticket, culminating in a patch to doc/native_protocol*.spec when 
consensus is achieved.



The advantage to this mode is that Cassandra developers can verify that 
the change is easily implementable; when they are ready to implement the 
feature, drivers that were already adapted to support it will just work.



2. The protocol change is developed outside the Cassandra process.


In this mode, we develop the change in a forked version of 
native_protocol*.spec; Cassandra can still retroactively merge that 
change when (and if) it is implemented, but the ability to influence the 
change during development is reduced.



If we agree on this, I'd like to allocate a prefix for feature names in 
the SUPPORTED message for our use.



3. No cooperation.


This requires the least amount of effort from Cassandra developers (just 
enough to reach this point in this email), but will cause duplication of 
effort for driver authors who wish to support both projects, and may 
cause Cassandra developers to redo work that we already did.



Looking forward to your views.


Avi


-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org