Re: Proposing an Apache Cassandra Management process

[Dinesh Joshi]

Thanks, Nate. I’ll create this request.

Dinesh

On Aug 17, 2018, at 5:09 PM, Nate McCall  wrote:

>> I'm not sure logistically how we get a new repo created and licensing and
>> such, but if someone helps make it we can cut the patch against it
>> 
> This is pretty straight forward. For precedent, see:
> https://issues.apache.org/jira/browse/CASSANDRA-13634
> 
> We currently have three repositories:
> https://git-wip-us.apache.org/repos/asf
> 
> I'm +0 on what approach we take.
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
> 


-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Proposing an Apache Cassandra Management process

[Nate McCall]

> I'm not sure logistically how we get a new repo created and licensing and
> such, but if someone helps make it we can cut the patch against it
>
This is pretty straight forward. For precedent, see:
https://issues.apache.org/jira/browse/CASSANDRA-13634

We currently have three repositories:
https://git-wip-us.apache.org/repos/asf

I'm +0 on what approach we take.

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: upgrade guava on trunk before 9/1?

[Sumanth Pasupuleti]

Thanks for the confirmation Andy.

Created a JIRA to track the guava upgrade
https://issues.apache.org/jira/browse/CASSANDRA-14655.

I current put target version as 4.x. If a compatible version of the driver
happens to be released soon enough, and if there is a consensus that we
should merge this into 4.0, will change the target version accordingly.
Otherwise, it seems unlikely for 4.0 purely from dependencies perspective.

Thanks,
Sumanth

On Thu, Aug 16, 2018 at 6:47 PM, Andy Tolbert 
wrote:

> Hi Sumanth,
>
> I gave a shot at upgrading to Guava 26.0; resolved build issues; about 63
> > Unit Tests fail - a lot of them due to a NPE and another bunch due to
> > the driver using a method that does not exist in latest Guava.
> >
>
> Unfortunately the driver doesn't currently work with Guava 26.0.  We have a
> fix (JAVA-1928 ) that
> addresses this that will be in version 3.6.0 which should be released in
> the coming weeks.  After that the driver will work with Guava 16.0.1
> through 26 (We've run our full test suite against all versions in that
> range).
>
> Thanks,
> Andy
>
> On Thu, Aug 16, 2018 at 6:55 PM, Sumanth Pasupuleti <
> sumanth.pasupuleti...@gmail.com> wrote:
>
> > I am +1 on upgrading to latest Guava, given that we would be on 4.0 for a
> > while, and we would have a relatively big testing phase that follows the
> > freeze date, which would hopefully iron out (as much as possible) any
> > issues we may have with the upgrade.
> >
> > I gave a shot at upgrading to Guava 26.0; resolved build issues; about 63
> > Unit Tests fail - a lot of them due to a NPE and another bunch due to
> > the driver using a method that does not exist in latest Guava.
> >
> > Github Branch:
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__github.
> > com_sumanth-2Dpasupuleti_cassandra_tree_guava-5F26-5Ftrunk=DwIFaQ=
> > adz96Xi0w1RHqtPMowiL2g=VHsWqsWT2MoX5jRZ0xZfdAWZxBsrn5KzowynGYCJaXE=
> > NBUK9LYIuBLTb2_dn7ZZnfqkFQuWmAr69kTsiLgthN8=
> > upEwsGydoKbFV9vnRLMO9LEy4YaSlVXBiPoChzsJfCE=
> > Failing Unit Tests: https://urldefense.proofpoint.
> com/v2/url?u=https-3A__
> > circleci.com_gh_sumanth-2Dpasupuleti_cassandra_84=DwIFaQ=
> > adz96Xi0w1RHqtPMowiL2g=VHsWqsWT2MoX5jRZ0xZfdAWZxBsrn5KzowynGYCJaXE=
> > NBUK9LYIuBLTb2_dn7ZZnfqkFQuWmAr69kTsiLgthN8&
> s=oKPrxl_zBsIcuZoHkzS2MZOi9U5_
> > lF3DaNldTbMQsBU=
> >
> > Thanks,
> > Sumanth
> >
> > On Thu, Aug 16, 2018 at 8:22 AM, Jonathan Haddad 
> > wrote:
> >
> > > Pushing it back means it’s a bigger risk later on. I’m +.5 on upgrading
> > now
> > >
> > > On Wed, Aug 15, 2018 at 11:46 PM dinesh.jo...@yahoo.com.INVALID
> > >  wrote:
> > >
> > > > Jason,
> > > > Given that we're so close to the 9/1 date, I would err on the side of
> > > > caution especially given the low value prop. If someone does run into
> > > Guava
> > > > compatibility issues (and someone somewhere will), we can revisit
> this
> > > > question then.
> > > > Dinesh
> > > >
> > > > On Wednesday, August 15, 2018, 11:42:31 PM PDT, Dinesh A. Joshi <
> > > > dinesh.jo...@gatech.edu> wrote:
> > > >
> > > >  Jason,
> > > > Given that we're so close to the 9/1 date, I would err on the side of
> > > > caution especially given the low value prop. If someone does run into
> > > Guava
> > > > compatibility issues (and someone somewhere will), we can revisit
> this
> > > > question then.
> > > > Dinesh
> > > >
> > > > On Wednesday, August 15, 2018, 8:22:28 AM PDT, Salih Gedik <
> > > > m...@salih.xyz> wrote:
> > > >
> > > >  Hi,
> > > >
> > > > Change logs are on Github releases page. It seems like only hash
> > flooding
> > > > protection which is added to ImmutableMap is relevant to Cassandra
> > code.
> > > I
> > > > haven’t checked whether we use deprecated APIs. But there isn’t much
> on
> > > > table from what I see.
> > > >
> > > > Salih
> > > > On 15 Aug 2018 17:55 +0300, Ariel Weisberg ,
> wrote:
> > > > > Hi,
> > > > >
> > > > > They don't even do release notes after 23. Also no API diffs. I
> mean
> > > I'm
> > > > fine with it, but it's mostly just changing to another arbitrary
> > version
> > > > that won't match what is in apps.
> > > > >
> > > > > Ariel
> > > > >
> > > > > On Wed, Aug 15, 2018, at 10:48 AM, Jason Brown wrote:
> > > > > > Hey Ariel,
> > > > > >
> > > > > > Tbqh, not that much. I was mostly thinking from the "I have
> > conflicts
> > > > on
> > > > > > guava versions in my app because I pull in cassandra and XYZ
> > > > libraries, and
> > > > > > the transitive dependencies on guava use different versions" POV.
> > > > Further,
> > > > > > we'll be on this version of guava for 4.0 for at least two years
> > from
> > > > now.
> > > > > >
> > > > > > As I asked, "does anybody feeling strongly?". Personally, I'm
> sorta
> > > +0
> > > > to
> > > > > > +0.5, but I was just throwing this out there in case someone does
> > > > really
> > > > > > think it best we upgrade (and wants to make a contribution).
> > > > > >
> > > > > > -Jason

Re: Apache Cassandra Blog is now live

[dinesh.jo...@yahoo.com.INVALID]

Hi Jeff,
Thanks for the patch. I assigned the ticket to you. I'll be happy to review the 
ticket later today.
Dinesh 

On Friday, August 17, 2018, 1:24:33 PM PDT, Jeff Beck  
wrote:  
 
 I submitted a patch for the feed to that ticket it also seemed like we
needed bundler to ensure we always use the same gems.

On Thu, Aug 9, 2018 at 2:44 AM Jacques-Henri Berthemet <
jacques-henri.berthe...@genesys.com> wrote:

> Hi Scott,
>
> Here is the ticket: https://issues.apache.org/jira/browse/CASSANDRA-14631
>
> Regards,
> --
> Jacques-Henri Berthemet
>
> -Original Message-
> From: Scott Andreas 
> Sent: Wednesday, August 08, 2018 6:53 PM
> To: dev@cassandra.apache.org
> Subject: Re: Apache Cassandra Blog is now live
>
> Please feel free to file a ticket (label: Documentation and Website).
>
> It looks like Jekyll, the static site generator used to build the website,
> has a plugin that generates Atom feeds if someone would like to work on
> adding one: https://github.com/jekyll/jekyll-feed
>
> 
> From: Jacques-Henri Berthemet 
> Sent: Wednesday, August 8, 2018 1:32:23 AM
> To: dev@cassandra.apache.org
> Subject: RE: Apache Cassandra Blog is now live
>
> It is possible to setup RSS on the blog? Should I create a Jira about that?
>
> --
> Jacques-Henri Berthemet
>
> -Original Message-
> From: Mick Semb Wever 
> Sent: Wednesday, August 08, 2018 8:17 AM
> To: dev@cassandra.apache.org
> Subject: Re: Apache Cassandra Blog is now live
>
>
> > We are also interested in contributing to the blog, what's the process?
>
>
> Dikang,
> it's part of the website,
> https://svn.apache.org/repos/asf/cassandra/site/src/README
>
> regards,
> Mick
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>
  

Re: Apache Cassandra Blog is now live

[Jeff Beck]

I submitted a patch for the feed to that ticket it also seemed like we
needed bundler to ensure we always use the same gems.

On Thu, Aug 9, 2018 at 2:44 AM Jacques-Henri Berthemet <
jacques-henri.berthe...@genesys.com> wrote:

> Hi Scott,
>
> Here is the ticket: https://issues.apache.org/jira/browse/CASSANDRA-14631
>
> Regards,
> --
> Jacques-Henri Berthemet
>
> -Original Message-
> From: Scott Andreas 
> Sent: Wednesday, August 08, 2018 6:53 PM
> To: dev@cassandra.apache.org
> Subject: Re: Apache Cassandra Blog is now live
>
> Please feel free to file a ticket (label: Documentation and Website).
>
> It looks like Jekyll, the static site generator used to build the website,
> has a plugin that generates Atom feeds if someone would like to work on
> adding one: https://github.com/jekyll/jekyll-feed
>
> 
> From: Jacques-Henri Berthemet 
> Sent: Wednesday, August 8, 2018 1:32:23 AM
> To: dev@cassandra.apache.org
> Subject: RE: Apache Cassandra Blog is now live
>
> It is possible to setup RSS on the blog? Should I create a Jira about that?
>
> --
> Jacques-Henri Berthemet
>
> -Original Message-
> From: Mick Semb Wever 
> Sent: Wednesday, August 08, 2018 8:17 AM
> To: dev@cassandra.apache.org
> Subject: Re: Apache Cassandra Blog is now live
>
>
> > We are also interested in contributing to the blog, what's the process?
>
>
> Dikang,
> it's part of the website,
> https://svn.apache.org/repos/asf/cassandra/site/src/README
>
> regards,
> Mick
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Proposing an Apache Cassandra Management process

[Vinay Chella]

You can still use the same repo and go with different release cadence as
Jeremiah mentioned, and yes, you can just pull the management sidecar,
test, and rollout without rolling out the server.

It looks like there are merits to both approaches, but it is going to come
down to the appetite of the team to manage several projects, it's releases,
coordination and without compromising on quality. we are fine with either
of the approaches. If this does not work out in the future, we always have
a decision log to refer, learn and move forward.

I'm not sure logistically how we get a new repo created and licensing and
such, but if someone helps make it we can cut the patch against it

Regards,
Vinay Chella


On Fri, Aug 17, 2018 at 10:46 AM Rahul Singh 
wrote:

> I understand the issues of managing different versions of two correlated
> components — but it is possible to create unit tests with core components
> of both. It takes more effort but it is possible.
>
> That being said, in my experience using Reaper and in the DataStax
> distribution , using OpsCenter , I prefer a separate project that is
> loosely tied to the system and not connected at the hips. Whenever there is
> an update to Reaper or OpsCenter, I can always pull it down and test it
> before rolling it out - and this is much more frequently than if I were
> rolling out updates to a C* cluster.
>
>
> Rahul
> On Aug 17, 2018, 9:41 AM -0700, Jonathan Haddad ,
> wrote:
> > Speaking from experience (Reaper), I can say that developing a sidecar
> > admin / repair tool out of tree that's compatible with multiple versions
> > really isn't that big of a problem. We've been doing it for a while now.
> >
> >
> https://github.com/thelastpickle/cassandra-reaper/blob/master/.travis.yml
> >
> > On Fri, Aug 17, 2018 at 9:39 AM Joseph Lynch 
> wrote:
> >
> > > While I would love to use a different build system (e.g. gradle) for
> the
> > > sidecar, I agree with Dinesh that a separate repo would make sidecar
> > > development much harder to verify, especially on the testing and
> > > compatibility front. As Jeremiah mentioned we can always choose later
> to
> > > release the sidecar artifact separately or more frequently than the
> main
> > > server regardless of repo choice and as per Roopa's point having a
> separate
> > > release artifact (jar or deb/rpm) is probably a good idea until the
> default
> > > Cassandra packages don't automatically stop and start Cassandra on
> install.
> > >
> > > While we were developing the repair scheduler in a separate repo we
> had a
> > > number of annoying issues that only started surfacing once we started
> > > merging it directly into the trunk tree:
> > > 1. It is hard to compile/test against unreleased versions of Cassandra
> > > (e.g. the JMX interfaces changed a lot with 4.x, and it was pretty
> tricky
> > > to compile against that as the main project doesn't release nightly
> > > snapshots or anything like that, so we had to build local trunk jars
> and
> > > then manually dep them).
> > > 2. Integration testing and cross version compatibility is extremely
> hard.
> > > The sidecar is going to be involved in multi node coordination (e.g.
> > > monitoring, metrics, maintenance) and will be tightly coupled to JMX
> > > interface choices in the server and trying to make sure that it all
> works
> > > with multiple versions of Cassandra is much harder if it's a separate
> repo
> > > that has to have a mirroring release cycle to Cassandra. It seems much
> > > easier to have it in tree and just be like "the in tree sidecar is
> tested
> > > against that version of Cassandra". Every time we cut a Cassandra
> server
> > > branch the sidecar branches with it.
> > >
> > > We experience these pains all the time with Priam being in a separate
> repo,
> > > where every time we support a new Cassandra version a bunch of JMX
> > > endpoints break and we have to refactor the code to either call JMX
> methods
> > > by string or cut a new Priam branch. A separate artifact is pretty
> > > important, but a separate repo just allows drifts. Furthermore from the
> > > Priam experience I also don't think it's realistic to say one version
> of a
> > > sidecar artifact can actually support multiple versions.
> > >
> > > -Joey
> > >
> > > On Fri, Aug 17, 2018 at 12:00 PM Jeremiah D Jordan <
> jerem...@datastax.com>
> > > wrote:
> > >
> > > > Not sure why the two things being in the same repo means they need
> the
> > > > same release process. You can always do interim releases of the
> > > management
> > > > artifact between server releases, or even have completely decoupled
> > > > releases.
> > > >
> > > > -Jeremiah
> > > >
> > > > > On Aug 17, 2018, at 10:52 AM, Blake Eggleston <
> beggles...@apple.com>
> > > > wrote:
> > > > >
> > > > > I'd be more in favor of making it a separate project, basically
> for all
> > > > the reasons listed below. I'm assuming we'd want a management
> process to
> > > > work across different versions, which will 

Re: Proposing an Apache Cassandra Management process

[dinesh.jo...@yahoo.com.INVALID]

Both approaches have merits. However the general consensus seems to be that we 
should put this in a separate repo and I agree.
Dinesh 

On Friday, August 17, 2018, 10:46:04 AM PDT, Rahul Singh 
 wrote:  
 
 I understand the issues of managing different versions of two correlated 
components — but it is possible to create unit tests with core components of 
both. It takes more effort but it is possible.

That being said, in my experience using Reaper and in the DataStax distribution 
, using OpsCenter , I prefer a separate project that is loosely tied to the 
system and not connected at the hips. Whenever there is an update to Reaper or 
OpsCenter, I can always pull it down and test it before rolling it out - and 
this is much more frequently than if I were rolling out updates to a C* cluster.


Rahul
On Aug 17, 2018, 9:41 AM -0700, Jonathan Haddad , wrote:
> Speaking from experience (Reaper), I can say that developing a sidecar
> admin / repair tool out of tree that's compatible with multiple versions
> really isn't that big of a problem. We've been doing it for a while now.
>
> https://github.com/thelastpickle/cassandra-reaper/blob/master/.travis.yml
>
> On Fri, Aug 17, 2018 at 9:39 AM Joseph Lynch  wrote:
>
> > While I would love to use a different build system (e.g. gradle) for the
> > sidecar, I agree with Dinesh that a separate repo would make sidecar
> > development much harder to verify, especially on the testing and
> > compatibility front. As Jeremiah mentioned we can always choose later to
> > release the sidecar artifact separately or more frequently than the main
> > server regardless of repo choice and as per Roopa's point having a separate
> > release artifact (jar or deb/rpm) is probably a good idea until the default
> > Cassandra packages don't automatically stop and start Cassandra on install.
> >
> > While we were developing the repair scheduler in a separate repo we had a
> > number of annoying issues that only started surfacing once we started
> > merging it directly into the trunk tree:
> > 1. It is hard to compile/test against unreleased versions of Cassandra
> > (e.g. the JMX interfaces changed a lot with 4.x, and it was pretty tricky
> > to compile against that as the main project doesn't release nightly
> > snapshots or anything like that, so we had to build local trunk jars and
> > then manually dep them).
> > 2. Integration testing and cross version compatibility is extremely hard.
> > The sidecar is going to be involved in multi node coordination (e.g.
> > monitoring, metrics, maintenance) and will be tightly coupled to JMX
> > interface choices in the server and trying to make sure that it all works
> > with multiple versions of Cassandra is much harder if it's a separate repo
> > that has to have a mirroring release cycle to Cassandra. It seems much
> > easier to have it in tree and just be like "the in tree sidecar is tested
> > against that version of Cassandra". Every time we cut a Cassandra server
> > branch the sidecar branches with it.
> >
> > We experience these pains all the time with Priam being in a separate repo,
> > where every time we support a new Cassandra version a bunch of JMX
> > endpoints break and we have to refactor the code to either call JMX methods
> > by string or cut a new Priam branch. A separate artifact is pretty
> > important, but a separate repo just allows drifts. Furthermore from the
> > Priam experience I also don't think it's realistic to say one version of a
> > sidecar artifact can actually support multiple versions.
> >
> > -Joey
> >
> > On Fri, Aug 17, 2018 at 12:00 PM Jeremiah D Jordan 
> > wrote:
> >
> > > Not sure why the two things being in the same repo means they need the
> > > same release process. You can always do interim releases of the
> > management
> > > artifact between server releases, or even have completely decoupled
> > > releases.
> > >
> > > -Jeremiah
> > >
> > > > On Aug 17, 2018, at 10:52 AM, Blake Eggleston 
> > > wrote:
> > > >
> > > > I'd be more in favor of making it a separate project, basically for all
> > > the reasons listed below. I'm assuming we'd want a management process to
> > > work across different versions, which will be more awkward if it's in
> > tree.
> > > Even if that's not the case, keeping it in a different repo at this point
> > > will make iteration easier than if it were in tree. I'd imagine (or at
> > > least hope) that validating the management process for release would be
> > > less difficult than the main project, so tying them to the Cassandra
> > > release cycle seems unnecessarily restrictive.
> > > >
> > > >
> > > > On August 17, 2018 at 12:07:18 AM, Dinesh Joshi (
> > dinesh.jo...@yahoo.com.invalid)
> > > wrote:
> > > >
> > > > > On Aug 16, 2018, at 9:27 PM, Sankalp Kohli 
> > > wrote:
> > > > >
> > > > > I am bumping this thread because patch has landed for this with repair
> > > functionality.
> > > > >
> > > > > I have a following proposal for this which I can put in the 

Re: Proposing an Apache Cassandra Management process

[Rahul Singh]

I understand the issues of managing different versions of two correlated 
components — but it is possible to create unit tests with core components of 
both. It takes more effort but it is possible.

That being said, in my experience using Reaper and in the DataStax distribution 
, using OpsCenter , I prefer a separate project that is loosely tied to the 
system and not connected at the hips. Whenever there is an update to Reaper or 
OpsCenter, I can always pull it down and test it before rolling it out - and 
this is much more frequently than if I were rolling out updates to a C* cluster.


Rahul
On Aug 17, 2018, 9:41 AM -0700, Jonathan Haddad , wrote:
> Speaking from experience (Reaper), I can say that developing a sidecar
> admin / repair tool out of tree that's compatible with multiple versions
> really isn't that big of a problem. We've been doing it for a while now.
>
> https://github.com/thelastpickle/cassandra-reaper/blob/master/.travis.yml
>
> On Fri, Aug 17, 2018 at 9:39 AM Joseph Lynch  wrote:
>
> > While I would love to use a different build system (e.g. gradle) for the
> > sidecar, I agree with Dinesh that a separate repo would make sidecar
> > development much harder to verify, especially on the testing and
> > compatibility front. As Jeremiah mentioned we can always choose later to
> > release the sidecar artifact separately or more frequently than the main
> > server regardless of repo choice and as per Roopa's point having a separate
> > release artifact (jar or deb/rpm) is probably a good idea until the default
> > Cassandra packages don't automatically stop and start Cassandra on install.
> >
> > While we were developing the repair scheduler in a separate repo we had a
> > number of annoying issues that only started surfacing once we started
> > merging it directly into the trunk tree:
> > 1. It is hard to compile/test against unreleased versions of Cassandra
> > (e.g. the JMX interfaces changed a lot with 4.x, and it was pretty tricky
> > to compile against that as the main project doesn't release nightly
> > snapshots or anything like that, so we had to build local trunk jars and
> > then manually dep them).
> > 2. Integration testing and cross version compatibility is extremely hard.
> > The sidecar is going to be involved in multi node coordination (e.g.
> > monitoring, metrics, maintenance) and will be tightly coupled to JMX
> > interface choices in the server and trying to make sure that it all works
> > with multiple versions of Cassandra is much harder if it's a separate repo
> > that has to have a mirroring release cycle to Cassandra. It seems much
> > easier to have it in tree and just be like "the in tree sidecar is tested
> > against that version of Cassandra". Every time we cut a Cassandra server
> > branch the sidecar branches with it.
> >
> > We experience these pains all the time with Priam being in a separate repo,
> > where every time we support a new Cassandra version a bunch of JMX
> > endpoints break and we have to refactor the code to either call JMX methods
> > by string or cut a new Priam branch. A separate artifact is pretty
> > important, but a separate repo just allows drifts. Furthermore from the
> > Priam experience I also don't think it's realistic to say one version of a
> > sidecar artifact can actually support multiple versions.
> >
> > -Joey
> >
> > On Fri, Aug 17, 2018 at 12:00 PM Jeremiah D Jordan 
> > wrote:
> >
> > > Not sure why the two things being in the same repo means they need the
> > > same release process. You can always do interim releases of the
> > management
> > > artifact between server releases, or even have completely decoupled
> > > releases.
> > >
> > > -Jeremiah
> > >
> > > > On Aug 17, 2018, at 10:52 AM, Blake Eggleston 
> > > wrote:
> > > >
> > > > I'd be more in favor of making it a separate project, basically for all
> > > the reasons listed below. I'm assuming we'd want a management process to
> > > work across different versions, which will be more awkward if it's in
> > tree.
> > > Even if that's not the case, keeping it in a different repo at this point
> > > will make iteration easier than if it were in tree. I'd imagine (or at
> > > least hope) that validating the management process for release would be
> > > less difficult than the main project, so tying them to the Cassandra
> > > release cycle seems unnecessarily restrictive.
> > > >
> > > >
> > > > On August 17, 2018 at 12:07:18 AM, Dinesh Joshi (
> > dinesh.jo...@yahoo.com.invalid)
> > > wrote:
> > > >
> > > > > On Aug 16, 2018, at 9:27 PM, Sankalp Kohli 
> > > wrote:
> > > > >
> > > > > I am bumping this thread because patch has landed for this with repair
> > > functionality.
> > > > >
> > > > > I have a following proposal for this which I can put in the JIRA or
> > doc
> > > > >
> > > > > 1. We should see if we can keep this in a separate repo like Dtest.
> > > >
> > > > This would imply a looser coupling between the two. Keeping things
> > > in-tree is my 

Re: Proposing an Apache Cassandra Management process

[Jonathan Haddad]

Speaking from experience (Reaper), I can say that developing a sidecar
admin / repair tool out of tree that's compatible with multiple versions
really isn't that big of a problem.  We've been doing it for a while now.

https://github.com/thelastpickle/cassandra-reaper/blob/master/.travis.yml

On Fri, Aug 17, 2018 at 9:39 AM Joseph Lynch  wrote:

> While I would love to use a different build system (e.g. gradle) for the
> sidecar, I agree with Dinesh that a separate repo would make sidecar
> development much harder to verify, especially on the testing and
> compatibility front. As Jeremiah mentioned we can always choose later to
> release the sidecar artifact separately or more frequently than the main
> server regardless of repo choice and as per Roopa's point having a separate
> release artifact (jar or deb/rpm) is probably a good idea until the default
> Cassandra packages don't automatically stop and start Cassandra on install.
>
> While we were developing the repair scheduler in a separate repo we had a
> number of annoying issues that only started surfacing once we started
> merging it directly into the trunk tree:
> 1. It is hard to compile/test against unreleased versions of Cassandra
> (e.g. the JMX interfaces changed a lot with 4.x, and it was pretty tricky
> to compile against that as the main project doesn't release nightly
> snapshots or anything like that, so we had to build local trunk jars and
> then manually dep them).
> 2. Integration testing and cross version compatibility is extremely hard.
> The sidecar is going to be involved in multi node coordination (e.g.
> monitoring, metrics, maintenance) and will be tightly coupled to JMX
> interface choices in the server and trying to make sure that it all works
> with multiple versions of Cassandra is much harder if it's a separate repo
> that has to have a mirroring release cycle to Cassandra. It seems much
> easier to have it in tree and just be like "the in tree sidecar is tested
> against that version of Cassandra". Every time we cut a Cassandra server
> branch the sidecar branches with it.
>
> We experience these pains all the time with Priam being in a separate repo,
> where every time we support a new Cassandra version a bunch of JMX
> endpoints break and we have to refactor the code to either call JMX methods
> by string or cut a new Priam branch. A separate artifact is pretty
> important, but a separate repo just allows drifts. Furthermore from the
> Priam experience I also don't think it's realistic to say one version of a
> sidecar artifact can actually support multiple versions.
>
> -Joey
>
> On Fri, Aug 17, 2018 at 12:00 PM Jeremiah D Jordan 
> wrote:
>
> > Not sure why the two things being in the same repo means they need the
> > same release process.  You can always do interim releases of the
> management
> > artifact between server releases, or even have completely decoupled
> > releases.
> >
> > -Jeremiah
> >
> > > On Aug 17, 2018, at 10:52 AM, Blake Eggleston 
> > wrote:
> > >
> > > I'd be more in favor of making it a separate project, basically for all
> > the reasons listed below. I'm assuming we'd want a management process to
> > work across different versions, which will be more awkward if it's in
> tree.
> > Even if that's not the case, keeping it in a different repo at this point
> > will make iteration easier than if it were in tree. I'd imagine (or at
> > least hope) that validating the management process for release would be
> > less difficult than the main project, so tying them to the Cassandra
> > release cycle seems unnecessarily restrictive.
> > >
> > >
> > > On August 17, 2018 at 12:07:18 AM, Dinesh Joshi (
> dinesh.jo...@yahoo.com.invalid)
> > wrote:
> > >
> > >> On Aug 16, 2018, at 9:27 PM, Sankalp Kohli 
> > wrote:
> > >>
> > >> I am bumping this thread because patch has landed for this with repair
> > functionality.
> > >>
> > >> I have a following proposal for this which I can put in the JIRA or
> doc
> > >>
> > >> 1. We should see if we can keep this in a separate repo like Dtest.
> > >
> > > This would imply a looser coupling between the two. Keeping things
> > in-tree is my preferred approach. It makes testing, dependency management
> > and code sharing easier.
> > >
> > >> 2. It should have its own release process.
> > >
> > > This means now there would be two releases that need to be managed and
> > coordinated.
> > >
> > >> 3. It should have integration tests for different versions of
> Cassandra
> > it will support.
> > >
> > > Given the lack of test infrastructure - this will be hard especially if
> > you have to qualify a matrix of builds.
> > >
> > > Dinesh
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > > For additional commands, e-mail: dev-h...@cassandra.apache.org
> > >
> >
> >
> > -
> > To unsubscribe, e-mail: 

Re: Proposing an Apache Cassandra Management process

[Joseph Lynch]

While I would love to use a different build system (e.g. gradle) for the
sidecar, I agree with Dinesh that a separate repo would make sidecar
development much harder to verify, especially on the testing and
compatibility front. As Jeremiah mentioned we can always choose later to
release the sidecar artifact separately or more frequently than the main
server regardless of repo choice and as per Roopa's point having a separate
release artifact (jar or deb/rpm) is probably a good idea until the default
Cassandra packages don't automatically stop and start Cassandra on install.

While we were developing the repair scheduler in a separate repo we had a
number of annoying issues that only started surfacing once we started
merging it directly into the trunk tree:
1. It is hard to compile/test against unreleased versions of Cassandra
(e.g. the JMX interfaces changed a lot with 4.x, and it was pretty tricky
to compile against that as the main project doesn't release nightly
snapshots or anything like that, so we had to build local trunk jars and
then manually dep them).
2. Integration testing and cross version compatibility is extremely hard.
The sidecar is going to be involved in multi node coordination (e.g.
monitoring, metrics, maintenance) and will be tightly coupled to JMX
interface choices in the server and trying to make sure that it all works
with multiple versions of Cassandra is much harder if it's a separate repo
that has to have a mirroring release cycle to Cassandra. It seems much
easier to have it in tree and just be like "the in tree sidecar is tested
against that version of Cassandra". Every time we cut a Cassandra server
branch the sidecar branches with it.

We experience these pains all the time with Priam being in a separate repo,
where every time we support a new Cassandra version a bunch of JMX
endpoints break and we have to refactor the code to either call JMX methods
by string or cut a new Priam branch. A separate artifact is pretty
important, but a separate repo just allows drifts. Furthermore from the
Priam experience I also don't think it's realistic to say one version of a
sidecar artifact can actually support multiple versions.

-Joey

On Fri, Aug 17, 2018 at 12:00 PM Jeremiah D Jordan 
wrote:

> Not sure why the two things being in the same repo means they need the
> same release process.  You can always do interim releases of the management
> artifact between server releases, or even have completely decoupled
> releases.
>
> -Jeremiah
>
> > On Aug 17, 2018, at 10:52 AM, Blake Eggleston 
> wrote:
> >
> > I'd be more in favor of making it a separate project, basically for all
> the reasons listed below. I'm assuming we'd want a management process to
> work across different versions, which will be more awkward if it's in tree.
> Even if that's not the case, keeping it in a different repo at this point
> will make iteration easier than if it were in tree. I'd imagine (or at
> least hope) that validating the management process for release would be
> less difficult than the main project, so tying them to the Cassandra
> release cycle seems unnecessarily restrictive.
> >
> >
> > On August 17, 2018 at 12:07:18 AM, Dinesh Joshi 
> > (dinesh.jo...@yahoo.com.invalid)
> wrote:
> >
> >> On Aug 16, 2018, at 9:27 PM, Sankalp Kohli 
> wrote:
> >>
> >> I am bumping this thread because patch has landed for this with repair
> functionality.
> >>
> >> I have a following proposal for this which I can put in the JIRA or doc
> >>
> >> 1. We should see if we can keep this in a separate repo like Dtest.
> >
> > This would imply a looser coupling between the two. Keeping things
> in-tree is my preferred approach. It makes testing, dependency management
> and code sharing easier.
> >
> >> 2. It should have its own release process.
> >
> > This means now there would be two releases that need to be managed and
> coordinated.
> >
> >> 3. It should have integration tests for different versions of Cassandra
> it will support.
> >
> > Given the lack of test infrastructure - this will be hard especially if
> you have to qualify a matrix of builds.
> >
> > Dinesh
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>

Re: Proposing an Apache Cassandra Management process

[Roopa]

+1 in maintaining a separate project and release cycle for side car. We have 
been running side car in production for 6+ years and the rate of changes to the 
side car is much higher than to the actual data store. This will enable faster 
iteration needed for the side car and help folks roll out maintenance fixes 
easily. 

Thanks
Roopa



> On Aug 17, 2018, at 8:52 AM, Blake Eggleston  wrote:
> 
> I'd be more in favor of making it a separate project, basically for all the 
> reasons listed below. I'm assuming we'd want a management process to work 
> across different versions, which will be more awkward if it's in tree. Even 
> if that's not the case, keeping it in a different repo at this point will 
> make iteration easier than if it were in tree. I'd imagine (or at least hope) 
> that validating the management process for release would be less difficult 
> than the main project, so tying them to the Cassandra release cycle seems 
> unnecessarily restrictive.
> 
> 
>> On August 17, 2018 at 12:07:18 AM, Dinesh Joshi 
>> (dinesh.jo...@yahoo.com.invalid) wrote:
>> 
>> On Aug 16, 2018, at 9:27 PM, Sankalp Kohli  wrote: 
>> 
>> I am bumping this thread because patch has landed for this with repair 
>> functionality. 
>> 
>> I have a following proposal for this which I can put in the JIRA or doc 
>> 
>> 1. We should see if we can keep this in a separate repo like Dtest. 
> 
> This would imply a looser coupling between the two. Keeping things in-tree is 
> my preferred approach. It makes testing, dependency management and code 
> sharing easier. 
> 
>> 2. It should have its own release process. 
> 
> This means now there would be two releases that need to be managed and 
> coordinated. 
> 
>> 3. It should have integration tests for different versions of Cassandra it 
>> will support. 
> 
> Given the lack of test infrastructure - this will be hard especially if you 
> have to qualify a matrix of builds. 
> 
> Dinesh 
> - 
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org 
> For additional commands, e-mail: dev-h...@cassandra.apache.org 
> 

-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Proposing an Apache Cassandra Management process

[Jeremiah D Jordan]

Not sure why the two things being in the same repo means they need the same 
release process.  You can always do interim releases of the management artifact 
between server releases, or even have completely decoupled releases.

-Jeremiah

> On Aug 17, 2018, at 10:52 AM, Blake Eggleston  wrote:
> 
> I'd be more in favor of making it a separate project, basically for all the 
> reasons listed below. I'm assuming we'd want a management process to work 
> across different versions, which will be more awkward if it's in tree. Even 
> if that's not the case, keeping it in a different repo at this point will 
> make iteration easier than if it were in tree. I'd imagine (or at least hope) 
> that validating the management process for release would be less difficult 
> than the main project, so tying them to the Cassandra release cycle seems 
> unnecessarily restrictive.
> 
> 
> On August 17, 2018 at 12:07:18 AM, Dinesh Joshi 
> (dinesh.jo...@yahoo.com.invalid) wrote:
> 
>> On Aug 16, 2018, at 9:27 PM, Sankalp Kohli  wrote: 
>> 
>> I am bumping this thread because patch has landed for this with repair 
>> functionality. 
>> 
>> I have a following proposal for this which I can put in the JIRA or doc 
>> 
>> 1. We should see if we can keep this in a separate repo like Dtest. 
> 
> This would imply a looser coupling between the two. Keeping things in-tree is 
> my preferred approach. It makes testing, dependency management and code 
> sharing easier. 
> 
>> 2. It should have its own release process. 
> 
> This means now there would be two releases that need to be managed and 
> coordinated. 
> 
>> 3. It should have integration tests for different versions of Cassandra it 
>> will support. 
> 
> Given the lack of test infrastructure - this will be hard especially if you 
> have to qualify a matrix of builds. 
> 
> Dinesh 
> - 
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org 
> For additional commands, e-mail: dev-h...@cassandra.apache.org 
> 


-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org

Re: Proposing an Apache Cassandra Management process

[Blake Eggleston]

I'd be more in favor of making it a separate project, basically for all the 
reasons listed below. I'm assuming we'd want a management process to work 
across different versions, which will be more awkward if it's in tree. Even if 
that's not the case, keeping it in a different repo at this point will make 
iteration easier than if it were in tree. I'd imagine (or at least hope) that 
validating the management process for release would be less difficult than the 
main project, so tying them to the Cassandra release cycle seems unnecessarily 
restrictive.


On August 17, 2018 at 12:07:18 AM, Dinesh Joshi 
(dinesh.jo...@yahoo.com.invalid) wrote:

> On Aug 16, 2018, at 9:27 PM, Sankalp Kohli  wrote: 
> 
> I am bumping this thread because patch has landed for this with repair 
> functionality. 
> 
> I have a following proposal for this which I can put in the JIRA or doc 
> 
> 1. We should see if we can keep this in a separate repo like Dtest. 

This would imply a looser coupling between the two. Keeping things in-tree is 
my preferred approach. It makes testing, dependency management and code sharing 
easier. 

> 2. It should have its own release process. 

This means now there would be two releases that need to be managed and 
coordinated. 

> 3. It should have integration tests for different versions of Cassandra it 
> will support. 

Given the lack of test infrastructure - this will be hard especially if you 
have to qualify a matrix of builds. 

Dinesh 
- 
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org 
For additional commands, e-mail: dev-h...@cassandra.apache.org 

Re: URGENT: CASSANDRA-14092 causes Data Loss

[kurt greaves]

I definitely think we should include it in 4.0. TBH I think it's reasonable
for it to get done after the feature freeze seeing as it is a bug.

On 17 August 2018 at 21:06, Anuj Wadehra 
wrote:

> Hi,
>
> I think CASSANDRA-14227 is pending for long time now. Though, the  data
> loss issue was addressed in CASSANDRA-14092, Cassandra users are still
> prohibited to use long TTLs (20+ years) as the maximum expiration timestamp
> that can be represented by the storage engine is 2038-01-19T03:14:06+00:00
> (due to the encoding of localExpirationTime as an int32). As per JIRA
> comments, the fix seems relatively simple. Considering high impact/returns
> and relatively less efforts, are there any plans to prioritize this fix for
> upcoming releases?
>
> Thanks
> Anuj
>
>
>
>
> On Saturday, 27 January, 2018, 8:35:20 PM IST, Anuj Wadehra <
> anujw_2...@yahoo.co.in.INVALID> wrote:
>
>
>
>
>
> Hi Paulo,
>
> Thanks for coming out with the Emergency Hot Fix!!
> The patch will help many Cassandra users in saving their precious data.
> I think the criticality and urgency of the bug is too high. How can we
> make sure that maximum Cassandra users are alerted about the silent
> deletion problem? What are formal ways of working for broadcasting such
> critical alerts?
> I still see that the JIRA is marked as a "Major" defect and not a
> "Blocker". What worst can happen to a database than irrecoverable silent
> deletion of successfully inserted data. I hope you understand.
>
>
>
> ThanksAnuj
>
>
>
>
>   On Fri, 26 Jan 2018 at 18:57, Paulo Motta
> wrote:  > I have serious concerns regarding reducing the TTL to 15 yrs.The
> patch will immediately break all existing applications in Production which
> are using 15+ yrs TTL.
>
> In order to prevent applications from breaking I will update the patch
> to automatically set the maximum TTL to '03:14:08 UTC 19 January 2038'
> when it overflows and log a warning as a initial measure.  We will
> work on extending this limit or lifting this limitation, probably for
> the 3.0+ series due to the large scale compatibility changes required
> on lower versions, but community patches are always welcome.
>
> Companies that cannot upgrade to a version with the proper fix will
> need to workaround this limitation in some other way: do a batch job
> to delete old data periodically, perform deletes with timestamps in
> the future, etc.
>
> > If its a 32 bit timestamp, can't we just save/read localDeletionTime as
> unsinged int?
>
> The proper fix will likely be along these lines, but this involve many
> changes throughout the codebase where localDeletionTime is consumed
> and extensive testing, reviewing, etc, so we're now looking into a
> emergency hot fix to prevent silent data loss while the permanent fix is
> not in place.
>
> 2018-01-26 6:27 GMT-02:00 Anuj Wadehra :
> > Hi Jeff,
> > One correction in my last message: "it may be more feasible to SUPPORT
> (not extend) the 20 year limit in Cassandra in 2.1/2.2".
> > I completely agree that the existing 20 years TTL support is okay for
> older versions.
> >
> > If I have understood your last message correctly, upcoming patches are
> on following lines :
> >
> > 1. New Patches shall be released for 2.1, 2.2 and 3.x.2. The patches for
> 2.1 & 2.2 would support the existing 20 year TTL limit and ensure that
> there is no data loss when 20 year is set as TTL.3. The patches for 2.1 and
> 2.2 are unlikely to update the sstable format.
> > 4. 3.x patches may even remove the 20 year TTL constraint (and extend
> TTL support beyond 2038).
> > I think that the JIRA priority should be increased from "Major" to
> "Blocker" as the JIRA may cause unexpected data loss. Also, all impacted
> versions should be included in the JIRA. This will attract the due
> attention of all Cassandra users.
> > ThanksAnuj
> >On Friday 26 January 2018, 12:47:18 PM IST, Anuj Wadehra <
> anujw_2...@yahoo.co.in.INVALID> wrote:
> >
> >  Hi Jeff,
> >
> > Thanks for the prompt action! I agree that patching an application MAY
> have a shorter life cycle than patching Cassandra in production. But, in
> the interest of the larger Cassandra user community, we should put our best
> effort to avoid breaking all the affected applications in production. We
> should also consider that updating business logic as per the new 15 year
> TTL constraint may have business implications for many users. I have a
> limited understanding about the complexity of the code patch, but it may be
> more feasible to extend the 20 year limit in Cassandra in 2.1/2.2 rather
> than asking all impacted users to do an immediate business logic
> adaptation. Moreover, now that we officially support Cassandra 2.1 & 2.2
> until 4.0 release and provide critical fixes for 2.1, it becomes even more
> reasonable to provide this extremely critical patch for 2.1 & 2.2 (unless
> its absolutely impossible). Still, many users use Cassandra 2.1 and 2.2 in
> their most critical production systems.
> >
> > Thanks
> > Anuj
> >
> > 

Re: URGENT: CASSANDRA-14092 causes Data Loss

[Anuj Wadehra]

Hi,

I think CASSANDRA-14227 is pending for long time now. Though, the  data loss 
issue was addressed in CASSANDRA-14092, Cassandra users are still prohibited to 
use long TTLs (20+ years) as the maximum expiration timestamp that can be 
represented by the storage engine is 2038-01-19T03:14:06+00:00 (due to the 
encoding of localExpirationTime as an int32). As per JIRA comments, the fix 
seems relatively simple. Considering high impact/returns and relatively less 
efforts, are there any plans to prioritize this fix for upcoming releases? 

Thanks
Anuj




On Saturday, 27 January, 2018, 8:35:20 PM IST, Anuj Wadehra 
 wrote: 





Hi Paulo,

Thanks for coming out with the Emergency Hot Fix!! 
The patch will help many Cassandra users in saving their precious data.
I think the criticality and urgency of the bug is too high. How can we make 
sure that maximum Cassandra users are alerted about the silent deletion 
problem? What are formal ways of working for broadcasting such critical alerts? 
I still see that the JIRA is marked as a "Major" defect and not a "Blocker". 
What worst can happen to a database than irrecoverable silent deletion of 
successfully inserted data. I hope you understand.



ThanksAnuj




  On Fri, 26 Jan 2018 at 18:57, Paulo Motta wrote:  > 
I have serious concerns regarding reducing the TTL to 15 yrs.The patch will 
immediately break all existing applications in Production which are using 15+ 
yrs TTL.

In order to prevent applications from breaking I will update the patch
to automatically set the maximum TTL to '03:14:08 UTC 19 January 2038'
when it overflows and log a warning as a initial measure.  We will
work on extending this limit or lifting this limitation, probably for
the 3.0+ series due to the large scale compatibility changes required
on lower versions, but community patches are always welcome.

Companies that cannot upgrade to a version with the proper fix will
need to workaround this limitation in some other way: do a batch job
to delete old data periodically, perform deletes with timestamps in
the future, etc.

> If its a 32 bit timestamp, can't we just save/read localDeletionTime as 
> unsinged int?

The proper fix will likely be along these lines, but this involve many
changes throughout the codebase where localDeletionTime is consumed
and extensive testing, reviewing, etc, so we're now looking into a
emergency hot fix to prevent silent data loss while the permanent fix is
not in place.

2018-01-26 6:27 GMT-02:00 Anuj Wadehra :
> Hi Jeff,
> One correction in my last message: "it may be more feasible to SUPPORT (not 
> extend) the 20 year limit in Cassandra in 2.1/2.2".
> I completely agree that the existing 20 years TTL support is okay for older 
> versions.
>
> If I have understood your last message correctly, upcoming patches are on 
> following lines :
>
> 1. New Patches shall be released for 2.1, 2.2 and 3.x.2. The patches for 2.1 
> & 2.2 would support the existing 20 year TTL limit and ensure that there is 
> no data loss when 20 year is set as TTL.3. The patches for 2.1 and 2.2 are 
> unlikely to update the sstable format.
> 4. 3.x patches may even remove the 20 year TTL constraint (and extend TTL 
> support beyond 2038).
> I think that the JIRA priority should be increased from "Major" to "Blocker" 
> as the JIRA may cause unexpected data loss. Also, all impacted versions 
> should be included in the JIRA. This will attract the due attention of all 
> Cassandra users.
> ThanksAnuj
>    On Friday 26 January 2018, 12:47:18 PM IST, Anuj Wadehra 
> wrote:
>
>  Hi Jeff,
>
> Thanks for the prompt action! I agree that patching an application MAY have a 
> shorter life cycle than patching Cassandra in production. But, in the 
> interest of the larger Cassandra user community, we should put our best 
> effort to avoid breaking all the affected applications in production. We 
> should also consider that updating business logic as per the new 15 year TTL 
> constraint may have business implications for many users. I have a limited 
> understanding about the complexity of the code patch, but it may be more 
> feasible to extend the 20 year limit in Cassandra in 2.1/2.2 rather than 
> asking all impacted users to do an immediate business logic adaptation. 
> Moreover, now that we officially support Cassandra 2.1 & 2.2 until 4.0 
> release and provide critical fixes for 2.1, it becomes even more reasonable 
> to provide this extremely critical patch for 2.1 & 2.2 (unless its absolutely 
> impossible). Still, many users use Cassandra 2.1 and 2.2 in their most 
> critical production systems.
>
> Thanks
> Anuj
>
>    On Friday 26 January 2018, 11:06:30 AM IST, Jeff Jirsa  
>wrote:
>
>  We’ll get patches out. They almost certainly aren’t going to change the 
>sstable format for old versions (unless whoever writes the patch makes a great 
>argument for it), so there’s probably not going to be post-2038 ttl support 
>for 2.1/2.2. For those old versions, we can definitely make it 

Re: Proposing an Apache Cassandra Management process

[Dinesh Joshi]

> On Aug 16, 2018, at 9:27 PM, Sankalp Kohli  wrote:
> 
> I am bumping this thread because patch has landed for this with repair 
> functionality. 
> 
> I have a following proposal for this which I can put in the JIRA or doc 
> 
> 1. We should see if we can keep this in a separate repo like Dtest. 

This would imply a looser coupling between the two. Keeping things in-tree is 
my preferred approach. It makes testing, dependency management and code sharing 
easier.

> 2. It should have its own release process.

This means now there would be two releases that need to be managed and 
coordinated.

> 3. It should have integration tests for different versions of Cassandra it 
> will support. 

Given the lack of test infrastructure - this will be hard especially if you 
have to qualify a matrix of builds. 

Dinesh
-
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org