Re: Proposing an Apache Cassandra Management process

[Sankalp Kohli]

I am bumping this thread because patch has landed for this with repair 
functionality. 

I have a following proposal for this which I can put in the JIRA or doc 

1. We should see if we can keep this in a separate repo like Dtest. 
2. It should have its own release process.
3. It should have integration tests for different versions of Cassandra it will 
support. 

Does anyone has any ideas on this ? 

Thanks
Sankalp 

> On Apr 18, 2018, at 19:20, Dinesh Joshi  
> wrote:
> 
> Thank you all for the feedback. Nate made a Google doc with the proposal in 
> it and is linked off of the ticket. I will try to flesh it out as I gather 
> your input.
> Dinesh 
> 
>On Wednesday, April 18, 2018, 5:12:49 PM PDT, kurt greaves 
>  wrote:  
> 
> For anyone looking Dinesh made a ticket already: CASSANDRA-14395
> 
> 
>> On 18 April 2018 at 18:14, Vinay Chella  wrote:
>> 
>> This is a good initiative. We have advocated for and run a sidecar for the
>> past 5+ years, and we've learned and improved it a lot. We look forward to
>> moving features from Priam (such as backup, HTTP -> JMX, etc) incrementally
>> to this sidecar as they make sense.
>> 
>> 
>> Thanks,
>> Vinay Chella
>> 
>> On Fri, Apr 13, 2018 at 7:01 AM, Eric Evans 
>> wrote:
>> 
>>> On Thu, Apr 12, 2018 at 4:41 PM, Dinesh Joshi
>>>  wrote:
 Hey all -
 With the uptick in discussion around Cassandra operability and after
>>> discussing potential solutions with various members of the community, we
>>> would like to propose the addition of a management process/sub-project
>> into
>>> Apache Cassandra. The process would be responsible for common operational
>>> tasks like bulk execution of nodetool commands, backup/restore, and
>> health
>>> checks, among others. We feel we have a proposal that will garner some
>>> discussion and debate but is likely to reach consensus.
 While the community, in large part, agrees that these features should
>>> exist “in the database”, there is debate on how they should be
>> implemented.
>>> Primarily, whether or not to use an external process or build on
>>> CassandraDaemon. This is an important architectural decision but we feel
>>> the most critical aspect is not where the code runs but that the operator
>>> still interacts with the notion of a single database. Multi-process
>>> databases are as old as Postgres and continue to be common in newer
>> systems
>>> like Druid. As such, we propose a separate management process for the
>>> following reasons:
 
 - Resource isolation & Safety: Features in the management process
>>> will not affect C*'s read/write path which is critical for stability. An
>>> isolated process has several technical advantages including preventing
>> use
>>> of unnecessary dependencies in CassandraDaemon, separation of JVM
>> resources
>>> like thread pools and heap, and preventing bugs from adversely affecting
>>> the main process. In particular, GC tuning can be done separately for the
>>> two processes, hopefully helping to improve, or at least not adversely
>>> affect, tail latencies of the main process.
 
 - Health Checks & Recovery: Currently users implement health checks
>>> in their own sidecar process. Implementing them in the serving process
>> does
>>> not make sense because if the JVM running the CassandraDaemon goes south,
>>> the healthchecks and potentially any recovery code may not be able to
>> run.
>>> Having a management process running in isolation opens up the possibility
>>> to not only report the health of the C* process such as long GC pauses or
>>> stuck JVM but also to recover from it. Having a list of basic health
>> checks
>>> that are tested with every C* release and officially supported will help
>>> boost confidence in C* quality and make it easier to operate.
 
 - Reduced Risk: By having a separate Daemon we open the possibility
>>> to contribute features that otherwise would not have been considered
>> before
>>> eg. a UI. A library that started many background threads and is operated
>>> completely differently would likely be considered too risky for
>>> CassandraDaemon but is a good candidate for the management process.
>>> 
>>> Makes sense IMO.
>>> 
 What can go into the management process?
 - Features that are non-essential for serving reads & writes for eg.
>>> Backup/Restore or Running Health Checks against the CassandraDaemon, etc.
 
 - Features that do not make the management process critical for
>>> functioning of the serving process. In other words, if someone does not
>>> wish to use this management process, they are free to disable it.
 
 We would like to initially build minimal set of features such as health
>>> checks and bulk commands into the first iteration of the management
>>> process. We would use the same software stack that is used to build the
>>> current CassandraDaemon binary. This would be critical for sharing code

Re: upgrade guava on trunk before 9/1?

[Andy Tolbert]

Hi Sumanth,

I gave a shot at upgrading to Guava 26.0; resolved build issues; about 63
> Unit Tests fail - a lot of them due to a NPE and another bunch due to
> the driver using a method that does not exist in latest Guava.
>

Unfortunately the driver doesn't currently work with Guava 26.0.  We have a
fix (JAVA-1928 ) that
addresses this that will be in version 3.6.0 which should be released in
the coming weeks.  After that the driver will work with Guava 16.0.1
through 26 (We've run our full test suite against all versions in that
range).

Thanks,
Andy

On Thu, Aug 16, 2018 at 6:55 PM, Sumanth Pasupuleti <
sumanth.pasupuleti...@gmail.com> wrote:

> I am +1 on upgrading to latest Guava, given that we would be on 4.0 for a
> while, and we would have a relatively big testing phase that follows the
> freeze date, which would hopefully iron out (as much as possible) any
> issues we may have with the upgrade.
>
> I gave a shot at upgrading to Guava 26.0; resolved build issues; about 63
> Unit Tests fail - a lot of them due to a NPE and another bunch due to
> the driver using a method that does not exist in latest Guava.
>
> Github Branch:
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.
> com_sumanth-2Dpasupuleti_cassandra_tree_guava-5F26-5Ftrunk=DwIFaQ=
> adz96Xi0w1RHqtPMowiL2g=VHsWqsWT2MoX5jRZ0xZfdAWZxBsrn5KzowynGYCJaXE=
> NBUK9LYIuBLTb2_dn7ZZnfqkFQuWmAr69kTsiLgthN8=
> upEwsGydoKbFV9vnRLMO9LEy4YaSlVXBiPoChzsJfCE=
> Failing Unit Tests: https://urldefense.proofpoint.com/v2/url?u=https-3A__
> circleci.com_gh_sumanth-2Dpasupuleti_cassandra_84=DwIFaQ=
> adz96Xi0w1RHqtPMowiL2g=VHsWqsWT2MoX5jRZ0xZfdAWZxBsrn5KzowynGYCJaXE=
> NBUK9LYIuBLTb2_dn7ZZnfqkFQuWmAr69kTsiLgthN8=oKPrxl_zBsIcuZoHkzS2MZOi9U5_
> lF3DaNldTbMQsBU=
>
> Thanks,
> Sumanth
>
> On Thu, Aug 16, 2018 at 8:22 AM, Jonathan Haddad 
> wrote:
>
> > Pushing it back means it’s a bigger risk later on. I’m +.5 on upgrading
> now
> >
> > On Wed, Aug 15, 2018 at 11:46 PM dinesh.jo...@yahoo.com.INVALID
> >  wrote:
> >
> > > Jason,
> > > Given that we're so close to the 9/1 date, I would err on the side of
> > > caution especially given the low value prop. If someone does run into
> > Guava
> > > compatibility issues (and someone somewhere will), we can revisit this
> > > question then.
> > > Dinesh
> > >
> > > On Wednesday, August 15, 2018, 11:42:31 PM PDT, Dinesh A. Joshi <
> > > dinesh.jo...@gatech.edu> wrote:
> > >
> > >  Jason,
> > > Given that we're so close to the 9/1 date, I would err on the side of
> > > caution especially given the low value prop. If someone does run into
> > Guava
> > > compatibility issues (and someone somewhere will), we can revisit this
> > > question then.
> > > Dinesh
> > >
> > > On Wednesday, August 15, 2018, 8:22:28 AM PDT, Salih Gedik <
> > > m...@salih.xyz> wrote:
> > >
> > >  Hi,
> > >
> > > Change logs are on Github releases page. It seems like only hash
> flooding
> > > protection which is added to ImmutableMap is relevant to Cassandra
> code.
> > I
> > > haven’t checked whether we use deprecated APIs. But there isn’t much on
> > > table from what I see.
> > >
> > > Salih
> > > On 15 Aug 2018 17:55 +0300, Ariel Weisberg , wrote:
> > > > Hi,
> > > >
> > > > They don't even do release notes after 23. Also no API diffs. I mean
> > I'm
> > > fine with it, but it's mostly just changing to another arbitrary
> version
> > > that won't match what is in apps.
> > > >
> > > > Ariel
> > > >
> > > > On Wed, Aug 15, 2018, at 10:48 AM, Jason Brown wrote:
> > > > > Hey Ariel,
> > > > >
> > > > > Tbqh, not that much. I was mostly thinking from the "I have
> conflicts
> > > on
> > > > > guava versions in my app because I pull in cassandra and XYZ
> > > libraries, and
> > > > > the transitive dependencies on guava use different versions" POV.
> > > Further,
> > > > > we'll be on this version of guava for 4.0 for at least two years
> from
> > > now.
> > > > >
> > > > > As I asked, "does anybody feeling strongly?". Personally, I'm sorta
> > +0
> > > to
> > > > > +0.5, but I was just throwing this out there in case someone does
> > > really
> > > > > think it best we upgrade (and wants to make a contribution).
> > > > >
> > > > > -Jason
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Wed, Aug 15, 2018 at 7:25 AM, Ariel Weisberg  >
> > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > What do we get from Guava in exchange for upgrading?
> > > > > >
> > > > > > Ariel
> > > > > >
> > > > > > On Wed, Aug 15, 2018, at 10:19 AM, Jason Brown wrote:
> > > > > > > Hey all,
> > > > > > >
> > > > > > > Does anyone feel strongly about upgrading guava on trunk before
> > > the 9/1
> > > > > > > feature freeze for 4.0? We are currently at 23.3 (thanks to
> > > > > > > CASSANDRA-13997), and the current is 26.0.
> > > > > > >
> > > > > > > I took a quick look, and there's about 17 compilation errors.
> > They
> > > fall
> > > > > > > into two categories, both of which appear 

Re: upgrade guava on trunk before 9/1?

[Sumanth Pasupuleti]

I am +1 on upgrading to latest Guava, given that we would be on 4.0 for a
while, and we would have a relatively big testing phase that follows the
freeze date, which would hopefully iron out (as much as possible) any
issues we may have with the upgrade.

I gave a shot at upgrading to Guava 26.0; resolved build issues; about 63
Unit Tests fail - a lot of them due to a NPE and another bunch due to
the driver using a method that does not exist in latest Guava.

Github Branch:
https://github.com/sumanth-pasupuleti/cassandra/tree/guava_26_trunk
Failing Unit Tests: https://circleci.com/gh/sumanth-pasupuleti/cassandra/84

Thanks,
Sumanth

On Thu, Aug 16, 2018 at 8:22 AM, Jonathan Haddad  wrote:

> Pushing it back means it’s a bigger risk later on. I’m +.5 on upgrading now
>
> On Wed, Aug 15, 2018 at 11:46 PM dinesh.jo...@yahoo.com.INVALID
>  wrote:
>
> > Jason,
> > Given that we're so close to the 9/1 date, I would err on the side of
> > caution especially given the low value prop. If someone does run into
> Guava
> > compatibility issues (and someone somewhere will), we can revisit this
> > question then.
> > Dinesh
> >
> > On Wednesday, August 15, 2018, 11:42:31 PM PDT, Dinesh A. Joshi <
> > dinesh.jo...@gatech.edu> wrote:
> >
> >  Jason,
> > Given that we're so close to the 9/1 date, I would err on the side of
> > caution especially given the low value prop. If someone does run into
> Guava
> > compatibility issues (and someone somewhere will), we can revisit this
> > question then.
> > Dinesh
> >
> > On Wednesday, August 15, 2018, 8:22:28 AM PDT, Salih Gedik <
> > m...@salih.xyz> wrote:
> >
> >  Hi,
> >
> > Change logs are on Github releases page. It seems like only hash flooding
> > protection which is added to ImmutableMap is relevant to Cassandra code.
> I
> > haven’t checked whether we use deprecated APIs. But there isn’t much on
> > table from what I see.
> >
> > Salih
> > On 15 Aug 2018 17:55 +0300, Ariel Weisberg , wrote:
> > > Hi,
> > >
> > > They don't even do release notes after 23. Also no API diffs. I mean
> I'm
> > fine with it, but it's mostly just changing to another arbitrary version
> > that won't match what is in apps.
> > >
> > > Ariel
> > >
> > > On Wed, Aug 15, 2018, at 10:48 AM, Jason Brown wrote:
> > > > Hey Ariel,
> > > >
> > > > Tbqh, not that much. I was mostly thinking from the "I have conflicts
> > on
> > > > guava versions in my app because I pull in cassandra and XYZ
> > libraries, and
> > > > the transitive dependencies on guava use different versions" POV.
> > Further,
> > > > we'll be on this version of guava for 4.0 for at least two years from
> > now.
> > > >
> > > > As I asked, "does anybody feeling strongly?". Personally, I'm sorta
> +0
> > to
> > > > +0.5, but I was just throwing this out there in case someone does
> > really
> > > > think it best we upgrade (and wants to make a contribution).
> > > >
> > > > -Jason
> > > >
> > > >
> > > >
> > > >
> > > > On Wed, Aug 15, 2018 at 7:25 AM, Ariel Weisberg 
> > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > What do we get from Guava in exchange for upgrading?
> > > > >
> > > > > Ariel
> > > > >
> > > > > On Wed, Aug 15, 2018, at 10:19 AM, Jason Brown wrote:
> > > > > > Hey all,
> > > > > >
> > > > > > Does anyone feel strongly about upgrading guava on trunk before
> > the 9/1
> > > > > > feature freeze for 4.0? We are currently at 23.3 (thanks to
> > > > > > CASSANDRA-13997), and the current is 26.0.
> > > > > >
> > > > > > I took a quick look, and there's about 17 compilation errors.
> They
> > fall
> > > > > > into two categories, both of which appear not too difficult to
> > resolve (I
> > > > > > didn't look too closely, tbh).
> > > > > >
> > > > > > If anyone wants to tackle this LHF I can rustle up some review
> > time.
> > > > > >
> > > > > > Thanks,
> > > > > >
> > > > > > -Jason
> > > > >
> > > > > 
> -
> > > > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > > > > For additional commands, e-mail: dev-h...@cassandra.apache.org
> > > > >
> > > > >
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > > For additional commands, e-mail: dev-h...@cassandra.apache.org
> > >
>
> --
> Jon Haddad
> http://www.rustyrazorblade.com
> twitter: rustyrazorblade
>

Re: upgrade guava on trunk before 9/1?

[Jonathan Haddad]

Pushing it back means it’s a bigger risk later on. I’m +.5 on upgrading now

On Wed, Aug 15, 2018 at 11:46 PM dinesh.jo...@yahoo.com.INVALID
 wrote:

> Jason,
> Given that we're so close to the 9/1 date, I would err on the side of
> caution especially given the low value prop. If someone does run into Guava
> compatibility issues (and someone somewhere will), we can revisit this
> question then.
> Dinesh
>
> On Wednesday, August 15, 2018, 11:42:31 PM PDT, Dinesh A. Joshi <
> dinesh.jo...@gatech.edu> wrote:
>
>  Jason,
> Given that we're so close to the 9/1 date, I would err on the side of
> caution especially given the low value prop. If someone does run into Guava
> compatibility issues (and someone somewhere will), we can revisit this
> question then.
> Dinesh
>
> On Wednesday, August 15, 2018, 8:22:28 AM PDT, Salih Gedik <
> m...@salih.xyz> wrote:
>
>  Hi,
>
> Change logs are on Github releases page. It seems like only hash flooding
> protection which is added to ImmutableMap is relevant to Cassandra code. I
> haven’t checked whether we use deprecated APIs. But there isn’t much on
> table from what I see.
>
> Salih
> On 15 Aug 2018 17:55 +0300, Ariel Weisberg , wrote:
> > Hi,
> >
> > They don't even do release notes after 23. Also no API diffs. I mean I'm
> fine with it, but it's mostly just changing to another arbitrary version
> that won't match what is in apps.
> >
> > Ariel
> >
> > On Wed, Aug 15, 2018, at 10:48 AM, Jason Brown wrote:
> > > Hey Ariel,
> > >
> > > Tbqh, not that much. I was mostly thinking from the "I have conflicts
> on
> > > guava versions in my app because I pull in cassandra and XYZ
> libraries, and
> > > the transitive dependencies on guava use different versions" POV.
> Further,
> > > we'll be on this version of guava for 4.0 for at least two years from
> now.
> > >
> > > As I asked, "does anybody feeling strongly?". Personally, I'm sorta +0
> to
> > > +0.5, but I was just throwing this out there in case someone does
> really
> > > think it best we upgrade (and wants to make a contribution).
> > >
> > > -Jason
> > >
> > >
> > >
> > >
> > > On Wed, Aug 15, 2018 at 7:25 AM, Ariel Weisberg 
> wrote:
> > >
> > > > Hi,
> > > >
> > > > What do we get from Guava in exchange for upgrading?
> > > >
> > > > Ariel
> > > >
> > > > On Wed, Aug 15, 2018, at 10:19 AM, Jason Brown wrote:
> > > > > Hey all,
> > > > >
> > > > > Does anyone feel strongly about upgrading guava on trunk before
> the 9/1
> > > > > feature freeze for 4.0? We are currently at 23.3 (thanks to
> > > > > CASSANDRA-13997), and the current is 26.0.
> > > > >
> > > > > I took a quick look, and there's about 17 compilation errors. They
> fall
> > > > > into two categories, both of which appear not too difficult to
> resolve (I
> > > > > didn't look too closely, tbh).
> > > > >
> > > > > If anyone wants to tackle this LHF I can rustle up some review
> time.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > -Jason
> > > >
> > > > -
> > > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > > > For additional commands, e-mail: dev-h...@cassandra.apache.org
> > > >
> > > >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >

-- 
Jon Haddad
http://www.rustyrazorblade.com
twitter: rustyrazorblade

Re: upgrade guava on trunk before 9/1?

[dinesh.jo...@yahoo.com.INVALID]

Jason,
Given that we're so close to the 9/1 date, I would err on the side of caution 
especially given the low value prop. If someone does run into Guava 
compatibility issues (and someone somewhere will), we can revisit this question 
then.
Dinesh 

On Wednesday, August 15, 2018, 11:42:31 PM PDT, Dinesh A. Joshi 
 wrote:  
 
 Jason,
Given that we're so close to the 9/1 date, I would err on the side of caution 
especially given the low value prop. If someone does run into Guava 
compatibility issues (and someone somewhere will), we can revisit this question 
then.
Dinesh 

On Wednesday, August 15, 2018, 8:22:28 AM PDT, Salih Gedik  
wrote:  
 
 Hi,

Change logs are on Github releases page. It seems like only hash flooding 
protection which is added to ImmutableMap is relevant to Cassandra code. I 
haven’t checked whether we use deprecated APIs. But there isn’t much on table 
from what I see.

Salih
On 15 Aug 2018 17:55 +0300, Ariel Weisberg , wrote:
> Hi,
>
> They don't even do release notes after 23. Also no API diffs. I mean I'm fine 
> with it, but it's mostly just changing to another arbitrary version that 
> won't match what is in apps.
>
> Ariel
>
> On Wed, Aug 15, 2018, at 10:48 AM, Jason Brown wrote:
> > Hey Ariel,
> >
> > Tbqh, not that much. I was mostly thinking from the "I have conflicts on
> > guava versions in my app because I pull in cassandra and XYZ libraries, and
> > the transitive dependencies on guava use different versions" POV. Further,
> > we'll be on this version of guava for 4.0 for at least two years from now.
> >
> > As I asked, "does anybody feeling strongly?". Personally, I'm sorta +0 to
> > +0.5, but I was just throwing this out there in case someone does really
> > think it best we upgrade (and wants to make a contribution).
> >
> > -Jason
> >
> >
> >
> >
> > On Wed, Aug 15, 2018 at 7:25 AM, Ariel Weisberg  wrote:
> >
> > > Hi,
> > >
> > > What do we get from Guava in exchange for upgrading?
> > >
> > > Ariel
> > >
> > > On Wed, Aug 15, 2018, at 10:19 AM, Jason Brown wrote:
> > > > Hey all,
> > > >
> > > > Does anyone feel strongly about upgrading guava on trunk before the 9/1
> > > > feature freeze for 4.0? We are currently at 23.3 (thanks to
> > > > CASSANDRA-13997), and the current is 26.0.
> > > >
> > > > I took a quick look, and there's about 17 compilation errors. They fall
> > > > into two categories, both of which appear not too difficult to resolve 
> > > > (I
> > > > didn't look too closely, tbh).
> > > >
> > > > If anyone wants to tackle this LHF I can rustle up some review time.
> > > >
> > > > Thanks,
> > > >
> > > > -Jason
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > > For additional commands, e-mail: dev-h...@cassandra.apache.org
> > >
> > >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>