Re: What kind of Automation you have for Cassandra related operations on AWS ?

2018-02-09 Thread vincent gromakowski 
It will clearly follow your colleagues approach on the postgresql operator
https://github.com/zalando-incubator/postgres-operator

Just watch my repo for a first beta working version in the next weeks
https://github.com/vgkowski/cassandra-operator


2018-02-09 15:20 GMT+01:00 Oleksandr Shulgin :

> On Fri, Feb 9, 2018 at 1:01 PM, vincent gromakowski <
> vincent.gromakow...@gmail.com> wrote:
>
>> Working on a Kubernetes operator for Cassandra (Alpha stage...)
>>
>
> I would love to learn more about your approach.  Do you have anything to
> show already?  Design docs / prototype?
>
> --
> Alex
>
>

Re: What kind of Automation you have for Cassandra related operations on AWS ?

2018-02-09 Thread Oleksandr Shulgin 
On Fri, Feb 9, 2018 at 1:01 PM, vincent gromakowski <
vincent.gromakow...@gmail.com> wrote:

> Working on a Kubernetes operator for Cassandra (Alpha stage...)
>

I would love to learn more about your approach.  Do you have anything to
show already?  Design docs / prototype?

--
Alex

Re: What kind of Automation you have for Cassandra related operations on AWS ?

2018-02-09 Thread vincent gromakowski 
Working on a Kubernetes operator for Cassandra (Alpha stage...)

Le 9 févr. 2018 12:56 PM, "Oleksandr Shulgin" 
a écrit :

> On Fri, Feb 9, 2018 at 12:46 AM, Krish Donald 
> wrote:
>
>> Hi All,
>>
>> What kind of Automation you have for Cassandra related operations on AWS
>> like restacking, restart of the cluster , changing cassandra.yaml
>> parameters etc ?
>>
>
> We wrote some scripts customized for Zalando's STUPS platform:
> https://github.com/zalando-stups/planb-cassandra  (Warning! messy Python
> inside)
>
> We deploy EBS-backed instances with AWS EC2 auto-recovery enabled.
> Cassandra runs inside Docker on the EC2 hosts.
>
> The EBS setup allows us to perform rolling restarts / binary updates
> without streaming.
>
> Updating configuration parameters is a bit tricky since there are many
> places where different stuff is configured: cassandra-env.sh, jvm.options,
> cassandra.yaml and environment variables.  We don't have a comprehensive
> answer to that yet.
>
> Cheers,
> --
> Oleksandr "Alex" Shulgin | Database Engineer | Zalando SE | Tel: +49 176
> 127-59-707 <+49%20176%2012759707>
>
>

Re: What kind of Automation you have for Cassandra related operations on AWS ?

2018-02-09 Thread Oleksandr Shulgin 
On Fri, Feb 9, 2018 at 12:46 AM, Krish Donald  wrote:

> Hi All,
>
> What kind of Automation you have for Cassandra related operations on AWS
> like restacking, restart of the cluster , changing cassandra.yaml
> parameters etc ?
>

We wrote some scripts customized for Zalando's STUPS platform:
https://github.com/zalando-stups/planb-cassandra  (Warning! messy Python
inside)

We deploy EBS-backed instances with AWS EC2 auto-recovery enabled.
Cassandra runs inside Docker on the EC2 hosts.

The EBS setup allows us to perform rolling restarts / binary updates
without streaming.

Updating configuration parameters is a bit tricky since there are many
places where different stuff is configured: cassandra-env.sh, jvm.options,
cassandra.yaml and environment variables.  We don't have a comprehensive
answer to that yet.

Cheers,
-- 
Oleksandr "Alex" Shulgin | Database Engineer | Zalando SE | Tel: +49 176
127-59-707

Re: What kind of Automation you have for Cassandra related operations on AWS ?

2018-02-08 Thread Lerh Chuan Low 
Terraform, Packer and Ansible does pretty decently, you may have to do some
smarts around replacing nodes and attaching the right volumes to replaced
nodes. If you could get Kubernetes working with Cassandra (beyond the
readily available guides) then I think you'll be a total baller.

On 9 February 2018 at 15:45, daemeon reiydelle  wrote:

> Terraform plus ansible. Put ok but messy. 5-30,000 nodes and infra
>
>
> Daemeon (Dæmœn) Reiydelle
> USA 1.415.501.0198 <(415)%20501-0198>
>
> On Thu, Feb 8, 2018, 15:57 Ben Wood  wrote:
>
>> Shameless plug of our (DC/OS) Apache Cassandra service: https://docs.
>> mesosphere.com/services/cassandra/2.0.3-3.0.14.
>>
>> You must run DC/OS, but it will handle:
>> Restarts
>> Replacement of nodes
>> Modification of configuration
>> Backups and Restores (to S3)
>>
>> On Thu, Feb 8, 2018 at 3:46 PM, Krish Donald 
>> wrote:
>>
>>> Hi All,
>>>
>>> What kind of Automation you have for Cassandra related operations on AWS
>>> like restacking, restart of the cluster , changing cassandra.yaml
>>> parameters etc ?
>>>
>>> Thanks
>>>
>>>
>>
>>
>> --
>> Ben Wood
>> Software Engineer - Data Agility
>> Mesosphere
>>
>

Re: What kind of Automation you have for Cassandra related operations on AWS ?

2018-02-08 Thread daemeon reiydelle 
Terraform plus ansible. Put ok but messy. 5-30,000 nodes and infra


Daemeon (Dæmœn) Reiydelle
USA 1.415.501.0198

On Thu, Feb 8, 2018, 15:57 Ben Wood  wrote:

> Shameless plug of our (DC/OS) Apache Cassandra service:
> https://docs.mesosphere.com/services/cassandra/2.0.3-3.0.14.
>
> You must run DC/OS, but it will handle:
> Restarts
> Replacement of nodes
> Modification of configuration
> Backups and Restores (to S3)
>
> On Thu, Feb 8, 2018 at 3:46 PM, Krish Donald  wrote:
>
>> Hi All,
>>
>> What kind of Automation you have for Cassandra related operations on AWS
>> like restacking, restart of the cluster , changing cassandra.yaml
>> parameters etc ?
>>
>> Thanks
>>
>>
>
>
> --
> Ben Wood
> Software Engineer - Data Agility
> Mesosphere
>

Re: What kind of Automation you have for Cassandra related operations on AWS ?

2018-02-08 Thread Romain Hardouin 
 
At Teads we use Terraform, Chef, Packer and Rundeck for our AWS infrastructure. 
I'll publish a blog post on Medium which talk about that, it's in the pipeline. 
Terraform is awesome.
Best,
RomainLe vendredi 9 février 2018 à 00:57:01 UTC+1, Ben Wood 
 a écrit :  
 
 Shameless plug of our (DC/OS) Apache Cassandra service: 
https://docs.mesosphere.com/services/cassandra/2.0.3-3.0.14.
You must run DC/OS, but it will handle:RestartsReplacement of nodesModification 
of configurationBackups and Restores (to S3)
On Thu, Feb 8, 2018 at 3:46 PM, Krish Donald  wrote:

Hi All,
What kind of Automation you have for Cassandra related operations on AWS like 
restacking, restart of the cluster , changing cassandra.yaml parameters etc ?

Thanks




-- 
Ben WoodSoftware Engineer - Data AgilityMesosphere

Re: What kind of Automation you have for Cassandra related operations on AWS ?

2018-02-08 Thread Ben Wood 
Shameless plug of our (DC/OS) Apache Cassandra service:
https://docs.mesosphere.com/services/cassandra/2.0.3-3.0.14.

You must run DC/OS, but it will handle:
Restarts
Replacement of nodes
Modification of configuration
Backups and Restores (to S3)

On Thu, Feb 8, 2018 at 3:46 PM, Krish Donald  wrote:

> Hi All,
>
> What kind of Automation you have for Cassandra related operations on AWS
> like restacking, restart of the cluster , changing cassandra.yaml
> parameters etc ?
>
> Thanks
>
>


-- 
Ben Wood
Software Engineer - Data Agility
Mesosphere

What kind of Automation you have for Cassandra related operations on AWS ?

2018-02-08 Thread Krish Donald 
Hi All,

What kind of Automation you have for Cassandra related operations on AWS
like restacking, restart of the cluster , changing cassandra.yaml
parameters etc ?

Thanks