RE: AWS Cassandra backup/Restore tools

2017-09-12 Thread Durity, Sean R 
Datos IO has a backup/restore product for Cassandra that another team here has 
used successfully. It solves many of the problems inherent with sstable 
captures. Without something like it, restores are a nightmare with any volume 
of data. The downtime required and the loss of data since the snapshot are 
usually not worth it.


Sean Durity

From: Alexander Dejanovski [mailto:a...@thelastpickle.com]
Sent: Friday, May 12, 2017 12:14 PM
To: Manikandan Srinivasan <msriniva...@datastax.com>; Nitan Kainth 
<ni...@bamlabs.com>
Cc: Blake Eggleston <beggles...@apple.com>; cass savy <casss...@gmail.com>; 
user@cassandra.apache.org
Subject: Re: AWS Cassandra backup/Restore tools

Hi,

here are the main techniques that I know of to perform backups for Cassandra :

  *   Tablesnap 
(https://github.com/JeremyGrosser/tablesnap<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_JeremyGrosser_tablesnap=DwMFaQ=MtgQEAMQGqekjTjiAhkudQ=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ=hrZJx9SNtdlofcJElpjcjw4rp4rlAq8nZKSSsefCvCc=DMw6BkRjlkS9LM5RzcvTamwv8fj6_Czd4RcKBmJnUxc=>)
 : performs continuous backups on S3. Comes with tableslurp to restore backups 
(one table at a time only) and tablechop to delete outdated sstables from S3.
  *   incremental backup : activate it in the cassandra.yaml file and it will 
create snapshots for all newly flushed SSTables. It's up to you to move the 
snapshots off-node and delete them. I don't really like that technique since it 
creates a lot of small sstables that eventually contain a lot of outdated data. 
Upon restore you'll have to wait until compaction catches up on compacting all 
the history (which could take a while and use a lot of power). Your backups 
could also grow indefinitely with this technique since there's no compaction, 
so no purge. You'll have to build the restore script/procedure.
  *   scheduled snapshots : you perform full snapshots by yourself and move 
them off node. You'll have to build the restore script/procedure.
  *   EBS snapshots : probably the easiest way to perform backups if you are 
using M4/R4 instances on AWS.

Cheers,

On Thu, May 11, 2017 at 11:01 PM Manikandan Srinivasan 
<msriniva...@datastax.com<mailto:msriniva...@datastax.com>> wrote:
Blake is correct. OpsCenter 6.0 and up doesn't work with OSS C*. @Nitan: We 
have made some substantial changes to the Opscenter 6.1 backup service, 
specifically when it comes to S3 backups. Having said this, I am not going to 
be sale-sy here. If folks need some help or need more clarity to know more 
about these improvements, please send me an email directly: 
msriniva...@datastax.com<mailto:msriniva...@datastax.com>

Regards
Mani

On Thu, May 11, 2017 at 1:54 PM, Nitan Kainth 
<ni...@bamlabs.com<mailto:ni...@bamlabs.com>> wrote:
Also , Opscenter backup/restore does not work for large databases

Sent from my iPhone

On May 11, 2017, at 3:41 PM, Blake Eggleston 
<beggles...@apple.com<mailto:beggles...@apple.com>> wrote:
OpsCenter 6.0 and up don't work with Cassandra.


On May 11, 2017 at 12:31:08 PM, cass savy 
(casss...@gmail.com<mailto:casss...@gmail.com>) wrote:
AWS Backup/Restore process/tools for C*/DSE C*:

Has anyone used Opscenter 6.1 backup tool to backup/restore data for larger 
datasets online ?

If yes, did you run into issues using that tool to backup/restore data in PROD 
that caused any performance or any other impact to the cluster?

If no, what are other tools that people have used or recommended for backup and 
restore of Cassandra keyspaces?

Please advice.





--
Regards,

Manikandan Srinivasan

Director, Product Management| +1.408.887.3686<tel:%2B1.408.887.3686> | 
manikandan.sriniva...@datastax.com<mailto:manikandan.sriniva...@datastax.com>


[Image removed by sender. 
linkedin.png]<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.linkedin.com_in_srinivm_=DwMFaQ=MtgQEAMQGqekjTjiAhkudQ=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ=hrZJx9SNtdlofcJElpjcjw4rp4rlAq8nZKSSsefCvCc=QVzYL31K-iWGptuTJeKSX2hMW9lrGn5HP3X9p-A8wO4=>[Image
 removed by sender. 
facebook.png]<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.facebook.com_datastax=DwMFaQ=MtgQEAMQGqekjTjiAhkudQ=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ=hrZJx9SNtdlofcJElpjcjw4rp4rlAq8nZKSSsefCvCc=tCXQZRynu6vGzUuBNtKyhKS0qf1FZcZPAlwGw_5HVBM=>[Image
 removed by sender. 
twitter.png]<https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_mani-5Fsrini=DwMFaQ=MtgQEAMQGqekjTjiAhkudQ=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ=hrZJx9SNtdlofcJElpjcjw4rp4rlAq8nZKSSsefCvCc=fH6hn8l2gJJVVmpCOoKdXA80OgFPqt6pt3bjR9pzjxI=>[Image
 removed by sender. 
g+.png]<https://urldefense.proofpoint.com/v2/url?u=https-3A__plus.google.com_-2BDatastax_about=DwMFaQ=MtgQEAMQGqekjTjiAhkudQ=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ=hrZJx9SNtdlofcJElpjcjw4rp4rlAq8nZKSSsefCvCc=2ZG39fHN9Oix46hmXFKRQH5M0AVqA7h-9bqZ7VWvguE=>[Im

Re: AWS Cassandra backup/Restore tools

2017-05-12 Thread Alexander Dejanovski 
Hi,

here are the main techniques that I know of to perform backups for
Cassandra :

   - Tablesnap (https://github.com/JeremyGrosser/tablesnap) : performs
   continuous backups on S3. Comes with tableslurp to restore backups (one
   table at a time only) and tablechop to delete outdated sstables from S3.
   - incremental backup : activate it in the cassandra.yaml file and it
   will create snapshots for all newly flushed SSTables. It's up to you to
   move the snapshots off-node and delete them. I don't really like that
   technique since it creates a lot of small sstables that eventually contain
   a lot of outdated data. Upon restore you'll have to wait until compaction
   catches up on compacting all the history (which could take a while and use
   a lot of power). Your backups could also grow indefinitely with this
   technique since there's no compaction, so no purge. You'll have to build
   the restore script/procedure.
   - scheduled snapshots : you perform full snapshots by yourself and move
   them off node. You'll have to build the restore script/procedure.
   - EBS snapshots : probably the easiest way to perform backups if you are
   using M4/R4 instances on AWS.


Cheers,

On Thu, May 11, 2017 at 11:01 PM Manikandan Srinivasan <
msriniva...@datastax.com> wrote:

> Blake is correct. OpsCenter 6.0 and up doesn't work with OSS C*. @Nitan:
> We have made some substantial changes to the Opscenter 6.1 backup service,
> specifically when it comes to S3 backups. Having said this, I am not going
> to be sale-sy here. If folks need some help or need more clarity to know
> more about these improvements, please send me an email directly:
> msriniva...@datastax.com
>
> Regards
> Mani
>
> On Thu, May 11, 2017 at 1:54 PM, Nitan Kainth  wrote:
>
>> Also , Opscenter backup/restore does not work for large databases
>>
>> Sent from my iPhone
>>
>> On May 11, 2017, at 3:41 PM, Blake Eggleston 
>> wrote:
>>
>> OpsCenter 6.0 and up don't work with Cassandra.
>>
>> On May 11, 2017 at 12:31:08 PM, cass savy (casss...@gmail.com) wrote:
>>
>> AWS Backup/Restore process/tools for C*/DSE C*:
>>
>> Has anyone used Opscenter 6.1 backup tool to backup/restore data for
>> larger datasets online ?
>>
>> If yes, did you run into issues using that tool to backup/restore data in
>> PROD that caused any performance or any other impact to the cluster?
>>
>> If no, what are other tools that people have used or recommended for
>> backup and restore of Cassandra keyspaces?
>>
>> Please advice.
>>
>>
>>
>
>
> --
> Regards,
>
> Manikandan Srinivasan
>
> Director, Product Management| +1.408.887.3686 |
> manikandan.sriniva...@datastax.com
>
> [image: linkedin.png]  [image:
> facebook.png]  [image: twitter.png]
>  [image: g+.png]
> 
> 
>
> --
-
Alexander Dejanovski
France
@alexanderdeja

Consultant
Apache Cassandra Consulting
http://www.thelastpickle.com

Re: AWS Cassandra backup/Restore tools

2017-05-11 Thread Manikandan Srinivasan 
Blake is correct. OpsCenter 6.0 and up doesn't work with OSS C*. @Nitan: We
have made some substantial changes to the Opscenter 6.1 backup service,
specifically when it comes to S3 backups. Having said this, I am not going
to be sale-sy here. If folks need some help or need more clarity to know
more about these improvements, please send me an email directly:
msriniva...@datastax.com

Regards
Mani

On Thu, May 11, 2017 at 1:54 PM, Nitan Kainth  wrote:

> Also , Opscenter backup/restore does not work for large databases
>
> Sent from my iPhone
>
> On May 11, 2017, at 3:41 PM, Blake Eggleston  wrote:
>
> OpsCenter 6.0 and up don't work with Cassandra.
>
> On May 11, 2017 at 12:31:08 PM, cass savy (casss...@gmail.com) wrote:
>
> AWS Backup/Restore process/tools for C*/DSE C*:
>
> Has anyone used Opscenter 6.1 backup tool to backup/restore data for
> larger datasets online ?
>
> If yes, did you run into issues using that tool to backup/restore data in
> PROD that caused any performance or any other impact to the cluster?
>
> If no, what are other tools that people have used or recommended for
> backup and restore of Cassandra keyspaces?
>
> Please advice.
>
>
>


-- 
Regards,

Manikandan Srinivasan

Director, Product Management| +1.408.887.3686 |
manikandan.sriniva...@datastax.com

[image: linkedin.png]  [image:
facebook.png]  [image: twitter.png]
 [image: g+.png]

Re: AWS Cassandra backup/Restore tools

2017-05-11 Thread Nitan Kainth 
Also , Opscenter backup/restore does not work for large databases 

Sent from my iPhone

> On May 11, 2017, at 3:41 PM, Blake Eggleston  wrote:
> 
> OpsCenter 6.0 and up don't work with Cassandra.
> 
>> On May 11, 2017 at 12:31:08 PM, cass savy (casss...@gmail.com) wrote:
>> 
>> AWS Backup/Restore process/tools for C*/DSE C*:
>> 
>> Has anyone used Opscenter 6.1 backup tool to backup/restore data for larger 
>> datasets online ?
>> 
>> If yes, did you run into issues using that tool to backup/restore data in 
>> PROD that caused any performance or any other impact to the cluster?
>> 
>> If no, what are other tools that people have used or recommended for backup 
>> and restore of Cassandra keyspaces?
>> 
>> Please advice.
>> 
>>

Re: AWS Cassandra backup/Restore tools

2017-05-11 Thread Blake Eggleston 
OpsCenter 6.0 and up don't work with Cassandra.

On May 11, 2017 at 12:31:08 PM, cass savy (casss...@gmail.com) wrote:

AWS Backup/Restore process/tools for C*/DSE C*:

Has anyone used Opscenter 6.1 backup tool to backup/restore data for larger 
datasets online ?

If yes, did you run into issues using that tool to backup/restore data in PROD 
that caused any performance or any other impact to the cluster?

If no, what are other tools that people have used or recommended for backup and 
restore of Cassandra keyspaces?

Please advice.

AWS Cassandra backup/Restore tools

2017-05-11 Thread cass savy 
AWS Backup/Restore process/tools for C*/DSE C*:

Has anyone used Opscenter 6.1 backup tool to backup/restore data for larger
datasets online ?

If yes, did you run into issues using that tool to backup/restore data in
PROD that caused any performance or any other impact to the cluster?

If no, what are other tools that people have used or recommended for backup
and restore of Cassandra keyspaces?

Please advice.