Re: Cassandra backup to alternate location

[Jeff Jirsa]

No - they'll hardlink into the snapshot folder on each data directory. They
are true hardlinks, so even if you could move it, it'd still be on the same
filesystem.

Typical behavior is to issue a snapshot, and then copy the data out as
needed (using something like https://github.com/JeremyGrosser/tablesnap ).

On Thu, Jun 28, 2018 at 10:00 AM, Lohchab, Sanjeev  wrote:

> Hi All,
>
>
>
> I am trying to backup Cassandra DB, but by default it is saving the
> snapshots in the default location.
>
> Is there any way we can specific the location where we want to store the
> snapshots.
>
>
>
> Regards
>
> Sanjeev
>

Cassandra backup to alternate location

[Lohchab, Sanjeev]

Hi All,

I am trying to backup Cassandra DB, but by default it is saving the snapshots 
in the default location.
Is there any way we can specific the location where we want to store the 
snapshots.

Regards
Sanjeev

RE: [EXTERNAL] Re: Any Cassandra Backup and Restore tool like Cassandra Reaper?

[Durity, Sean R]

Datos IO solves many of the problems inherent in Cassandra backups (primarily 
issues with acceptable restores). It is worth considering. Other groups in my 
company are happy with it.


Sean Durity

From: Lerh Chuan Low [mailto:l...@instaclustr.com]
Sent: Thursday, December 14, 2017 4:13 PM
To: user@cassandra.apache.org
Subject: [EXTERNAL] Re: Any Cassandra Backup and Restore tool like Cassandra 
Reaper?

Tablesnap assumes S3, and tableslurp can set up the stage for restoring by 
downloading the relevant SSTables (but then it's up to the operator to complete 
the restore from there). Restoring (especially point-in-time restore) isn't 
easy to handle so there aren't a lot available out there.

There's also Netflix's Priam 
https://github.com/Netflix/Priam<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Netflix_Priam=DwMFaQ=MtgQEAMQGqekjTjiAhkudQ=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ=sRV37KK5PaTUR9CK_K51nLT891UrXhpxDLeFISAY1w4=CkFf4Zlib0sjVBkVOiWgYY7G1VYKHrBluvU35Cc9bDc=>
 but I think it's a little bit old and is meant to run alongside C* as an agent 
and be the agent for repairs, monitoring, backups and restores, configuring 
Cassandra YAMLs...

One other one I've heard of is 
https://github.com/tbarbugli/cassandra_snapshotter<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_tbarbugli_cassandra-5Fsnapshotter=DwMFaQ=MtgQEAMQGqekjTjiAhkudQ=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ=sRV37KK5PaTUR9CK_K51nLT891UrXhpxDLeFISAY1w4=0kRuPvbtgWNr2yV4VOpDk4vqthk_L9CNGX0uxFKFiuc=>
 but there's no restore yet.

On 15 December 2017 at 07:52, Rutvij Bhatt 
<rut...@sense.com<mailto:rut...@sense.com>> wrote:
There is tablesnap/tablechop/tableslurp - 
https://github.com/JeremyGrosser/tablesnap<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_JeremyGrosser_tablesnap=DwMFaQ=MtgQEAMQGqekjTjiAhkudQ=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ=sRV37KK5PaTUR9CK_K51nLT891UrXhpxDLeFISAY1w4=xsx3AL8JrDNgaikDMFEBqTcWpZa9M0h5hevQi4kVB4M=>.


On Thu, Dec 14, 2017 at 3:49 PM Roger Brown 
<roger.br...@perfectsearchcorp.com<mailto:roger.br...@perfectsearchcorp.com>> 
wrote:
I've found nothing affordable that works with vnodes. If you have money, you 
could use DataStax OpsCenter or Datos.io Recoverx.

I ended up creating a cron job to make snapshots along with 
incremental_backups: true in the cassandra.yaml. And I'm thinking of setting up 
a replication strategy so that one rack contains 1  replica of each keyspace 
and then using r1soft to image each of those servers to tape for offsite backup.


On Thu, Dec 14, 2017 at 1:30 PM Harika Vangapelli -T (hvangape - AKRAYA INC at 
Cisco) <hvang...@cisco.com<mailto:hvang...@cisco.com>> wrote:
Any Cassandra Backup and Restore tool like Cassandra Reaper for Repairs?




Harika Vangapelli
Engineer - IT
hvang...@cisco.com<mailto:hvang...@cisco.com>
Tel:

Cisco Systems, Inc.



United States
cisco.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__cisco.com=DwMFaQ=MtgQEAMQGqekjTjiAhkudQ=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ=sRV37KK5PaTUR9CK_K51nLT891UrXhpxDLeFISAY1w4=usAzFb-boMNjQssKMHYtwrE4ba8eoBMETpRqqLwVjxs=>


Think before you print.

This email may contain confidential and privileged material for the sole use of 
the intended recipient. Any review, use, distribution or disclosure by others 
is strictly prohibited. If you are not the intended recipient (or authorized to 
receive for the recipient), please contact the sender by reply email and delete 
all copies of this message.
Please click 
here<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.cisco.com_web_about_doing-5Fbusiness_legal_cri_index.html=DwMFaQ=MtgQEAMQGqekjTjiAhkudQ=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ=sRV37KK5PaTUR9CK_K51nLT891UrXhpxDLeFISAY1w4=2zOTFslDaReXQeF2gOKx4ax3gH4aLr9WJgcKUvtJUSU=>
 for Company Registration Information.






The information in this Internet Email is confidential and may be legally 
privileged. It is intended solely for the addressee. Access to this Email by 
anyone else is unauthorized. If you are not the intended recipient, any 
disclosure, copying, distribution or any action taken or omitted to be taken in 
reliance on it, is prohibited and may be unlawful. When addressed to our 
clients any opinions or advice contained in this Email are subject to the terms 
and conditions expressed in any applicable governing The Home Depot terms of 
business or client engagement letter. The Home Depot disclaims all 
responsibility and liability for the accuracy and content of this attachment 
and for any damages or losses arising from any inaccuracies, errors, viruses, 
e.g., worms, trojan horses, etc., or other items of a destructive nature, which 
may be contained in this attachment and shall not be liable for direct, 
indirect, consequential or special damages in connection with this e-mail 
message or its attachment.

Re: Any Cassandra Backup and Restore tool like Cassandra Reaper?

[Lerh Chuan Low]

Tablesnap assumes S3, and tableslurp can set up the stage for restoring by
downloading the relevant SSTables (but then it's up to the operator to
complete the restore from there). Restoring (especially point-in-time
restore) isn't easy to handle so there aren't a lot available out there.

There's also Netflix's Priam https://github.com/Netflix/Priam but I think
it's a little bit old and is meant to run alongside C* as an agent and be
the agent for repairs, monitoring, backups and restores, configuring
Cassandra YAMLs...

One other one I've heard of is
https://github.com/tbarbugli/cassandra_snapshotter but there's no restore
yet.

On 15 December 2017 at 07:52, Rutvij Bhatt <rut...@sense.com> wrote:

> There is tablesnap/tablechop/tableslurp - https://github.com/
> JeremyGrosser/tablesnap.
>
>
> On Thu, Dec 14, 2017 at 3:49 PM Roger Brown <roger.brown@
> perfectsearchcorp.com> wrote:
>
>> I've found nothing affordable that works with vnodes. If you have money,
>> you could use DataStax OpsCenter or Datos.io Recoverx.
>>
>> I ended up creating a cron job to make snapshots along with
>> incremental_backups: true in the cassandra.yaml. And I'm thinking of
>> setting up a replication strategy so that one rack contains 1  replica of
>> each keyspace and then using r1soft to image each of those servers to tape
>> for offsite backup.
>>
>>
>> On Thu, Dec 14, 2017 at 1:30 PM Harika Vangapelli -T (hvangape - AKRAYA
>> INC at Cisco) <hvang...@cisco.com> wrote:
>>
>>> Any Cassandra Backup and Restore tool like Cassandra Reaper for Repairs?
>>>
>>>
>>>
>>> [image:
>>> http://wwwin.cisco.com/c/dam/cec/organizations/gmcc/services-tools/signaturetool/images/logo/logo_gradient.png]
>>>
>>>
>>>
>>> *Harika Vangapelli*
>>>
>>> Engineer - IT
>>>
>>> hvang...@cisco.com
>>>
>>> Tel:
>>>
>>> *Cisco Systems, Inc.*
>>>
>>>
>>>
>>>
>>> United States
>>> cisco.com
>>>
>>>
>>>
>>> [image: http://www.cisco.com/assets/swa/img/thinkbeforeyouprint.gif]Think
>>> before you print.
>>>
>>> This email may contain confidential and privileged material for the sole
>>> use of the intended recipient. Any review, use, distribution or disclosure
>>> by others is strictly prohibited. If you are not the intended recipient (or
>>> authorized to receive for the recipient), please contact the sender by
>>> reply email and delete all copies of this message.
>>>
>>> Please click here
>>> <http://www.cisco.com/web/about/doing_business/legal/cri/index.html>
>>> for Company Registration Information.
>>>
>>>
>>>
>>

Re: Any Cassandra Backup and Restore tool like Cassandra Reaper?

[Rutvij Bhatt]

There is tablesnap/tablechop/tableslurp -
https://github.com/JeremyGrosser/tablesnap.


On Thu, Dec 14, 2017 at 3:49 PM Roger Brown <
roger.br...@perfectsearchcorp.com> wrote:

> I've found nothing affordable that works with vnodes. If you have money,
> you could use DataStax OpsCenter or Datos.io Recoverx.
>
> I ended up creating a cron job to make snapshots along with
> incremental_backups: true in the cassandra.yaml. And I'm thinking of
> setting up a replication strategy so that one rack contains 1  replica of
> each keyspace and then using r1soft to image each of those servers to tape
> for offsite backup.
>
>
> On Thu, Dec 14, 2017 at 1:30 PM Harika Vangapelli -T (hvangape - AKRAYA
> INC at Cisco) <hvang...@cisco.com> wrote:
>
>> Any Cassandra Backup and Restore tool like Cassandra Reaper for Repairs?
>>
>>
>>
>> [image:
>> http://wwwin.cisco.com/c/dam/cec/organizations/gmcc/services-tools/signaturetool/images/logo/logo_gradient.png]
>>
>>
>>
>> *Harika Vangapelli*
>>
>> Engineer - IT
>>
>> hvang...@cisco.com
>>
>> Tel:
>>
>> *Cisco Systems, Inc.*
>>
>>
>>
>>
>> United States
>> cisco.com
>>
>>
>>
>> [image: http://www.cisco.com/assets/swa/img/thinkbeforeyouprint.gif]Think
>> before you print.
>>
>> This email may contain confidential and privileged material for the sole
>> use of the intended recipient. Any review, use, distribution or disclosure
>> by others is strictly prohibited. If you are not the intended recipient (or
>> authorized to receive for the recipient), please contact the sender by
>> reply email and delete all copies of this message.
>>
>> Please click here
>> <http://www.cisco.com/web/about/doing_business/legal/cri/index.html> for
>> Company Registration Information.
>>
>>
>>
>

Re: Any Cassandra Backup and Restore tool like Cassandra Reaper?

[Roger Brown]

I've found nothing affordable that works with vnodes. If you have money,
you could use DataStax OpsCenter or Datos.io Recoverx.

I ended up creating a cron job to make snapshots along with
incremental_backups: true in the cassandra.yaml. And I'm thinking of
setting up a replication strategy so that one rack contains 1  replica of
each keyspace and then using r1soft to image each of those servers to tape
for offsite backup.


On Thu, Dec 14, 2017 at 1:30 PM Harika Vangapelli -T (hvangape - AKRAYA INC
at Cisco) <hvang...@cisco.com> wrote:

> Any Cassandra Backup and Restore tool like Cassandra Reaper for Repairs?
>
>
>
> [image:
> http://wwwin.cisco.com/c/dam/cec/organizations/gmcc/services-tools/signaturetool/images/logo/logo_gradient.png]
>
>
>
> *Harika Vangapelli*
>
> Engineer - IT
>
> hvang...@cisco.com
>
> Tel:
>
> *Cisco Systems, Inc.*
>
>
>
>
> United States
> cisco.com
>
>
>
> [image: http://www.cisco.com/assets/swa/img/thinkbeforeyouprint.gif]Think
> before you print.
>
> This email may contain confidential and privileged material for the sole
> use of the intended recipient. Any review, use, distribution or disclosure
> by others is strictly prohibited. If you are not the intended recipient (or
> authorized to receive for the recipient), please contact the sender by
> reply email and delete all copies of this message.
>
> Please click here
> <http://www.cisco.com/web/about/doing_business/legal/cri/index.html> for
> Company Registration Information.
>
>
>

Any Cassandra Backup and Restore tool like Cassandra Reaper?

[Harika Vangapelli -T (hvangape - AKRAYA INC at Cisco)]

Any Cassandra Backup and Restore tool like Cassandra Reaper for Repairs?

[http://wwwin.cisco.com/c/dam/cec/organizations/gmcc/services-tools/signaturetool/images/logo/logo_gradient.png]



Harika Vangapelli
Engineer - IT
hvang...@cisco.com<mailto:hvang...@cisco.com>
Tel:

Cisco Systems, Inc.



United States
cisco.com


[http://www.cisco.com/assets/swa/img/thinkbeforeyouprint.gif]Think before you 
print.

This email may contain confidential and privileged material for the sole use of 
the intended recipient. Any review, use, distribution or disclosure by others 
is strictly prohibited. If you are not the intended recipient (or authorized to 
receive for the recipient), please contact the sender by reply email and delete 
all copies of this message.
Please click 
here<http://www.cisco.com/web/about/doing_business/legal/cri/index.html> for 
Company Registration Information.

RE: AWS Cassandra backup/Restore tools

[Durity, Sean R]

Datos IO has a backup/restore product for Cassandra that another team here has 
used successfully. It solves many of the problems inherent with sstable 
captures. Without something like it, restores are a nightmare with any volume 
of data. The downtime required and the loss of data since the snapshot are 
usually not worth it.


Sean Durity

From: Alexander Dejanovski [mailto:a...@thelastpickle.com]
Sent: Friday, May 12, 2017 12:14 PM
To: Manikandan Srinivasan <msriniva...@datastax.com>; Nitan Kainth 
<ni...@bamlabs.com>
Cc: Blake Eggleston <beggles...@apple.com>; cass savy <casss...@gmail.com>; 
user@cassandra.apache.org
Subject: Re: AWS Cassandra backup/Restore tools

Hi,

here are the main techniques that I know of to perform backups for Cassandra :

  *   Tablesnap 
(https://github.com/JeremyGrosser/tablesnap<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_JeremyGrosser_tablesnap=DwMFaQ=MtgQEAMQGqekjTjiAhkudQ=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ=hrZJx9SNtdlofcJElpjcjw4rp4rlAq8nZKSSsefCvCc=DMw6BkRjlkS9LM5RzcvTamwv8fj6_Czd4RcKBmJnUxc=>)
 : performs continuous backups on S3. Comes with tableslurp to restore backups 
(one table at a time only) and tablechop to delete outdated sstables from S3.
  *   incremental backup : activate it in the cassandra.yaml file and it will 
create snapshots for all newly flushed SSTables. It's up to you to move the 
snapshots off-node and delete them. I don't really like that technique since it 
creates a lot of small sstables that eventually contain a lot of outdated data. 
Upon restore you'll have to wait until compaction catches up on compacting all 
the history (which could take a while and use a lot of power). Your backups 
could also grow indefinitely with this technique since there's no compaction, 
so no purge. You'll have to build the restore script/procedure.
  *   scheduled snapshots : you perform full snapshots by yourself and move 
them off node. You'll have to build the restore script/procedure.
  *   EBS snapshots : probably the easiest way to perform backups if you are 
using M4/R4 instances on AWS.

Cheers,

On Thu, May 11, 2017 at 11:01 PM Manikandan Srinivasan 
<msriniva...@datastax.com<mailto:msriniva...@datastax.com>> wrote:
Blake is correct. OpsCenter 6.0 and up doesn't work with OSS C*. @Nitan: We 
have made some substantial changes to the Opscenter 6.1 backup service, 
specifically when it comes to S3 backups. Having said this, I am not going to 
be sale-sy here. If folks need some help or need more clarity to know more 
about these improvements, please send me an email directly: 
msriniva...@datastax.com<mailto:msriniva...@datastax.com>

Regards
Mani

On Thu, May 11, 2017 at 1:54 PM, Nitan Kainth 
<ni...@bamlabs.com<mailto:ni...@bamlabs.com>> wrote:
Also , Opscenter backup/restore does not work for large databases

Sent from my iPhone

On May 11, 2017, at 3:41 PM, Blake Eggleston 
<beggles...@apple.com<mailto:beggles...@apple.com>> wrote:
OpsCenter 6.0 and up don't work with Cassandra.


On May 11, 2017 at 12:31:08 PM, cass savy 
(casss...@gmail.com<mailto:casss...@gmail.com>) wrote:
AWS Backup/Restore process/tools for C*/DSE C*:

Has anyone used Opscenter 6.1 backup tool to backup/restore data for larger 
datasets online ?

If yes, did you run into issues using that tool to backup/restore data in PROD 
that caused any performance or any other impact to the cluster?

If no, what are other tools that people have used or recommended for backup and 
restore of Cassandra keyspaces?

Please advice.





--
Regards,

Manikandan Srinivasan

Director, Product Management| +1.408.887.3686<tel:%2B1.408.887.3686> | 
manikandan.sriniva...@datastax.com<mailto:manikandan.sriniva...@datastax.com>


[Image removed by sender. 
linkedin.png]<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.linkedin.com_in_srinivm_=DwMFaQ=MtgQEAMQGqekjTjiAhkudQ=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ=hrZJx9SNtdlofcJElpjcjw4rp4rlAq8nZKSSsefCvCc=QVzYL31K-iWGptuTJeKSX2hMW9lrGn5HP3X9p-A8wO4=>[Image
 removed by sender. 
facebook.png]<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.facebook.com_datastax=DwMFaQ=MtgQEAMQGqekjTjiAhkudQ=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ=hrZJx9SNtdlofcJElpjcjw4rp4rlAq8nZKSSsefCvCc=tCXQZRynu6vGzUuBNtKyhKS0qf1FZcZPAlwGw_5HVBM=>[Image
 removed by sender. 
twitter.png]<https://urldefense.proofpoint.com/v2/url?u=https-3A__twitter.com_mani-5Fsrini=DwMFaQ=MtgQEAMQGqekjTjiAhkudQ=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ=hrZJx9SNtdlofcJElpjcjw4rp4rlAq8nZKSSsefCvCc=fH6hn8l2gJJVVmpCOoKdXA80OgFPqt6pt3bjR9pzjxI=>[Image
 removed by sender. 
g+.png]<https://urldefense.proofpoint.com/v2/url?u=https-3A__plus.google.com_-2BDatastax_about=DwMFaQ=MtgQEAMQGqekjTjiAhkudQ=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ=hrZJx9SNtdlofcJElpjcjw4rp4rlAq8nZKSSsefCvCc=2ZG39fHN9Oix46hmXFKRQH5M0AVqA7h-9bqZ7VWvguE=>[Im

Re: AWS Cassandra backup/Restore tools

[Alexander Dejanovski]

Hi,

here are the main techniques that I know of to perform backups for
Cassandra :

   - Tablesnap (https://github.com/JeremyGrosser/tablesnap) : performs
   continuous backups on S3. Comes with tableslurp to restore backups (one
   table at a time only) and tablechop to delete outdated sstables from S3.
   - incremental backup : activate it in the cassandra.yaml file and it
   will create snapshots for all newly flushed SSTables. It's up to you to
   move the snapshots off-node and delete them. I don't really like that
   technique since it creates a lot of small sstables that eventually contain
   a lot of outdated data. Upon restore you'll have to wait until compaction
   catches up on compacting all the history (which could take a while and use
   a lot of power). Your backups could also grow indefinitely with this
   technique since there's no compaction, so no purge. You'll have to build
   the restore script/procedure.
   - scheduled snapshots : you perform full snapshots by yourself and move
   them off node. You'll have to build the restore script/procedure.
   - EBS snapshots : probably the easiest way to perform backups if you are
   using M4/R4 instances on AWS.


Cheers,

On Thu, May 11, 2017 at 11:01 PM Manikandan Srinivasan <
msriniva...@datastax.com> wrote:

> Blake is correct. OpsCenter 6.0 and up doesn't work with OSS C*. @Nitan:
> We have made some substantial changes to the Opscenter 6.1 backup service,
> specifically when it comes to S3 backups. Having said this, I am not going
> to be sale-sy here. If folks need some help or need more clarity to know
> more about these improvements, please send me an email directly:
> msriniva...@datastax.com
>
> Regards
> Mani
>
> On Thu, May 11, 2017 at 1:54 PM, Nitan Kainth  wrote:
>
>> Also , Opscenter backup/restore does not work for large databases
>>
>> Sent from my iPhone
>>
>> On May 11, 2017, at 3:41 PM, Blake Eggleston 
>> wrote:
>>
>> OpsCenter 6.0 and up don't work with Cassandra.
>>
>> On May 11, 2017 at 12:31:08 PM, cass savy (casss...@gmail.com) wrote:
>>
>> AWS Backup/Restore process/tools for C*/DSE C*:
>>
>> Has anyone used Opscenter 6.1 backup tool to backup/restore data for
>> larger datasets online ?
>>
>> If yes, did you run into issues using that tool to backup/restore data in
>> PROD that caused any performance or any other impact to the cluster?
>>
>> If no, what are other tools that people have used or recommended for
>> backup and restore of Cassandra keyspaces?
>>
>> Please advice.
>>
>>
>>
>
>
> --
> Regards,
>
> Manikandan Srinivasan
>
> Director, Product Management| +1.408.887.3686 |
> manikandan.sriniva...@datastax.com
>
> [image: linkedin.png]  [image:
> facebook.png]  [image: twitter.png]
>  [image: g+.png]
> 
> 
>
> --
-
Alexander Dejanovski
France
@alexanderdeja

Consultant
Apache Cassandra Consulting
http://www.thelastpickle.com

Re: AWS Cassandra backup/Restore tools

[Manikandan Srinivasan]

Blake is correct. OpsCenter 6.0 and up doesn't work with OSS C*. @Nitan: We
have made some substantial changes to the Opscenter 6.1 backup service,
specifically when it comes to S3 backups. Having said this, I am not going
to be sale-sy here. If folks need some help or need more clarity to know
more about these improvements, please send me an email directly:
msriniva...@datastax.com

Regards
Mani

On Thu, May 11, 2017 at 1:54 PM, Nitan Kainth  wrote:

> Also , Opscenter backup/restore does not work for large databases
>
> Sent from my iPhone
>
> On May 11, 2017, at 3:41 PM, Blake Eggleston  wrote:
>
> OpsCenter 6.0 and up don't work with Cassandra.
>
> On May 11, 2017 at 12:31:08 PM, cass savy (casss...@gmail.com) wrote:
>
> AWS Backup/Restore process/tools for C*/DSE C*:
>
> Has anyone used Opscenter 6.1 backup tool to backup/restore data for
> larger datasets online ?
>
> If yes, did you run into issues using that tool to backup/restore data in
> PROD that caused any performance or any other impact to the cluster?
>
> If no, what are other tools that people have used or recommended for
> backup and restore of Cassandra keyspaces?
>
> Please advice.
>
>
>


-- 
Regards,

Manikandan Srinivasan

Director, Product Management| +1.408.887.3686 |
manikandan.sriniva...@datastax.com

[image: linkedin.png]  [image:
facebook.png]  [image: twitter.png]
 [image: g+.png]

Re: AWS Cassandra backup/Restore tools

[Nitan Kainth]

Also , Opscenter backup/restore does not work for large databases 

Sent from my iPhone

> On May 11, 2017, at 3:41 PM, Blake Eggleston  wrote:
> 
> OpsCenter 6.0 and up don't work with Cassandra.
> 
>> On May 11, 2017 at 12:31:08 PM, cass savy (casss...@gmail.com) wrote:
>> 
>> AWS Backup/Restore process/tools for C*/DSE C*:
>> 
>> Has anyone used Opscenter 6.1 backup tool to backup/restore data for larger 
>> datasets online ?
>> 
>> If yes, did you run into issues using that tool to backup/restore data in 
>> PROD that caused any performance or any other impact to the cluster?
>> 
>> If no, what are other tools that people have used or recommended for backup 
>> and restore of Cassandra keyspaces?
>> 
>> Please advice.
>> 
>> 

Re: AWS Cassandra backup/Restore tools

[Blake Eggleston]

OpsCenter 6.0 and up don't work with Cassandra.

On May 11, 2017 at 12:31:08 PM, cass savy (casss...@gmail.com) wrote:

AWS Backup/Restore process/tools for C*/DSE C*:

Has anyone used Opscenter 6.1 backup tool to backup/restore data for larger 
datasets online ?

If yes, did you run into issues using that tool to backup/restore data in PROD 
that caused any performance or any other impact to the cluster?

If no, what are other tools that people have used or recommended for backup and 
restore of Cassandra keyspaces?

Please advice.

AWS Cassandra backup/Restore tools

[cass savy]

AWS Backup/Restore process/tools for C*/DSE C*:

Has anyone used Opscenter 6.1 backup tool to backup/restore data for larger
datasets online ?

If yes, did you run into issues using that tool to backup/restore data in
PROD that caused any performance or any other impact to the cluster?

If no, what are other tools that people have used or recommended for backup
and restore of Cassandra keyspaces?

Please advice.

Re: Verify cassandra backup and restore in C * 2.1

[Riccardo Ferrari]

Hi Indranil,

I think it really depends on what makes a backup "correct" for you. Do you
have some test you can run on that data?
When I want to test my data I usually restore it in a new cluster (ie. on
AWS) and use Spark to perform some cross-tests. This is a bit cumbersome
nevertheless does the trick.

Best,

On Tue, Aug 9, 2016 at 8:50 AM, INDRANIL BASU <indranil...@yahoo.com> wrote:

> Thanks Rajath. We have atleast 20 millions of records, so count(*) with
> limit and token is getting cumbersome.
>  Yes using nodetool snapshot. I know there is a nodetool verify, but i m
> using C * 2.1
> Copy commands sometimes fails as well.
>
> So any better easy mechanism?
>
> *Thanks and regards,*
> *-- Indranil Basu*
>
>
> --
> *From:* Rajath Subramanyam <rajat...@gmail.com>
> *To:* user@cassandra.apache.org; INDRANIL BASU <indranil...@yahoo.com>
> *Sent:* Monday, 8 August 2016 11:41 PM
> *Subject:* Re: Verify cassandra backup and restore in C * 2.1
>
> Hi Indranil,
>
> One approach is to do a row count on the original source table and the
> table that is restored from backup. How are you backing up data ? I am
> assuming you are issuing snapshot commands (either incremental or
> otherwise).
>
> I hope this helps.
>
> - Rajath
>
> 
> Rajath Subramanyam
>
>
> On Mon, Aug 8, 2016 at 4:29 AM, INDRANIL BASU <indranil...@yahoo.com>
> wrote:
>
> Hello,
>
>How can we verify that cassandra data is backed up and restored
> correctly?
>
> *-- IB*
>
>
>
>
>
>
>

Re: Verify cassandra backup and restore in C * 2.1

[INDRANIL BASU]

Thanks Rajath. We have atleast 20 millions of records, so count(*) with limit 
and token is getting cumbersome.
 Yes using nodetool snapshot. I know there is a nodetool verify, but i m using 
C * 2.1
Copy commands sometimes fails as well.
So any better easy mechanism?
  Thanks and regards,-- Indranil Basu

  From: Rajath Subramanyam <rajat...@gmail.com>
 To: user@cassandra.apache.org; INDRANIL BASU <indranil...@yahoo.com> 
 Sent: Monday, 8 August 2016 11:41 PM
 Subject: Re: Verify cassandra backup and restore in C * 2.1
   
Hi Indranil, 
One approach is to do a row count on the original source table and the table 
that is restored from backup. How are you backing up data ? I am assuming you 
are issuing snapshot commands (either incremental or otherwise).
I hope this helps. 
- Rajath

Rajath Subramanyam


On Mon, Aug 8, 2016 at 4:29 AM, INDRANIL BASU <indranil...@yahoo.com> wrote:

Hello,
   How can we verify that cassandra data is backed up and restored correctly?

-- IB


   



  

Re: Verify cassandra backup and restore in C * 2.1

[Rajath Subramanyam]

Hi Indranil,

One approach is to do a row count on the original source table and the
table that is restored from backup. How are you backing up data ? I am
assuming you are issuing snapshot commands (either incremental or
otherwise).

I hope this helps.

- Rajath


Rajath Subramanyam


On Mon, Aug 8, 2016 at 4:29 AM, INDRANIL BASU  wrote:

> Hello,
>
>How can we verify that cassandra data is backed up and restored
> correctly?
>
> *-- IB*
>
>
>
>

Verify cassandra backup and restore in C * 2.1

[INDRANIL BASU]

Hello,
   How can we verify that cassandra data is backed up and restored correctly?

-- IB


  

Re: Cassandra backup via snapshots in production

[Robert Coli]

On Thu, Nov 27, 2014 at 2:34 AM, Jens Rantil jens.ran...@tink.se wrote:

 Late answer; You can find my backup script here:
 https://gist.github.com/JensRantil/a8150e998250edfcd1a3


Why not use the much more robustly designed and maintained community based
project, tablesnap?

https://github.com/JeremyGrosser/tablesnap

=Rob

Re: Cassandra backup via snapshots in production

[Jens Rantil]

On Mon, Dec 1, 2014 at 8:39 PM, Robert Coli rc...@eventbrite.com wrote:

 Why not use the much more robustly designed and maintained community based
 project, tablesnap?


For two reasons:

   - Because I am tired of the deployment model of Python apps which
   require me to set up virtual environments.
   - Because it did, AFAIK, not support (asymmetric) encryption before
   uploading.

-- 
Jens Rantil
Backend engineer
Tink AB

Email: jens.ran...@tink.se
Phone: +46 708 84 18 32
Web: www.tink.se

Facebook https://www.facebook.com/#!/tink.se Linkedin
http://www.linkedin.com/company/2735919?trk=vsrp_companies_res_phototrkInfo=VSRPsearchId%3A1057023381369207406670%2CVSRPtargetId%3A2735919%2CVSRPcmpt%3Aprimary
 Twitter https://twitter.com/tink

RE: Cassandra backup via snapshots in production

[Ngoc Minh VO]

Thanks a lot for your answers!

What we plan to do is:

-  auto_snapshot = true

-  if the human errors happened on D-5:

o   we will bring the cluster offline

o   purge all data

o   import snapshots prior D-5 (and delete snapshots after D-5)

o   upload all missing data between D-5 and D

o   bring the cluster online

Do you think it would work?

From: Jens Rantil [mailto:jens.ran...@tink.se]
Sent: mardi 25 novembre 2014 10:03
To: user@cassandra.apache.org
Subject: Re: Cassandra backup via snapshots in production

 Truncate does trigger snapshot creation though

Doesn’t it? With “auto_snapshot: true” it should.

——— Jens Rantil Backend engineer Tink AB Email: 
jens.ran...@tink.semailto:jens.ran...@tink.se Phone: +46 708 84 18 32 Web: 
www.tink.sehttp://www.tink.se Facebook Linkedin Twitter


On Tue, Nov 25, 2014 at 9:21 AM, DuyHai Doan 
doanduy...@gmail.commailto:doanduy...@gmail.com wrote:

True

Delete in CQL just create tombstone so from the storage engine pov it's just 
adding some physical columns

Truncate does trigger snapshot creation though
Le 21 nov. 2014 19:29, Robert Coli 
rc...@eventbrite.commailto:rc...@eventbrite.com a écrit :
On Fri, Nov 21, 2014 at 8:40 AM, Jens Rantil 
jens.ran...@tink.semailto:jens.ran...@tink.se wrote:
 The main purpose is to protect us from human errors (eg. unexpected 
 manipulations: delete, drop tables, …).

If that is the main purpose, having auto_snapshot: true” in cassandra.yaml 
will be enough to protect you.

OP includes delete in their list of unexpected manipulations, and 
auto_snapshot: true will not protect you in any way from DELETE.

=Rob
http://twitter.com/rcolidba



This message and any attachments (the message) is
intended solely for the intended addressees and is confidential. 
If you receive this message in error,or are not the intended recipient(s), 
please delete it and any copies from your systems and immediately notify
the sender. Any unauthorized view, use that does not comply with its purpose, 
dissemination or disclosure, either whole or partial, is prohibited. Since the 
internet 
cannot guarantee the integrity of this message which may not be reliable, BNP 
PARIBAS 
(and its subsidiaries) shall not be liable for the message if modified, changed 
or falsified. 
Do not print this message unless it is necessary,consider the environment.

--

Ce message et toutes les pieces jointes (ci-apres le message) 
sont etablis a l'intention exclusive de ses destinataires et sont confidentiels.
Si vous recevez ce message par erreur ou s'il ne vous est pas destine,
merci de le detruire ainsi que toute copie de votre systeme et d'en avertir
immediatement l'expediteur. Toute lecture non autorisee, toute utilisation de 
ce message qui n'est pas conforme a sa destination, toute diffusion ou toute 
publication, totale ou partielle, est interdite. L'Internet ne permettant pas 
d'assurer
l'integrite de ce message electronique susceptible d'alteration, BNP Paribas 
(et ses filiales) decline(nt) toute responsabilite au titre de ce message dans 
l'hypothese
ou il aurait ete modifie, deforme ou falsifie. 
N'imprimez ce message que si necessaire, pensez a l'environnement.

Re: Cassandra backup via snapshots in production

[Jens Rantil]

Late answer; You can find my backup script here: 
https://gist.github.com/JensRantil/a8150e998250edfcd1a3


Basically you need to set S3_BUCKET, PGP_KEY_RECIPIENT, configure s3cmd (using 
s3cmd --configure) and then issue `./backup-keyspace.sh your-keyspace` to 
backup it to S3. We run the script is run periodically on every node.




Regarding “s3cmd --configure”, I executed it once and then copied “~/.s3cfg” to 
all nodes.




Like I said, there’s lots of love that can be put into a backup system. Note 
that the script has the following limitations:

 * It does not checksum the files. However s3cmd website states that it by 
default compares MD5 and file size on upload.

 * It does not do purging of files on S3 (which you could configure using 
“Object Lifecycles”).

 * It does not warn you that a backup fails. Check your logs periodically.

 * It does not do any advanced logging. Make sure to pipe the output to a file 
or the `syslog` utility.

 * It does not do continuous/point-in-time backup.




That said, it does its job for us for now.




Feel free to propose improvements!




Cheers,

Jens


———
Jens Rantil
Backend engineer
Tink AB

Email: jens.ran...@tink.se
Phone: +46 708 84 18 32
Web: www.tink.se

Facebook Linkedin Twitter

On Fri, Nov 21, 2014 at 7:36 PM, William Arbaugh w...@cs.umd.edu wrote:

 Jens,
 I'd be interested in seeing your script. We've been thinking of doing exactly 
 that but uploading to Glacier instead.
 Thanks, Bill
 On Nov 21, 2014, at 11:40 AM, Jens Rantil jens.ran...@tink.se wrote:
 
  The main purpose is to protect us from human errors (eg. unexpected 
  manipulations: delete, drop tables, …).
 
 If that is the main purpose, having auto_snapshot: true” in cassandra.yaml 
 will be enough to protect you.
 
 Regarding backup, I have a small script that creates a named snapshot and 
 for each sstable; encrypts, uploads to S3 and deletes the snapshotted 
 sstable. It took me an hour to write and roll out to all our nodes. The 
 whole process is currently logged, but eventually I will also send an e-mail 
 if backup fails.
 
 ——— Jens Rantil Backend engineer Tink AB Email: jens.ran...@tink.se Phone: 
 +46 708 84 18 32 Web: www.tink.se Facebook Linkedin Twitter
 
 
 On Tue, Nov 18, 2014 at 3:52 PM, Ngoc Minh VO ngocminh...@bnpparibas.com 
 wrote:
 
 Hello all,
 
 
 
 
  
 
 We are looking for a solution to backup data in our C* cluster (v2.0.x, 16 
 nodes, 4 x 500GB SSD, RF = 6 over 2 datacenters).
 
 
 
 The main purpose is to protect us from human errors (eg. unexpected 
 manipulations: delete, drop tables, …).
 
 
 
 
  
 
 We are thinking of:
 
 
 
 -  Backup: add a 2TB HDD on each node for C* daily/weekly snapshots.
 
 
 
 -  Restore: load the most recent snapshots or latest “non-corrupted” 
 ones and replay missing data imports from other data source.
 
 
 
 
  
 
 We would like to know if somebody are using Cassandra’s backup feature in 
 production and could share your experience with us.
 
 
 
 
  
 
 Your help would be greatly appreciated.
 
 
 
 Best regards,
 
 
 
 Minh
 
 
 
 
 This message and any attachments (the message) is
 intended solely for the intended addressees and is confidential. 
 If you receive this message in error,or are not the intended recipient(s), 
 please delete it and any copies from your systems and immediately notify
 the sender. Any unauthorized view, use that does not comply with its 
 purpose, 
 dissemination or disclosure, either whole or partial, is prohibited. Since 
 the internet 
 cannot guarantee the integrity of this message which may not be reliable, 
 BNP PARIBAS 
 (and its subsidiaries) shall not be liable for the message if modified, 
 changed or falsified. 
 Do not print this message unless it is necessary,consider the environment.
 
 --
 
 Ce message et toutes les pieces jointes (ci-apres le message) 
 sont etablis a l'intention exclusive de ses destinataires et sont 
 confidentiels.
 Si vous recevez ce message par erreur ou s'il ne vous est pas destine,
 merci de le detruire ainsi que toute copie de votre systeme et d'en avertir
 immediatement l'expediteur. Toute lecture non autorisee, toute utilisation 
 de 
 ce message qui n'est pas conforme a sa destination, toute diffusion ou toute 
 publication, totale ou partielle, est interdite. L'Internet ne permettant 
 pas d'assurer
 l'integrite de ce message electronique susceptible d'alteration, BNP Paribas 
 (et ses filiales) decline(nt) toute responsabilite au titre de ce message 
 dans l'hypothese
 ou il aurait ete modifie, deforme ou falsifie. 
 N'imprimez ce message que si necessaire, pensez a l'environnement.
 
 

Re: Cassandra backup via snapshots in production

[DuyHai Doan]

True

Delete in CQL just create tombstone so from the storage engine pov it's
just adding some physical columns

Truncate does trigger snapshot creation though
Le 21 nov. 2014 19:29, Robert Coli rc...@eventbrite.com a écrit :

 On Fri, Nov 21, 2014 at 8:40 AM, Jens Rantil jens.ran...@tink.se wrote:

  The main purpose is to protect us from human errors (eg. unexpected
 manipulations: delete, drop tables, …).

 If that is the main purpose, having auto_snapshot: true” in
 cassandra.yaml will be enough to protect you.


 OP includes delete in their list of unexpected manipulations, and
 auto_snapshot: true will not protect you in any way from DELETE.

 =Rob
 http://twitter.com/rcolidba

Re: Cassandra backup via snapshots in production

[Jens Rantil]

 Truncate does trigger snapshot creation though




Doesn’t it? With “auto_snapshot: true” it should.




———
Jens Rantil
Backend engineer
Tink AB

Email: jens.ran...@tink.se
Phone: +46 708 84 18 32
Web: www.tink.se

Facebook Linkedin Twitter

On Tue, Nov 25, 2014 at 9:21 AM, DuyHai Doan doanduy...@gmail.com wrote:

 True
 Delete in CQL just create tombstone so from the storage engine pov it's
 just adding some physical columns
 Truncate does trigger snapshot creation though
 Le 21 nov. 2014 19:29, Robert Coli rc...@eventbrite.com a écrit :
 On Fri, Nov 21, 2014 at 8:40 AM, Jens Rantil jens.ran...@tink.se wrote:

  The main purpose is to protect us from human errors (eg. unexpected
 manipulations: delete, drop tables, …).

 If that is the main purpose, having auto_snapshot: true” in
 cassandra.yaml will be enough to protect you.


 OP includes delete in their list of unexpected manipulations, and
 auto_snapshot: true will not protect you in any way from DELETE.

 =Rob
 http://twitter.com/rcolidba

Re: Cassandra backup via snapshots in production

[Jens Rantil]

 The main purpose is to protect us from human errors (eg. unexpected 
 manipulations: delete, drop tables, …).




If that is the main purpose, having auto_snapshot: true” in cassandra.yaml 
will be enough to protect you.




Regarding backup, I have a small script that creates a named snapshot and for 
each sstable; encrypts, uploads to S3 and deletes the snapshotted sstable. It 
took me an hour to write and roll out to all our nodes. The whole process is 
currently logged, but eventually I will also send an e-mail if backup fails.


———
Jens Rantil
Backend engineer
Tink AB

Email: jens.ran...@tink.se
Phone: +46 708 84 18 32
Web: www.tink.se

Facebook Linkedin Twitter

On Tue, Nov 18, 2014 at 3:52 PM, Ngoc Minh VO ngocminh...@bnpparibas.com
wrote:

 Hello all,
 We are looking for a solution to backup data in our C* cluster (v2.0.x, 16 
 nodes, 4 x 500GB SSD, RF = 6 over 2 datacenters).
 The main purpose is to protect us from human errors (eg. unexpected 
 manipulations: delete, drop tables, …).
 We are thinking of:
 -  Backup: add a 2TB HDD on each node for C* daily/weekly snapshots.
 -  Restore: load the most recent snapshots or latest “non-corrupted” 
 ones and replay missing data imports from other data source.
 We would like to know if somebody are using Cassandra’s backup feature in 
 production and could share your experience with us.
 Your help would be greatly appreciated.
 Best regards,
 Minh
 This message and any attachments (the message) is
 intended solely for the intended addressees and is confidential. 
 If you receive this message in error,or are not the intended recipient(s), 
 please delete it and any copies from your systems and immediately notify
 the sender. Any unauthorized view, use that does not comply with its purpose, 
 dissemination or disclosure, either whole or partial, is prohibited. Since 
 the internet 
 cannot guarantee the integrity of this message which may not be reliable, BNP 
 PARIBAS 
 (and its subsidiaries) shall not be liable for the message if modified, 
 changed or falsified. 
 Do not print this message unless it is necessary,consider the environment.
 --
 Ce message et toutes les pieces jointes (ci-apres le message) 
 sont etablis a l'intention exclusive de ses destinataires et sont 
 confidentiels.
 Si vous recevez ce message par erreur ou s'il ne vous est pas destine,
 merci de le detruire ainsi que toute copie de votre systeme et d'en avertir
 immediatement l'expediteur. Toute lecture non autorisee, toute utilisation de 
 ce message qui n'est pas conforme a sa destination, toute diffusion ou toute 
 publication, totale ou partielle, est interdite. L'Internet ne permettant pas 
 d'assurer
 l'integrite de ce message electronique susceptible d'alteration, BNP Paribas 
 (et ses filiales) decline(nt) toute responsabilite au titre de ce message 
 dans l'hypothese
 ou il aurait ete modifie, deforme ou falsifie. 
 N'imprimez ce message que si necessaire, pensez a l'environnement.

Re: Cassandra backup via snapshots in production

[Robert Coli]

On Fri, Nov 21, 2014 at 8:40 AM, Jens Rantil jens.ran...@tink.se wrote:

  The main purpose is to protect us from human errors (eg. unexpected
 manipulations: delete, drop tables, …).

 If that is the main purpose, having auto_snapshot: true” in
 cassandra.yaml will be enough to protect you.


OP includes delete in their list of unexpected manipulations, and
auto_snapshot: true will not protect you in any way from DELETE.

=Rob
http://twitter.com/rcolidba

Re: Cassandra backup via snapshots in production

[Robert Coli]

On Tue, Nov 18, 2014 at 6:50 AM, Ngoc Minh VO ngocminh...@bnpparibas.com
wrote:

   We are looking for a solution to backup data in our C* cluster (v2.0.x,
 16 nodes, 4 x 500GB SSD, RF = 6 over 2 datacenters).

 The main purpose is to protect us from human errors (eg. unexpected
 manipulations: delete, drop tables, …).


https://github.com/JeremyGrosser/tablesnap

=Rob

Cassandra backup via snapshots in production

[Ngoc Minh VO]

Hello all,

We are looking for a solution to backup data in our C* cluster (v2.0.x, 16 
nodes, 4 x 500GB SSD, RF = 6 over 2 datacenters).
The main purpose is to protect us from human errors (eg. unexpected 
manipulations: delete, drop tables, …).

We are thinking of:

-  Backup: add a 2TB HDD on each node for C* daily/weekly snapshots.

-  Restore: load the most recent snapshots or latest “non-corrupted” 
ones and replay missing data imports from other data source.

We would like to know if somebody are using Cassandra’s backup feature in 
production and could share your experience with us.

Your help would be greatly appreciated.
Best regards,
Minh


This message and any attachments (the message) is
intended solely for the intended addressees and is confidential. 
If you receive this message in error,or are not the intended recipient(s), 
please delete it and any copies from your systems and immediately notify
the sender. Any unauthorized view, use that does not comply with its purpose, 
dissemination or disclosure, either whole or partial, is prohibited. Since the 
internet 
cannot guarantee the integrity of this message which may not be reliable, BNP 
PARIBAS 
(and its subsidiaries) shall not be liable for the message if modified, changed 
or falsified. 
Do not print this message unless it is necessary,consider the environment.

--

Ce message et toutes les pieces jointes (ci-apres le message) 
sont etablis a l'intention exclusive de ses destinataires et sont confidentiels.
Si vous recevez ce message par erreur ou s'il ne vous est pas destine,
merci de le detruire ainsi que toute copie de votre systeme et d'en avertir
immediatement l'expediteur. Toute lecture non autorisee, toute utilisation de 
ce message qui n'est pas conforme a sa destination, toute diffusion ou toute 
publication, totale ou partielle, est interdite. L'Internet ne permettant pas 
d'assurer
l'integrite de ce message electronique susceptible d'alteration, BNP Paribas 
(et ses filiales) decline(nt) toute responsabilite au titre de ce message dans 
l'hypothese
ou il aurait ete modifie, deforme ou falsifie. 
N'imprimez ce message que si necessaire, pensez a l'environnement.

cassandra backup

[Marcelo Elias Del Valle]

Hello everyone,

I am trying to create backups of my data on AWS. My goal is to store
the backups on S3 or glacier, as it's cheap to store this kind of data. So,
if I have a cluster with N nodes, I would like to copy data from all N
nodes to S3 and be able to restore later. I know Priam does that (we were
using it), but I am using the latest cassandra version and we plan to use
DSE some time, I am not sure Priam fits this case.
I took a look at the docs:
http://www.datastax.com/documentation/cassandra/2.0/webhelp/index.html#cassandra/operations/../../cassandra/operations/ops_backup_takes_snapshot_t.html

And I am trying to understand if it's really needed to take a snapshot
to create my backup. Suppose I do a flush and copy the sstables from each
node, 1 by one, to s3. Not all at the same time, but one by one.
When I try to restore my backup, data from node 1 will be older than
data from node 2. Will this cause problems? AFAIK, if I am using a
replication factor of 2, for instance, and Cassandra sees data from node X
only, it will automatically copy it to other nodes, right? Is there any
chance of cassandra nodes become corrupt somehow if I do my backups this
way?

Best regards,
Marcelo Valle.

Re: cassandra backup

[Michael Theroux]

Hi Marcelo,

Cassandra provides and eventually consistent model for backups.  You can do 
staggered backups of data, with the idea that if you restore a node, and then 
do a repair, your data will be once again consistent.  Cassandra will not 
automatically copy the data to other nodes (other than via hinted handoff).  
You should manually run repair after restoring a node.
  
You should take snapshots when doing a backup, as it keeps the data you are 
backing up relevant to a single point in time, otherwise compaction could 
add/delete files one you mid-backup, or worse, I imagine attempt to access a 
SSTable mid-write.  Snapshots work by using links, and don't take additional 
storage to perform.  In our process we create the snapshot, perform the backup, 
and then clear the snapshot.

One thing to keep in mind in your S3 cost analysis is that, even though storage 
is cheap, reads/writes to S3 are not (especially writes).  If you are using 
LeveledCompaction, or otherwise have a ton of SSTables, some people have 
encountered increased costs moving the data to S3.

Ourselves, we maintain backup EBS volumes that we regularly snaphot/rsync data 
too.  Thus far this has worked very well for us.

-Mike



On Friday, December 6, 2013 8:14 AM, Marcelo Elias Del Valle 
marc...@s1mbi0se.com.br wrote:
 
Hello everyone,

    I am trying to create backups of my data on AWS. My goal is to store the 
backups on S3 or glacier, as it's cheap to store this kind of data. So, if I 
have a cluster with N nodes, I would like to copy data from all N nodes to S3 
and be able to restore later. I know Priam does that (we were using it), but I 
am using the latest cassandra version and we plan to use DSE some time, I am 
not sure Priam fits this case.
    I took a look at the docs: 
http://www.datastax.com/documentation/cassandra/2.0/webhelp/index.html#cassandra/operations/../../cassandra/operations/ops_backup_takes_snapshot_t.html
 
    And I am trying to understand if it's really needed to take a snapshot to 
create my backup. Suppose I do a flush and copy the sstables from each node, 1 
by one, to s3. Not all at the same time, but one by one. 
    When I try to restore my backup, data from node 1 will be older than data 
from node 2. Will this cause problems? AFAIK, if I am using a replication 
factor of 2, for instance, and Cassandra sees data from node X only, it will 
automatically copy it to other nodes, right? Is there any chance of cassandra 
nodes become corrupt somehow if I do my backups this way?

Best regards,
Marcelo Valle.

Re: cassandra backup

[Rahul Menon]

You should look at this - https://github.com/amorton/cassback i dont
believe its setup to use 1.2.10 and above but i believe is just small
tweeks to get it running.

Thanks
Rahul


On Fri, Dec 6, 2013 at 7:09 PM, Michael Theroux mthero...@yahoo.com wrote:

 Hi Marcelo,

 Cassandra provides and eventually consistent model for backups.  You can
 do staggered backups of data, with the idea that if you restore a node, and
 then do a repair, your data will be once again consistent.  Cassandra will
 not automatically copy the data to other nodes (other than via hinted
 handoff).  You should manually run repair after restoring a node.

 You should take snapshots when doing a backup, as it keeps the data you
 are backing up relevant to a single point in time, otherwise compaction
 could add/delete files one you mid-backup, or worse, I imagine attempt to
 access a SSTable mid-write.  Snapshots work by using links, and don't take
 additional storage to perform.  In our process we create the snapshot,
 perform the backup, and then clear the snapshot.

 One thing to keep in mind in your S3 cost analysis is that, even though
 storage is cheap, reads/writes to S3 are not (especially writes).  If you
 are using LeveledCompaction, or otherwise have a ton of SSTables, some
 people have encountered increased costs moving the data to S3.

 Ourselves, we maintain backup EBS volumes that we regularly snaphot/rsync
 data too.  Thus far this has worked very well for us.

 -Mike


   On Friday, December 6, 2013 8:14 AM, Marcelo Elias Del Valle 
 marc...@s1mbi0se.com.br wrote:
   Hello everyone,

 I am trying to create backups of my data on AWS. My goal is to store
 the backups on S3 or glacier, as it's cheap to store this kind of data. So,
 if I have a cluster with N nodes, I would like to copy data from all N
 nodes to S3 and be able to restore later. I know Priam does that (we were
 using it), but I am using the latest cassandra version and we plan to use
 DSE some time, I am not sure Priam fits this case.
 I took a look at the docs:
 http://www.datastax.com/documentation/cassandra/2.0/webhelp/index.html#cassandra/operations/../../cassandra/operations/ops_backup_takes_snapshot_t.html

 And I am trying to understand if it's really needed to take a snapshot
 to create my backup. Suppose I do a flush and copy the sstables from each
 node, 1 by one, to s3. Not all at the same time, but one by one.
 When I try to restore my backup, data from node 1 will be older than
 data from node 2. Will this cause problems? AFAIK, if I am using a
 replication factor of 2, for instance, and Cassandra sees data from node X
 only, it will automatically copy it to other nodes, right? Is there any
 chance of cassandra nodes become corrupt somehow if I do my backups this
 way?

 Best regards,
 Marcelo Valle.

Re: cassandra backup

[Jonathan Haddad]

I believe SSTables are written to a temporary file then moved.  If I
remember correctly, tools like tablesnap listen for the inotify event
IN_MOVED_TO.  This should handle the try to back up sstable while in
mid-write issue.


On Fri, Dec 6, 2013 at 5:39 AM, Michael Theroux mthero...@yahoo.com wrote:

 Hi Marcelo,

 Cassandra provides and eventually consistent model for backups.  You can
 do staggered backups of data, with the idea that if you restore a node, and
 then do a repair, your data will be once again consistent.  Cassandra will
 not automatically copy the data to other nodes (other than via hinted
 handoff).  You should manually run repair after restoring a node.

 You should take snapshots when doing a backup, as it keeps the data you
 are backing up relevant to a single point in time, otherwise compaction
 could add/delete files one you mid-backup, or worse, I imagine attempt to
 access a SSTable mid-write.  Snapshots work by using links, and don't take
 additional storage to perform.  In our process we create the snapshot,
 perform the backup, and then clear the snapshot.

 One thing to keep in mind in your S3 cost analysis is that, even though
 storage is cheap, reads/writes to S3 are not (especially writes).  If you
 are using LeveledCompaction, or otherwise have a ton of SSTables, some
 people have encountered increased costs moving the data to S3.

 Ourselves, we maintain backup EBS volumes that we regularly snaphot/rsync
 data too.  Thus far this has worked very well for us.

 -Mike


   On Friday, December 6, 2013 8:14 AM, Marcelo Elias Del Valle 
 marc...@s1mbi0se.com.br wrote:
  Hello everyone,

 I am trying to create backups of my data on AWS. My goal is to store
 the backups on S3 or glacier, as it's cheap to store this kind of data. So,
 if I have a cluster with N nodes, I would like to copy data from all N
 nodes to S3 and be able to restore later. I know Priam does that (we were
 using it), but I am using the latest cassandra version and we plan to use
 DSE some time, I am not sure Priam fits this case.
 I took a look at the docs:
 http://www.datastax.com/documentation/cassandra/2.0/webhelp/index.html#cassandra/operations/../../cassandra/operations/ops_backup_takes_snapshot_t.html

 And I am trying to understand if it's really needed to take a snapshot
 to create my backup. Suppose I do a flush and copy the sstables from each
 node, 1 by one, to s3. Not all at the same time, but one by one.
 When I try to restore my backup, data from node 1 will be older than
 data from node 2. Will this cause problems? AFAIK, if I am using a
 replication factor of 2, for instance, and Cassandra sees data from node X
 only, it will automatically copy it to other nodes, right? Is there any
 chance of cassandra nodes become corrupt somehow if I do my backups this
 way?

 Best regards,
 Marcelo Valle.





-- 
Jon Haddad
http://www.rustyrazorblade.com
skype: rustyrazorblade

Re: cassandra backup

[Robert Coli]

On Fri, Dec 6, 2013 at 5:13 AM, Marcelo Elias Del Valle 
marc...@s1mbi0se.com.br wrote:

 I am trying to create backups of my data on AWS. My goal is to store
 the backups on S3 or glacier, as it's cheap to store this kind of data. So,
 if I have a cluster with N nodes, I would like to copy data from all N
 nodes to S3 and be able to restore later.


https://github.com/synack/tablesnap

Automated backup, restore, purging, intended for use with Cassandra.

=Rob

Cassandra backup

[Kanwar Sangha]

Hi - We have a req to store around 90 days of data per user. Last 7 days of 
data is going to be accessed frequently. Is there a way we can have the recent 
data (7 days) in SSD and the rest of the data in the
HDD ? Do we take a snapshot every 7 days and use a separate 'archive' cluster 
to serve the old data and a 'active' cluster to serve recent data ?

Any links/thoughts would be helpful.

Thanks,
Kanwar

Re: Cassandra backup

[Michael Kjellman]

There is this:

http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-1-flexible-data-file-placement

But you'll need to design your data model around the fact that this is only as 
granular as 1 column family

Best,
michael

From: Kanwar Sangha kan...@mavenir.commailto:kan...@mavenir.com
Reply-To: user@cassandra.apache.orgmailto:user@cassandra.apache.org 
user@cassandra.apache.orgmailto:user@cassandra.apache.org
Date: Monday, February 18, 2013 6:06 PM
To: user@cassandra.apache.orgmailto:user@cassandra.apache.org 
user@cassandra.apache.orgmailto:user@cassandra.apache.org
Subject: Cassandra backup

Hi – We have a req to store around 90 days of data per user. Last 7 days of 
data is going to be accessed frequently. Is there a way we can have the recent 
data (7 days) in SSD and the rest of the data in the
HDD ? Do we take a snapshot every 7 days and use a separate ‘archive’ cluster 
to serve the old data and a ‘active’ cluster to serve recent data ?

Any links/thoughts would be helpful.

Thanks,
Kanwar

RE: Cassandra backup

[Kanwar Sangha]

Thanks. I will look into the details.

One issue I see is that if I have only one column family which needs only the 
last 7 days data to be on SSD and the rest to be on the HDD, how will that work.

From: Michael Kjellman [mailto:mkjell...@barracuda.com]
Sent: 18 February 2013 20:08
To: user@cassandra.apache.org
Subject: Re: Cassandra backup

There is this:

http://www.datastax.com/dev/blog/whats-new-in-cassandra-1-1-flexible-data-file-placement

But you'll need to design your data model around the fact that this is only as 
granular as 1 column family

Best,
michael

From: Kanwar Sangha kan...@mavenir.commailto:kan...@mavenir.com
Reply-To: user@cassandra.apache.orgmailto:user@cassandra.apache.org 
user@cassandra.apache.orgmailto:user@cassandra.apache.org
Date: Monday, February 18, 2013 6:06 PM
To: user@cassandra.apache.orgmailto:user@cassandra.apache.org 
user@cassandra.apache.orgmailto:user@cassandra.apache.org
Subject: Cassandra backup

Hi - We have a req to store around 90 days of data per user. Last 7 days of 
data is going to be accessed frequently. Is there a way we can have the recent 
data (7 days) in SSD and the rest of the data in the
HDD ? Do we take a snapshot every 7 days and use a separate 'archive' cluster 
to serve the old data and a 'active' cluster to serve recent data ?

Any links/thoughts would be helpful.

Thanks,
Kanwar

Re: Cassandra backup question regarding commitlogs

[Vijay]

The incremental backups are generated when the flush is complete (Only
during the flush), If the node crash before the flush completes then the
commit logs in the local node's backup for the data in memory.
It wouldn't help to copy the Commit log across because they are not
immutable (They are recycled).

There is commit log backup in 1.1.1 (Yet to be released)
https://issues.apache.org/jira/browse/CASSANDRA-3690

Thanks,
/VJ



On Sun, Apr 29, 2012 at 3:29 PM, Roshan codeva...@gmail.com wrote:

 Hi

 Currently I am taking daily snapshot on my keyspace in production and
 already enable the incremental backups as well.

 According to the documentation, the incremental backup option will create
 an
 hard-link to the backup folder when new sstable is flushed. Snapshot will
 copy all the data/index/etc. files to a new folder.

 Question:
 What will happen (with enabling the incremental backup) when crash (due to
 any reason) the Cassandra before flushing the data as a SSTable (inserted
 data still in commitlog). In this case how can I backup/restore data?

 Do I need to backup the commitlogs as well and and replay during the server
 start to restore the data in commitlog files?

 Thanks.

 --
 View this message in context:
 http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Cassandra-backup-question-regarding-commitlogs-tp7511918.html
 Sent from the cassandra-u...@incubator.apache.org mailing list archive at
 Nabble.com.

Re: Cassandra backup queston regarding commitlogs

[aaron morton]

If you delete the commit logs you are rolling back to exactly what was in the 
snapshot. When you take a snapshot it flushes the memtables first, so there is 
nothing in the commit log that is not in the snapshot. Rolling back to a 
snapshot is rollback to that point in time. 

If you want to restore to any point in time you need snapshots + incremental 
snapshot + commit log (for things that have not made it to sstables). Otherwise 
there is a potential loss of data that has not been flushed to disk. This is 
different to what the DS docs are talking about. I'm not sure why they are 
saying delete the commit log, try asking on their forum 
http://www.datastax.com/support-forums/

Cheers

-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 2/05/2012, at 12:02 PM, Roshan wrote:

 Any help regarding this is appreciated.
 
 --
 View this message in context: 
 http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Cassandra-backup-queston-regarding-commitlogs-tp7508823p7518544.html
 Sent from the cassandra-u...@incubator.apache.org mailing list archive at 
 Nabble.com.

Re: Cassandra backup queston regarding commitlogs

[Roshan]

Many thanks Aaron. I will post a support issue for them. But will keep the
snapshot + incremental backups + commitlogs to recover any failure
situation.

--
View this message in context: 
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Cassandra-backup-queston-regarding-commitlogs-tp7508823p7518866.html
Sent from the cassandra-u...@incubator.apache.org mailing list archive at 
Nabble.com.

Re: Cassandra backup queston regarding commitlogs

[Roshan]

Hi Aaron

Thanks for the comments. Yes for the durability will keep them in a safe
place. But such crash situation, how can I restore the data (because those
are not in a SSTable and only in commit log). 

Do I need to replay only that commit log when server starts after crash?
Will it override the same keys with values?

Appreciate your reply on this.

Kind Regards

/Roshan

--
View this message in context: 
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/deleted-tp7508823p7512499.html
Sent from the cassandra-u...@incubator.apache.org mailing list archive at 
Nabble.com.

Re: Cassandra backup queston regarding commitlogs

[aaron morton]

When the server starts it reads the SSTables then applies the Commit Logs. 

There is nothing you need to do other than leave the commit logs where they 
are. 

Cheers

-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 30/04/2012, at 6:02 PM, Roshan wrote:

 Hi Aaron
 
 Thanks for the comments. Yes for the durability will keep them in a safe
 place. But such crash situation, how can I restore the data (because those
 are not in a SSTable and only in commit log). 
 
 Do I need to replay only that commit log when server starts after crash?
 Will it override the same keys with values?
 
 Appreciate your reply on this.
 
 Kind Regards
 
 /Roshan
 
 --
 View this message in context: 
 http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/deleted-tp7508823p7512499.html
 Sent from the cassandra-u...@incubator.apache.org mailing list archive at 
 Nabble.com.

Re: Cassandra backup queston regarding commitlogs

[Roshan]

Many Thanks Aaron. 

According to the datastax restore documentation, they ask to remove the
commitlogs before restoring (Clear all files the
/var/lib/cassandra/commitlog (by default)). 

In that case better not to follow this step in a server rash situation.

Thanks

/Roshan 

--
View this message in context: 
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Cassandra-backup-queston-regarding-commitlogs-tp7508823p7515217.html
Sent from the cassandra-u...@incubator.apache.org mailing list archive at 
Nabble.com.

Re: Cassandra backup queston regarding commitlogs

[aaron morton]

Can you provide a link to that page ?

Cheers
-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 1/05/2012, at 10:12 AM, Roshan wrote:

 Many Thanks Aaron. 
 
 According to the datastax restore documentation, they ask to remove the
 commitlogs before restoring (Clear all files the
 /var/lib/cassandra/commitlog (by default)). 
 
 In that case better not to follow this step in a server rash situation.
 
 Thanks
 
 /Roshan 
 
 --
 View this message in context: 
 http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Cassandra-backup-queston-regarding-commitlogs-tp7508823p7515217.html
 Sent from the cassandra-u...@incubator.apache.org mailing list archive at 
 Nabble.com.

Re: Cassandra backup queston regarding commitlogs

[Tamar Fraenkel]

I want to add a couple of questions regrading incremental backups:
1. If I already have a Cassandra cluster running, would changing the  i
ncremental_backups parameter in the cassandra.yaml of each node, and then
restart it do the trick?
2. Assuming I am creating a daily snapshot, what is the gain from setting
incremental backup to true?

Thanks,
Tamar

*Tamar Fraenkel *
Senior Software Engineer, TOK Media

[image: Inline image 1]

ta...@tok-media.com
Tel:   +972 2 6409736
Mob:  +972 54 8356490
Fax:   +972 2 5612956





On Sat, Apr 28, 2012 at 4:04 PM, Roshan codeva...@gmail.com wrote:

 Hi

 Currently I am taking daily snapshot on my keyspace in production and
 already enable the incremental backups as well.

 According to the documentation, the incremental backup option will create
 an
 hard-link to the backup folder when new sstable is flushed. Snapshot will
 copy all the data/index/etc. files to a new folder.

 *Question:*
 What will happen (with enabling the incremental backup) when crash (due to
 any reason) the Cassandra before flushing the data as a SSTable (inserted
 data still in commitlog). In this case how can I backup/restore data?

 Do I need to backup the commitlogs as well and and replay during the server
 start to restore the data in commitlog files?

 Thanks.



 --
 View this message in context:
 http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Cassandra-backup-queston-regarding-commitlogs-tp7508823.html
 Sent from the cassandra-u...@incubator.apache.org mailing list archive at
 Nabble.com.

tokLogo.png

Re: Cassandra backup queston regarding commitlogs

[Roshan]

Tamar

Please don't jump to other users discussions. If you want to ask any issue,
create a new one, please.

Thanks. 


--
View this message in context: 
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Cassandra-backup-question-regarding-commitlogs-tp7508823p7511913.html
Sent from the cassandra-u...@incubator.apache.org mailing list archive at 
Nabble.com.

Cassandra backup question regarding commitlogs

[Roshan]

Hi

Currently I am taking daily snapshot on my keyspace in production and
already enable the incremental backups as well.

According to the documentation, the incremental backup option will create an
hard-link to the backup folder when new sstable is flushed. Snapshot will
copy all the data/index/etc. files to a new folder.

Question:
What will happen (with enabling the incremental backup) when crash (due to
any reason) the Cassandra before flushing the data as a SSTable (inserted
data still in commitlog). In this case how can I backup/restore data?

Do I need to backup the commitlogs as well and and replay during the server
start to restore the data in commitlog files?

Thanks. 

--
View this message in context: 
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Cassandra-backup-question-regarding-commitlogs-tp7511918.html
Sent from the cassandra-u...@incubator.apache.org mailing list archive at 
Nabble.com.

Re: Cassandra backup queston regarding commitlogs

[aaron morton]

Each mutation is applied to the commit log before being applied to the 
memtable. On server start the SSTables are read before replaying the commit 
logs. This is part of the crash only software design and happens for every 
start.

AFAIk there is no facility to snapshot commit log files as they are closed. The 
best advice would be to to keep them on a mirror set for durability. 

Cheers

-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 29/04/2012, at 1:04 AM, Roshan wrote:

 Hi
 
 Currently I am taking daily snapshot on my keyspace in production and
 already enable the incremental backups as well.
 
 According to the documentation, the incremental backup option will create an
 hard-link to the backup folder when new sstable is flushed. Snapshot will
 copy all the data/index/etc. files to a new folder.
 
 *Question:*
 What will happen (with enabling the incremental backup) when crash (due to
 any reason) the Cassandra before flushing the data as a SSTable (inserted
 data still in commitlog). In this case how can I backup/restore data?
 
 Do I need to backup the commitlogs as well and and replay during the server
 start to restore the data in commitlog files?
 
 Thanks.
 
 
 
 --
 View this message in context: 
 http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Cassandra-backup-queston-regarding-commitlogs-tp7508823.html
 Sent from the cassandra-u...@incubator.apache.org mailing list archive at 
 Nabble.com.

Re: Cassandra backup queston regarding commitlogs

[aaron morton]

 1. If I already have a Cassandra cluster running, would changing the  
 incremental_backups parameter in the cassandra.yaml of each node, and then 
 restart it do the trick?
Yes it is a per node setting. 

 2. Assuming I am creating a daily snapshot, what is the gain from setting 
 incremental backup to true?

Better point in time recovery on a node. 

Cheers

-
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 29/04/2012, at 6:41 PM, Tamar Fraenkel wrote:

 I want to add a couple of questions regrading incremental backups:
 1. If I already have a Cassandra cluster running, would changing the  
 incremental_backups parameter in the cassandra.yaml of each node, and then 
 restart it do the trick?
 2. Assuming I am creating a daily snapshot, what is the gain from setting 
 incremental backup to true?
 
 Thanks,
 Tamar
 
 Tamar Fraenkel 
 Senior Software Engineer, TOK Media 
 
 tokLogo.png
 
 ta...@tok-media.com
 Tel:   +972 2 6409736 
 Mob:  +972 54 8356490 
 Fax:   +972 2 5612956 
 
 
 
 
 
 On Sat, Apr 28, 2012 at 4:04 PM, Roshan codeva...@gmail.com wrote:
 Hi
 
 Currently I am taking daily snapshot on my keyspace in production and
 already enable the incremental backups as well.
 
 According to the documentation, the incremental backup option will create an
 hard-link to the backup folder when new sstable is flushed. Snapshot will
 copy all the data/index/etc. files to a new folder.
 
 *Question:*
 What will happen (with enabling the incremental backup) when crash (due to
 any reason) the Cassandra before flushing the data as a SSTable (inserted
 data still in commitlog). In this case how can I backup/restore data?
 
 Do I need to backup the commitlogs as well and and replay during the server
 start to restore the data in commitlog files?
 
 Thanks.
 
 
 
 --
 View this message in context: 
 http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Cassandra-backup-queston-regarding-commitlogs-tp7508823.html
 Sent from the cassandra-u...@incubator.apache.org mailing list archive at 
 Nabble.com.
 

Cassandra backup queston regarding commitlogs

[Roshan]

Hi

Currently I am taking daily snapshot on my keyspace in production and
already enable the incremental backups as well.

According to the documentation, the incremental backup option will create an
hard-link to the backup folder when new sstable is flushed. Snapshot will
copy all the data/index/etc. files to a new folder.

*Question:*
What will happen (with enabling the incremental backup) when crash (due to
any reason) the Cassandra before flushing the data as a SSTable (inserted
data still in commitlog). In this case how can I backup/restore data?

Do I need to backup the commitlogs as well and and replay during the server
start to restore the data in commitlog files?

Thanks.



--
View this message in context: 
http://cassandra-user-incubator-apache-org.3065146.n2.nabble.com/Cassandra-backup-queston-regarding-commitlogs-tp7508823.html
Sent from the cassandra-u...@incubator.apache.org mailing list archive at 
Nabble.com.