[jira] [Commented] (CASSANDRA-16451) Add ability to ttl snapshots
[ https://issues.apache.org/jira/browse/CASSANDRA-16451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17347185#comment-17347185 ] Paulo Motta commented on CASSANDRA-16451: - Linking tickets for similar feature in Hbase for reference. > Add ability to ttl snapshots > > > Key: CASSANDRA-16451 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16451 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Paulo Motta >Priority: Normal > Labels: gsoc2021, mentor > > It should be possible to add a TTL to snapshots, after which it automatically > cleans itself up. > This will be useful together with the {{auto_snapshot}} option, where you > want to keep an emergency snapshot in case of accidental drop or truncation > but automatically remove it after a specified period when it's no longer > useful. So in addition to allowing a user to specify a snapshot ttl on > {{nodetool snapshot}} we should have a {{auto_snapshot_ttl}} option that > allows a user to set a ttl for automatic snaphots on drop/truncate. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16451) Add ability to ttl snapshots
[ https://issues.apache.org/jira/browse/CASSANDRA-16451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17315761#comment-17315761 ] Abuli Palagashvili commented on CASSANDRA-16451: [~paulo] replied into google doc > Add ability to ttl snapshots > > > Key: CASSANDRA-16451 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16451 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Paulo Motta >Priority: Normal > Labels: gsoc2021, mentor > > It should be possible to add a TTL to snapshots, after which it automatically > cleans itself up. > This will be useful together with the {{auto_snapshot}} option, where you > want to keep an emergency snapshot in case of accidental drop or truncation > but automatically remove it after a specified period when it's no longer > useful. So in addition to allowing a user to specify a snapshot ttl on > {{nodetool snapshot}} we should have a {{auto_snapshot_ttl}} option that > allows a user to set a ttl for automatic snaphots on drop/truncate. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16451) Add ability to ttl snapshots
[ https://issues.apache.org/jira/browse/CASSANDRA-16451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17313586#comment-17313586 ] maxwellguo commented on CASSANDRA-16451: [~paulo] so I think besides just add a option when make snapshot ,we also need to doing some setting when truncate table (if the table will make a snapshot when the table is truncated). And this setting may be of table level. Different table should be of different snapshot ttl . what is your opinion ? [~paulo][~clohfink][~fibersel] > Add ability to ttl snapshots > > > Key: CASSANDRA-16451 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16451 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Paulo Motta >Priority: Normal > Labels: gsoc2021, mentor > > It should be possible to add a TTL to snapshots, after which it automatically > cleans itself up. > This will be useful together with the {{auto_snapshot}} option, where you > want to keep an emergency snapshot in case of accidental drop or truncation > but automatically remove it after a specified period when it's no longer > useful. So in addition to allowing a user to specify a snapshot ttl on > {{nodetool snapshot}} we should have a {{auto_snapshot_ttl}} option that > allows a user to set a ttl for automatic snaphots on drop/truncate. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16451) Add ability to ttl snapshots
[ https://issues.apache.org/jira/browse/CASSANDRA-16451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17313585#comment-17313585 ] maxwellguo commented on CASSANDRA-16451: [~clohfink] "Great idea! should be sure to add a throttle in the deletion to prevent trashing drives", I remember that jeff have done some similar work , we may use the code or the delete function . besides I think the ttl for snapshot may be usefull for the table that be truncate when the auto_snapshot is set true . for the snapshot we make mannually , we also know that the snapshot is exists, so we can clean it mannually. so the ttl for this kind of snapshot may not be of some meaning. But for the truncate table's snapshot we may need a recycle bin, and the Recycle bin may need a ttl . In this way ,I think snapshot ttl is of much meanning. > Add ability to ttl snapshots > > > Key: CASSANDRA-16451 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16451 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Paulo Motta >Priority: Normal > Labels: gsoc2021, mentor > > It should be possible to add a TTL to snapshots, after which it automatically > cleans itself up. > This will be useful together with the {{auto_snapshot}} option, where you > want to keep an emergency snapshot in case of accidental drop or truncation > but automatically remove it after a specified period when it's no longer > useful. So in addition to allowing a user to specify a snapshot ttl on > {{nodetool snapshot}} we should have a {{auto_snapshot_ttl}} option that > allows a user to set a ttl for automatic snaphots on drop/truncate. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16451) Add ability to ttl snapshots
[ https://issues.apache.org/jira/browse/CASSANDRA-16451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17313480#comment-17313480 ] Paulo Motta commented on CASSANDRA-16451: - Thanks [~fibersel], added some comments to the doc. > Add ability to ttl snapshots > > > Key: CASSANDRA-16451 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16451 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Paulo Motta >Priority: Normal > Labels: gsoc2021, mentor > > It should be possible to add a TTL to snapshots, after which it automatically > cleans itself up. > This will be useful together with the {{auto_snapshot}} option, where you > want to keep an emergency snapshot in case of accidental drop or truncation > but automatically remove it after a specified period when it's no longer > useful. So in addition to allowing a user to specify a snapshot ttl on > {{nodetool snapshot}} we should have a {{auto_snapshot_ttl}} option that > allows a user to set a ttl for automatic snaphots on drop/truncate. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16451) Add ability to ttl snapshots
[ https://issues.apache.org/jira/browse/CASSANDRA-16451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17312421#comment-17312421 ] Abuli Palagashvili commented on CASSANDRA-16451: I will add all design documentation here: https://docs.google.com/document/d/1OqPaXT-4g6wqEIGzrG2UMadfVNtPT2d8BfY4DAzjIlk/edit?usp=sharing Check if you have an access. > Add ability to ttl snapshots > > > Key: CASSANDRA-16451 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16451 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Paulo Motta >Priority: Normal > Labels: gsoc2021, mentor > > It should be possible to add a TTL to snapshots, after which it automatically > cleans itself up. > This will be useful together with the {{auto_snapshot}} option, where you > want to keep an emergency snapshot in case of accidental drop or truncation > but automatically remove it after a specified period when it's no longer > useful. So in addition to allowing a user to specify a snapshot ttl on > {{nodetool snapshot}} we should have a {{auto_snapshot_ttl}} option that > allows a user to set a ttl for automatic snaphots on drop/truncate. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16451) Add ability to ttl snapshots
[ https://issues.apache.org/jira/browse/CASSANDRA-16451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17308264#comment-17308264 ] Paulo Motta commented on CASSANDRA-16451: - {quote}Can you please tell me, where I can write design documentation? {quote} Any editor of your choice.. notepad, vim, google docs, github.. you can attach the link or file to this JIRA when you have it. {quote}I will take CASSANDRA-14582 as first issue. Can you please guide me on how to commit changes? {quote} Sounds good! A good practice before submitting the patch is to assign yourself to the issue and describe how do you plan to implement it before diving into coding in order to validate the implementation with a reviewer. Once you have a patch ready you can follow the instructions [here|https://cassandra.apache.org/doc/latest/development/patches.html] to create the patch. You can either attach the patch to the JIRA or submit a pull request to [apache/cassandra|https://github.com/apache/cassandra] and add the link to the ticket. {quote}Should I work with github's mirror repo or with gitbox origin? {quote} You can use the github mirror of apache/cassandra. > Add ability to ttl snapshots > > > Key: CASSANDRA-16451 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16451 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Paulo Motta >Priority: Normal > Labels: gsoc2021, mentor > > It should be possible to add a TTL to snapshots, after which it automatically > cleans itself up. > This will be useful together with the {{auto_snapshot}} option, where you > want to keep an emergency snapshot in case of accidental drop or truncation > but automatically remove it after a specified period when it's no longer > useful. So in addition to allowing a user to specify a snapshot ttl on > {{nodetool snapshot}} we should have a {{auto_snapshot_ttl}} option that > allows a user to set a ttl for automatic snaphots on drop/truncate. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16451) Add ability to ttl snapshots
[ https://issues.apache.org/jira/browse/CASSANDRA-16451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17308209#comment-17308209 ] Abuli Palagashvili commented on CASSANDRA-16451: [~paulo] Very nice! Can you please tell me, where I can write design documentation? I will take [CASSANDRA-14582|https://issues.apache.org/jira/browse/CASSANDRA-14582] as first issue.Can you please guide me on how to commit changes?Should I work with github's mirror repo or with gitbox origin? > Add ability to ttl snapshots > > > Key: CASSANDRA-16451 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16451 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Paulo Motta >Priority: Normal > Labels: gsoc2021, mentor > > It should be possible to add a TTL to snapshots, after which it automatically > cleans itself up. > This will be useful together with the {{auto_snapshot}} option, where you > want to keep an emergency snapshot in case of accidental drop or truncation > but automatically remove it after a specified period when it's no longer > useful. So in addition to allowing a user to specify a snapshot ttl on > {{nodetool snapshot}} we should have a {{auto_snapshot_ttl}} option that > allows a user to set a ttl for automatic snaphots on drop/truncate. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16451) Add ability to ttl snapshots
[ https://issues.apache.org/jira/browse/CASSANDRA-16451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17308205#comment-17308205 ] Abuli Palagashvili commented on CASSANDRA-16451: [#comment-17305808] Nice! > Add ability to ttl snapshots > > > Key: CASSANDRA-16451 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16451 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Paulo Motta >Priority: Normal > Labels: gsoc2021, mentor > > It should be possible to add a TTL to snapshots, after which it automatically > cleans itself up. > This will be useful together with the {{auto_snapshot}} option, where you > want to keep an emergency snapshot in case of accidental drop or truncation > but automatically remove it after a specified period when it's no longer > useful. So in addition to allowing a user to specify a snapshot ttl on > {{nodetool snapshot}} we should have a {{auto_snapshot_ttl}} option that > allows a user to set a ttl for automatic snaphots on drop/truncate. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16451) Add ability to ttl snapshots
[ https://issues.apache.org/jira/browse/CASSANDRA-16451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17308204#comment-17308204 ] Abuli Palagashvili commented on CASSANDRA-16451: [#якорь]https://issues.apache.org/jira/browse/CASSANDRA-16451?focusedCommentId=17305808=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17305808 > Add ability to ttl snapshots > > > Key: CASSANDRA-16451 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16451 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Paulo Motta >Priority: Normal > Labels: gsoc2021, mentor > > It should be possible to add a TTL to snapshots, after which it automatically > cleans itself up. > This will be useful together with the {{auto_snapshot}} option, where you > want to keep an emergency snapshot in case of accidental drop or truncation > but automatically remove it after a specified period when it's no longer > useful. So in addition to allowing a user to specify a snapshot ttl on > {{nodetool snapshot}} we should have a {{auto_snapshot_ttl}} option that > allows a user to set a ttl for automatic snaphots on drop/truncate. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16451) Add ability to ttl snapshots
[ https://issues.apache.org/jira/browse/CASSANDRA-16451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17305808#comment-17305808 ] Paulo Motta commented on CASSANDRA-16451: - {quote}As I understand right now, after snapshot creation it is not managed by cassandra.Also, each snapshot has it's own manifest.json file with metadata.So, we can write expiration timestamp to each such file and make cassandra regularly scanning folders with keyspaces/column families and detecting/removing outdated snapshots. {quote} Sounds like a good approach overall, it would be nice if you could detail this approach a bit more in a short design doc (ie. nodetool API, manifest format, how the expiration scan works, testing plan). It would probably help in your GSoC application if you work on some other ticket prior to your application do demonstrate your basic familiarity with the cassandra codebase. Some good starter tickets are CASSANDRA-14587 or CASSANDRA-14582, but you can find others on this list. > Add ability to ttl snapshots > > > Key: CASSANDRA-16451 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16451 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Paulo Motta >Priority: Normal > Labels: gsoc2021, mentor > > It should be possible to add a TTL to snapshots, after which it automatically > cleans itself up. > This will be useful together with the {{auto_snapshot}} option, where you > want to keep an emergency snapshot in case of accidental drop or truncation > but automatically remove it after a specified period when it's no longer > useful. So in addition to allowing a user to specify a snapshot ttl on > {{nodetool snapshot}} we should have a {{auto_snapshot_ttl}} option that > allows a user to set a ttl for automatic snaphots on drop/truncate. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16451) Add ability to ttl snapshots
[ https://issues.apache.org/jira/browse/CASSANDRA-16451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17305524#comment-17305524 ] Abuli Palagashvili commented on CASSANDRA-16451: [~paulo] I have played a bit with CCM and manually-made snapshots.As I understand right now, after snapshot creation it is not managed by cassandra.Also, each snapshot has it's own manifest.json file with metadata.So, we can write expiration timestamp to each such file and make cassandra regularly scanning folders with keyspaces/column families and detecting/removing outdated snapshots.Correct me if I'm wrong, please. > Add ability to ttl snapshots > > > Key: CASSANDRA-16451 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16451 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Paulo Motta >Priority: Normal > Labels: gsoc2021, mentor > > It should be possible to add a TTL to snapshots, after which it automatically > cleans itself up. > This will be useful together with the {{auto_snapshot}} option, where you > want to keep an emergency snapshot in case of accidental drop or truncation > but automatically remove it after a specified period when it's no longer > useful. So in addition to allowing a user to specify a snapshot ttl on > {{nodetool snapshot}} we should have a {{auto_snapshot_ttl}} option that > allows a user to set a ttl for automatic snaphots on drop/truncate. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16451) Add ability to ttl snapshots
[ https://issues.apache.org/jira/browse/CASSANDRA-16451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17305202#comment-17305202 ] Paulo Motta commented on CASSANDRA-16451: - Hi [~fibersel], Thanks for your interest in working on this project. Before diving into the code I suggest to get acquainted with the snapshot feature, here are a few links to get started: * [About snapshots|https://docs.datastax.com/en/cassandra-oss/3.0/cassandra/operations/opsAboutSnapshots.html] * [Taking a snapshot|https://docs.datastax.com/en/cassandra-oss/3.0/cassandra/operations/opsBackupTakesSnapshot.html] * [All about cassandra snapshots|https://dev.to/ecnepsnai/all-about-apache-cassandra-snapshots-3oo2] Once you are familiar I suggest you to create a local cassandra cluster with [CCM|https://github.com/riptano/ccm], try inserting some data and creating snapshots manually and also test the [auto_snapshot|https://cassandra.apache.org/doc/latest/configuration/cassandra_config_file.html#auto-snapshot] feature. This exercise will make it easier to dive into the code when you're ready. A good starting point is the [SnapshotCommand|https://github.com/apache/cassandra/blob/79e693e16e2152097c5b27d2d7aaa1763e34f594/src/java/org/apache/cassandra/db/SnapshotCommand.java] class. Once you have a good understanding of how snapshot works, then we can collaboratively discuss how to best implement this feature and you can submit your GSoC proposal based on that discussion. If you have any questions don't hesitate to ask here or on the [community slack channels|https://cassandra.apache.org/community/]. > Add ability to ttl snapshots > > > Key: CASSANDRA-16451 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16451 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Paulo Motta >Priority: Normal > Labels: gsoc2021, mentor > > It should be possible to add a TTL to snapshots, after which it automatically > cleans itself up. > This will be useful together with the {{auto_snapshot}} option, where you > want to keep an emergency snapshot in case of accidental drop or truncation > but automatically remove it after a specified period when it's no longer > useful. So in addition to allowing a user to specify a snapshot ttl on > {{nodetool snapshot}} we should have a {{auto_snapshot_ttl}} option that > allows a user to set a ttl for automatic snaphots on drop/truncate. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16451) Add ability to ttl snapshots
[ https://issues.apache.org/jira/browse/CASSANDRA-16451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17305134#comment-17305134 ] Abuli Palagashvili commented on CASSANDRA-16451: [~paulo] , Hi! I have successfully built cassandra from repo on my machine and run it. Can you please show me the point in code to start inspecting Cassandra's Snapshot inner structure and it's life cycle?And what should I do by April 13 to successfully participate GSoC 2021? > Add ability to ttl snapshots > > > Key: CASSANDRA-16451 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16451 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Paulo Motta >Priority: Normal > Labels: gsoc2021, mentor > > It should be possible to add a TTL to snapshots, after which it automatically > cleans itself up. > This will be useful together with the {{auto_snapshot}} option, where you > want to keep an emergency snapshot in case of accidental drop or truncation > but automatically remove it after a specified period when it's no longer > useful. So in addition to allowing a user to specify a snapshot ttl on > {{nodetool snapshot}} we should have a {{auto_snapshot_ttl}} option that > allows a user to set a ttl for automatic snaphots on drop/truncate. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org
[jira] [Commented] (CASSANDRA-16451) Add ability to ttl snapshots
[ https://issues.apache.org/jira/browse/CASSANDRA-16451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17286575#comment-17286575 ] Chris Lohfink commented on CASSANDRA-16451: --- Great idea! should be sure to add a throttle in the deletion to prevent trashing drives > Add ability to ttl snapshots > > > Key: CASSANDRA-16451 > URL: https://issues.apache.org/jira/browse/CASSANDRA-16451 > Project: Cassandra > Issue Type: Improvement > Components: Tool/nodetool >Reporter: Paulo Motta >Priority: Normal > Labels: gsoc2021, mentor > > It should be possible to add a TTL to snapshots, after which it automatically > cleans itself up. > This will be useful together with the {{auto_snapshot}} option, where you > want to keep an emergency snapshot in case of accidental drop or truncation > but automatically remove it after a specified period when it's no longer > useful. So in addition to allowing a user to specify a snapshot ttl on > {{nodetool snapshot}} we should have a {{auto_snapshot_ttl}} option that > allows a user to set a ttl for automatic snaphots on drop/truncate. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: commits-unsubscr...@cassandra.apache.org For additional commands, e-mail: commits-h...@cassandra.apache.org