Hey Andi, The backup implementation is up to the storage providers. As for lunr, we are working on a very similar feature set to back up volumes to swift (and only backing up the changes since the last backup).
-- Chuck On Thu, Jul 21, 2011 at 1:21 PM, andi abes <[email protected]> wrote: > hmm - they definitely muddy the waters, but provide a really cool feature > set: > > Amazon EBS Snapshots > > Amazon EBS provides the ability to back up point-in-time snapshots of your > data to Amazon S3 for durable recovery. Amazon EBS snapshots are incremental > backups, meaning that only the blocks on the device that have changed since > your last snapshot will be saved. If you have a device with 100 GBs of data, > but only 5 GBs of data has changed since your last snapshot, only the 5 > additional GBs of snapshot data will be stored back to Amazon S3. Even > though the snapshots are saved incrementally, when you delete a snapshot, > only the data not needed for any other snapshot is removed. So regardless of > which prior snapshots have been deleted, all active snapshots will contain > all the information needed to restore the volume. In addition, the time to > restore the volume is the same for all snapshots, offering the restore time > of full backups with the space savings of incremental > > > That quoted - it's not exactly a low bar to meet in terms of capability. > Chuck - are you proposing that as the target for Diablo? > > p.s - typing on a real keyboard is so much easier than an iPad, and leads to > much better grammar... > On Thu, Jul 21, 2011 at 12:19 PM, Chuck Thier <[email protected]> wrote: >> >> Hey Andi, >> >> Perhaps it would be better to re-frame the question. >> >> What should the base functionality of the Openstack API for >> backup/snapshot functionality be? >> >> I'm looking at it from the perspective of initially providing the >> capabilities that EC2/EBS currently provides (which they call >> snapshots). To me, this is the absolute base of what is needed, and >> is what I am basically proposing as the idea of backups. >> >> I also see that allowing for true volume snapshot capabilities are >> desirable down the road. The difficulty with snapshots, is that their >> properties can vary greatly between different storage systems, and >> thus needs some care in defining what a Nova Volume snapshot should >> support. I would expect that the different storage providers would >> initially provide this support through extensions to the API. At that >> point it may be easier to find what commonalities there are, and to >> find what types of features are most demanded in the cloud. >> >> -- >> Chuck >> >> On Thu, Jul 21, 2011 at 5:57 AM, Andiabes <[email protected]> wrote: >> > I think vish pointed out the main differences between the 2 entities, >> > and maybe that can lead to name disambiguation... >> > >> > Backup is a full copy, and usable without the original object being >> > available in any state ( original or modified). It's expensive, since it's >> > a >> > full copy. Main use cases are dr and recovery. >> > >> > Snapshot represents a point in time state of the object. It's relatively >> > cheap ( with the expectation that some copy-on-write or differencing >> > technique is used). Only usable if the reference point of the snapshot is >> > available (could be thought of as an incremental backup); what that >> > reference point is depends on the underlying implementation technology. >> > Main >> > use case is rewinding to so a historic state some time in the future. >> > >> > That said, with the prereqs met, both can probably be used to mount a >> > new volume. >> > Reasonable? >> > >> > On Jul 20, 2011, at 5:27 PM, Chuck Thier <[email protected]> wrote: >> > >> >> Yeah, I think you are illustrating how this generates much confusion :) >> >> >> >> To try to be more specific, the base functionality should be: >> >> >> >> 1. Create a point in time backup of a volume >> >> 2. Create a new volume from a backup (I guess it seems reasonable to >> >> call this a clone) >> >> >> >> This emulates the behavior of what EC2/EBS provide with volume >> >> snapshots. In this scenario, a "restore" is create a new volume from >> >> the backup, and delete the old volume. >> >> >> >> In the Storage world, much more can generally be done with snapshots. >> >> For example in most storage system snapshots are treated just like a >> >> normal volume and can be mounted directly. A snapshot is often used >> >> when creating a backup to ensure that you have a consistent point in >> >> time backup, which I think most of the confusion comes from. >> >> >> >> What we finally call it doesn't matter as much to me, as long as we >> >> paint a consistent story that isn't confusing, and that we get it in >> >> the Openstack API. >> >> >> >> -- >> >> Chuck >> >> >> >> On Wed, Jul 20, 2011 at 3:33 PM, Vishvananda Ishaya >> >> <[email protected]> wrote: >> >>> In rereading this i'm noticing that you are actually suggesting >> >>> alternative usage: >> >>> >> >>> backup/clone >> >>> >> >>> snapshot/restore >> >>> >> >>> Correct? >> >>> >> >>> It seems like backup and snapshot are kind of interchangable. This is >> >>> quite confusing, perhaps we should refer to them as: >> >>> >> >>> partial-snapshot >> >>> >> >>> whole-snapshot >> >>> >> >>> or something along those lines that conveys that one is a differencing >> >>> image and one is a copy of the entire object? >> >>> >> >>> On Jul 20, 2011, at 12:01 PM, Chuck Thier wrote: >> >>> >> >>>> At the last developers summit, it was noted by many, that the idea of >> >>>> a volume snaphsot in the cloud is highly overloaded. EBS uses the >> >>>> notion of snapshots for making point in time backups of a volume that >> >>>> can be used to create a new volume from. These are not true >> >>>> snapshots >> >>>> though from a storage world view. Because of this I would like to >> >>>> make the following proposal: >> >>>> >> >>>> Add a backup API to the Openstack API for Nova Volume. This is to >> >>>> provide EBS style snapshot functionality in the Openstack API. I'm >> >>>> proposing to name it backup instead of snapshot as that seems to >> >>>> better describe what is happening. It also allows room for other >> >>>> storage backends to expose real snapshot capabilities down the road. >> >>>> >> >>>> In the case of Lunr, we would be making backups of volumes to swift >> >>>> (possibly abstracted through glance in the future). >> >>>> >> >>>> I have started a blueprint and spec at: >> >>>> >> >>>> https://blueprints.launchpad.net/nova/+spec/backups-api >> >>>> http://etherpad.openstack.org/volume-backup >> >>>> >> >>>> Please feel free to comment and contribute. >> >>>> >> >>>> -- >> >>>> Chuck >> >>>> >> >>>> _______________________________________________ >> >>>> Mailing list: https://launchpad.net/~openstack >> >>>> Post to : [email protected] >> >>>> Unsubscribe : https://launchpad.net/~openstack >> >>>> More help : https://help.launchpad.net/ListHelp >> >>> >> >>> >> >> >> >> _______________________________________________ >> >> Mailing list: https://launchpad.net/~openstack >> >> Post to : [email protected] >> >> Unsubscribe : https://launchpad.net/~openstack >> >> More help : https://help.launchpad.net/ListHelp >> > > > _______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : [email protected] Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
