I think that all sounds reasonable then - thanks! On Thu, Feb 4, 2016 at 6:52 PM, Syed Mushtaq <syed1.mush...@gmail.com> wrote:
> You are correct Mike in terms of the requirements. One of our earlier > iterations on this was to have an argument to the create snapshot API which > decides whether to backup the volume to sec storage but we realized it > would make management of snapshots quite messy so we proposed a new api > instead. > > On Thu, Feb 4, 2016, 8:29 PM Mike Tutkowski <mike.tutkow...@solidfire.com> > wrote: > >> Hi, >> >> Just to make sure I understand all the requirements here: >> >> 1) This relates only to managed storage (1:1 mapping between a virtual >> disk and a backend SAN volume). >> >> 2) We want to take the current (introduced in 4.6) functionality, which >> creates a snapshot on the SAN, and extend it via a config option (or >> something) to not only take the SAN snapshot, but to copy the underlying >> VHD (XenServer only) to NFS. >> >> 3) The SAN snapshot is always taken. It's the backup to NFS that is >> optional. >> >> 4) Templates can be created from the snapshot that's on the SAN (already >> works). >> >> 5) CloudStack volumes can be created from the snapshot that's on the SAN >> (already works as long as the new CloudStack volume ends up on the same >> primary storage). >> >> Would we have a need for a storage snapshot API then or would that just >> be the standard volume snapshot without the backup to NFS? >> >> Thanks! >> Mike >> >> On Thu, Feb 4, 2016 at 5:24 PM, Syed Mushtaq <syed1.mush...@gmail.com> >> wrote: >> >>> Is it possible to have both functionalities (snapshot on SAN & Sec >>> Storage) coexist? Because Ideally, we would like to have both. >>> For example, some of our customers want to implement their own backup >>> strategies and do encryption to their backups which is a perfect >>> use case for Storage Snapshot while our other customers will still keep >>> using the standard volume snapshot. >>> >>> To keep things backward compatible, we can add a setting which says to >>> not upload on secondary storage, because, after all, you would take a >>> SAN snapshot first when doing a Volume Snapshot. You could stop the >>> process there and not do the upload. >>> >>> What do you think about this approach? >>> >>> Thanks, >>> -Syed >>> >>> On Thu, Feb 4, 2016 at 4:25 PM, Mike Tutkowski < >>> mike.tutkow...@solidfire.com> wrote: >>> >>>> So, this is just me thinking out load here, but if a given CloudStack >>>> cloud doesn't actually need to provide both the ability to take a SAN >>>> snapshot and export it to NFS (if just taking a SAN snapshot is OK), then >>>> we might be able to get away with no new API calls and simply implement a >>>> new custom snapshot strategy and data motion strategy to handle the case >>>> where the CloudStack cloud does want both a SAN snapshot and >>>> exported-to-NFS backup. >>>> >>>> In other words, the "default" behavior would be to use the snapshot >>>> strategy and data motion strategy that we already have (the one that only >>>> takes a SAN snapshot). >>>> >>>> If your CloudStack cloud, however, wants to take a SAN snapshot and >>>> have the data exported to NFS, then we could have you manipulate a Swing >>>> config file to make use of a new snapshot strategy and data motion strategy >>>> that performs both of these activities. >>>> >>>> This way, the old behavior is still the default for users, but >>>> CloudStack admins can change this behavior via configuration. >>>> >>>> Thoughts? >>>> >>>> On Thu, Feb 4, 2016 at 11:55 AM, Mike Tutkowski < >>>> mike.tutkow...@solidfire.com> wrote: >>>> >>>>> Right...I think we will need to come up with a viable upgrade path or >>>>> some reasonable way for them to move from the old way to the new way (and >>>>> some obvious way that they will know they need to do this). >>>>> >>>>> On Thu, Feb 4, 2016 at 11:45 AM, Syed Mushtaq <syed1.mush...@gmail.com >>>>> > wrote: >>>>> >>>>>> I'm not really sure about the upgrade path however, customers who are >>>>>> using 4.6 and are on a managed storage would no longer have the same >>>>>> functionality with Volume Snapshots. >>>>>> >>>>>> On Thu, Feb 4, 2016 at 1:43 PM, Syed Mushtaq <syed1.mush...@gmail.com >>>>>> > wrote: >>>>>> >>>>>>> So if I understand correctly, currently taking a Volume Snapshots of >>>>>>> a volume on a managed storage keeps it on the storage array. As a part >>>>>>> of >>>>>>> this feature, we can make sure that Volume Snapshots on managed storage >>>>>>> are >>>>>>> uploaded to the secondary storage. This would make the Volume Snapshot >>>>>>> feature behave the same regardless of the storage (managed or >>>>>>> non-managed) >>>>>>> And, for utilizing the efficient backend storage capabilities, we can >>>>>>> use >>>>>>> the new storage snapshots API. >>>>>>> >>>>>>> On Thu, Feb 4, 2016 at 1:36 PM, Mike Tutkowski < >>>>>>> mike.tutkow...@solidfire.com> wrote: >>>>>>> >>>>>>>> Hi everyone, >>>>>>>> >>>>>>>> Whatever we do here, we need to have a plan to deal with the fact >>>>>>>> that we already have a feature (in 4.6 and later) that allows you to >>>>>>>> use >>>>>>>> the existing volume-snapshot APIs to create a volume snapshot (for >>>>>>>> managed >>>>>>>> storage) that resides on a backend SAN (using a custom snapshot >>>>>>>> strategy >>>>>>>> and a custom data motion strategy). >>>>>>>> >>>>>>>> If these new APIs go in, then how should the original >>>>>>>> implementation (present in 4.6 and later) be changed? If it is >>>>>>>> changed, how >>>>>>>> do we support customers who are already using the original >>>>>>>> volume-snapshot >>>>>>>> API to take snapshots on a backend SAN? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Mike >>>>>>>> >>>>>>>> On Thu, Feb 4, 2016 at 11:27 AM, Will Stevens < >>>>>>>> wstev...@cloudops.com> wrote: >>>>>>>> >>>>>>>>> Will you be able to create a Template from a StorageSnapshot? If >>>>>>>>> yes, will the template be stored in the secondary storage like normal >>>>>>>>> templates or will that be handled somehow on the vendor side? >>>>>>>>> >>>>>>>>> *Will STEVENS* >>>>>>>>> Lead Developer >>>>>>>>> >>>>>>>>> *CloudOps* *| *Cloud Solutions Experts >>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 >>>>>>>>> w cloudops.com *|* tw @CloudOps_ >>>>>>>>> >>>>>>>>> On Thu, Feb 4, 2016 at 1:22 PM, Syed Mushtaq < >>>>>>>>> syed1.mush...@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Thanks Will!!! >>>>>>>>>> >>>>>>>>>> On Thu, Feb 4, 2016 at 1:19 PM, Will Stevens < >>>>>>>>>> wstev...@cloudops.com> wrote: >>>>>>>>>> >>>>>>>>>>> I explicitly linked the Design Spec in the Jira ticket because >>>>>>>>>>> it was not clear in the 'mention' section because it shows as a >>>>>>>>>>> page 'you >>>>>>>>>>> do not have permission to'. >>>>>>>>>>> >>>>>>>>>>> *Will STEVENS* >>>>>>>>>>> Lead Developer >>>>>>>>>>> >>>>>>>>>>> *CloudOps* *| *Cloud Solutions Experts >>>>>>>>>>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 >>>>>>>>>>> w cloudops.com *|* tw @CloudOps_ >>>>>>>>>>> >>>>>>>>>>> On Thu, Feb 4, 2016 at 1:02 PM, Syed Ahmed <sah...@cloudops.com> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Design Spec: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/StorageSnapshot++API >>>>>>>>>>>> >>>>>>>>>>>> Jira Ticket >>>>>>>>>>>> >>>>>>>>>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-9278 >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Hi All, >>>>>>>>>>>> >>>>>>>>>>>> We plan to propose a new set of APIs to do snapshots on managed >>>>>>>>>>>> storage backends like SolidFire. Snapshots on current managed >>>>>>>>>>>> storage stay >>>>>>>>>>>> on the >>>>>>>>>>>> device which is contrary to what CloudStack calls snpshots. But >>>>>>>>>>>> taking snapshots >>>>>>>>>>>> on storage and keeping it there has its own advantages and we >>>>>>>>>>>> would ideally like >>>>>>>>>>>> to have both ways of doing snapshots. This proposal adds 4 new >>>>>>>>>>>> APIs to >>>>>>>>>>>> create snapshots on backend storage. >>>>>>>>>>>> >>>>>>>>>>>> What do you guys think of this feature? I would love to have >>>>>>>>>>>> some feedback. I am working on making the design spec more >>>>>>>>>>>> concrete but >>>>>>>>>>>> wanted to have >>>>>>>>>>>> a high level feedback first before starting to work on it. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> -Syed >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> *Mike Tutkowski* >>>>>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>>>>> e: mike.tutkow...@solidfire.com >>>>>>>> o: 303.746.7302 >>>>>>>> Advancing the way the world uses the cloud >>>>>>>> <http://solidfire.com/solution/overview/?video=play>*™* >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> *Mike Tutkowski* >>>>> *Senior CloudStack Developer, SolidFire Inc.* >>>>> e: mike.tutkow...@solidfire.com >>>>> o: 303.746.7302 >>>>> Advancing the way the world uses the cloud >>>>> <http://solidfire.com/solution/overview/?video=play>*™* >>>>> >>>> >>>> >>>> >>>> -- >>>> *Mike Tutkowski* >>>> *Senior CloudStack Developer, SolidFire Inc.* >>>> e: mike.tutkow...@solidfire.com >>>> o: 303.746.7302 >>>> Advancing the way the world uses the cloud >>>> <http://solidfire.com/solution/overview/?video=play>*™* >>>> >>> >>> >> >> >> -- >> *Mike Tutkowski* >> *Senior CloudStack Developer, SolidFire Inc.* >> e: mike.tutkow...@solidfire.com >> o: 303.746.7302 >> Advancing the way the world uses the cloud >> <http://solidfire.com/solution/overview/?video=play>*™* >> > -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloud <http://solidfire.com/solution/overview/?video=play>*™*
