Re: XCP-ng Backup Cloudstack 4.15
+1 what Yordan shared, in addition The B framework is extensible with the provider model, that is you can write plugins. The framework assumes that plugin is responsible for actually carrying out the operation, i.e. execution. The plugin however can either choose to do those operations or delegate to a service that performs the operation. There is a dummy provider/plugin which is for testing purposes of the framework. We've only one backup vendor support that is Veeam and the Veeam B provider/plugin simply delegates B framework operations to the Veeam manager/service. At the time of writing it was only tested with Veeam & VMware, but now Veeam has added support for other hypervisors (I've not checked the list myself so not sure if Xen/Xcp-ng is already on it or not). So, in theory with some changes, the Veeam B provider can support any hypervisor that Veeam supports. Regards From: Yordan Kostov Sent: Friday, September 3, 2021 15:00 To: users@cloudstack.apache.org Subject: RE: XCP-ng Backup Cloudstack 4.15 Hey Benoit, I am also interested is such integration to external vendor for XCP hypervisor. Lately my attention is in other matters but I do begun thinking on an external solution for a cold backup. Here is a design concept: Backup: 1. ACS framework does occasional volume backups and keeps them on NFS share (hot backup) 2. Veeam collects the backup folder structure and contents once every X and keeps it on tape (cold backup) In this case ACS framework is what the backup solution for hot backup and Veeam is for cold. Hot restore: 1. Happens through ACS Framework and is done by the end user. Cold restore: 1. admin dumps a cold backup to a NFS 2. in an automated manner single or all volumes are imported into ACS and assigned to the proper account owners of the original VMs. 3. ACS user restore VMs from volumes as Hot restore. From everything above what is not possible is how the restored folder structure and volumes can be automatically imported and assigned to the proper accounts? What I came up it as least effort solution that also is resilient is on the backup NFS server to make a cron script that executes once an hour and scans the /backup partition. ACS NFS backup partition has the following structure //snapshots/account_id/VOLUME_ID /master_and_delta_files In each VOLUME_ID directory a YAML file will be created listing: - owner acc id - origin volume id - origin volume name (taken from ACS DB) - list of mater and delta files Consecutive runs of the script will compare if VOLUME_ID folder contents are different than the YAML and will update it. Usually that will before Veeam backup occurs. Then Veeam backup will occur. There is one caveat here - hot backups can occur anytime depending on user settings while cold backup happens once per time frame. There can be the case where cold backup occurs while hot backup jobs are running. This should be avoided. When restore is required specific volume will be restored from Veeam and with restore shell script it will take the volume YAML conif: - converge deltas - set new volume name as RESTORED_ - import the volume to ACS under the origin account ID Moreover a secondary script can create VMs from volumes for a case of mass cold restore but that will be at later stage. Again that is just a concept and if anyone has an idea on how to improve or simplify that will be great! Best regards, Jordan -Original Message- From: benoit lair Sent: 02 септември 2021 г. 18:36 To: users@cloudstack.apache.org Subject: Re: XCP-ng Backup Cloudstack 4.15 [X] This message came from outside your organization Is there a way to implement ourself a custom external provider in order to backup VMs ? Le jeu. 2 sept. 2021 à 15:52, benoit lair a écrit : > Hello, > > I am interested too in doing Backup VM for Xcp-NG Would you have a > solution for using Veeam like Yordan aims ? > > Le lun. 12 juil. 2021 à 13:30, Abishek Budhathoki > a écrit : > >> Thank You for the response. Really apricated. >> >> On 2021/07/12 09:41:07, Rohit Yadav wrote: >> > Hi Abishek, >> > >> > That's right, the current Backup & Recovery framework only supports >> Veeam provider on VMware. >> > >> > For XenServer/xcpng, we don't have a plugin/provider, however >> > volume >> snapshots can be used to backup snapshots on secondary storage. >> > >> > Regards. >> > >> > Regards, >> > Rohit Yadav >> > >> > >> > From: Abishek Budhathoki
RE: XCP-ng Backup Cloudstack 4.15
Hey Benoit, I am also interested is such integration to external vendor for XCP hypervisor. Lately my attention is in other matters but I do begun thinking on an external solution for a cold backup. Here is a design concept: Backup: 1. ACS framework does occasional volume backups and keeps them on NFS share (hot backup) 2. Veeam collects the backup folder structure and contents once every X and keeps it on tape (cold backup) In this case ACS framework is what the backup solution for hot backup and Veeam is for cold. Hot restore: 1. Happens through ACS Framework and is done by the end user. Cold restore: 1. admin dumps a cold backup to a NFS 2. in an automated manner single or all volumes are imported into ACS and assigned to the proper account owners of the original VMs. 3. ACS user restore VMs from volumes as Hot restore. From everything above what is not possible is how the restored folder structure and volumes can be automatically imported and assigned to the proper accounts? What I came up it as least effort solution that also is resilient is on the backup NFS server to make a cron script that executes once an hour and scans the /backup partition. ACS NFS backup partition has the following structure //snapshots/account_id/VOLUME_ID /master_and_delta_files In each VOLUME_ID directory a YAML file will be created listing: - owner acc id - origin volume id - origin volume name (taken from ACS DB) - list of mater and delta files Consecutive runs of the script will compare if VOLUME_ID folder contents are different than the YAML and will update it. Usually that will before Veeam backup occurs. Then Veeam backup will occur. There is one caveat here - hot backups can occur anytime depending on user settings while cold backup happens once per time frame. There can be the case where cold backup occurs while hot backup jobs are running. This should be avoided. When restore is required specific volume will be restored from Veeam and with restore shell script it will take the volume YAML conif: - converge deltas - set new volume name as RESTORED_ - import the volume to ACS under the origin account ID Moreover a secondary script can create VMs from volumes for a case of mass cold restore but that will be at later stage. Again that is just a concept and if anyone has an idea on how to improve or simplify that will be great! Best regards, Jordan -Original Message- From: benoit lair Sent: 02 септември 2021 г. 18:36 To: users@cloudstack.apache.org Subject: Re: XCP-ng Backup Cloudstack 4.15 [X] This message came from outside your organization Is there a way to implement ourself a custom external provider in order to backup VMs ? Le jeu. 2 sept. 2021 à 15:52, benoit lair a écrit : > Hello, > > I am interested too in doing Backup VM for Xcp-NG Would you have a > solution for using Veeam like Yordan aims ? > > Le lun. 12 juil. 2021 à 13:30, Abishek Budhathoki > a écrit : > >> Thank You for the response. Really apricated. >> >> On 2021/07/12 09:41:07, Rohit Yadav wrote: >> > Hi Abishek, >> > >> > That's right, the current Backup & Recovery framework only supports >> Veeam provider on VMware. >> > >> > For XenServer/xcpng, we don't have a plugin/provider, however >> > volume >> snapshots can be used to backup snapshots on secondary storage. >> > >> > Regards. >> > >> > Regards, >> > Rohit Yadav >> > >> > >> > From: Abishek Budhathoki >> > Sent: Saturday, July 10, 2021 7:42:12 PM >> > To: users@cloudstack.apache.org >> > Subject: XCP-ng Backup Cloudstack 4.15 >> > >> > Hello EveryOne, >> > >> > I am trying cloudstack with xen environment. I was trying out the >> backup feature of the cloudstack and was not able to achieve it. Does >> the backup work in xen environment or it strictly only works with vmware >> only. >> > >> > >> > >> > >> > >> > >> >
Re: XCP-ng Backup Cloudstack 4.15
Is there a way to implement ourself a custom external provider in order to backup VMs ? Le jeu. 2 sept. 2021 à 15:52, benoit lair a écrit : > Hello, > > I am interested too in doing Backup VM for Xcp-NG > Would you have a solution for using Veeam like Yordan aims ? > > Le lun. 12 juil. 2021 à 13:30, Abishek Budhathoki a > écrit : > >> Thank You for the response. Really apricated. >> >> On 2021/07/12 09:41:07, Rohit Yadav wrote: >> > Hi Abishek, >> > >> > That's right, the current Backup & Recovery framework only supports >> Veeam provider on VMware. >> > >> > For XenServer/xcpng, we don't have a plugin/provider, however volume >> snapshots can be used to backup snapshots on secondary storage. >> > >> > Regards. >> > >> > Regards, >> > Rohit Yadav >> > >> > >> > From: Abishek Budhathoki >> > Sent: Saturday, July 10, 2021 7:42:12 PM >> > To: users@cloudstack.apache.org >> > Subject: XCP-ng Backup Cloudstack 4.15 >> > >> > Hello EveryOne, >> > >> > I am trying cloudstack with xen environment. I was trying out the >> backup feature of the cloudstack and was not able to achieve it. Does the >> backup work in xen environment or it strictly only works with vmware only. >> > >> > >> > >> > >> > >> > >> >
Re: XCP-ng Backup Cloudstack 4.15
Hello, I am interested too in doing Backup VM for Xcp-NG Would you have a solution for using Veeam like Yordan aims ? Le lun. 12 juil. 2021 à 13:30, Abishek Budhathoki a écrit : > Thank You for the response. Really apricated. > > On 2021/07/12 09:41:07, Rohit Yadav wrote: > > Hi Abishek, > > > > That's right, the current Backup & Recovery framework only supports > Veeam provider on VMware. > > > > For XenServer/xcpng, we don't have a plugin/provider, however volume > snapshots can be used to backup snapshots on secondary storage. > > > > Regards. > > > > Regards, > > Rohit Yadav > > > > > > From: Abishek Budhathoki > > Sent: Saturday, July 10, 2021 7:42:12 PM > > To: users@cloudstack.apache.org > > Subject: XCP-ng Backup Cloudstack 4.15 > > > > Hello EveryOne, > > > > I am trying cloudstack with xen environment. I was trying out the backup > feature of the cloudstack and was not able to achieve it. Does the backup > work in xen environment or it strictly only works with vmware only. > > > > > > > > > > > > >
Re: XCP-ng Backup Cloudstack 4.15
Thank You for the response. Really apricated. On 2021/07/12 09:41:07, Rohit Yadav wrote: > Hi Abishek, > > That's right, the current Backup & Recovery framework only supports Veeam > provider on VMware. > > For XenServer/xcpng, we don't have a plugin/provider, however volume > snapshots can be used to backup snapshots on secondary storage. > > Regards. > > Regards, > Rohit Yadav > > > From: Abishek Budhathoki > Sent: Saturday, July 10, 2021 7:42:12 PM > To: users@cloudstack.apache.org > Subject: XCP-ng Backup Cloudstack 4.15 > > Hello EveryOne, > > I am trying cloudstack with xen environment. I was trying out the backup > feature of the cloudstack and was not able to achieve it. Does the backup > work in xen environment or it strictly only works with vmware only. > > > > > >
RE: XCP-ng Backup Cloudstack 4.15
I will be digging into that this matter week. We are trying to do some kind of disaster recovery setup for XCP-NG/XenServer setup and Veeam. The raw design idea we are having is Backup is done through ACS, Veeam collects the backup once every X days for DR purposes. Restore is a bit more problematic because dumping the VHDs back is fine but it wont show in ACS GUI under the proper volume. A good thing is that when volume backup is removed due to retention policy expiration the ACS DB entry in table " snapshots" is kept but marked from "Backup" to "Destroyed" So during restore of the VHD if the name of the VHD can be used to revert that entry change in the DB will probably make the backup appear in the GUI. What I mention above is the worst case scenario where every step should be automated. I need to dig into the backup framework and the api to see how they can aid this process. Best regards, Jordan -Original Message- From: Rohit Yadav Sent: 12 юли 2021 г. 12:41 To: users@cloudstack.apache.org Subject: Re: XCP-ng Backup Cloudstack 4.15 [X] This message came from outside your organization Hi Abishek, That's right, the current Backup & Recovery framework only supports Veeam provider on VMware. For XenServer/xcpng, we don't have a plugin/provider, however volume snapshots can be used to backup snapshots on secondary storage. Regards. Regards, Rohit Yadav From: Abishek Budhathoki Sent: Saturday, July 10, 2021 7:42:12 PM To: users@cloudstack.apache.org Subject: XCP-ng Backup Cloudstack 4.15 Hello EveryOne, I am trying cloudstack with xen environment. I was trying out the backup feature of the cloudstack and was not able to achieve it. Does the backup work in xen environment or it strictly only works with vmware only.
Re: XCP-ng Backup Cloudstack 4.15
Hi Abishek, That's right, the current Backup & Recovery framework only supports Veeam provider on VMware. For XenServer/xcpng, we don't have a plugin/provider, however volume snapshots can be used to backup snapshots on secondary storage. Regards. Regards, Rohit Yadav From: Abishek Budhathoki Sent: Saturday, July 10, 2021 7:42:12 PM To: users@cloudstack.apache.org Subject: XCP-ng Backup Cloudstack 4.15 Hello EveryOne, I am trying cloudstack with xen environment. I was trying out the backup feature of the cloudstack and was not able to achieve it. Does the backup work in xen environment or it strictly only works with vmware only.
Re: XCP-ng Backup Cloudstack 4.15
Abishek, If you mean to use the backup framework it is generic but only a framework. It needs an agent to do the actual backing up. As of now it has only an implementation for veeem which works with vmware. regards, On Sun, Jul 11, 2021 at 8:05 PM Abishek Budhathoki wrote: > Hello EveryOne, > > I am trying cloudstack with xen environment. I was trying out the backup > feature of the cloudstack and was not able to achieve it. Does the backup > work in xen environment or it strictly only works with vmware only. > > > -- Daan
