Re: {ANNOUNCE] 4.11.1 RC2 cut

[ilya musayev]

Hi Daan

I've tried installing 4.11 RC1 (i know i'm in the wrong thread) and noticed
it does something funky - like installing marvin and going to pypi to get
dependencies. If its already fixed in RC2 - please ignore this request -
but if not...

Unfortunately some environments dont have a luxury of open outbound
internet connection.

Do you think we can refine the requirements and drop packages that arent
needed

Here is an example:

  Installing : cloudstack-marvin-4.11.1.0-rc1.el7.centos.x86_64



191/297

Collecting
http://cdn.mysql.com/Downloads/Connector-Python/mysql-connector-python-2.0.4.zip#md5=3df394d89300db95163f17c843ef49df

  Downloading
http://cdn.mysql.com/Downloads/Connector-Python/mysql-connector-python-2.0.4.zip
(277kB)

Installing collected packages: mysql-connector-python

  Found existing installation: mysql-connector-python 1.1.6

DEPRECATION: Uninstalling a distutils installed project
(mysql-connector-python) has been deprecated and will be removed in a
future version. This is due to the fact that uninstalling a distutils
project will only partially uninstall the project.

Uninstalling mysql-connector-python-1.1.6:

  Successfully uninstalled mysql-connector-python-1.1.6

  Running setup.py install for mysql-connector-python: started

Running setup.py install for mysql-connector-python: finished with
status 'done'

Successfully installed mysql-connector-python-2.0.4

You are using pip version 8.1.2, however version 10.0.1 is available.

You should consider upgrading via the 'pip install --upgrade pip' command.

Processing /usr/share/cloudstack-marvin/Marvin-4.11.1.0.tar.gz

Collecting mysql-connector-python>=1.1.6 (from Marvin==4.11.1.0)

  Downloading
https://files.pythonhosted.org/packages/dc/48/32c715d2cef42d0791c5b2f21b4f1f280c8e45afa66a02f4d1828c77f3ea/mysql_connector_python-8.0.11-cp27-cp27mu-manylinux1_x86_64.whl
(8.1MB)

Collecting requests>=2.2.1 (from Marvin==4.11.1.0)

  Downloading
https://files.pythonhosted.org/packages/49/df/50aa1999ab9bde74656c2919d9c0c085fd2b3775fd3eca826012bef76d8c/requests-2.18.4-py2.py3-none-any.whl
(88kB)

Collecting paramiko>=1.13.0 (from Marvin==4.11.1.0)

  Downloading
https://files.pythonhosted.org/packages/3e/db/cb7b6656e0e7387637ce850689084dc0b94b44df31cc52e5fc5c2c4fd2c1/paramiko-2.4.1-py2.py3-none-any.whl
(194kB)

Collecting nose>=1.3.3 (from Marvin==4.11.1.0)

  Downloading
https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
(154kB)

Collecting ddt>=0.4.0 (from Marvin==4.11.1.0)

  Downloading
https://files.pythonhosted.org/packages/54/eb/b39eec5f24414cb5d7393ed9cb1bafac740d005846019473d6fc8df18db2/ddt-1.1.3-py2.py3-none-any.whl

Collecting pyvmomi>=5.5.0 (from Marvin==4.11.1.0)

  Downloading
https://files.pythonhosted.org/packages/ba/45/d6e4a87004f1c87bdee2942a8896289684e660dbd76e868047d3319b245f/pyvmomi-6.7.0-py2.py3-none-any.whl
(249kB)

Collecting netaddr>=0.7.14 (from Marvin==4.11.1.0)

  Downloading
https://files.pythonhosted.org/packages/ba/97/ce14451a9fd7bdb5a397abf99b24a1a6bb7a1a440b019bebd2e9a0dbec74/netaddr-0.7.19-py2.py3-none-any.whl
(1.6MB)

Collecting dnspython (from Marvin==4.11.1.0)

  Downloading
https://files.pythonhosted.org/packages/a6/72/209e18bdfedfd78c6994e9ec96981624a5ad7738524dd474237268422cb8/dnspython-1.15.0-py2.py3-none-any.whl
(177kB)

Collecting ipmisim>=0.7 (from Marvin==4.11.1.0)

  Downloading
https://files.pythonhosted.org/packages/58/95/6acd215ec4eaa523b1bfd3b9e16f1defaaf03717a2ed7193077ecf96fa7e/ipmisim-0.7.tar.gz

Collecting protobuf>=3.0.0 (from
mysql-connector-python>=1.1.6->Marvin==4.11.1.0)

  Downloading
https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
(6.4MB)

Collecting urllib3<1.23,>=1.21.1 (from requests>=2.2.1->Marvin==4.11.1.0)

  Downloading
https://files.pythonhosted.org/packages/63/cb/6965947c13a94236f6d4b8223e21beb4d576dc72e8130bd7880f600839b8/urllib3-1.22-py2.py3-none-any.whl
(132kB)

Collecting idna<2.7,>=2.5 (from requests>=2.2.1->Marvin==4.11.1.0)

  Downloading
https://files.pythonhosted.org/packages/27/cc/6dd9a3869f15c2edfab863b992838277279ce92663d334df9ecf5106f5c6/idna-2.6-py2.py3-none-any.whl
(56kB)

Collecting chardet<3.1.0,>=3.0.2 (from requests>=2.2.1->Marvin==4.11.1.0)

  Downloading
https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
(133kB)

Collecting certifi>=2017.4.17 (from requests>=2.2.1->Marvin==4.11.1.0)

  Downloading
https://files.pythonhosted.org/packages/7c/e6/92ad559b7192d846975fc916b65f667c7b8c3a32bea7372340bfe9a15fa5/certifi-2018.4.16-py2.py3-none-any.whl
(150kB)

Collecting cryptography>=1.5 (from paramiko>=1.13.0->Marvin==4.11.1.0)

  Downloading

Re: [PROPOSE] Backup and Recovery Framework

[Will Stevens]

Thanks for clarifying Nicolas.  I just wanted to make sure that there
wasn't any confusion in this conversation as a 'snapshot' has different
meanings to different people/vendors, so I wanted to make sure we were all
on the same page.

BTW, for what it is worth, I love this initiative.  This is an important
feature integration which I know a LOT of organizations will appreciate.

Cheers,

*Will Stevens*
Chief Technology Officer
c 514.826.0190




On Thu, Jun 7, 2018 at 12:57 PM Nicolas Vazquez <
nicolas.vazq...@shapeblue.com> wrote:

> First of all, the framework is agnostic of any backup vendor/provider. It
> just provides the way to communicate with the provider to allow users to
> backup their VMs.
>
>
> Will is right, backups are not the same as snapshots, I got confused with
> the Veeam backup which starts from taking a snapshot to generate the
> backup. The way they are generated are up to each vendor, and it is not
> relevant to the framework. Sorry for the confusion, we are also introducing
> the Veeam plugin in top of this framework.
>
>
> So, in terms of this framework, it does not matter what a backup is, we
> just keep a reference to an existing backup on each vendor.
>
>
> I would also like to state that this framework is not using a snapshot
> mechanism for backups in CloudStack. It is just providing interfaces and
> mechanisms for vendors/providers plugins in CloudStack. Implementation of
> each plugin should be relevant to the vendor and any backup handling will
> be external to CloudStack.
>
>
> I think that the diagram in the FS could explain what the task of the
> framework is:
> https://cwiki.apache.org/confluence/download/attachments/85476500/Backup%20and%20Recovery%20UML.png?version=1=1528367734183=v2.
> The framework simply allows management server to communicate with backup
> and recovery vendors/providers. To allow CloudStack communication with an
> specific vendor, a plugin should be implemented for it (as we are doing
> with Veeam for example).
>
>
> Please let me know if it is still confusing and I'll try to explain better
>
> 
> From: Todd Pigram 
> Sent: Thursday, June 7, 2018 12:11:18 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [PROPOSE] Backup and Recovery Framework
>
> Now outside of our ACS, I use Veeam for vCloud and vSphere and use Alike
> for our XS pools. Inside ACS, the VMs use MaxBackup
>
>
> nicolas.vazq...@shapeblue.com
> www.shapeblue.com
> ,
> @shapeblue
>
>
>
> On Thu, Jun 7, 2018 at 11:09 AM, Todd Pigram  wrote:
>
> > I agree completely. That is why I use inside VM backup products
> currently.
> >
> > On Thu, Jun 7, 2018 at 11:07 AM, Will Stevens 
> > wrote:
> >
> >> I have not been following this thread entirely, but I would not count on
> >> our snapshot mechanism for backups.  If something like Veeam is used,
> then
> >> an agent is in play and that agent will quiesce the running processes,
> >> such
> >> as MySQL.  This is important in order to do backup and DR as you will
> lose
> >> transactions in memory and can potentially have a corrupted backup.
> >>
> >> Snapshots are not backups!!!
> >>
> >> *Will Stevens*
> >> Chief Technology Officer
> >> c 514.826.0190
> >>
> >> 
> >>
> >>
> >> On Thu, Jun 7, 2018 at 10:13 AM Rafael Weingärtner <
> >> rafaelweingart...@gmail.com> wrote:
> >>
> >> > Ok, now I start to understand.  So, we will be handling snapshots and
> >> they
> >> > will be sent to a third-party system.
> >> >
> >> > Now let’s proceed. If I configure this “feature” with external=false.
> >> What
> >> > does it do? Nothing? I mean, what is the purpose of this “external”
> >> > parameter?
> >> >
> >> >
> >> > On Thu, Jun 7, 2018 at 4:09 PM, Nicolas Vazquez <
> >> > nicolas.vazq...@shapeblue.com> wrote:
> >> >
> >> > > Its format may depend on the provider, but under the hood it is a
> >> > snapshot
> >> > > which can be full or incremental and it is copied and stored in a
> >> backup
> >> > > repository in one of the backup servers. For example, if you use
> >> Veeam,
> >> > you
> >> > > will find your backups stored as .vbk (for full backups) and .vib
> (for
> >> > > incremental backups). It is the task of the backup provider to
> create
> >> and
> >> > > move these files to the backup repository.
> >> > >
> >> > > 
> >> > > From: Rafael Weingärtner 
> >> > > Sent: Thursday, June 7, 2018 10:39:35 AM
> >> > > To: dev
> >> > > Subject: Re: [PROPOSE] Backup and Recovery Framework
> >> > >
> >> > > Ok, there are a lot of things, which are kind of blurry to me. So,
> >> let’s
> >> > > try to go step by step.
> >> > >
> >> > > Let me ask again. What is a backup of the VM? You mention
> >> restore-point.
> >> > > What is it? Is it a snapshot in the hypervisor (complete/partial
> >> > snapshot)?
> >> > > If it is not a snapshot,  what is it then? Are you installing some
> >> agent
> >> > in
> >> > > the VM and then copying/mirroring the 

Re: {ANNOUNCE] 4.11.1 RC2 cut

[Rohit Yadav]

There are few PRs in pipeline, let's try to review/merge them tomorrow. Then 
RC2 can be cut either by tomorrow Eod or on Monday.

Regards.

Get Outlook for Android


From: Rafael Weingärtner 
Sent: Thursday, June 7, 2018 8:30:32 PM
To: dev
Subject: Re: {ANNOUNCE] 4.11.1 RC2 cut

Sure let's do that, but first, let's merge these PRs we have been
discussing today.

PS. I thought this was an e-mail announcing the RC2, and not an inquiry one


On Wed, Jun 6, 2018 at 4:16 PM, Daan Hoogland 
wrote:

> Devs,
>
> We have a number of issues out for milestone 4.11.1 but non of them are
> blockers asaik. So unless there are any i don't know of, lets cut RC2
> tomorrow.
>
> ​any opinions?​
>
> --
> Daan
>



--
Rafael Weingärtner

rohit.ya...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 

Re: [PROPOSE] Backup and Recovery Framework

[Nicolas Vazquez]

First of all, the framework is agnostic of any backup vendor/provider. It just 
provides the way to communicate with the provider to allow users to backup 
their VMs.


Will is right, backups are not the same as snapshots, I got confused with the 
Veeam backup which starts from taking a snapshot to generate the backup. The 
way they are generated are up to each vendor, and it is not relevant to the 
framework. Sorry for the confusion, we are also introducing the Veeam plugin in 
top of this framework.


So, in terms of this framework, it does not matter what a backup is, we just 
keep a reference to an existing backup on each vendor.


I would also like to state that this framework is not using a snapshot 
mechanism for backups in CloudStack. It is just providing interfaces and 
mechanisms for vendors/providers plugins in CloudStack. Implementation of each 
plugin should be relevant to the vendor and any backup handling will be 
external to CloudStack.


I think that the diagram in the FS could explain what the task of the framework 
is: 
https://cwiki.apache.org/confluence/download/attachments/85476500/Backup%20and%20Recovery%20UML.png?version=1=1528367734183=v2.
 The framework simply allows management server to communicate with backup and 
recovery vendors/providers. To allow CloudStack communication with an specific 
vendor, a plugin should be implemented for it (as we are doing with Veeam for 
example).


Please let me know if it is still confusing and I'll try to explain better


From: Todd Pigram 
Sent: Thursday, June 7, 2018 12:11:18 PM
To: dev@cloudstack.apache.org
Subject: Re: [PROPOSE] Backup and Recovery Framework

Now outside of our ACS, I use Veeam for vCloud and vSphere and use Alike
for our XS pools. Inside ACS, the VMs use MaxBackup


nicolas.vazq...@shapeblue.com 
www.shapeblue.com
,   
@shapeblue
  
 

On Thu, Jun 7, 2018 at 11:09 AM, Todd Pigram  wrote:

> I agree completely. That is why I use inside VM backup products currently.
>
> On Thu, Jun 7, 2018 at 11:07 AM, Will Stevens 
> wrote:
>
>> I have not been following this thread entirely, but I would not count on
>> our snapshot mechanism for backups.  If something like Veeam is used, then
>> an agent is in play and that agent will quiesce the running processes,
>> such
>> as MySQL.  This is important in order to do backup and DR as you will lose
>> transactions in memory and can potentially have a corrupted backup.
>>
>> Snapshots are not backups!!!
>>
>> *Will Stevens*
>> Chief Technology Officer
>> c 514.826.0190
>>
>> 
>>
>>
>> On Thu, Jun 7, 2018 at 10:13 AM Rafael Weingärtner <
>> rafaelweingart...@gmail.com> wrote:
>>
>> > Ok, now I start to understand.  So, we will be handling snapshots and
>> they
>> > will be sent to a third-party system.
>> >
>> > Now let’s proceed. If I configure this “feature” with external=false.
>> What
>> > does it do? Nothing? I mean, what is the purpose of this “external”
>> > parameter?
>> >
>> >
>> > On Thu, Jun 7, 2018 at 4:09 PM, Nicolas Vazquez <
>> > nicolas.vazq...@shapeblue.com> wrote:
>> >
>> > > Its format may depend on the provider, but under the hood it is a
>> > snapshot
>> > > which can be full or incremental and it is copied and stored in a
>> backup
>> > > repository in one of the backup servers. For example, if you use
>> Veeam,
>> > you
>> > > will find your backups stored as .vbk (for full backups) and .vib (for
>> > > incremental backups). It is the task of the backup provider to create
>> and
>> > > move these files to the backup repository.
>> > >
>> > > 
>> > > From: Rafael Weingärtner 
>> > > Sent: Thursday, June 7, 2018 10:39:35 AM
>> > > To: dev
>> > > Subject: Re: [PROPOSE] Backup and Recovery Framework
>> > >
>> > > Ok, there are a lot of things, which are kind of blurry to me. So,
>> let’s
>> > > try to go step by step.
>> > >
>> > > Let me ask again. What is a backup of the VM? You mention
>> restore-point.
>> > > What is it? Is it a snapshot in the hypervisor (complete/partial
>> > snapshot)?
>> > > If it is not a snapshot,  what is it then? Are you installing some
>> agent
>> > in
>> > > the VM and then copying/mirroring the files in the VM’s file system?
>> > >
>> > > On Thu, Jun 7, 2018 at 3:26 PM, Nicolas Vazquez <
>> > > nicolas.vazq...@shapeblue.com> wrote:
>> > >
>> > > > Hi Daniel and Rafael,
>> > > >
>> > > >
>> > > > I have updated the diagram picture on the FS, please let me know if
>> the
>> > > > problem persists.
>> > > >
>> > > >
>> > > > @Daniel you are right
>> about
>> > > > restoring VMs, please let me update the FS. Regarding your
>> question, we
>> > > > have considered the backups as a first class citizens, meaning that
>> a
>> > > > backup would continue existing unless its originating VM has been
>> > > > removed/expunged. This way, listBackups API method would list them
>> in
>> > > order
>> > > > to allow a user to 

Re: [PROPOSE] Backup and Recovery Framework

[Will Stevens]

Yes, this is important to note as most people don't understand this
detail.  It is VERY important to this discussion though.

*Will Stevens*
Chief Technology Officer
c 514.826.0190




On Thu, Jun 7, 2018 at 11:10 AM Todd Pigram  wrote:

> I agree completely. That is why I use inside VM backup products currently.
>
> On Thu, Jun 7, 2018 at 11:07 AM, Will Stevens 
> wrote:
>
> > I have not been following this thread entirely, but I would not count on
> > our snapshot mechanism for backups.  If something like Veeam is used,
> then
> > an agent is in play and that agent will quiesce the running processes,
> such
> > as MySQL.  This is important in order to do backup and DR as you will
> lose
> > transactions in memory and can potentially have a corrupted backup.
> >
> > Snapshots are not backups!!!
> >
> > *Will Stevens*
> > Chief Technology Officer
> > c 514.826.0190
> >
> > 
> >
> >
> > On Thu, Jun 7, 2018 at 10:13 AM Rafael Weingärtner <
> > rafaelweingart...@gmail.com> wrote:
> >
> > > Ok, now I start to understand.  So, we will be handling snapshots and
> > they
> > > will be sent to a third-party system.
> > >
> > > Now let’s proceed. If I configure this “feature” with external=false.
> > What
> > > does it do? Nothing? I mean, what is the purpose of this “external”
> > > parameter?
> > >
> > >
> > > On Thu, Jun 7, 2018 at 4:09 PM, Nicolas Vazquez <
> > > nicolas.vazq...@shapeblue.com> wrote:
> > >
> > > > Its format may depend on the provider, but under the hood it is a
> > > snapshot
> > > > which can be full or incremental and it is copied and stored in a
> > backup
> > > > repository in one of the backup servers. For example, if you use
> Veeam,
> > > you
> > > > will find your backups stored as .vbk (for full backups) and .vib
> (for
> > > > incremental backups). It is the task of the backup provider to create
> > and
> > > > move these files to the backup repository.
> > > >
> > > > 
> > > > From: Rafael Weingärtner 
> > > > Sent: Thursday, June 7, 2018 10:39:35 AM
> > > > To: dev
> > > > Subject: Re: [PROPOSE] Backup and Recovery Framework
> > > >
> > > > Ok, there are a lot of things, which are kind of blurry to me. So,
> > let’s
> > > > try to go step by step.
> > > >
> > > > Let me ask again. What is a backup of the VM? You mention
> > restore-point.
> > > > What is it? Is it a snapshot in the hypervisor (complete/partial
> > > snapshot)?
> > > > If it is not a snapshot,  what is it then? Are you installing some
> > agent
> > > in
> > > > the VM and then copying/mirroring the files in the VM’s file system?
> > > >
> > > > On Thu, Jun 7, 2018 at 3:26 PM, Nicolas Vazquez <
> > > > nicolas.vazq...@shapeblue.com> wrote:
> > > >
> > > > > Hi Daniel and Rafael,
> > > > >
> > > > >
> > > > > I have updated the diagram picture on the FS, please let me know if
> > the
> > > > > problem persists.
> > > > >
> > > > >
> > > > > @Daniel you are right
> about
> > > > > restoring VMs, please let me update the FS. Regarding your
> question,
> > we
> > > > > have considered the backups as a first class citizens, meaning
> that a
> > > > > backup would continue existing unless its originating VM has been
> > > > > removed/expunged. This way, listBackups API method would list them
> in
> > > > order
> > > > > to allow a user to restore a deleted VM from a backup.
> > > > >
> > > > >
> > > > > @Rafael the idea of the Dummy provider is simply returning the
> > expected
> > > > > entities from a backup provider. For example, the BackupManager
> could
> > > ask
> > > > > the BackupProvider to list its backups and would expect a list of
> > > > backups.
> > > > > As the Dummy Provider implements the BackupProvider interface, it
> has
> > > its
> > > > > own implementation of listBackups() method and simply returns some
> > mock
> > > > > entities to the Backup Manager. It does not store any backup, it
> > simply
> > > > > creates them in execution time and returns them.
> > > > >
> > > > >
> > > > > From the CloudStack point of view, Backups are stored externally,
> on
> > > the
> > > > > Backup and Recovery provider servers and are handled by them. We
> keep
> > > > track
> > > > > of the existing backups in the Backup provider. Implementation may
> > > depend
> > > > > on the Backup and Recovery provider, would consist on snapshots,
> but
> > > are
> > > > > handled by the provider and not by CloudStack.
> > > > >
> > > > >
> > > > > Backup policies are created on the Backup and Recovery provider and
> > > > > provide retention options and periods as well as backup times. For
> > > > example,
> > > > > the provider may expose Golden, Silver and Bronze policies and have
> > > > > different options for each of them.
> > > > >
> > > > >
> > > > > The 'external' parameter on the listBackupPolicies API method would
> > > only
> > > > > ask the Backup and Recovery provider to list their policies if it
> 

Re: [VOTE] Apache CloudStack 4.11.1.0 LTS [RC1]

[Gabriel Beims Bräscher]

I am testing 4.11.1.0 LTS [RC1] and found a bug, reported it on #2702 
(https://github.com/apache/cloudstack/issues/2702)

> On June 4, 2018 at 6:04 PM Rafael Weingärtner wrote:
> 
> 
> Would you mind using Github issues instead of Jira?
> 
> On Mon, Jun 4, 2018 at 1:03 PM, Frank Maximus <
> frank.maxi...@nuagenetworks.net mailto:frank.maxi...@nuagenetworks.net > 
> wrote:
> 
> > > I created CLOUDSTACK-10381
> > 
> > 
> > On Mon, Jun 4, 2018 at 2:10 PM Frank Maximus <
> > frank.maxi...@nuagenetworks.net 
> > mailto:frank.maxi...@nuagenetworks.net > wrote:
> > 
> > > > > I know that is the expected behavior.
> > > We stop the VM, do password reset, start the VM, check the 
> > > config drive
> > > disk: no password file.
> > > When starting the VM, Cloudstack doesn't seem to remember the 
> > > fact the
> > > password reset was executed.
> > > A strategy similar to what is done on VRElement in case VR 
> > > isn't running
> > > might be needed.
> > > 
> > > 
> > > On Mon, Jun 4, 2018 at 1:42 PM Rohit Yadav
> > > wrote:
> > > 
> > > > > > > Frank - wrt config drive, the reset password API 
> > > would require the VM is
> > > > stopped. After VM is stopped, reset password and start 
> > > > the VM, the new
> > > > password should be put in the config drive iso:
> > > > 
> > > > 
> > > > http://cloudstack.apache.org/api/apidocs-4.11/apis/
> > > > 
> > > > > > > 
> > > > > resetPasswordForVirtualMachine.html
> > >>
> > >> zx
> > >>
> > >>
> > >> The previous implementation would not require vm to be stopped 
> > per ap,
> > >> and instead do a detach+update iso + attach operation which is
> > fail-prone
> > >> and may not work for all cases/VMs.
> > >>
> > >>
> > >> - Rohit
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> 
> > >> From: Frank Maximus
> > >> Sent: Monday, June 4, 2018 3:47:11 PM
> > >> To: dev@cloudstack.apache mailto:dev@cloudstack.apache .org
> > >> Cc: users@cloudstack.apache mailto:users@cloudstack.apache .org
> > >> Subject: Re: [VOTE] Apache CloudStack 4.11.1.0 LTS [RC1]
> > >>
> > >> -1.
> > >>
> > >> Reset password with ConfigDrive seems to broken.
> > >> The old password stays in use.
> > >> Will make a ticket soon.
> > >> Also the example scripts in setup/bindir need to be changed.
> > >>
> > >>
> > >> On Mon, May 28, 2018 at 10:26 AM Daan Hoogland 
> > mailto:daan.hoogl...@gmail.com
> > >
> > >> wrote:
> > >>
> > >> > I checked three verification files, unpacked and build the 
> > code, and
> > am
> > >> > trusting the process otherwise:
> > >> > 0 (binding)
> > >> > The reason I am not giving a +1 is because the code presently 
> > does not
> > >> > build on anything but linux, due to the configdrive test not 
> > building
> > on
> > >> > macosx (and I presume windows) If we add that to release notes 
> > I am
> > fine
> > >> > with it.
> > >> >
> > >> > On Sat, May 26, 2018 at 5:27 AM, Tutkowski, Mike <
> > >> > mike.tutkow...@netapp.com mailto:mike.tutkow...@netapp.com >
> > >> > wrote:
> > >> >
> > >> > > +1 (binding)
> > >> > >
> > >> > > I created a new cloud using commit
> > >> > 5f48487dc62fd1decaabc4ab2a10f549d6c82400
> > >> > > (RC1). I ran the automated regression tests for managed 
> > storage. All
> > >> > tests
> > >> > > passed.
> > >> > >
> > >> > > On 5/24/18, 9:56 AM, "Paul Angus" wrote:
> > >> > >
> > >> > > Hi All,
> > >> > >
> > >> > >
> > >> > >
> > >> > > I've created a 4.11.1.0 release (RC1), with the following
> > >> artefacts
> > >> > up
> > >> > > for testing and a vote:
> > >> > >
> > >> > > [NB we know there are issues for Nuage to sort in this RC, 
> > but
> > >> they
> > >> > > will be well contained, so let’s test everything else  ]
> > >> > >
> > >> > >
> > >> > >
> > >> > > Git Branch and Commit SH:
> > >> > >
> > >> > > https://gitbox.apache.org/repos/asf?p=cloudstack.git;a=
> > >> > > shortlog;h=refs/heads/4.11.1.0-RC20180524T1000
> > >> > >
> > >> > > Commit: 5f48487dc62fd1decaabc4ab2a10f549d6c82400
> > >> > >
> > >> > >
> > >> > >
> > >> > > Source release (checksums and signatures are available at the
> > same
> > 

Re: [PROPOSE] Backup and Recovery Framework

[Todd Pigram]

Now outside of our ACS, I use Veeam for vCloud and vSphere and use Alike
for our XS pools. Inside ACS, the VMs use MaxBackup

On Thu, Jun 7, 2018 at 11:09 AM, Todd Pigram  wrote:

> I agree completely. That is why I use inside VM backup products currently.
>
> On Thu, Jun 7, 2018 at 11:07 AM, Will Stevens 
> wrote:
>
>> I have not been following this thread entirely, but I would not count on
>> our snapshot mechanism for backups.  If something like Veeam is used, then
>> an agent is in play and that agent will quiesce the running processes,
>> such
>> as MySQL.  This is important in order to do backup and DR as you will lose
>> transactions in memory and can potentially have a corrupted backup.
>>
>> Snapshots are not backups!!!
>>
>> *Will Stevens*
>> Chief Technology Officer
>> c 514.826.0190
>>
>> 
>>
>>
>> On Thu, Jun 7, 2018 at 10:13 AM Rafael Weingärtner <
>> rafaelweingart...@gmail.com> wrote:
>>
>> > Ok, now I start to understand.  So, we will be handling snapshots and
>> they
>> > will be sent to a third-party system.
>> >
>> > Now let’s proceed. If I configure this “feature” with external=false.
>> What
>> > does it do? Nothing? I mean, what is the purpose of this “external”
>> > parameter?
>> >
>> >
>> > On Thu, Jun 7, 2018 at 4:09 PM, Nicolas Vazquez <
>> > nicolas.vazq...@shapeblue.com> wrote:
>> >
>> > > Its format may depend on the provider, but under the hood it is a
>> > snapshot
>> > > which can be full or incremental and it is copied and stored in a
>> backup
>> > > repository in one of the backup servers. For example, if you use
>> Veeam,
>> > you
>> > > will find your backups stored as .vbk (for full backups) and .vib (for
>> > > incremental backups). It is the task of the backup provider to create
>> and
>> > > move these files to the backup repository.
>> > >
>> > > 
>> > > From: Rafael Weingärtner 
>> > > Sent: Thursday, June 7, 2018 10:39:35 AM
>> > > To: dev
>> > > Subject: Re: [PROPOSE] Backup and Recovery Framework
>> > >
>> > > Ok, there are a lot of things, which are kind of blurry to me. So,
>> let’s
>> > > try to go step by step.
>> > >
>> > > Let me ask again. What is a backup of the VM? You mention
>> restore-point.
>> > > What is it? Is it a snapshot in the hypervisor (complete/partial
>> > snapshot)?
>> > > If it is not a snapshot,  what is it then? Are you installing some
>> agent
>> > in
>> > > the VM and then copying/mirroring the files in the VM’s file system?
>> > >
>> > > On Thu, Jun 7, 2018 at 3:26 PM, Nicolas Vazquez <
>> > > nicolas.vazq...@shapeblue.com> wrote:
>> > >
>> > > > Hi Daniel and Rafael,
>> > > >
>> > > >
>> > > > I have updated the diagram picture on the FS, please let me know if
>> the
>> > > > problem persists.
>> > > >
>> > > >
>> > > > @Daniel you are right
>> about
>> > > > restoring VMs, please let me update the FS. Regarding your
>> question, we
>> > > > have considered the backups as a first class citizens, meaning that
>> a
>> > > > backup would continue existing unless its originating VM has been
>> > > > removed/expunged. This way, listBackups API method would list them
>> in
>> > > order
>> > > > to allow a user to restore a deleted VM from a backup.
>> > > >
>> > > >
>> > > > @Rafael the idea of the Dummy provider is simply returning the
>> expected
>> > > > entities from a backup provider. For example, the BackupManager
>> could
>> > ask
>> > > > the BackupProvider to list its backups and would expect a list of
>> > > backups.
>> > > > As the Dummy Provider implements the BackupProvider interface, it
>> has
>> > its
>> > > > own implementation of listBackups() method and simply returns some
>> mock
>> > > > entities to the Backup Manager. It does not store any backup, it
>> simply
>> > > > creates them in execution time and returns them.
>> > > >
>> > > >
>> > > > From the CloudStack point of view, Backups are stored externally, on
>> > the
>> > > > Backup and Recovery provider servers and are handled by them. We
>> keep
>> > > track
>> > > > of the existing backups in the Backup provider. Implementation may
>> > depend
>> > > > on the Backup and Recovery provider, would consist on snapshots, but
>> > are
>> > > > handled by the provider and not by CloudStack.
>> > > >
>> > > >
>> > > > Backup policies are created on the Backup and Recovery provider and
>> > > > provide retention options and periods as well as backup times. For
>> > > example,
>> > > > the provider may expose Golden, Silver and Bronze policies and have
>> > > > different options for each of them.
>> > > >
>> > > >
>> > > > The 'external' parameter on the listBackupPolicies API method would
>> > only
>> > > > ask the Backup and Recovery provider to list their policies if it is
>> > set
>> > > to
>> > > > true. If it is not set, or set to false, it just simply returns the
>> > > > existing mapped policies in CloudStack. Initially, CloudStack does
>> not
>> > > > store any 

Re: [PROPOSE] Backup and Recovery Framework

[Todd Pigram]

I agree completely. That is why I use inside VM backup products currently.

On Thu, Jun 7, 2018 at 11:07 AM, Will Stevens  wrote:

> I have not been following this thread entirely, but I would not count on
> our snapshot mechanism for backups.  If something like Veeam is used, then
> an agent is in play and that agent will quiesce the running processes, such
> as MySQL.  This is important in order to do backup and DR as you will lose
> transactions in memory and can potentially have a corrupted backup.
>
> Snapshots are not backups!!!
>
> *Will Stevens*
> Chief Technology Officer
> c 514.826.0190
>
> 
>
>
> On Thu, Jun 7, 2018 at 10:13 AM Rafael Weingärtner <
> rafaelweingart...@gmail.com> wrote:
>
> > Ok, now I start to understand.  So, we will be handling snapshots and
> they
> > will be sent to a third-party system.
> >
> > Now let’s proceed. If I configure this “feature” with external=false.
> What
> > does it do? Nothing? I mean, what is the purpose of this “external”
> > parameter?
> >
> >
> > On Thu, Jun 7, 2018 at 4:09 PM, Nicolas Vazquez <
> > nicolas.vazq...@shapeblue.com> wrote:
> >
> > > Its format may depend on the provider, but under the hood it is a
> > snapshot
> > > which can be full or incremental and it is copied and stored in a
> backup
> > > repository in one of the backup servers. For example, if you use Veeam,
> > you
> > > will find your backups stored as .vbk (for full backups) and .vib (for
> > > incremental backups). It is the task of the backup provider to create
> and
> > > move these files to the backup repository.
> > >
> > > 
> > > From: Rafael Weingärtner 
> > > Sent: Thursday, June 7, 2018 10:39:35 AM
> > > To: dev
> > > Subject: Re: [PROPOSE] Backup and Recovery Framework
> > >
> > > Ok, there are a lot of things, which are kind of blurry to me. So,
> let’s
> > > try to go step by step.
> > >
> > > Let me ask again. What is a backup of the VM? You mention
> restore-point.
> > > What is it? Is it a snapshot in the hypervisor (complete/partial
> > snapshot)?
> > > If it is not a snapshot,  what is it then? Are you installing some
> agent
> > in
> > > the VM and then copying/mirroring the files in the VM’s file system?
> > >
> > > On Thu, Jun 7, 2018 at 3:26 PM, Nicolas Vazquez <
> > > nicolas.vazq...@shapeblue.com> wrote:
> > >
> > > > Hi Daniel and Rafael,
> > > >
> > > >
> > > > I have updated the diagram picture on the FS, please let me know if
> the
> > > > problem persists.
> > > >
> > > >
> > > > @Daniel you are right about
> > > > restoring VMs, please let me update the FS. Regarding your question,
> we
> > > > have considered the backups as a first class citizens, meaning that a
> > > > backup would continue existing unless its originating VM has been
> > > > removed/expunged. This way, listBackups API method would list them in
> > > order
> > > > to allow a user to restore a deleted VM from a backup.
> > > >
> > > >
> > > > @Rafael the idea of the Dummy provider is simply returning the
> expected
> > > > entities from a backup provider. For example, the BackupManager could
> > ask
> > > > the BackupProvider to list its backups and would expect a list of
> > > backups.
> > > > As the Dummy Provider implements the BackupProvider interface, it has
> > its
> > > > own implementation of listBackups() method and simply returns some
> mock
> > > > entities to the Backup Manager. It does not store any backup, it
> simply
> > > > creates them in execution time and returns them.
> > > >
> > > >
> > > > From the CloudStack point of view, Backups are stored externally, on
> > the
> > > > Backup and Recovery provider servers and are handled by them. We keep
> > > track
> > > > of the existing backups in the Backup provider. Implementation may
> > depend
> > > > on the Backup and Recovery provider, would consist on snapshots, but
> > are
> > > > handled by the provider and not by CloudStack.
> > > >
> > > >
> > > > Backup policies are created on the Backup and Recovery provider and
> > > > provide retention options and periods as well as backup times. For
> > > example,
> > > > the provider may expose Golden, Silver and Bronze policies and have
> > > > different options for each of them.
> > > >
> > > >
> > > > The 'external' parameter on the listBackupPolicies API method would
> > only
> > > > ask the Backup and Recovery provider to list their policies if it is
> > set
> > > to
> > > > true. If it is not set, or set to false, it just simply returns the
> > > > existing mapped policies in CloudStack. Initially, CloudStack does
> not
> > > > store any backup policy, those should be imported/mapped by the
> > > > 'importBackupPolicy' API method.
> > > >
> > > >
> > > > Finally, we consider backups as restore points. As mentioned above,
> > > > backups are stored on the Backup and Recovery provider servers, so we
> > > keep
> > > > a mapping between the CloudStack entity and the backup, 

Re: [PROPOSE] Backup and Recovery Framework

[Will Stevens]

I have not been following this thread entirely, but I would not count on
our snapshot mechanism for backups.  If something like Veeam is used, then
an agent is in play and that agent will quiesce the running processes, such
as MySQL.  This is important in order to do backup and DR as you will lose
transactions in memory and can potentially have a corrupted backup.

Snapshots are not backups!!!

*Will Stevens*
Chief Technology Officer
c 514.826.0190




On Thu, Jun 7, 2018 at 10:13 AM Rafael Weingärtner <
rafaelweingart...@gmail.com> wrote:

> Ok, now I start to understand.  So, we will be handling snapshots and they
> will be sent to a third-party system.
>
> Now let’s proceed. If I configure this “feature” with external=false. What
> does it do? Nothing? I mean, what is the purpose of this “external”
> parameter?
>
>
> On Thu, Jun 7, 2018 at 4:09 PM, Nicolas Vazquez <
> nicolas.vazq...@shapeblue.com> wrote:
>
> > Its format may depend on the provider, but under the hood it is a
> snapshot
> > which can be full or incremental and it is copied and stored in a backup
> > repository in one of the backup servers. For example, if you use Veeam,
> you
> > will find your backups stored as .vbk (for full backups) and .vib (for
> > incremental backups). It is the task of the backup provider to create and
> > move these files to the backup repository.
> >
> > 
> > From: Rafael Weingärtner 
> > Sent: Thursday, June 7, 2018 10:39:35 AM
> > To: dev
> > Subject: Re: [PROPOSE] Backup and Recovery Framework
> >
> > Ok, there are a lot of things, which are kind of blurry to me. So, let’s
> > try to go step by step.
> >
> > Let me ask again. What is a backup of the VM? You mention restore-point.
> > What is it? Is it a snapshot in the hypervisor (complete/partial
> snapshot)?
> > If it is not a snapshot,  what is it then? Are you installing some agent
> in
> > the VM and then copying/mirroring the files in the VM’s file system?
> >
> > On Thu, Jun 7, 2018 at 3:26 PM, Nicolas Vazquez <
> > nicolas.vazq...@shapeblue.com> wrote:
> >
> > > Hi Daniel and Rafael,
> > >
> > >
> > > I have updated the diagram picture on the FS, please let me know if the
> > > problem persists.
> > >
> > >
> > > @Daniel you are right about
> > > restoring VMs, please let me update the FS. Regarding your question, we
> > > have considered the backups as a first class citizens, meaning that a
> > > backup would continue existing unless its originating VM has been
> > > removed/expunged. This way, listBackups API method would list them in
> > order
> > > to allow a user to restore a deleted VM from a backup.
> > >
> > >
> > > @Rafael the idea of the Dummy provider is simply returning the expected
> > > entities from a backup provider. For example, the BackupManager could
> ask
> > > the BackupProvider to list its backups and would expect a list of
> > backups.
> > > As the Dummy Provider implements the BackupProvider interface, it has
> its
> > > own implementation of listBackups() method and simply returns some mock
> > > entities to the Backup Manager. It does not store any backup, it simply
> > > creates them in execution time and returns them.
> > >
> > >
> > > From the CloudStack point of view, Backups are stored externally, on
> the
> > > Backup and Recovery provider servers and are handled by them. We keep
> > track
> > > of the existing backups in the Backup provider. Implementation may
> depend
> > > on the Backup and Recovery provider, would consist on snapshots, but
> are
> > > handled by the provider and not by CloudStack.
> > >
> > >
> > > Backup policies are created on the Backup and Recovery provider and
> > > provide retention options and periods as well as backup times. For
> > example,
> > > the provider may expose Golden, Silver and Bronze policies and have
> > > different options for each of them.
> > >
> > >
> > > The 'external' parameter on the listBackupPolicies API method would
> only
> > > ask the Backup and Recovery provider to list their policies if it is
> set
> > to
> > > true. If it is not set, or set to false, it just simply returns the
> > > existing mapped policies in CloudStack. Initially, CloudStack does not
> > > store any backup policy, those should be imported/mapped by the
> > > 'importBackupPolicy' API method.
> > >
> > >
> > > Finally, we consider backups as restore points. As mentioned above,
> > > backups are stored on the Backup and Recovery provider servers, so we
> > keep
> > > a mapping between the CloudStack entity and the backup, which can be
> used
> > > to restore VMs.
> > >
> > >
> > > Please let me know if anything.
> > >
> > >
> > > Regards,
> > >
> > > Nicolas Vazquez
> > >
> > > 
> > > From: Rafael Weingärtner 
> > > Sent: Thursday, June 7, 2018 6:55:57 AM
> > > To: dev
> > > Subject: Re: [PROPOSE] Backup and Recovery Framework
> > >
> > > Hi Nicolas,
> > >
> > > The 

Re: [PROPOSE] Backup and Recovery Framework

[Todd Pigram]

Rafael,

I would agree with that as well as how Quadric spins up a ABD at time of
backup. I have tried both Alike and Unitrends (old PHD virtual) and ACS
removed the device before it works, but that was back on v4.3.



On Thu, Jun 7, 2018 at 11:03 AM, Rafael Weingärtner <
rafaelweingart...@gmail.com> wrote:

> I believe the framework is going to be agnostic, but we need to implement
> "driver" to deal with different vendors API, right?
>
> On Thu, Jun 7, 2018 at 5:01 PM, Todd Pigram  wrote:
>
> > Nicolas,
> >
> > Given you Veeam example, this framework is backup vendor independent? So
> I
> > could use Veeam for vSphere or Hyper-V or use Quadric (Alike) for
> XenServer
> > or Hyper-V. This would sure beat the way I am doing it now, inside the
> VM.
> >
> > I will be following this closely.
> >
> >
> >
> > On Thu, Jun 7, 2018 at 10:13 AM, Rafael Weingärtner <
> > rafaelweingart...@gmail.com> wrote:
> >
> > > Ok, now I start to understand.  So, we will be handling snapshots and
> > they
> > > will be sent to a third-party system.
> > >
> > > Now let’s proceed. If I configure this “feature” with external=false.
> > What
> > > does it do? Nothing? I mean, what is the purpose of this “external”
> > > parameter?
> > >
> > >
> > > On Thu, Jun 7, 2018 at 4:09 PM, Nicolas Vazquez <
> > > nicolas.vazq...@shapeblue.com> wrote:
> > >
> > > > Its format may depend on the provider, but under the hood it is a
> > > snapshot
> > > > which can be full or incremental and it is copied and stored in a
> > backup
> > > > repository in one of the backup servers. For example, if you use
> Veeam,
> > > you
> > > > will find your backups stored as .vbk (for full backups) and .vib
> (for
> > > > incremental backups). It is the task of the backup provider to create
> > and
> > > > move these files to the backup repository.
> > > >
> > > > 
> > > > From: Rafael Weingärtner 
> > > > Sent: Thursday, June 7, 2018 10:39:35 AM
> > > > To: dev
> > > > Subject: Re: [PROPOSE] Backup and Recovery Framework
> > > >
> > > > Ok, there are a lot of things, which are kind of blurry to me. So,
> > let’s
> > > > try to go step by step.
> > > >
> > > > Let me ask again. What is a backup of the VM? You mention
> > restore-point.
> > > > What is it? Is it a snapshot in the hypervisor (complete/partial
> > > snapshot)?
> > > > If it is not a snapshot,  what is it then? Are you installing some
> > agent
> > > in
> > > > the VM and then copying/mirroring the files in the VM’s file system?
> > > >
> > > > On Thu, Jun 7, 2018 at 3:26 PM, Nicolas Vazquez <
> > > > nicolas.vazq...@shapeblue.com> wrote:
> > > >
> > > > > Hi Daniel and Rafael,
> > > > >
> > > > >
> > > > > I have updated the diagram picture on the FS, please let me know if
> > the
> > > > > problem persists.
> > > > >
> > > > >
> > > > > @Daniel you are right
> about
> > > > > restoring VMs, please let me update the FS. Regarding your
> question,
> > we
> > > > > have considered the backups as a first class citizens, meaning
> that a
> > > > > backup would continue existing unless its originating VM has been
> > > > > removed/expunged. This way, listBackups API method would list them
> in
> > > > order
> > > > > to allow a user to restore a deleted VM from a backup.
> > > > >
> > > > >
> > > > > @Rafael the idea of the Dummy provider is simply returning the
> > expected
> > > > > entities from a backup provider. For example, the BackupManager
> could
> > > ask
> > > > > the BackupProvider to list its backups and would expect a list of
> > > > backups.
> > > > > As the Dummy Provider implements the BackupProvider interface, it
> has
> > > its
> > > > > own implementation of listBackups() method and simply returns some
> > mock
> > > > > entities to the Backup Manager. It does not store any backup, it
> > simply
> > > > > creates them in execution time and returns them.
> > > > >
> > > > >
> > > > > From the CloudStack point of view, Backups are stored externally,
> on
> > > the
> > > > > Backup and Recovery provider servers and are handled by them. We
> keep
> > > > track
> > > > > of the existing backups in the Backup provider. Implementation may
> > > depend
> > > > > on the Backup and Recovery provider, would consist on snapshots,
> but
> > > are
> > > > > handled by the provider and not by CloudStack.
> > > > >
> > > > >
> > > > > Backup policies are created on the Backup and Recovery provider and
> > > > > provide retention options and periods as well as backup times. For
> > > > example,
> > > > > the provider may expose Golden, Silver and Bronze policies and have
> > > > > different options for each of them.
> > > > >
> > > > >
> > > > > The 'external' parameter on the listBackupPolicies API method would
> > > only
> > > > > ask the Backup and Recovery provider to list their policies if it
> is
> > > set
> > > > to
> > > > > true. If it is not set, or set to false, it just simply returns the
> > > > > existing 

Re: [PROPOSE] Backup and Recovery Framework

[Rafael Weingärtner]

I believe the framework is going to be agnostic, but we need to implement
"driver" to deal with different vendors API, right?

On Thu, Jun 7, 2018 at 5:01 PM, Todd Pigram  wrote:

> Nicolas,
>
> Given you Veeam example, this framework is backup vendor independent? So I
> could use Veeam for vSphere or Hyper-V or use Quadric (Alike) for XenServer
> or Hyper-V. This would sure beat the way I am doing it now, inside the VM.
>
> I will be following this closely.
>
>
>
> On Thu, Jun 7, 2018 at 10:13 AM, Rafael Weingärtner <
> rafaelweingart...@gmail.com> wrote:
>
> > Ok, now I start to understand.  So, we will be handling snapshots and
> they
> > will be sent to a third-party system.
> >
> > Now let’s proceed. If I configure this “feature” with external=false.
> What
> > does it do? Nothing? I mean, what is the purpose of this “external”
> > parameter?
> >
> >
> > On Thu, Jun 7, 2018 at 4:09 PM, Nicolas Vazquez <
> > nicolas.vazq...@shapeblue.com> wrote:
> >
> > > Its format may depend on the provider, but under the hood it is a
> > snapshot
> > > which can be full or incremental and it is copied and stored in a
> backup
> > > repository in one of the backup servers. For example, if you use Veeam,
> > you
> > > will find your backups stored as .vbk (for full backups) and .vib (for
> > > incremental backups). It is the task of the backup provider to create
> and
> > > move these files to the backup repository.
> > >
> > > 
> > > From: Rafael Weingärtner 
> > > Sent: Thursday, June 7, 2018 10:39:35 AM
> > > To: dev
> > > Subject: Re: [PROPOSE] Backup and Recovery Framework
> > >
> > > Ok, there are a lot of things, which are kind of blurry to me. So,
> let’s
> > > try to go step by step.
> > >
> > > Let me ask again. What is a backup of the VM? You mention
> restore-point.
> > > What is it? Is it a snapshot in the hypervisor (complete/partial
> > snapshot)?
> > > If it is not a snapshot,  what is it then? Are you installing some
> agent
> > in
> > > the VM and then copying/mirroring the files in the VM’s file system?
> > >
> > > On Thu, Jun 7, 2018 at 3:26 PM, Nicolas Vazquez <
> > > nicolas.vazq...@shapeblue.com> wrote:
> > >
> > > > Hi Daniel and Rafael,
> > > >
> > > >
> > > > I have updated the diagram picture on the FS, please let me know if
> the
> > > > problem persists.
> > > >
> > > >
> > > > @Daniel you are right about
> > > > restoring VMs, please let me update the FS. Regarding your question,
> we
> > > > have considered the backups as a first class citizens, meaning that a
> > > > backup would continue existing unless its originating VM has been
> > > > removed/expunged. This way, listBackups API method would list them in
> > > order
> > > > to allow a user to restore a deleted VM from a backup.
> > > >
> > > >
> > > > @Rafael the idea of the Dummy provider is simply returning the
> expected
> > > > entities from a backup provider. For example, the BackupManager could
> > ask
> > > > the BackupProvider to list its backups and would expect a list of
> > > backups.
> > > > As the Dummy Provider implements the BackupProvider interface, it has
> > its
> > > > own implementation of listBackups() method and simply returns some
> mock
> > > > entities to the Backup Manager. It does not store any backup, it
> simply
> > > > creates them in execution time and returns them.
> > > >
> > > >
> > > > From the CloudStack point of view, Backups are stored externally, on
> > the
> > > > Backup and Recovery provider servers and are handled by them. We keep
> > > track
> > > > of the existing backups in the Backup provider. Implementation may
> > depend
> > > > on the Backup and Recovery provider, would consist on snapshots, but
> > are
> > > > handled by the provider and not by CloudStack.
> > > >
> > > >
> > > > Backup policies are created on the Backup and Recovery provider and
> > > > provide retention options and periods as well as backup times. For
> > > example,
> > > > the provider may expose Golden, Silver and Bronze policies and have
> > > > different options for each of them.
> > > >
> > > >
> > > > The 'external' parameter on the listBackupPolicies API method would
> > only
> > > > ask the Backup and Recovery provider to list their policies if it is
> > set
> > > to
> > > > true. If it is not set, or set to false, it just simply returns the
> > > > existing mapped policies in CloudStack. Initially, CloudStack does
> not
> > > > store any backup policy, those should be imported/mapped by the
> > > > 'importBackupPolicy' API method.
> > > >
> > > >
> > > > Finally, we consider backups as restore points. As mentioned above,
> > > > backups are stored on the Backup and Recovery provider servers, so we
> > > keep
> > > > a mapping between the CloudStack entity and the backup, which can be
> > used
> > > > to restore VMs.
> > > >
> > > >
> > > > Please let me know if anything.
> > > >
> > > >
> > > > Regards,
> > > >
> > > > Nicolas 

Re: [PROPOSE] Backup and Recovery Framework

[Todd Pigram]

Nicolas,

Given you Veeam example, this framework is backup vendor independent? So I
could use Veeam for vSphere or Hyper-V or use Quadric (Alike) for XenServer
or Hyper-V. This would sure beat the way I am doing it now, inside the VM.

I will be following this closely.



On Thu, Jun 7, 2018 at 10:13 AM, Rafael Weingärtner <
rafaelweingart...@gmail.com> wrote:

> Ok, now I start to understand.  So, we will be handling snapshots and they
> will be sent to a third-party system.
>
> Now let’s proceed. If I configure this “feature” with external=false. What
> does it do? Nothing? I mean, what is the purpose of this “external”
> parameter?
>
>
> On Thu, Jun 7, 2018 at 4:09 PM, Nicolas Vazquez <
> nicolas.vazq...@shapeblue.com> wrote:
>
> > Its format may depend on the provider, but under the hood it is a
> snapshot
> > which can be full or incremental and it is copied and stored in a backup
> > repository in one of the backup servers. For example, if you use Veeam,
> you
> > will find your backups stored as .vbk (for full backups) and .vib (for
> > incremental backups). It is the task of the backup provider to create and
> > move these files to the backup repository.
> >
> > 
> > From: Rafael Weingärtner 
> > Sent: Thursday, June 7, 2018 10:39:35 AM
> > To: dev
> > Subject: Re: [PROPOSE] Backup and Recovery Framework
> >
> > Ok, there are a lot of things, which are kind of blurry to me. So, let’s
> > try to go step by step.
> >
> > Let me ask again. What is a backup of the VM? You mention restore-point.
> > What is it? Is it a snapshot in the hypervisor (complete/partial
> snapshot)?
> > If it is not a snapshot,  what is it then? Are you installing some agent
> in
> > the VM and then copying/mirroring the files in the VM’s file system?
> >
> > On Thu, Jun 7, 2018 at 3:26 PM, Nicolas Vazquez <
> > nicolas.vazq...@shapeblue.com> wrote:
> >
> > > Hi Daniel and Rafael,
> > >
> > >
> > > I have updated the diagram picture on the FS, please let me know if the
> > > problem persists.
> > >
> > >
> > > @Daniel you are right about
> > > restoring VMs, please let me update the FS. Regarding your question, we
> > > have considered the backups as a first class citizens, meaning that a
> > > backup would continue existing unless its originating VM has been
> > > removed/expunged. This way, listBackups API method would list them in
> > order
> > > to allow a user to restore a deleted VM from a backup.
> > >
> > >
> > > @Rafael the idea of the Dummy provider is simply returning the expected
> > > entities from a backup provider. For example, the BackupManager could
> ask
> > > the BackupProvider to list its backups and would expect a list of
> > backups.
> > > As the Dummy Provider implements the BackupProvider interface, it has
> its
> > > own implementation of listBackups() method and simply returns some mock
> > > entities to the Backup Manager. It does not store any backup, it simply
> > > creates them in execution time and returns them.
> > >
> > >
> > > From the CloudStack point of view, Backups are stored externally, on
> the
> > > Backup and Recovery provider servers and are handled by them. We keep
> > track
> > > of the existing backups in the Backup provider. Implementation may
> depend
> > > on the Backup and Recovery provider, would consist on snapshots, but
> are
> > > handled by the provider and not by CloudStack.
> > >
> > >
> > > Backup policies are created on the Backup and Recovery provider and
> > > provide retention options and periods as well as backup times. For
> > example,
> > > the provider may expose Golden, Silver and Bronze policies and have
> > > different options for each of them.
> > >
> > >
> > > The 'external' parameter on the listBackupPolicies API method would
> only
> > > ask the Backup and Recovery provider to list their policies if it is
> set
> > to
> > > true. If it is not set, or set to false, it just simply returns the
> > > existing mapped policies in CloudStack. Initially, CloudStack does not
> > > store any backup policy, those should be imported/mapped by the
> > > 'importBackupPolicy' API method.
> > >
> > >
> > > Finally, we consider backups as restore points. As mentioned above,
> > > backups are stored on the Backup and Recovery provider servers, so we
> > keep
> > > a mapping between the CloudStack entity and the backup, which can be
> used
> > > to restore VMs.
> > >
> > >
> > > Please let me know if anything.
> > >
> > >
> > > Regards,
> > >
> > > Nicolas Vazquez
> > >
> > > 
> > > From: Rafael Weingärtner 
> > > Sent: Thursday, June 7, 2018 6:55:57 AM
> > > To: dev
> > > Subject: Re: [PROPOSE] Backup and Recovery Framework
> > >
> > > Hi Nicolas,
> > >
> > > The idea is great. I have some questions though.
> > >
> > > Before getting to the questions, your pictures are broken, or at least
> > they
> > > seem broken to me.
> > >
> > > How would the “dummy” backup 

Re: {ANNOUNCE] 4.11.1 RC2 cut

[Rafael Weingärtner]

Sure let's do that, but first, let's merge these PRs we have been
discussing today.

PS. I thought this was an e-mail announcing the RC2, and not an inquiry one


On Wed, Jun 6, 2018 at 4:16 PM, Daan Hoogland 
wrote:

> Devs,
>
> We have a number of issues out for milestone 4.11.1 but non of them are
> blockers asaik. So unless there are any i don't know of, lets cut RC2
> tomorrow.
>
> ​any opinions?​
>
> --
> Daan
>



-- 
Rafael Weingärtner

Re: [PROPOSE] Backup and Recovery Framework

[Rafael Weingärtner]

Ok, now I start to understand.  So, we will be handling snapshots and they
will be sent to a third-party system.

Now let’s proceed. If I configure this “feature” with external=false. What
does it do? Nothing? I mean, what is the purpose of this “external”
parameter?


On Thu, Jun 7, 2018 at 4:09 PM, Nicolas Vazquez <
nicolas.vazq...@shapeblue.com> wrote:

> Its format may depend on the provider, but under the hood it is a snapshot
> which can be full or incremental and it is copied and stored in a backup
> repository in one of the backup servers. For example, if you use Veeam, you
> will find your backups stored as .vbk (for full backups) and .vib (for
> incremental backups). It is the task of the backup provider to create and
> move these files to the backup repository.
>
> 
> From: Rafael Weingärtner 
> Sent: Thursday, June 7, 2018 10:39:35 AM
> To: dev
> Subject: Re: [PROPOSE] Backup and Recovery Framework
>
> Ok, there are a lot of things, which are kind of blurry to me. So, let’s
> try to go step by step.
>
> Let me ask again. What is a backup of the VM? You mention restore-point.
> What is it? Is it a snapshot in the hypervisor (complete/partial snapshot)?
> If it is not a snapshot,  what is it then? Are you installing some agent in
> the VM and then copying/mirroring the files in the VM’s file system?
>
> On Thu, Jun 7, 2018 at 3:26 PM, Nicolas Vazquez <
> nicolas.vazq...@shapeblue.com> wrote:
>
> > Hi Daniel and Rafael,
> >
> >
> > I have updated the diagram picture on the FS, please let me know if the
> > problem persists.
> >
> >
> > @Daniel you are right about
> > restoring VMs, please let me update the FS. Regarding your question, we
> > have considered the backups as a first class citizens, meaning that a
> > backup would continue existing unless its originating VM has been
> > removed/expunged. This way, listBackups API method would list them in
> order
> > to allow a user to restore a deleted VM from a backup.
> >
> >
> > @Rafael the idea of the Dummy provider is simply returning the expected
> > entities from a backup provider. For example, the BackupManager could ask
> > the BackupProvider to list its backups and would expect a list of
> backups.
> > As the Dummy Provider implements the BackupProvider interface, it has its
> > own implementation of listBackups() method and simply returns some mock
> > entities to the Backup Manager. It does not store any backup, it simply
> > creates them in execution time and returns them.
> >
> >
> > From the CloudStack point of view, Backups are stored externally, on the
> > Backup and Recovery provider servers and are handled by them. We keep
> track
> > of the existing backups in the Backup provider. Implementation may depend
> > on the Backup and Recovery provider, would consist on snapshots, but are
> > handled by the provider and not by CloudStack.
> >
> >
> > Backup policies are created on the Backup and Recovery provider and
> > provide retention options and periods as well as backup times. For
> example,
> > the provider may expose Golden, Silver and Bronze policies and have
> > different options for each of them.
> >
> >
> > The 'external' parameter on the listBackupPolicies API method would only
> > ask the Backup and Recovery provider to list their policies if it is set
> to
> > true. If it is not set, or set to false, it just simply returns the
> > existing mapped policies in CloudStack. Initially, CloudStack does not
> > store any backup policy, those should be imported/mapped by the
> > 'importBackupPolicy' API method.
> >
> >
> > Finally, we consider backups as restore points. As mentioned above,
> > backups are stored on the Backup and Recovery provider servers, so we
> keep
> > a mapping between the CloudStack entity and the backup, which can be used
> > to restore VMs.
> >
> >
> > Please let me know if anything.
> >
> >
> > Regards,
> >
> > Nicolas Vazquez
> >
> > 
> > From: Rafael Weingärtner 
> > Sent: Thursday, June 7, 2018 6:55:57 AM
> > To: dev
> > Subject: Re: [PROPOSE] Backup and Recovery Framework
> >
> > Hi Nicolas,
> >
> > The idea is great. I have some questions though.
> >
> > Before getting to the questions, your pictures are broken, or at least
> they
> > seem broken to me.
> >
> > How would the “dummy” backup provider work? I mean, what does it do?
> where
> > does it storage the "backup"?
> > What is the difference from this Backup to a Snapshot? Is it simply
> storing
> > a snapshot outside the cloud structure?
> > How would these “backup policies” look like? I mean, what can I
> configure?
> > Regarding that parameter “external” of your backup policy, how would it
> > work if we set it to false?
> > What are you defining as a backup (what do you mean by backup in the
> > context of this feature?)? I mean, is it simply a snapshot file that you
> > store somewhere else?
> >
> >
> > nicolas.vazq...@shapeblue.com
> > 

Re: [PROPOSE] Backup and Recovery Framework

[Nicolas Vazquez]

Its format may depend on the provider, but under the hood it is a snapshot 
which can be full or incremental and it is copied and stored in a backup 
repository in one of the backup servers. For example, if you use Veeam, you 
will find your backups stored as .vbk (for full backups) and .vib (for 
incremental backups). It is the task of the backup provider to create and move 
these files to the backup repository.


From: Rafael Weingärtner 
Sent: Thursday, June 7, 2018 10:39:35 AM
To: dev
Subject: Re: [PROPOSE] Backup and Recovery Framework

Ok, there are a lot of things, which are kind of blurry to me. So, let’s
try to go step by step.

Let me ask again. What is a backup of the VM? You mention restore-point.
What is it? Is it a snapshot in the hypervisor (complete/partial snapshot)?
If it is not a snapshot,  what is it then? Are you installing some agent in
the VM and then copying/mirroring the files in the VM’s file system?

On Thu, Jun 7, 2018 at 3:26 PM, Nicolas Vazquez <
nicolas.vazq...@shapeblue.com> wrote:

> Hi Daniel and Rafael,
>
>
> I have updated the diagram picture on the FS, please let me know if the
> problem persists.
>
>
> @Daniel you are right about
> restoring VMs, please let me update the FS. Regarding your question, we
> have considered the backups as a first class citizens, meaning that a
> backup would continue existing unless its originating VM has been
> removed/expunged. This way, listBackups API method would list them in order
> to allow a user to restore a deleted VM from a backup.
>
>
> @Rafael the idea of the Dummy provider is simply returning the expected
> entities from a backup provider. For example, the BackupManager could ask
> the BackupProvider to list its backups and would expect a list of backups.
> As the Dummy Provider implements the BackupProvider interface, it has its
> own implementation of listBackups() method and simply returns some mock
> entities to the Backup Manager. It does not store any backup, it simply
> creates them in execution time and returns them.
>
>
> From the CloudStack point of view, Backups are stored externally, on the
> Backup and Recovery provider servers and are handled by them. We keep track
> of the existing backups in the Backup provider. Implementation may depend
> on the Backup and Recovery provider, would consist on snapshots, but are
> handled by the provider and not by CloudStack.
>
>
> Backup policies are created on the Backup and Recovery provider and
> provide retention options and periods as well as backup times. For example,
> the provider may expose Golden, Silver and Bronze policies and have
> different options for each of them.
>
>
> The 'external' parameter on the listBackupPolicies API method would only
> ask the Backup and Recovery provider to list their policies if it is set to
> true. If it is not set, or set to false, it just simply returns the
> existing mapped policies in CloudStack. Initially, CloudStack does not
> store any backup policy, those should be imported/mapped by the
> 'importBackupPolicy' API method.
>
>
> Finally, we consider backups as restore points. As mentioned above,
> backups are stored on the Backup and Recovery provider servers, so we keep
> a mapping between the CloudStack entity and the backup, which can be used
> to restore VMs.
>
>
> Please let me know if anything.
>
>
> Regards,
>
> Nicolas Vazquez
>
> 
> From: Rafael Weingärtner 
> Sent: Thursday, June 7, 2018 6:55:57 AM
> To: dev
> Subject: Re: [PROPOSE] Backup and Recovery Framework
>
> Hi Nicolas,
>
> The idea is great. I have some questions though.
>
> Before getting to the questions, your pictures are broken, or at least they
> seem broken to me.
>
> How would the “dummy” backup provider work? I mean, what does it do? where
> does it storage the "backup"?
> What is the difference from this Backup to a Snapshot? Is it simply storing
> a snapshot outside the cloud structure?
> How would these “backup policies” look like? I mean, what can I configure?
> Regarding that parameter “external” of your backup policy, how would it
> work if we set it to false?
> What are you defining as a backup (what do you mean by backup in the
> context of this feature?)? I mean, is it simply a snapshot file that you
> store somewhere else?
>
>
> nicolas.vazq...@shapeblue.com
> www.shapeblue.com
> ,
> @shapeblue
>
>
>

nicolas.vazq...@shapeblue.com 
www.shapeblue.com
,   
@shapeblue
  
 

> On Thu, Jun 7, 2018 at 6:05 AM,  wrote:
>
> > Hi all,
> >
> > +1 for this feature request, thank you for bringing this to life! Volume
> > snapshots are what our users currently use as backup solution, which has
> > some inherent problems such as handling large volumes and the handling of
> > multiple volumes (consistency).
> >
> > One thought: when restoring a VM, I think there are two cases to
> consider:
> >
> > - The VM is still in CS 

Re: [PROPOSE] Backup and Recovery Framework

[Rafael Weingärtner]

Ok, there are a lot of things, which are kind of blurry to me. So, let’s
try to go step by step.

Let me ask again. What is a backup of the VM? You mention restore-point.
What is it? Is it a snapshot in the hypervisor (complete/partial snapshot)?
If it is not a snapshot,  what is it then? Are you installing some agent in
the VM and then copying/mirroring the files in the VM’s file system?

On Thu, Jun 7, 2018 at 3:26 PM, Nicolas Vazquez <
nicolas.vazq...@shapeblue.com> wrote:

> Hi Daniel and Rafael,
>
>
> I have updated the diagram picture on the FS, please let me know if the
> problem persists.
>
>
> @Daniel you are right about
> restoring VMs, please let me update the FS. Regarding your question, we
> have considered the backups as a first class citizens, meaning that a
> backup would continue existing unless its originating VM has been
> removed/expunged. This way, listBackups API method would list them in order
> to allow a user to restore a deleted VM from a backup.
>
>
> @Rafael the idea of the Dummy provider is simply returning the expected
> entities from a backup provider. For example, the BackupManager could ask
> the BackupProvider to list its backups and would expect a list of backups.
> As the Dummy Provider implements the BackupProvider interface, it has its
> own implementation of listBackups() method and simply returns some mock
> entities to the Backup Manager. It does not store any backup, it simply
> creates them in execution time and returns them.
>
>
> From the CloudStack point of view, Backups are stored externally, on the
> Backup and Recovery provider servers and are handled by them. We keep track
> of the existing backups in the Backup provider. Implementation may depend
> on the Backup and Recovery provider, would consist on snapshots, but are
> handled by the provider and not by CloudStack.
>
>
> Backup policies are created on the Backup and Recovery provider and
> provide retention options and periods as well as backup times. For example,
> the provider may expose Golden, Silver and Bronze policies and have
> different options for each of them.
>
>
> The 'external' parameter on the listBackupPolicies API method would only
> ask the Backup and Recovery provider to list their policies if it is set to
> true. If it is not set, or set to false, it just simply returns the
> existing mapped policies in CloudStack. Initially, CloudStack does not
> store any backup policy, those should be imported/mapped by the
> 'importBackupPolicy' API method.
>
>
> Finally, we consider backups as restore points. As mentioned above,
> backups are stored on the Backup and Recovery provider servers, so we keep
> a mapping between the CloudStack entity and the backup, which can be used
> to restore VMs.
>
>
> Please let me know if anything.
>
>
> Regards,
>
> Nicolas Vazquez
>
> 
> From: Rafael Weingärtner 
> Sent: Thursday, June 7, 2018 6:55:57 AM
> To: dev
> Subject: Re: [PROPOSE] Backup and Recovery Framework
>
> Hi Nicolas,
>
> The idea is great. I have some questions though.
>
> Before getting to the questions, your pictures are broken, or at least they
> seem broken to me.
>
> How would the “dummy” backup provider work? I mean, what does it do? where
> does it storage the "backup"?
> What is the difference from this Backup to a Snapshot? Is it simply storing
> a snapshot outside the cloud structure?
> How would these “backup policies” look like? I mean, what can I configure?
> Regarding that parameter “external” of your backup policy, how would it
> work if we set it to false?
> What are you defining as a backup (what do you mean by backup in the
> context of this feature?)? I mean, is it simply a snapshot file that you
> store somewhere else?
>
>
> nicolas.vazq...@shapeblue.com
> www.shapeblue.com
> ,
> @shapeblue
>
>
>
> On Thu, Jun 7, 2018 at 6:05 AM,  wrote:
>
> > Hi all,
> >
> > +1 for this feature request, thank you for bringing this to life! Volume
> > snapshots are what our users currently use as backup solution, which has
> > some inherent problems such as handling large volumes and the handling of
> > multiple volumes (consistency).
> >
> > One thought: when restoring a VM, I think there are two cases to
> consider:
> >
> > - The VM is still in CS and the user just wants to restore an old version
> > of the VM
> > - The VM has already been deleted and the user wants to restore the
> > deleted VM
> >
> > As listBackups is described as " listBackups API method. List existing
> > backups for a VM". Is the second use case supported at all?
> >
> > I like the approach not to define the backup policies in CS but in
> > whatever tool is used in the end, as otherwise a lot of features might be
> > lost. We as services providers need to create them in the first place,
> but
> > I think this is the better solution.
> >
> > Regards
> > Daniel
> >
> > On 06.06.18, 14:43, "Nicolas Vazquez" 
> > wrote:
> >
> > Hi all,
> >
> 

Re: [PROPOSE] Backup and Recovery Framework

[Nicolas Vazquez]

Hi Daniel and Rafael,


I have updated the diagram picture on the FS, please let me know if the problem 
persists.


@Daniel you are right about restoring 
VMs, please let me update the FS. Regarding your question, we have considered 
the backups as a first class citizens, meaning that a backup would continue 
existing unless its originating VM has been removed/expunged. This way, 
listBackups API method would list them in order to allow a user to restore a 
deleted VM from a backup.


@Rafael the idea of the Dummy provider is simply returning the expected 
entities from a backup provider. For example, the BackupManager could ask the 
BackupProvider to list its backups and would expect a list of backups. As the 
Dummy Provider implements the BackupProvider interface, it has its own 
implementation of listBackups() method and simply returns some mock entities to 
the Backup Manager. It does not store any backup, it simply creates them in 
execution time and returns them.


From the CloudStack point of view, Backups are stored externally, on the Backup 
and Recovery provider servers and are handled by them. We keep track of the 
existing backups in the Backup provider. Implementation may depend on the 
Backup and Recovery provider, would consist on snapshots, but are handled by 
the provider and not by CloudStack.


Backup policies are created on the Backup and Recovery provider and provide 
retention options and periods as well as backup times. For example, the 
provider may expose Golden, Silver and Bronze policies and have different 
options for each of them.


The 'external' parameter on the listBackupPolicies API method would only ask 
the Backup and Recovery provider to list their policies if it is set to true. 
If it is not set, or set to false, it just simply returns the existing mapped 
policies in CloudStack. Initially, CloudStack does not store any backup policy, 
those should be imported/mapped by the 'importBackupPolicy' API method.


Finally, we consider backups as restore points. As mentioned above, backups are 
stored on the Backup and Recovery provider servers, so we keep a mapping 
between the CloudStack entity and the backup, which can be used to restore VMs.


Please let me know if anything.


Regards,

Nicolas Vazquez


From: Rafael Weingärtner 
Sent: Thursday, June 7, 2018 6:55:57 AM
To: dev
Subject: Re: [PROPOSE] Backup and Recovery Framework

Hi Nicolas,

The idea is great. I have some questions though.

Before getting to the questions, your pictures are broken, or at least they
seem broken to me.

How would the “dummy” backup provider work? I mean, what does it do? where
does it storage the "backup"?
What is the difference from this Backup to a Snapshot? Is it simply storing
a snapshot outside the cloud structure?
How would these “backup policies” look like? I mean, what can I configure?
Regarding that parameter “external” of your backup policy, how would it
work if we set it to false?
What are you defining as a backup (what do you mean by backup in the
context of this feature?)? I mean, is it simply a snapshot file that you
store somewhere else?


nicolas.vazq...@shapeblue.com 
www.shapeblue.com
,   
@shapeblue
  
 

On Thu, Jun 7, 2018 at 6:05 AM,  wrote:

> Hi all,
>
> +1 for this feature request, thank you for bringing this to life! Volume
> snapshots are what our users currently use as backup solution, which has
> some inherent problems such as handling large volumes and the handling of
> multiple volumes (consistency).
>
> One thought: when restoring a VM, I think there are two cases to consider:
>
> - The VM is still in CS and the user just wants to restore an old version
> of the VM
> - The VM has already been deleted and the user wants to restore the
> deleted VM
>
> As listBackups is described as " listBackups API method. List existing
> backups for a VM". Is the second use case supported at all?
>
> I like the approach not to define the backup policies in CS but in
> whatever tool is used in the end, as otherwise a lot of features might be
> lost. We as services providers need to create them in the first place, but
> I think this is the better solution.
>
> Regards
> Daniel
>
> On 06.06.18, 14:43, "Nicolas Vazquez" 
> wrote:
>
> Hi all,
>
>
> We would like to introduce a new framework into CloudStack that will
> allow adding Backup and Recovery providers as plugins. This framework goal
> is allowing users to backup their guest VMs for recovery purposes.
>
>
> Please find the FS on this link: https://cwiki.apache.org/
> confluence/display/CLOUDSTACK/Backup+and+Recovery+Framework
>
>
> Regards,
>
> Nicolas Vazquez
>
> nicolas.vazq...@shapeblue.com
> www.shapeblue.com
> ,
> @shapeblue
>
>
>
>
>
>


--
Rafael Weingärtner

Re: [PROPOSE] Backup and Recovery Framework

[Rafael Weingärtner]

Hi Nicolas,

The idea is great. I have some questions though.

Before getting to the questions, your pictures are broken, or at least they
seem broken to me.

How would the “dummy” backup provider work? I mean, what does it do? where
does it storage the "backup"?
What is the difference from this Backup to a Snapshot? Is it simply storing
a snapshot outside the cloud structure?
How would these “backup policies” look like? I mean, what can I configure?
Regarding that parameter “external” of your backup policy, how would it
work if we set it to false?
What are you defining as a backup (what do you mean by backup in the
context of this feature?)? I mean, is it simply a snapshot file that you
store somewhere else?

On Thu, Jun 7, 2018 at 6:05 AM,  wrote:

> Hi all,
>
> +1 for this feature request, thank you for bringing this to life! Volume
> snapshots are what our users currently use as backup solution, which has
> some inherent problems such as handling large volumes and the handling of
> multiple volumes (consistency).
>
> One thought: when restoring a VM, I think there are two cases to consider:
>
> - The VM is still in CS and the user just wants to restore an old version
> of the VM
> - The VM has already been deleted and the user wants to restore the
> deleted VM
>
> As listBackups is described as " listBackups API method. List existing
> backups for a VM". Is the second use case supported at all?
>
> I like the approach not to define the backup policies in CS but in
> whatever tool is used in the end, as otherwise a lot of features might be
> lost. We as services providers need to create them in the first place, but
> I think this is the better solution.
>
> Regards
> Daniel
>
> On 06.06.18, 14:43, "Nicolas Vazquez" 
> wrote:
>
> Hi all,
>
>
> We would like to introduce a new framework into CloudStack that will
> allow adding Backup and Recovery providers as plugins. This framework goal
> is allowing users to backup their guest VMs for recovery purposes.
>
>
> Please find the FS on this link: https://cwiki.apache.org/
> confluence/display/CLOUDSTACK/Backup+and+Recovery+Framework
>
>
> Regards,
>
> Nicolas Vazquez
>
> nicolas.vazq...@shapeblue.com
> www.shapeblue.com
> ,
> @shapeblue
>
>
>
>
>
>


-- 
Rafael Weingärtner

Re: [PROPOSE] Backup and Recovery Framework

[daniel.herrmann]

Hi all,

+1 for this feature request, thank you for bringing this to life! Volume 
snapshots are what our users currently use as backup solution, which has some 
inherent problems such as handling large volumes and the handling of multiple 
volumes (consistency).

One thought: when restoring a VM, I think there are two cases to consider:

- The VM is still in CS and the user just wants to restore an old version of 
the VM
- The VM has already been deleted and the user wants to restore the deleted VM

As listBackups is described as " listBackups API method. List existing backups 
for a VM". Is the second use case supported at all?

I like the approach not to define the backup policies in CS but in whatever 
tool is used in the end, as otherwise a lot of features might be lost. We as 
services providers need to create them in the first place, but I think this is 
the better solution.

Regards
Daniel

On 06.06.18, 14:43, "Nicolas Vazquez"  wrote:

Hi all,


We would like to introduce a new framework into CloudStack that will allow 
adding Backup and Recovery providers as plugins. This framework goal is 
allowing users to backup their guest VMs for recovery purposes.


Please find the FS on this link: 
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Backup+and+Recovery+Framework


Regards,

Nicolas Vazquez

nicolas.vazq...@shapeblue.com 
www.shapeblue.com
,   
@shapeblue