Re: Snapshots only on Primary Storage feature

[Will Stevens]

I think reverting the change in 4.11.1 is probably a good idea.

*Will Stevens*
Chief Technology Officer
c 514.826.0190




On Fri, May 18, 2018 at 2:52 PM ilya musayev 
wrote:

> Perhaps bring it back into 4.11.1?
>
> On Fri, May 18, 2018 at 9:28 AM Suresh Kumar Anaparti <
> sureshkumar.anapa...@gmail.com> wrote:
>
> > Si / Will,
> >
> > That is just FYI, if anyone uses VMware with that flag set to false. I'm
> > neither against the feature nor telling to rip that out.
> >
> > You are correct, the PR 2081 supports KVM and Xen as the volume snapshots
> > are directly supported on them and backup operation is not tightly
> coupled
> > with the create operation.
> >
> > -Suresh
> >
> > On Fri, May 18, 2018 at 7:38 PM, Simon Weller 
> > wrote:
> >
> > > There are plenty of features in ACS that are particular to a certain
> > > hypervisor (or hypervisor set), including VMware specific items.
> > >
> > > It was never claimed this feature worked across all hypervisors. In
> > > addition to that, the default was to leave the existing functionality
> > > exactly the way it was originally implemented and if a user wished to
> > > change the functionality they could via a global config variable.
> > >
> > > Your original spec for PR 2081 in confluence states that the PR was
> > > targeted towards KVM and Xen, so I'm confused as to why VMware is even
> > > being mentioned here.
> > >
> > >
> > > This is a major feature regression that a number of
> organizations/service
> > > providers are relying on and it wasn't called out when the PR was
> > submitted.
> > >
> > >
> > > 
> > > From: Will Stevens 
> > > Sent: Friday, May 18, 2018 6:12 AM
> > > To: dev@cloudstack.apache.org
> > > Subject: Re: Snapshots only on Primary Storage feature
> > >
> > > Just because it does not work for VMware should not a reason to rip out
> > the
> > > functionality for other hypervisors where it is being used though.
> > >
> > > I know we also have the requirement that snapshots are not
> automatically
> > > replicated to secondary storage, so this feature is useful to us.
> > >
> > > I don't understand the rational for removing the feature just because
> it
> > > does not work on VMware.
> > >
> > > On Fri, May 18, 2018, 6:27 AM Suresh Kumar Anaparti, <
> > > sureshkumar.anapa...@gmail.com> wrote:
> > >
> > > > Si,
> > > >
> > > > The PR# 1697 with the global setting *snapshot.backup.rightafter** -
> > > > false* doesn't
> > > > work for VMware as create snapshot never takes a snapshot in Primary
> > > pool,
> > > > it just returns the snapshot uuid. The backup snapshot does the
> > complete
> > > > job - creates a VM snapshot with the uuid, extracts and exports the
> > > target
> > > > volume to secondary. On demand backup snapshot doesn't work as there
> is
> > > no
> > > > snapshot in primary. Also, there'll be only one entry with Primary
> > store
> > > > role in snapshot_store_ref, which is the latest snapshot taken for
> that
> > > > volume.
> > > >
> > > > -Suresh
> > > >
> > > > On Fri, May 18, 2018 at 1:03 AM, Simon Weller
>  > >
> > > > wrote:
> > > >
> > > > > The whole point of the original PR was to optionally disable this
> > > > > functionality.
> > > > >
> > > > > We don't expose views of the backup state to our customers (we have
> > our
> > > > > own customer interfaces) and it's a large waste of space for us to
> be
> > > > > backing up tons of VM images when we have a solid primary storage
> > > > > infrastructure that already has lots of resiliency.
> > > > >
> > > > >
> > > > > I guess we're going to have to revisit this again before we can
> > > consider
> > > > > rebasing on 4.11.
> > > > >
> > > > > 
> > > > > From: Suresh Kumar Anaparti 
> > > > > Sent: Thursday, May 17, 2018 2:21 PM
> > > > > To: dev
> > > > > Subject: Re: Snapshots only on Primary Storage feature
> > > > >
> > > > > Hi Si,
> > > > >
> > > > > No. not possible to disable the backup to secondary. It copies the
> > > volume
> > > > > snapshot to secondary in a background thread using asyncBackup
> param
> > > (set
> > > > > to true) and allows other operations during that time.
> > > > >
> > > > > I understand that the backup was on demand when any operations are
> > > > > performed on the snapshot. But, backup during that time may take
> > > > > considerable time (depending on the snapshot size and the network
> > > > > bandwidth), which can result in the job timeout and the User may
> > assume
> > > > > that it is already Backed up based on its state, unless it is
> > > documented.
> > > > >
> > > > > -Suresh
> > > > >
> > > > > On Fri, May 18, 2018 at 12:23 AM, Simon Weller
> >  > > >
> > > > > wrote:
> > > > >
> > > > > > Suresh,
> > > > > >
> > > > > >
> > > > > > With this new merged  PR, is it 

Re: Snapshots only on Primary Storage feature

[ilya musayev]

Perhaps bring it back into 4.11.1?

On Fri, May 18, 2018 at 9:28 AM Suresh Kumar Anaparti <
sureshkumar.anapa...@gmail.com> wrote:

> Si / Will,
>
> That is just FYI, if anyone uses VMware with that flag set to false. I'm
> neither against the feature nor telling to rip that out.
>
> You are correct, the PR 2081 supports KVM and Xen as the volume snapshots
> are directly supported on them and backup operation is not tightly coupled
> with the create operation.
>
> -Suresh
>
> On Fri, May 18, 2018 at 7:38 PM, Simon Weller 
> wrote:
>
> > There are plenty of features in ACS that are particular to a certain
> > hypervisor (or hypervisor set), including VMware specific items.
> >
> > It was never claimed this feature worked across all hypervisors. In
> > addition to that, the default was to leave the existing functionality
> > exactly the way it was originally implemented and if a user wished to
> > change the functionality they could via a global config variable.
> >
> > Your original spec for PR 2081 in confluence states that the PR was
> > targeted towards KVM and Xen, so I'm confused as to why VMware is even
> > being mentioned here.
> >
> >
> > This is a major feature regression that a number of organizations/service
> > providers are relying on and it wasn't called out when the PR was
> submitted.
> >
> >
> > 
> > From: Will Stevens 
> > Sent: Friday, May 18, 2018 6:12 AM
> > To: dev@cloudstack.apache.org
> > Subject: Re: Snapshots only on Primary Storage feature
> >
> > Just because it does not work for VMware should not a reason to rip out
> the
> > functionality for other hypervisors where it is being used though.
> >
> > I know we also have the requirement that snapshots are not automatically
> > replicated to secondary storage, so this feature is useful to us.
> >
> > I don't understand the rational for removing the feature just because it
> > does not work on VMware.
> >
> > On Fri, May 18, 2018, 6:27 AM Suresh Kumar Anaparti, <
> > sureshkumar.anapa...@gmail.com> wrote:
> >
> > > Si,
> > >
> > > The PR# 1697 with the global setting *snapshot.backup.rightafter** -
> > > false* doesn't
> > > work for VMware as create snapshot never takes a snapshot in Primary
> > pool,
> > > it just returns the snapshot uuid. The backup snapshot does the
> complete
> > > job - creates a VM snapshot with the uuid, extracts and exports the
> > target
> > > volume to secondary. On demand backup snapshot doesn't work as there is
> > no
> > > snapshot in primary. Also, there'll be only one entry with Primary
> store
> > > role in snapshot_store_ref, which is the latest snapshot taken for that
> > > volume.
> > >
> > > -Suresh
> > >
> > > On Fri, May 18, 2018 at 1:03 AM, Simon Weller  >
> > > wrote:
> > >
> > > > The whole point of the original PR was to optionally disable this
> > > > functionality.
> > > >
> > > > We don't expose views of the backup state to our customers (we have
> our
> > > > own customer interfaces) and it's a large waste of space for us to be
> > > > backing up tons of VM images when we have a solid primary storage
> > > > infrastructure that already has lots of resiliency.
> > > >
> > > >
> > > > I guess we're going to have to revisit this again before we can
> > consider
> > > > rebasing on 4.11.
> > > >
> > > > 
> > > > From: Suresh Kumar Anaparti 
> > > > Sent: Thursday, May 17, 2018 2:21 PM
> > > > To: dev
> > > > Subject: Re: Snapshots only on Primary Storage feature
> > > >
> > > > Hi Si,
> > > >
> > > > No. not possible to disable the backup to secondary. It copies the
> > volume
> > > > snapshot to secondary in a background thread using asyncBackup param
> > (set
> > > > to true) and allows other operations during that time.
> > > >
> > > > I understand that the backup was on demand when any operations are
> > > > performed on the snapshot. But, backup during that time may take
> > > > considerable time (depending on the snapshot size and the network
> > > > bandwidth), which can result in the job timeout and the User may
> assume
> > > > that it is already Backed up based on its state, unless it is
> > documented.
> > > >
> > > > -Suresh
> > > >
> > > > On Fri, May 18, 2018 at 12:23 AM, Simon Weller
>  > >
> > > > wrote:
> > > >
> > > > > Suresh,
> > > > >
> > > > >
> > > > > With this new merged  PR, is it possible to disable the backup to
> > > > > secondary completely? I can't tell from the reference spec and
> we're
> > > not
> > > > on
> > > > > a 4.10/4.11 base yet.
> > > > >
> > > > > For the record, in the instances where a volume or template from
> > > snapshot
> > > > > was required, the backup image was copied on demand to secondary.
> > > > >
> > > > > In an ideal world, secondary storage wouldn't even be involved in
> > most
> > > of
> > > > > these options, instead using the native 

Re: Snapshots only on Primary Storage feature

[Suresh Kumar Anaparti]

Si / Will,

That is just FYI, if anyone uses VMware with that flag set to false. I'm
neither against the feature nor telling to rip that out.

You are correct, the PR 2081 supports KVM and Xen as the volume snapshots
are directly supported on them and backup operation is not tightly coupled
with the create operation.

-Suresh

On Fri, May 18, 2018 at 7:38 PM, Simon Weller 
wrote:

> There are plenty of features in ACS that are particular to a certain
> hypervisor (or hypervisor set), including VMware specific items.
>
> It was never claimed this feature worked across all hypervisors. In
> addition to that, the default was to leave the existing functionality
> exactly the way it was originally implemented and if a user wished to
> change the functionality they could via a global config variable.
>
> Your original spec for PR 2081 in confluence states that the PR was
> targeted towards KVM and Xen, so I'm confused as to why VMware is even
> being mentioned here.
>
>
> This is a major feature regression that a number of organizations/service
> providers are relying on and it wasn't called out when the PR was submitted.
>
>
> 
> From: Will Stevens 
> Sent: Friday, May 18, 2018 6:12 AM
> To: dev@cloudstack.apache.org
> Subject: Re: Snapshots only on Primary Storage feature
>
> Just because it does not work for VMware should not a reason to rip out the
> functionality for other hypervisors where it is being used though.
>
> I know we also have the requirement that snapshots are not automatically
> replicated to secondary storage, so this feature is useful to us.
>
> I don't understand the rational for removing the feature just because it
> does not work on VMware.
>
> On Fri, May 18, 2018, 6:27 AM Suresh Kumar Anaparti, <
> sureshkumar.anapa...@gmail.com> wrote:
>
> > Si,
> >
> > The PR# 1697 with the global setting *snapshot.backup.rightafter** -
> > false* doesn't
> > work for VMware as create snapshot never takes a snapshot in Primary
> pool,
> > it just returns the snapshot uuid. The backup snapshot does the complete
> > job - creates a VM snapshot with the uuid, extracts and exports the
> target
> > volume to secondary. On demand backup snapshot doesn't work as there is
> no
> > snapshot in primary. Also, there'll be only one entry with Primary store
> > role in snapshot_store_ref, which is the latest snapshot taken for that
> > volume.
> >
> > -Suresh
> >
> > On Fri, May 18, 2018 at 1:03 AM, Simon Weller 
> > wrote:
> >
> > > The whole point of the original PR was to optionally disable this
> > > functionality.
> > >
> > > We don't expose views of the backup state to our customers (we have our
> > > own customer interfaces) and it's a large waste of space for us to be
> > > backing up tons of VM images when we have a solid primary storage
> > > infrastructure that already has lots of resiliency.
> > >
> > >
> > > I guess we're going to have to revisit this again before we can
> consider
> > > rebasing on 4.11.
> > >
> > > 
> > > From: Suresh Kumar Anaparti 
> > > Sent: Thursday, May 17, 2018 2:21 PM
> > > To: dev
> > > Subject: Re: Snapshots only on Primary Storage feature
> > >
> > > Hi Si,
> > >
> > > No. not possible to disable the backup to secondary. It copies the
> volume
> > > snapshot to secondary in a background thread using asyncBackup param
> (set
> > > to true) and allows other operations during that time.
> > >
> > > I understand that the backup was on demand when any operations are
> > > performed on the snapshot. But, backup during that time may take
> > > considerable time (depending on the snapshot size and the network
> > > bandwidth), which can result in the job timeout and the User may assume
> > > that it is already Backed up based on its state, unless it is
> documented.
> > >
> > > -Suresh
> > >
> > > On Fri, May 18, 2018 at 12:23 AM, Simon Weller  >
> > > wrote:
> > >
> > > > Suresh,
> > > >
> > > >
> > > > With this new merged  PR, is it possible to disable the backup to
> > > > secondary completely? I can't tell from the reference spec and we're
> > not
> > > on
> > > > a 4.10/4.11 base yet.
> > > >
> > > > For the record, in the instances where a volume or template from
> > snapshot
> > > > was required, the backup image was copied on demand to secondary.
> > > >
> > > > In an ideal world, secondary storage wouldn't even be involved in
> most
> > of
> > > > these options, instead using the native clone features of the
> > underlying
> > > > storage.
> > > >
> > > >
> > > > - Si
> > > >
> > > > 
> > > > From: Suresh Kumar Anaparti 
> > > > Sent: Thursday, May 17, 2018 1:37 PM
> > > > To: dev@cloudstack.apache.org
> > > > Cc: Nathan Johnson
> > > > Subject: Re: Snapshots only on Primary Storage feature
> > > >
> > > > Hi Glen 

Re: Snapshots only on Primary Storage feature

[Simon Weller]

There are plenty of features in ACS that are particular to a certain hypervisor 
(or hypervisor set), including VMware specific items.

It was never claimed this feature worked across all hypervisors. In addition to 
that, the default was to leave the existing functionality exactly the way it 
was originally implemented and if a user wished to change the functionality 
they could via a global config variable.

Your original spec for PR 2081 in confluence states that the PR was targeted 
towards KVM and Xen, so I'm confused as to why VMware is even being mentioned 
here.


This is a major feature regression that a number of organizations/service 
providers are relying on and it wasn't called out when the PR was submitted.



From: Will Stevens 
Sent: Friday, May 18, 2018 6:12 AM
To: dev@cloudstack.apache.org
Subject: Re: Snapshots only on Primary Storage feature

Just because it does not work for VMware should not a reason to rip out the
functionality for other hypervisors where it is being used though.

I know we also have the requirement that snapshots are not automatically
replicated to secondary storage, so this feature is useful to us.

I don't understand the rational for removing the feature just because it
does not work on VMware.

On Fri, May 18, 2018, 6:27 AM Suresh Kumar Anaparti, <
sureshkumar.anapa...@gmail.com> wrote:

> Si,
>
> The PR# 1697 with the global setting *snapshot.backup.rightafter** -
> false* doesn't
> work for VMware as create snapshot never takes a snapshot in Primary pool,
> it just returns the snapshot uuid. The backup snapshot does the complete
> job - creates a VM snapshot with the uuid, extracts and exports the target
> volume to secondary. On demand backup snapshot doesn't work as there is no
> snapshot in primary. Also, there'll be only one entry with Primary store
> role in snapshot_store_ref, which is the latest snapshot taken for that
> volume.
>
> -Suresh
>
> On Fri, May 18, 2018 at 1:03 AM, Simon Weller 
> wrote:
>
> > The whole point of the original PR was to optionally disable this
> > functionality.
> >
> > We don't expose views of the backup state to our customers (we have our
> > own customer interfaces) and it's a large waste of space for us to be
> > backing up tons of VM images when we have a solid primary storage
> > infrastructure that already has lots of resiliency.
> >
> >
> > I guess we're going to have to revisit this again before we can consider
> > rebasing on 4.11.
> >
> > 
> > From: Suresh Kumar Anaparti 
> > Sent: Thursday, May 17, 2018 2:21 PM
> > To: dev
> > Subject: Re: Snapshots only on Primary Storage feature
> >
> > Hi Si,
> >
> > No. not possible to disable the backup to secondary. It copies the volume
> > snapshot to secondary in a background thread using asyncBackup param (set
> > to true) and allows other operations during that time.
> >
> > I understand that the backup was on demand when any operations are
> > performed on the snapshot. But, backup during that time may take
> > considerable time (depending on the snapshot size and the network
> > bandwidth), which can result in the job timeout and the User may assume
> > that it is already Backed up based on its state, unless it is documented.
> >
> > -Suresh
> >
> > On Fri, May 18, 2018 at 12:23 AM, Simon Weller 
> > wrote:
> >
> > > Suresh,
> > >
> > >
> > > With this new merged  PR, is it possible to disable the backup to
> > > secondary completely? I can't tell from the reference spec and we're
> not
> > on
> > > a 4.10/4.11 base yet.
> > >
> > > For the record, in the instances where a volume or template from
> snapshot
> > > was required, the backup image was copied on demand to secondary.
> > >
> > > In an ideal world, secondary storage wouldn't even be involved in most
> of
> > > these options, instead using the native clone features of the
> underlying
> > > storage.
> > >
> > >
> > > - Si
> > >
> > > 
> > > From: Suresh Kumar Anaparti 
> > > Sent: Thursday, May 17, 2018 1:37 PM
> > > To: dev@cloudstack.apache.org
> > > Cc: Nathan Johnson
> > > Subject: Re: Snapshots only on Primary Storage feature
> > >
> > > Hi Glen / Si,
> > >
> > > In PR# 1697, the global setting *snapshot.backup.rightafter* if set to
> > > true, it'll be the default behaviour and snapshot is copied to the
> > > secondary storage. If set to false, then the snapshot state transitions
> > are
> > > mocked and Snapshot would be in BackedUp state even though it is not
> > really
> > > in Secondary storage, which doesn't make sense. Also, that will enable
> to
> > > create a volume or template from the snapshot, which will obviously
> fail.
> > >
> > > This behavior was changed with the PR
> > > https://github.com/apache/cloudstack/pull/2081. There is a clear
> > > separation
> > > of 

Re: Snapshots only on Primary Storage feature

[Will Stevens]

Just because it does not work for VMware should not a reason to rip out the
functionality for other hypervisors where it is being used though.

I know we also have the requirement that snapshots are not automatically
replicated to secondary storage, so this feature is useful to us.

I don't understand the rational for removing the feature just because it
does not work on VMware.

On Fri, May 18, 2018, 6:27 AM Suresh Kumar Anaparti, <
sureshkumar.anapa...@gmail.com> wrote:

> Si,
>
> The PR# 1697 with the global setting *snapshot.backup.rightafter** -
> false* doesn't
> work for VMware as create snapshot never takes a snapshot in Primary pool,
> it just returns the snapshot uuid. The backup snapshot does the complete
> job - creates a VM snapshot with the uuid, extracts and exports the target
> volume to secondary. On demand backup snapshot doesn't work as there is no
> snapshot in primary. Also, there'll be only one entry with Primary store
> role in snapshot_store_ref, which is the latest snapshot taken for that
> volume.
>
> -Suresh
>
> On Fri, May 18, 2018 at 1:03 AM, Simon Weller 
> wrote:
>
> > The whole point of the original PR was to optionally disable this
> > functionality.
> >
> > We don't expose views of the backup state to our customers (we have our
> > own customer interfaces) and it's a large waste of space for us to be
> > backing up tons of VM images when we have a solid primary storage
> > infrastructure that already has lots of resiliency.
> >
> >
> > I guess we're going to have to revisit this again before we can consider
> > rebasing on 4.11.
> >
> > 
> > From: Suresh Kumar Anaparti 
> > Sent: Thursday, May 17, 2018 2:21 PM
> > To: dev
> > Subject: Re: Snapshots only on Primary Storage feature
> >
> > Hi Si,
> >
> > No. not possible to disable the backup to secondary. It copies the volume
> > snapshot to secondary in a background thread using asyncBackup param (set
> > to true) and allows other operations during that time.
> >
> > I understand that the backup was on demand when any operations are
> > performed on the snapshot. But, backup during that time may take
> > considerable time (depending on the snapshot size and the network
> > bandwidth), which can result in the job timeout and the User may assume
> > that it is already Backed up based on its state, unless it is documented.
> >
> > -Suresh
> >
> > On Fri, May 18, 2018 at 12:23 AM, Simon Weller 
> > wrote:
> >
> > > Suresh,
> > >
> > >
> > > With this new merged  PR, is it possible to disable the backup to
> > > secondary completely? I can't tell from the reference spec and we're
> not
> > on
> > > a 4.10/4.11 base yet.
> > >
> > > For the record, in the instances where a volume or template from
> snapshot
> > > was required, the backup image was copied on demand to secondary.
> > >
> > > In an ideal world, secondary storage wouldn't even be involved in most
> of
> > > these options, instead using the native clone features of the
> underlying
> > > storage.
> > >
> > >
> > > - Si
> > >
> > > 
> > > From: Suresh Kumar Anaparti 
> > > Sent: Thursday, May 17, 2018 1:37 PM
> > > To: dev@cloudstack.apache.org
> > > Cc: Nathan Johnson
> > > Subject: Re: Snapshots only on Primary Storage feature
> > >
> > > Hi Glen / Si,
> > >
> > > In PR# 1697, the global setting *snapshot.backup.rightafter* if set to
> > > true, it'll be the default behaviour and snapshot is copied to the
> > > secondary storage. If set to false, then the snapshot state transitions
> > are
> > > mocked and Snapshot would be in BackedUp state even though it is not
> > really
> > > in Secondary storage, which doesn't make sense. Also, that will enable
> to
> > > create a volume or template from the snapshot, which will obviously
> fail.
> > >
> > > This behavior was changed with the PR
> > > https://github.com/apache/cloudstack/pull/2081. There is a clear
> > > separation
> > > of create and backup volume snapshot operations. The global setting
> > > *snapshot.backup.rightafter* has been removed in PR# 2081.
> > >
> > > -Suresh
> > >
> > > On Thu, May 17, 2018 at 8:40 PM, Simon Weller  >
> > > wrote:
> > >
> > > > Glen,
> > > >
> > > >
> > > > This feature was implemented in 4.9 by my colleague Nathan Johnson.
> > You
> > > > enable it by changing the global setting  snapshot.backup.rightafter
> to
> > > > false.
> > > >
> > > >
> > > > The PR is reference here: https://github.com/apache/
> > cloudstack/pull/1697
> > > >
> > > >
> > > > We have the exact same use case as you, as we also use Ceph.
> > > >
> > > >
> > > > - Si
> > > >
> > > >
> > > > 
> > > > From: Glen Baars 
> > > > Sent: Thursday, May 17, 2018 9:46 AM
> > > > To: dev@cloudstack.apache.org
> > > > Subject: Snapshots only on Primary Storage 

Re: [DISCUSS][ASK] Should agent wait for pending tasks on (mgmt server) disconnection?

[Suresh Kumar Anaparti]

Hi Rohit,

I checked that. Thanks for the details!

-Suresh

On Wed, May 16, 2018 at 4:55 PM, Rohit Yadav 
wrote:

> Hi Suresh,
>
>
> As explained earlier and advised to look at code on the PR, perhaps you
> did not get time so have a look here:
>
> https://github.com/apache/cloudstack/blob/4.11/agent/
> src/com/cloud/agent/Agent.java#L488
>
>
> The reconnect() historically sets the link to null. Therefore, any answer
> from pending tasks end up failing here:
>
> https://github.com/apache/cloudstack/blob/4.11/agent/
> src/com/cloud/agent/Agent.java#L868
>
> and,
>
> https://github.com/apache/cloudstack/blob/4.11/agent/
> src/com/cloud/agent/Agent.java#L893
>
>
> Do note that reconnect() only cancels watch tasks but does not
> cancel/shutdown any running task. Also, in case of network error, the mgmt
> server will fail at thread/context where is has done a agent.send() and
> expecting an answer.
>
>
> You can also perform a small test by doing a while or sleep around this
> code to see how getLink().send() behave when agent does reconnect. When it
> does not reconnect, i.e. the agent is blocked by pending tasks to complete
> such tasks always fail.
>
>
> - Rohit
>
> 
>
>
>
> 
> From: Suresh Kumar Anaparti 
> Sent: Wednesday, May 16, 2018 4:27:36 PM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS][ASK] Should agent wait for pending tasks on (mgmt
> server) disconnection?
>
> Hi Rohit,
>
> When Management Server and Agent are up and running and there is a network
> failure, I think it is better to wait for some time for the pending tasks
> to complete, instead of failing them and try reconnecting. If network delay
> is minimal, there can be a valid thread/context in the management server to
> handle the answers.
>
> It would be great if there are no major side-effects with this PR changes.
>
> Thanks,
> Suresh
>
> On Wed, May 16, 2018 at 3:40 PM, Rohit Yadav 
> wrote:
>
> > All,
> >
> >
> > Based on testing against KVM, XenServer and VMware and this discussion,
> > I'll merged the PR based on code reviews and tests. I investigated both
> > code-wise and against live environment for possible side-effects of
> letting
> > agent connect without being blocked on pending tasks and I found no new
> > fault behaviour.
> >
> >
> > If there are any objections or bugs, please share in which case we'll
> > revert the change to continue legacy/historic behaviour. Thanks.
> >
> >
> > - Rohit
> >
> > 
> >
> >
> >
> > 
> > From: Rohit Yadav 
> > Sent: Tuesday, May 15, 2018 2:37:58 PM
> > To: dev@cloudstack.apache.org
> > Subject: Re: [DISCUSS][ASK] Should agent wait for pending tasks on (mgmt
> > server) disconnection?
> >
> > Hi Suresh,
> >
> >
> > I've replied to your comment on the PR. In addition, when (i) management
> > server is restarted any pending operation on KVM/SSVM agent side will
> fail
> > fail to be communicated back in the correct thread/context and it depends
> > on a specific feature whether is supports sync or cleanup mechanism, in
> > most cases, the async/job timeout may kick in or cause queue/concurrent
> > failure seen in logs. When (ii) agent is reconnected, it reconnects only
> > after any pending job finishes therefore such jobs finish and fail to be
> > communicated back to the mgmt server (the answer instance is failed to be
> > sent on the link, as link is no longer valid and causes exception).
> >
> >
> > - Rohit
> >
> > 
> >
> >
> >
> > 
> > From: Suresh Kumar Anaparti 
> > Sent: Tuesday, May 15, 2018 12:06:14 AM
> > To: dev@cloudstack.apache.org
> > Subject: Re: [DISCUSS][ASK] Should agent wait for pending tasks on (mgmt
> > server) disconnection?
> >
> > Hi,
> >
> > @rhtyd, I checked the PR changes. Good that the agent is not waiting for
> > the pending jobs and retrying connection to management server. This might
> > have impact on ssvm and kvm agent tasks, not much on cpvm. Any sync or
> > cleanup mechanism for Volumes/VMs to address the failed/pending agent
> jobs
> > after (i) management server restart and (ii) agent connected ?
> >
> > -Suresh
> >
> > On Mon, May 14, 2018 at 8:05 PM, Marc-Aurèle Brothier  >
> > wrote:
> >
> > > Correct about the thread context, so if the answer is coming into a
> > > management server that doesn't have the context and drops it, it should
> > be
> > > fine then. The PR is then already a good improvement to let the agent
> > > reconnect even when it's doing a long processing request, so it can
> keeps
> > > on completing other jobs too.
> > >
> > > Regarding the restart/shutdown operation, yes I have to push now the
> > > changes to be able to stop some processing tasks (fetching new async
> jobs
> 

Re: Snapshots only on Primary Storage feature

[Suresh Kumar Anaparti]

Si,

The PR# 1697 with the global setting *snapshot.backup.rightafter** -
false* doesn't
work for VMware as create snapshot never takes a snapshot in Primary pool,
it just returns the snapshot uuid. The backup snapshot does the complete
job - creates a VM snapshot with the uuid, extracts and exports the target
volume to secondary. On demand backup snapshot doesn't work as there is no
snapshot in primary. Also, there'll be only one entry with Primary store
role in snapshot_store_ref, which is the latest snapshot taken for that
volume.

-Suresh

On Fri, May 18, 2018 at 1:03 AM, Simon Weller 
wrote:

> The whole point of the original PR was to optionally disable this
> functionality.
>
> We don't expose views of the backup state to our customers (we have our
> own customer interfaces) and it's a large waste of space for us to be
> backing up tons of VM images when we have a solid primary storage
> infrastructure that already has lots of resiliency.
>
>
> I guess we're going to have to revisit this again before we can consider
> rebasing on 4.11.
>
> 
> From: Suresh Kumar Anaparti 
> Sent: Thursday, May 17, 2018 2:21 PM
> To: dev
> Subject: Re: Snapshots only on Primary Storage feature
>
> Hi Si,
>
> No. not possible to disable the backup to secondary. It copies the volume
> snapshot to secondary in a background thread using asyncBackup param (set
> to true) and allows other operations during that time.
>
> I understand that the backup was on demand when any operations are
> performed on the snapshot. But, backup during that time may take
> considerable time (depending on the snapshot size and the network
> bandwidth), which can result in the job timeout and the User may assume
> that it is already Backed up based on its state, unless it is documented.
>
> -Suresh
>
> On Fri, May 18, 2018 at 12:23 AM, Simon Weller 
> wrote:
>
> > Suresh,
> >
> >
> > With this new merged  PR, is it possible to disable the backup to
> > secondary completely? I can't tell from the reference spec and we're not
> on
> > a 4.10/4.11 base yet.
> >
> > For the record, in the instances where a volume or template from snapshot
> > was required, the backup image was copied on demand to secondary.
> >
> > In an ideal world, secondary storage wouldn't even be involved in most of
> > these options, instead using the native clone features of the underlying
> > storage.
> >
> >
> > - Si
> >
> > 
> > From: Suresh Kumar Anaparti 
> > Sent: Thursday, May 17, 2018 1:37 PM
> > To: dev@cloudstack.apache.org
> > Cc: Nathan Johnson
> > Subject: Re: Snapshots only on Primary Storage feature
> >
> > Hi Glen / Si,
> >
> > In PR# 1697, the global setting *snapshot.backup.rightafter* if set to
> > true, it'll be the default behaviour and snapshot is copied to the
> > secondary storage. If set to false, then the snapshot state transitions
> are
> > mocked and Snapshot would be in BackedUp state even though it is not
> really
> > in Secondary storage, which doesn't make sense. Also, that will enable to
> > create a volume or template from the snapshot, which will obviously fail.
> >
> > This behavior was changed with the PR
> > https://github.com/apache/cloudstack/pull/2081. There is a clear
> > separation
> > of create and backup volume snapshot operations. The global setting
> > *snapshot.backup.rightafter* has been removed in PR# 2081.
> >
> > -Suresh
> >
> > On Thu, May 17, 2018 at 8:40 PM, Simon Weller 
> > wrote:
> >
> > > Glen,
> > >
> > >
> > > This feature was implemented in 4.9 by my colleague Nathan Johnson.
> You
> > > enable it by changing the global setting  snapshot.backup.rightafter to
> > > false.
> > >
> > >
> > > The PR is reference here: https://github.com/apache/
> cloudstack/pull/1697
> > >
> > >
> > > We have the exact same use case as you, as we also use Ceph.
> > >
> > >
> > > - Si
> > >
> > >
> > > 
> > > From: Glen Baars 
> > > Sent: Thursday, May 17, 2018 9:46 AM
> > > To: dev@cloudstack.apache.org
> > > Subject: Snapshots only on Primary Storage feature
> > >
> > >
> > > Hello Devs,
> > >
> > >
> > >
> > > I have been thinking about a feature request and want to see what
> people
> > > think about the use case.
> > >
> > >
> > >
> > > We use KVM + Ceph RBD as storage.
> > >
> > >
> > >
> > > Currently, when a client takes a snapshot, Cloudstack takes a Ceph
> > > snapshot and then uses qemu-img to export to secondary storage. This
> > > creates a full backup of the server. Clients want to use this as a
> daily
> > > snapshot and it isn’t feasible due to the space requirements.
> > >
> > >
> > >
> > > We would like create the snapshot only on primary storage. It is
> > > replicated offsite and fault tolerant. I can see that the download
> > snapshot
> > > and create 

Re: Dynamic roles question

[Daan Hoogland]

depends on the use case, for changing departments the move user is right,
for upgrading rights within a department probably not.

On Fri, May 18, 2018 at 8:59 AM, Ivan Kudryavtsev 
wrote:

> Hello, Daan, thank you for the clarification. It doesn't seems like the
> thing I'm talking about and actually, I don't think the approach enables
> the solving of the problem because VMs are owned by the account, and other
> related information like billing is connected with the account and, thus,
> changing the account is not the best way to fix the issue with changing the
> role. Don't think it's good for business cases.
>
> 2018-05-18 15:41 GMT+07:00 Daan Hoogland :
>
> > actually whet is in place is moving users between accounts, and thus
> > allowing for their role to change. The accounts would stay the same, but
> > the effective role of the users changes with it's account. MoverUserCmd
> is
> > the API.
> >
> > On Fri, May 18, 2018 at 7:44 AM, Ivan Kudryavtsev <
> > kudryavtsev...@bw-sw.com>
> > wrote:
> >
> > > Hello, Ilya.
> > > If it is already in place, then it's awesome. Thank you very much for
> > > assistance. I'll check it soon.
> > >
> > > 2018-05-18 14:37 GMT+07:00 ilya musayev 
> :
> > >
> > > > Ivan
> > > >
> > > > This is already done in 4.11, I’m not next to comp to check but
> > ShapeBlue
> > > > has a feature created that would allow for movement between different
> > > > roles.
> > > >
> > > > Regards
> > > > ilya
> > > >
> > > > On Thu, May 17, 2018 at 6:03 AM Ivan Kudryavtsev <
> > > kudryavtsev...@bw-sw.com
> > > > >
> > > > wrote:
> > > >
> > > > > Hello, community.
> > > > >
> > > > > I'm thinking about implementing the feature for accounts which
> > permits
> > > to
> > > > > change account role. Basically, the rationale is trials or
> > > demonstration
> > > > > modes which restricts users from doing extra stuff, like VM
> creation,
> > > > > service offering changes, etc. Basically, after trial the account
> > > should
> > > > be
> > > > > switched to a normal mode or removed. By permitting such role
> > switching
> > > > we
> > > > > can support the feature, otherwise we have to create unique role
> for
> > > > every
> > > > > user and manage it separately. Please, let me know your thoughts
> > about
> > > > > that. Have a good day.
> > > > >
> > > > > --
> > > > > With best regards, Ivan Kudryavtsev
> > > > > Bitworks Software, Ltd.
> > > > > Cell: +7-923-414-1515
> > > > > WWW: http://bitworks.software/ 
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > With best regards, Ivan Kudryavtsev
> > > Bitworks Software, Ltd.
> > > Cell: +7-923-414-1515
> > > WWW: http://bitworks.software/ 
> > >
> >
> >
> >
> > --
> > Daan
> >
>
>
>
> --
> With best regards, Ivan Kudryavtsev
> Bitworks Software, Ltd.
> Cell: +7-923-414-1515
> WWW: http://bitworks.software/ 
>



-- 
Daan

Re: Dynamic roles question

[Ivan Kudryavtsev]

Hello, Daan, thank you for the clarification. It doesn't seems like the
thing I'm talking about and actually, I don't think the approach enables
the solving of the problem because VMs are owned by the account, and other
related information like billing is connected with the account and, thus,
changing the account is not the best way to fix the issue with changing the
role. Don't think it's good for business cases.

2018-05-18 15:41 GMT+07:00 Daan Hoogland :

> actually whet is in place is moving users between accounts, and thus
> allowing for their role to change. The accounts would stay the same, but
> the effective role of the users changes with it's account. MoverUserCmd is
> the API.
>
> On Fri, May 18, 2018 at 7:44 AM, Ivan Kudryavtsev <
> kudryavtsev...@bw-sw.com>
> wrote:
>
> > Hello, Ilya.
> > If it is already in place, then it's awesome. Thank you very much for
> > assistance. I'll check it soon.
> >
> > 2018-05-18 14:37 GMT+07:00 ilya musayev :
> >
> > > Ivan
> > >
> > > This is already done in 4.11, I’m not next to comp to check but
> ShapeBlue
> > > has a feature created that would allow for movement between different
> > > roles.
> > >
> > > Regards
> > > ilya
> > >
> > > On Thu, May 17, 2018 at 6:03 AM Ivan Kudryavtsev <
> > kudryavtsev...@bw-sw.com
> > > >
> > > wrote:
> > >
> > > > Hello, community.
> > > >
> > > > I'm thinking about implementing the feature for accounts which
> permits
> > to
> > > > change account role. Basically, the rationale is trials or
> > demonstration
> > > > modes which restricts users from doing extra stuff, like VM creation,
> > > > service offering changes, etc. Basically, after trial the account
> > should
> > > be
> > > > switched to a normal mode or removed. By permitting such role
> switching
> > > we
> > > > can support the feature, otherwise we have to create unique role for
> > > every
> > > > user and manage it separately. Please, let me know your thoughts
> about
> > > > that. Have a good day.
> > > >
> > > > --
> > > > With best regards, Ivan Kudryavtsev
> > > > Bitworks Software, Ltd.
> > > > Cell: +7-923-414-1515
> > > > WWW: http://bitworks.software/ 
> > > >
> > >
> >
> >
> >
> > --
> > With best regards, Ivan Kudryavtsev
> > Bitworks Software, Ltd.
> > Cell: +7-923-414-1515
> > WWW: http://bitworks.software/ 
> >
>
>
>
> --
> Daan
>



-- 
With best regards, Ivan Kudryavtsev
Bitworks Software, Ltd.
Cell: +7-923-414-1515
WWW: http://bitworks.software/ 

Re: Dynamic roles question

[Daan Hoogland]

actually whet is in place is moving users between accounts, and thus
allowing for their role to change. The accounts would stay the same, but
the effective role of the users changes with it's account. MoverUserCmd is
the API.

On Fri, May 18, 2018 at 7:44 AM, Ivan Kudryavtsev 
wrote:

> Hello, Ilya.
> If it is already in place, then it's awesome. Thank you very much for
> assistance. I'll check it soon.
>
> 2018-05-18 14:37 GMT+07:00 ilya musayev :
>
> > Ivan
> >
> > This is already done in 4.11, I’m not next to comp to check but ShapeBlue
> > has a feature created that would allow for movement between different
> > roles.
> >
> > Regards
> > ilya
> >
> > On Thu, May 17, 2018 at 6:03 AM Ivan Kudryavtsev <
> kudryavtsev...@bw-sw.com
> > >
> > wrote:
> >
> > > Hello, community.
> > >
> > > I'm thinking about implementing the feature for accounts which permits
> to
> > > change account role. Basically, the rationale is trials or
> demonstration
> > > modes which restricts users from doing extra stuff, like VM creation,
> > > service offering changes, etc. Basically, after trial the account
> should
> > be
> > > switched to a normal mode or removed. By permitting such role switching
> > we
> > > can support the feature, otherwise we have to create unique role for
> > every
> > > user and manage it separately. Please, let me know your thoughts about
> > > that. Have a good day.
> > >
> > > --
> > > With best regards, Ivan Kudryavtsev
> > > Bitworks Software, Ltd.
> > > Cell: +7-923-414-1515
> > > WWW: http://bitworks.software/ 
> > >
> >
>
>
>
> --
> With best regards, Ivan Kudryavtsev
> Bitworks Software, Ltd.
> Cell: +7-923-414-1515
> WWW: http://bitworks.software/ 
>



-- 
Daan

Re: Dynamic roles question

[Ivan Kudryavtsev]

Hello, Ilya.
If it is already in place, then it's awesome. Thank you very much for
assistance. I'll check it soon.

2018-05-18 14:37 GMT+07:00 ilya musayev :

> Ivan
>
> This is already done in 4.11, I’m not next to comp to check but ShapeBlue
> has a feature created that would allow for movement between different
> roles.
>
> Regards
> ilya
>
> On Thu, May 17, 2018 at 6:03 AM Ivan Kudryavtsev  >
> wrote:
>
> > Hello, community.
> >
> > I'm thinking about implementing the feature for accounts which permits to
> > change account role. Basically, the rationale is trials or demonstration
> > modes which restricts users from doing extra stuff, like VM creation,
> > service offering changes, etc. Basically, after trial the account should
> be
> > switched to a normal mode or removed. By permitting such role switching
> we
> > can support the feature, otherwise we have to create unique role for
> every
> > user and manage it separately. Please, let me know your thoughts about
> > that. Have a good day.
> >
> > --
> > With best regards, Ivan Kudryavtsev
> > Bitworks Software, Ltd.
> > Cell: +7-923-414-1515
> > WWW: http://bitworks.software/ 
> >
>



-- 
With best regards, Ivan Kudryavtsev
Bitworks Software, Ltd.
Cell: +7-923-414-1515
WWW: http://bitworks.software/ 

Re: Dynamic roles question

[ilya musayev]

Ivan

This is already done in 4.11, I’m not next to comp to check but ShapeBlue
has a feature created that would allow for movement between different roles.

Regards
ilya

On Thu, May 17, 2018 at 6:03 AM Ivan Kudryavtsev 
wrote:

> Hello, community.
>
> I'm thinking about implementing the feature for accounts which permits to
> change account role. Basically, the rationale is trials or demonstration
> modes which restricts users from doing extra stuff, like VM creation,
> service offering changes, etc. Basically, after trial the account should be
> switched to a normal mode or removed. By permitting such role switching we
> can support the feature, otherwise we have to create unique role for every
> user and manage it separately. Please, let me know your thoughts about
> that. Have a good day.
>
> --
> With best regards, Ivan Kudryavtsev
> Bitworks Software, Ltd.
> Cell: +7-923-414-1515
> WWW: http://bitworks.software/ 
>