Re: [openstack-dev] [cinder] Dependencies of snapshots on volumes

[Duncan Thomas]

On 10 December 2015 at 04:48, Li, Xiaoyan  wrote:

>
> This leads to a problem like extending volume. Extending a volume in an
> incremental snapshot fails
> in vendor storage.  And then the cinder volume goes into error_extending
> status. From my opinion this is not good.
>

In any case where you see this, please file a bug. If you have time, please
propose a tempest test to check that this functionality works on all
drivers - all drivers *must* support the common Cinder API, the core
behaviours are not optional (though they are poorly documented and will
take time to fix).

I'm just starting a week's vacation, but I'll push up a tempest test for
this case when I get back if nobody else has, then we can see which CIs
fail and start raising bugs.


-- 
Duncan Thomas
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder] Dependencies of snapshots on volumes

[John Griffith]

On Tue, Dec 8, 2015 at 9:10 PM, Li, Xiaoyan  wrote:

> Hi all,
>
> Currently when deleting a volume, it checks whether there are snapshots
> created from it. If yes deletion is prohibited.  But it allows to extend
> the volume, no check whether there are snapshots from it.
>
​Correct​


>
> The two behaviors in Cinder are not consistent from my viewpoint.
>
​Well, your snapshot was taken at a point in time; and if you do a create
from snapshot the whole point is you want what you HAD when the snapshot
command was issued and NOT what happened afterwards.  So in my opinion this
is not inconsistent at all.
​


>
> In backend storage, their behaviors are same.
>
​Which backend storage are you referring to in this case?
​


> For full snapshot, if still in copying progress, both extend and deletion
> are not allowed. If snapshot copying finishes, both extend and deletion are
> allowed.
> For incremental snapshot, both extend and deletion are not allowed.
>

​So your particular backend has "different/specific" rules/requirements
around snapshots.  That's pretty common, I don't suppose theres any way to
hack around this internally?  In other words do things on your backend like
clones as snaps etc to make up for the differences in behavior?​


>
> As a result, this raises two concerns here:
> 1. Let such operations behavior same in Cinder.
> 2. I prefer to let storage driver decide the dependencies, not in the
> general core codes.
>

​I have and always will strongly disagree with this approach and your
proposal.  Sadly we've already started to allow more and more vendor
drivers just "do their own thing" and implement their own special API
methods.  This is in my opinion a horrible path and defeats the entire
purpose of have a Cinder abstraction layer.

This will make it impossible to have compatibility between clouds for those
that care about it, it will make it impossible for operators/deployers to
understand exactly what they can and should expect in terms of the usage of
their cloud.  Finally, it will also mean that not OpenStack API
functionality is COMPLETELY dependent on backend device.  I know people are
sick of hearing me say this, so I'll keep it short and say it one more time:
"Compatibility in the API matters and should always be our priority"


> Meanwhile, if we let driver to decide the dependencies, the following
> changes need to do in Cinder:
> 1. When creating a snapshot from volume, it needs copy all metadata of
> volume to snapshot. Currently it doesn't.
> Any other potential issues please let me know.
>
> Any input will be appreciated.
>
> Best wishes
> Lisa
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder] Dependencies of snapshots on volumes

[Arkady_Kanevsky]

You can do lazy copy that happens only when volume or snapshot is deleted.
You will need to have refcount on metadata.

-Original Message-
From: Li, Xiaoyan [mailto:xiaoyan...@intel.com]
Sent: Tuesday, December 08, 2015 10:11 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [cinder] Dependencies of snapshots on volumes

Hi all,

Currently when deleting a volume, it checks whether there are snapshots created 
from it. If yes deletion is prohibited. But it allows to extend the volume, no 
check whether there are snapshots from it.

The two behaviors in Cinder are not consistent from my viewpoint.

In backend storage, their behaviors are same.
For full snapshot, if still in copying progress, both extend and deletion are 
not allowed. If snapshot copying finishes, both extend and deletion are allowed.
For incremental snapshot, both extend and deletion are not allowed.

As a result, this raises two concerns here:
1. Let such operations behavior same in Cinder.
2. I prefer to let storage driver decide the dependencies, not in the general 
core codes.

Meanwhile, if we let driver to decide the dependencies, the following changes 
need to do in Cinder:
1. When creating a snapshot from volume, it needs copy all metadata of volume 
to snapshot. Currently it doesn't.
Any other potential issues please let me know.

Any input will be appreciated.

Best wishes
Lisa


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder] Dependencies of snapshots on volumes

[Mike Perez]

On 09:27 Dec 09, John Griffith wrote:
> On Tue, Dec 8, 2015 at 9:10 PM, Li, Xiaoyan  wrote:



> > As a result, this raises two concerns here:
> > 1. Let such operations behavior same in Cinder.
> > 2. I prefer to let storage driver decide the dependencies, not in the
> > general core codes.
> >
> 
> ​I have and always will strongly disagree with this approach and your
> proposal.  Sadly we've already started to allow more and more vendor
> drivers just "do their own thing" and implement their own special API
> methods.  This is in my opinion a horrible path and defeats the entire
> purpose of have a Cinder abstraction layer.
> 
> This will make it impossible to have compatibility between clouds for those
> that care about it, it will make it impossible for operators/deployers to
> understand exactly what they can and should expect in terms of the usage of
> their cloud.  Finally, it will also mean that not OpenStack API
> functionality is COMPLETELY dependent on backend device.  I know people are
> sick of hearing me say this, so I'll keep it short and say it one more time:
> "Compatibility in the API matters and should always be our priority"

+1

-- 
Mike Perez

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder] Dependencies of snapshots on volumes

[Li, Xiaoyan]

On Dec 10, 2015 06:34, Mike Perez wrote:
> On 09:27 Dec 09, John Griffith wrote:
>> On Tue, Dec 8, 2015 at 9:10 PM, Li, Xiaoyan 
> wrote:
> 
> 
> 
>>> As a result, this raises two concerns here:
>>> 1. Let such operations behavior same in Cinder.
>>> 2. I prefer to let storage driver decide the dependencies, not in
>>> the general core codes.
>>> 
>> 
>> ​I have and always will strongly disagree with this approach and your
>> proposal.  Sadly we've already started to allow more and more vendor
>> drivers just "do their own thing" and implement their own special API
>> methods.  This is in my opinion a horrible path and defeats the entire
>> purpose of have a Cinder abstraction layer.
>> 
>> This will make it impossible to have compatibility between clouds for
>> those that care about it, it will make it impossible for
>> operators/deployers to understand exactly what they can and should
>> expect in terms of the usage of their cloud.  Finally, it will also
>> mean that not OpenStack API functionality is COMPLETELY dependent on
>> backend device.  I know people are sick of hearing me say this, so I'll
>> keep it short and say it one more time: "Compatibility in the API
>> matters and should always be our priority"
> 
> +1
>

This seems that cinder needs to take more and more works, and vendor storages 
do what cinder asks them to. 
For example, cinder supports full and incremental snapshots in core codes, and 
it keeps the dependencies like backups. 

More general example is that storage vendors supports kinds of volumes, like 
normal, provisioned, and compressed etc. 
Cinder needs to implement such functions in core codes. Every storage vendor 
report their capability to cinder scheduler. 
When users create a volume, scheduler finds a storage vendor based on their 
capacities. 
(Of course these functions in cinder should be general and implemented by most 
of vendor storages. )

But currently cinder core codes do little, lots of this are in extra_specs, 
conf file which are handled in vendor drivers. 

This leads to a problem like extending volume. Extending a volume in an 
incremental snapshot fails
in vendor storage.  And then the cinder volume goes into error_extending 
status. From my opinion this is not good. 

Best wishes
Lisa


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder] Dependencies of snapshots on volumes

[Chris Friesen]

On 12/09/2015 10:27 AM, John Griffith wrote:



On Tue, Dec 8, 2015 at 9:10 PM, Li, Xiaoyan > wrote:

Hi all,

Currently when deleting a volume, it checks whether there are snapshots
created from it. If yes deletion is prohibited.  But it allows to extend
the volume, no check whether there are snapshots from it.

​Correct​


The two behaviors in Cinder are not consistent from my viewpoint.

​Well, your snapshot was taken at a point in time; and if you do a create from
snapshot the whole point is you want what you HAD when the snapshot command was
issued and NOT what happened afterwards.  So in my opinion this is not
inconsistent at all.


If we look at it a different way...suppose that the snapshot is linked in a 
copy-on-write manner with the original volume.  If someone deletes the original 
volume then the snapshot is in trouble.  However, if someone modifies the 
original volume then a new chunk of backing store is allocated for the original 
volume and the snapshot still references the original contents.


If we did allow deletion of the volume we'd have to either keep the volume 
backing store around as long as any snapshots are around, or else flatten any 
snapshots so they're no longer copy-on-write.


Chris

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder] Dependencies of snapshots on volumes

[Jordan Pittier]

Hi,
FWIW, I completely agree with what John said. All of it.

Please don't do that.

Jordan
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [cinder] Dependencies of snapshots on volumes

[Li, Xiaoyan]

Hi all, 

Currently when deleting a volume, it checks whether there are snapshots created 
from it. If yes deletion is prohibited.  But it allows to extend  the volume, 
no check whether there are snapshots from it.

The two behaviors in Cinder are not consistent from my viewpoint. 

In backend storage, their behaviors are same.
For full snapshot, if still in copying progress, both extend and deletion are 
not allowed. If snapshot copying finishes, both extend and deletion are allowed.
For incremental snapshot, both extend and deletion are not allowed. 

As a result, this raises two concerns here: 
1. Let such operations behavior same in Cinder.
2. I prefer to let storage driver decide the dependencies, not in the general 
core codes. 

Meanwhile, if we let driver to decide the dependencies, the following changes 
need to do in Cinder: 
1. When creating a snapshot from volume, it needs copy all metadata of volume 
to snapshot. Currently it doesn't. 
Any other potential issues please let me know.

Any input will be appreciated. 

Best wishes
Lisa


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev