Re: [openstack-dev] [tempest][cinder] Clone feature toggle not in clone tests

[Jay S. Bryant]

Ben,

Thank you very much for the history on the driver here.  This was all 
news to me.  It helps me to understand why cloning was listed as 
supported but was no longer implemented.  Also explains why the simpler 
'cp' based solution hasn't been implemented.  I was wondering why that 
hadn't just been done.


IBM is planning to take over the process of developing/maintaining the 
NFS driver, at least in the near term. Erlon has also been helping with 
this.


We would really like to avoid the driver being removed.  Perhaps I will 
send a separate 'State of the NFS driver' update e-mail to the mailing 
list to explain what I think the state of the driver is and what I see 
to be the plan going forward.


Thanks,

Jay


On 08/25/2016 12:06 PM, Ben Swartzlander wrote:


Originally the NFS driver did support snapshots, but it was 
implemented by just 'cp'ing the file containing the raw bits. This 
works fine (if inefficiently) for unattached volumes, but if you do 
this on an attached volume the snapshot won't be crash consistent at all.


It was decided that we could do better for attached volumes by 
switching to qcow2 and relying on nova to perform the snapshots. Based 
on this, the bad snapshot implementation was removed.


However, for a variety of reasons the nova-assisted snapshot 
implementation has remained unmerged for 2+ years and the NFS driver 
has been an exception to the rules for that whole time.


I would like to see that exception end in the near future with either 
the removal of the driver or the completion of the Nova-assisted 
snapshot implementation, and it doesn't really matter to me which.


There is a 3rd alternative which would be to modify the NFS driver to 
require a specific filesystem that supports snapshots (there are a few 
choices here, but definitely NOT ext4). Unfortunately those of us who 
work for storage vendors aren't motivated to make such a modification 
because it would be effectively creating more competition for 
ourselves. The only way this could happen is if someone not working 
for a storage vendor takes this on.


-Ben

On August 25, 2016 10:39:35 AM Erlon Cruz  wrote:


Hi Jordan, Slade,

Currently NFS driver does not support cloning neither snapshots 
(which are the base for implementing cloning). AFAIC, the NFS driver 
was in Cinder before the minimum requirements being discussed and 
set, so, it just stood there with the features it already supported.


There is currently this job 
'gate-tempest-dsvm-full-devstack-plugin-nfs-nv'[1] that by the way 
are failing in the same test you mentioned tough passing the snapshot 
tests (not shure how the configuration is doing that) and a work[2] 
in progress to support the snapshot feature.


So, Jordan, I think its OK to allow tempest to skip this tests, 
provided that at least in the NFS driver, tempest isn't being an 
enforcement to Cinder minimum features requirements.


Erlon


[1] 
http://logs.openstack.org/86/147186/25/experimental/gate-tempest-dsvm-full-devstack-plugin-nfs-nv/b149960/

[2] https://review.openstack.org/#/c/147186/

On Wed, Aug 24, 2016 at 6:34 PM, Jordan Pittier 
> wrote:



On Wed, Aug 24, 2016 at 6:06 PM, Slade Baumann
> wrote:

I am attempting to disable clone tests in tempest as they aren't
functioning in NFS. But the tests test_volumes_clone.py and
test_volumes_clone_negative.py don't have the "clone" feature
toggle in them. I thought it obvious that if clone is disabled
in tempest, the tests that simply clone should be disabled.

So I put up a bug and fix for it, but have been talking with
Jordan Pittier and he suggested I come to the mailing list to
get this figured out.

I'm not asking for reviews, unless you want to give them.
I'm simply asking if this is the right way to go about this
or if there is something else I need to do to get this into
Tempest.

Here are the bug and fix:
https://bugs.launchpad.net/tempest/+bug/1615770

https://review.openstack.org/#/c/358813/


I would appreciate any suggestion or direction in this problem.

For extra reference, the clone toggle flag was added here:
https://bugs.launchpad.net/tempest/+bug/1488274


Hi,
Thanks for starting this thread. My point about this patch is, as
"volume clone" is part of the core requirements [1] every Cinder
drive must support, I don't see a need for a feature flag. The
feature flag already exists, but that doesn't mean we should
encourage its usage.

Now, if this really helps the NFS driver (although I don"t know
why we couldn't support clone with 

Re: [openstack-dev] [tempest][cinder] Clone feature toggle not in clone tests

[Erlon Cruz]

He is mentioning about the Cinder side: https://review.openstack.org/#
/c/147186/

On Fri, Aug 26, 2016 at 7:01 AM, Jordan Pittier 
wrote:

>
>
> On Thu, Aug 25, 2016 at 7:06 PM, Ben Swartzlander 
> wrote:
>
>> Originally the NFS driver did support snapshots, but it was implemented
>> by just 'cp'ing the file containing the raw bits. This works fine (if
>> inefficiently) for unattached volumes, but if you do this on an attached
>> volume the snapshot won't be crash consistent at all.
>>
>> It was decided that we could do better for attached volumes by switching
>> to qcow2 and relying on nova to perform the snapshots. Based on this, the
>> bad snapshot implementation was removed.
>>
>> However, for a variety of reasons the nova-assisted snapshot
>> implementation has remained unmerged for 2+ years and the NFS driver has
>> been an exception to the rules for that whole time.
>>
> I am not sure to understand what you mean by "the nova-assisted snapshot
> implementation has remained unmerged for 2+ years". It looks merged to me
> [1] and several Cinder drivers dependent on it as far as I know.
>
> [1]: http://developer.openstack.org/api-ref-compute-
> v2.1.html#os-assisted-volume-snapshots-v2.1
>
> 
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tempest][cinder] Clone feature toggle not in clone tests

[Jordan Pittier]

On Thu, Aug 25, 2016 at 7:06 PM, Ben Swartzlander 
wrote:

> Originally the NFS driver did support snapshots, but it was implemented by
> just 'cp'ing the file containing the raw bits. This works fine (if
> inefficiently) for unattached volumes, but if you do this on an attached
> volume the snapshot won't be crash consistent at all.
>
> It was decided that we could do better for attached volumes by switching
> to qcow2 and relying on nova to perform the snapshots. Based on this, the
> bad snapshot implementation was removed.
>
> However, for a variety of reasons the nova-assisted snapshot
> implementation has remained unmerged for 2+ years and the NFS driver has
> been an exception to the rules for that whole time.
>
I am not sure to understand what you mean by "the nova-assisted snapshot
implementation has remained unmerged for 2+ years". It looks merged to me
[1] and several Cinder drivers dependent on it as far as I know.

[1]:
http://developer.openstack.org/api-ref-compute-v2.1.html#os-assisted-volume-snapshots-v2.1

-- 
 
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tempest][cinder] Clone feature toggle not in clone tests

[Slade Baumann]

Ben,In response to your comments, we are working on getting NFS up to date at IBM as we are using it as our storage backend (long story, don't ask lol). A few of use used to focus on Cinder development, so we have a good chance of getting things working. So I agree wholeheartedly that this needs to be resolved and we are working towards that goal (as well as fixing clone, etc).P.S. Thanks for you history of the NFS driver. I didn't know a lot of that.Slade-Ben Swartzlander <b...@swartzlander.org> wrote: -To: "OpenStack Development Mailing List (not for usage questions)" <openstack-dev@lists.openstack.org>, Erlon Cruz <sombra...@gmail.com>From: Ben Swartzlander <b...@swartzlander.org>Date: 08/25/2016 12:10PMSubject: Re: [openstack-dev] [tempest][cinder] Clone feature toggle not inclone tests


Originally the NFS driver did
support snapshots, but it was implemented by just 'cp'ing the file
containing the raw bits. This works fine (if inefficiently) for unattached
volumes, but if you do this on an attached volume the snapshot won't be
crash consistent at all.It was decided that we could do
better for attached volumes by switching to qcow2 and relying on nova to
perform the snapshots. Based on this, the bad snapshot implementation was
removed.However, for a variety of
reasons the nova-assisted snapshot implementation has remained unmerged for
2+ years and the NFS driver has been an exception to the rules for that
whole time.I would like to see that
exception end in the near future with either the removal of the driver or
the completion of the Nova-assisted snapshot implementation, and it doesn't
really matter to me which.There is a 3rd alternative
which would be to modify the NFS driver to require a specific filesystem
that supports snapshots (there are a few choices here, but definitely NOT
ext4). Unfortunately those of us who work for storage vendors aren't
motivated to make such a modification because it would be effectively
creating more competition for ourselves. The only way this could happen is
if someone not working for a storage vendor takes this on.-BenOn
August 25, 2016 10:39:35 AM Erlon Cruz <sombra...@gmail.com> wrote:Hi Jordan, Slade,Currently NFS driver
does not support cloning neither snapshots (which are the base for
implementing cloning). AFAIC, the NFS driver was in Cinder before the
minimum requirements being discussed and set, so, it just stood there with
the features it already supported.There is
currently this job
'gate-tempest-dsvm-full-devstack-plugin-nfs-nv'[1] that by the way
are failing in the same test you mentioned tough passing the snapshot tests
(not shure how the configuration is doing that) and a work[2] in progress
to support the snapshot feature.So, Jordan, I
think its OK to allow tempest to skip this tests, provided that at least in
the NFS driver, tempest isn't being an enforcement to Cinder minimum
features
requirements.Erlon[1]
http://logs.openstack.org/86/147186/25/experimental/gate-tempest-dsvm-full-devstack-plugin-nfs-nv/b149960/[2]
https://review.openstack.org/#/c/147186/ On Wed, Aug 24, 2016 at
6:34 PM, Jordan Pittier <jordan.pitt...@scality.com>wrote:On Wed, Aug 24, 2016 at 6:06 PM, Slade Baumann <baum...@us.ibm.com> wrote:I am attempting to
disable clone tests in tempest as they aren'tfunctioning in NFS.
But the tests test_volumes_clone.py andtest_volumes_clone_negative.py
don't have the "clone" featuretoggle in them. I thought
it obvious that if clone is disabledin tempest, the tests that simply
clone should be disabled.So I put up a bug and fix for it, but have
been talking withJordan Pittier and he suggested I come to the mailing
list toget this figured out. I'm not asking for reviews,
unless you want to give them.I'm simply asking if this is the right
way to go about thisor if there is something else I need to do to get
this intoTempest.Here are the bug and fix:https://bugs.launchpad.net/tempest/+bug/1615770https://review.openstack.org/#/c/358813/I
would appreciate any suggestion or direction in this problem.For
extra reference, the clone toggle flag was added here:https://bugs.launchpad.net/tempest/+bug/1488274

Hi, Thanks for starting this
thread. My point about this patch is, as "volume clone" is part
of the core requirements [1] every Cinder drive must support, I don't
see a need for a feature flag. The feature flag already exists, but that
doesn't mean we should encourage its
usage.Now, if this really helps the NFS driver
(although I don"t know why we couldn't support clone with NFS)...
I don't have a strong opinion on this patch.I
-1ed the patch for consistency: I agree that there should be a minimum set
of features expected from a Cinder driver.[1] http://docs.openstack.org/developer/cinder/devref/drivers.html#core-functionalityCheers,Jordan 
__OpenStack Development Mailing List (n

Re: [openstack-dev] [tempest][cinder] Clone feature toggle not in clone tests

[Ben Swartzlander]

Originally the NFS driver did support snapshots, but it was implemented by 
just 'cp'ing the file containing the raw bits. This works fine (if 
inefficiently) for unattached volumes, but if you do this on an attached 
volume the snapshot won't be crash consistent at all.


It was decided that we could do better for attached volumes by switching to 
qcow2 and relying on nova to perform the snapshots. Based on this, the bad 
snapshot implementation was removed.


However, for a variety of reasons the nova-assisted snapshot implementation 
has remained unmerged for 2+ years and the NFS driver has been an exception 
to the rules for that whole time.


I would like to see that exception end in the near future with either the 
removal of the driver or the completion of the Nova-assisted snapshot 
implementation, and it doesn't really matter to me which.


There is a 3rd alternative which would be to modify the NFS driver to 
require a specific filesystem that supports snapshots (there are a few 
choices here, but definitely NOT ext4). Unfortunately those of us who work 
for storage vendors aren't motivated to make such a modification because it 
would be effectively creating more competition for ourselves. The only way 
this could happen is if someone not working for a storage vendor takes this on.


-Ben


On August 25, 2016 10:39:35 AM Erlon Cruz  wrote:


Hi Jordan, Slade,

Currently NFS driver does not support cloning neither snapshots (which are
the base for implementing cloning). AFAIC, the NFS driver was in Cinder
before the minimum requirements being discussed and set, so, it just stood
there with the features it already supported.

There is currently this job
'gate-tempest-dsvm-full-devstack-plugin-nfs-nv'[1] that by the way are
failing in the same test you mentioned tough passing the snapshot tests
(not shure how the configuration is doing that) and a work[2] in progress
to support the snapshot feature.

So, Jordan, I think its OK to allow tempest to skip this tests, provided
that at least in the NFS driver, tempest isn't being an enforcement to
Cinder minimum features requirements.

Erlon


[1]
http://logs.openstack.org/86/147186/25/experimental/gate-tempest-dsvm-full-devstack-plugin-nfs-nv/b149960/
[2] https://review.openstack.org/#/c/147186/

On Wed, Aug 24, 2016 at 6:34 PM, Jordan Pittier 
wrote:



On Wed, Aug 24, 2016 at 6:06 PM, Slade Baumann  wrote:


I am attempting to disable clone tests in tempest as they aren't
functioning in NFS. But the tests test_volumes_clone.py and
test_volumes_clone_negative.py don't have the "clone" feature
toggle in them. I thought it obvious that if clone is disabled
in tempest, the tests that simply clone should be disabled.

So I put up a bug and fix for it, but have been talking with
Jordan Pittier and he suggested I come to the mailing list to
get this figured out.

I'm not asking for reviews, unless you want to give them.
I'm simply asking if this is the right way to go about this
or if there is something else I need to do to get this into
Tempest.

Here are the bug and fix:
https://bugs.launchpad.net/tempest/+bug/1615770
https://review.openstack.org/#/c/358813/

I would appreciate any suggestion or direction in this problem.

For extra reference, the clone toggle flag was added here:
https://bugs.launchpad.net/tempest/+bug/1488274

Hi,

Thanks for starting this thread. My point about this patch is, as "volume
clone" is part of the core requirements [1] every Cinder drive must
support, I don't see a need for a feature flag. The feature flag already
exists, but that doesn't mean we should encourage its usage.

Now, if this really helps the NFS driver (although I don"t know why we
couldn't support clone with NFS)... I don't have a strong opinion on this
patch.

I -1ed the patch for consistency: I agree that there should be a minimum
set of features expected from a Cinder driver.

[1] http://docs.openstack.org/developer/cinder/devref/drivers.html#core-
functionality

Cheers,
Jordan


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev






--
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tempest][cinder] Clone feature toggle not in clone tests

[Erlon Cruz]

Hi Jordan, Slade,

Currently NFS driver does not support cloning neither snapshots (which are
the base for implementing cloning). AFAIC, the NFS driver was in Cinder
before the minimum requirements being discussed and set, so, it just stood
there with the features it already supported.

There is currently this job
'gate-tempest-dsvm-full-devstack-plugin-nfs-nv'[1] that by the way are
failing in the same test you mentioned tough passing the snapshot tests
(not shure how the configuration is doing that) and a work[2] in progress
to support the snapshot feature.

So, Jordan, I think its OK to allow tempest to skip this tests, provided
that at least in the NFS driver, tempest isn't being an enforcement to
Cinder minimum features requirements.

Erlon


[1]
http://logs.openstack.org/86/147186/25/experimental/gate-tempest-dsvm-full-devstack-plugin-nfs-nv/b149960/
[2] https://review.openstack.org/#/c/147186/

On Wed, Aug 24, 2016 at 6:34 PM, Jordan Pittier 
wrote:

>
> On Wed, Aug 24, 2016 at 6:06 PM, Slade Baumann  wrote:
>
>> I am attempting to disable clone tests in tempest as they aren't
>> functioning in NFS. But the tests test_volumes_clone.py and
>> test_volumes_clone_negative.py don't have the "clone" feature
>> toggle in them. I thought it obvious that if clone is disabled
>> in tempest, the tests that simply clone should be disabled.
>>
>> So I put up a bug and fix for it, but have been talking with
>> Jordan Pittier and he suggested I come to the mailing list to
>> get this figured out.
>>
>> I'm not asking for reviews, unless you want to give them.
>> I'm simply asking if this is the right way to go about this
>> or if there is something else I need to do to get this into
>> Tempest.
>>
>> Here are the bug and fix:
>> https://bugs.launchpad.net/tempest/+bug/1615770
>> https://review.openstack.org/#/c/358813/
>>
>> I would appreciate any suggestion or direction in this problem.
>>
>> For extra reference, the clone toggle flag was added here:
>> https://bugs.launchpad.net/tempest/+bug/1488274
>>
>> Hi,
> Thanks for starting this thread. My point about this patch is, as "volume
> clone" is part of the core requirements [1] every Cinder drive must
> support, I don't see a need for a feature flag. The feature flag already
> exists, but that doesn't mean we should encourage its usage.
>
> Now, if this really helps the NFS driver (although I don"t know why we
> couldn't support clone with NFS)... I don't have a strong opinion on this
> patch.
>
> I -1ed the patch for consistency: I agree that there should be a minimum
> set of features expected from a Cinder driver.
>
> [1] http://docs.openstack.org/developer/cinder/devref/drivers.html#core-
> functionality
>
> Cheers,
> Jordan
>
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tempest][cinder] Clone feature toggle not in clone tests

[Jordan Pittier]

On Wed, Aug 24, 2016 at 6:06 PM, Slade Baumann  wrote:

> I am attempting to disable clone tests in tempest as they aren't
> functioning in NFS. But the tests test_volumes_clone.py and
> test_volumes_clone_negative.py don't have the "clone" feature
> toggle in them. I thought it obvious that if clone is disabled
> in tempest, the tests that simply clone should be disabled.
>
> So I put up a bug and fix for it, but have been talking with
> Jordan Pittier and he suggested I come to the mailing list to
> get this figured out.
>
> I'm not asking for reviews, unless you want to give them.
> I'm simply asking if this is the right way to go about this
> or if there is something else I need to do to get this into
> Tempest.
>
> Here are the bug and fix:
> https://bugs.launchpad.net/tempest/+bug/1615770
> https://review.openstack.org/#/c/358813/
>
> I would appreciate any suggestion or direction in this problem.
>
> For extra reference, the clone toggle flag was added here:
> https://bugs.launchpad.net/tempest/+bug/1488274
>
> Hi,
Thanks for starting this thread. My point about this patch is, as "volume
clone" is part of the core requirements [1] every Cinder drive must
support, I don't see a need for a feature flag. The feature flag already
exists, but that doesn't mean we should encourage its usage.

Now, if this really helps the NFS driver (although I don"t know why we
couldn't support clone with NFS)... I don't have a strong opinion on this
patch.

I -1ed the patch for consistency: I agree that there should be a minimum
set of features expected from a Cinder driver.

[1]
http://docs.openstack.org/developer/cinder/devref/drivers.html#core-functionality

Cheers,
Jordan

-- 
 
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [tempest][cinder] Clone feature toggle not in clone tests

[Slade Baumann]

I am attempting to disable clone tests in tempest as they aren'tfunctioning in NFS. But the tests test_volumes_clone.py andtest_volumes_clone_negative.py don't have the "clone" featuretoggle in them. I thought it obvious that if clone is disabledin tempest, the tests that simply clone should be disabled.So I put up a bug and fix for it, but have been talking withJordan Pittier and he suggested I come to the mailing list toget this figured out. I'm not asking for reviews, unless you want to give them.I'm simply asking if this is the right way to go about thisor if there is something else I need to do to get this intoTempest.Here are the bug and fix:https://bugs.launchpad.net/tempest/+bug/1615770https://review.openstack.org/#/c/358813/I would appreciate any suggestion or direction in this problem.For extra reference, the clone toggle flag was added here:https://bugs.launchpad.net/tempest/+bug/1488274


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev