[openstack-dev] [Cinder] [Manila] NetApp CI will be offline next week

[Kerr, Andrew]

Hi all,

The physical equipment that runs the NetApp CI system is scheduled to be moved 
to a new building next week.  In order to accommodate this move we will be 
taking the NetApp CI system offline at EOB on Friday Jan 15 and plan to have it 
back up and running by EOB on Monday Jan 25.

I just wanted to give a heads up in case anyone notices the radio silence from 
the netapp-ci account.

Andrew Kerr
QA, OpenStack
Data Fabric Group
NetApp
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder] Taskflow 0.10.0 incompatible with NetApp NFS drivers

[Kerr, Andrew]

The problem is in the version of taskflow that is downloaded from pypi by 
devstack.  You will need to wait until a new version >0.10.0 is available [1]

[1] https://pypi.python.org/pypi/taskflow/

Andrew Kerr
OpenStack QA
Cloud Solutions Group
NetApp

From: Bharat Kumar 
mailto:bharat.kobag...@redhat.com>>
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
mailto:openstack-dev@lists.openstack.org>>
Date: Friday, May 8, 2015 at 7:37 AM
To: 
"openstack-dev@lists.openstack.org" 
mailto:openstack-dev@lists.openstack.org>>
Subject: Re: [openstack-dev] [cinder] Taskflow 0.10.0 incompatible with NetApp 
NFS drivers

GlusterFS CI job is still failing with the same issue.

I gave couple of "recheck"s on [1], after 
https://review.openstack.org/#/c/181288/ patch got merged.

But still GlusterFS CI job is failing with below error [2]:
ObjectDereferencedError: Can't emit change event for attribute 
'Volume.provider_location' - parent object of type  has been garbage 
collected.

Also I found the same behaviour with NetApp CI also.


[1] https://review.openstack.org/#/c/165424/
[2] 
http://logs.openstack.org/24/165424/6/check/check-tempest-dsvm-full-glusterfs-nv/f386477/logs/screen-c-vol.txt.gz


On 05/08/2015 10:21 AM, Joshua Harlow wrote:
Alright, it was as I had a hunch for, a small bug found in the new algorithm to 
make the storage layer copy-original,mutate-copy,save-copy,update-original (vs 
update-original,save-original) more reliable.

https://bugs.launchpad.net/taskflow/+bug/1452978 opened and a one line fix made 
@ https://review.openstack.org/#/c/181288/ to stop trying to copy task results 
(which was activating logic that must of caused the reference to drop out of 
existence and therefore the issue noted below).

Will get that released in 0.10.1 once it flushes through the pipeline.

Thanks alex for helping double check, if others want to check to that'd be 
nice, can make sure that's the root cause (overzealous usage of copy.copy, ha).

Overall I'd still *highly* recommend that the following still happen:

>> One way to get around whatever the issue is would be to change the
>> drivers to not update the object directly as it is not needed. But
>> this should not fail. Perhaps a more proper fix is for the volume
>> manager to not pass around sqlalchemy objects.

But that can be a later tweak that cinder does; using any taskflow engine that 
isn't the greenthreaded/threaded/serial engine will require results to be 
serializable, and therefore copyable, so that those results can go across IPC 
or MQ/other boundaries. Sqlalchemy objects won't fit either of these cases 
(obviously).

-Josh

Joshua Harlow wrote:
Are we sure this is taskflow? I'm wondering since those errors are more
from task code (which is in cinder) and the following seems to be a
general garbage collection issue (not connected to taskflow?):

'Exception during message handling: Can't emit change event for
attribute 'Volume.provider_location' - parent object of type 
has been garbage collected.'''

Or:

'''2015-05-07 22:42:51.142 17040 TRACE oslo_messaging.rpc.dispatcher
ObjectDereferencedError: Can't emit change event for attribute
'Volume.provider_location' - parent object of type  has been
garbage collected.'''

Alex Meade wrote:
So it seems that this will break a number of drivers, I see that
glusterfs does the same thing.

On Thu, May 7, 2015 at 10:29 PM, Alex Meade 
mailto:mr.alex.me...@gmail.com>
> wrote:

It appears that the release of taskflow 0.10.0 exposed an issue in
the NetApp NFS drivers. Something changed that caused the sqlalchemy
Volume object to be garbage collected even though it is passed into
create_volume()

An example error can be found in the c-vol logs here:

http://dcf901611175aa43f968-c54047c910227e27e1d6f03bb1796fd7.r95.cf5.rackcdn.com/57/181157/1/check/cinder-cDOT-NFS/0473c54/


One way to get around whatever the issue is would be to change the
drivers to not update the object directly as it is not needed. But
this should not fail. Perhaps a more proper fix is for the volume
manager to not pass around sqlalchemy objects.

+1


Something changed in taskflow, however, and we should just
understand if that has other impact.

I'd like to understand that also: the only one commit that touched this
stuff is https://github.com/openstack/taskflow/commit/227cf52 (which
basically ensured that a storage object copy is modified, then saved,
then the local object is updated vs updating the local object, and then
saving, which has problems/inconsistencies if the save fails).


-Alex


__

OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mai

Re: [openstack-dev] [Cinder] 3'rd party CI systems: Not Whitelisted Volume type with name * could not be found.

[Kerr, Andrew]

Can the default_volume_type be left empty and get the original "None"
type, so we don't have to create the volume type prior to running tempest
tests?

Andrew Kerr
OpenStack QA
Cloud Solutions Group
NetApp



On 8/14/14, 5:23 PM, "Martin, Kurt Frederick (ESSN Storage MSDU)"
 wrote:

>They need to be manually added.
>
>-Original Message-
>From: Asselin, Ramy
>Sent: Thursday, August 14, 2014 2:17 PM
>To: OpenStack Development Mailing List (not for usage questions)
>Subject: Re: [openstack-dev] [Cinder] 3'rd party CI systems: Not
>Whitelisted Volume type with name * could not be found.
>
>Both configurations have that set as you described. [1][2] Who actually
>creates that volume type? Is that supposed to be added manually to
>local.sh, or is this a bug in devstack?
>
>[1] 
>http://publiclogs.emc.com/vnx_ostack/EMC_VNX_FC/212/logs/etc/cinder/cinder
>.conf.txt.gz
>[DEFAULT]
>default_volume_type = cat_1
>
>[2] 
>http://15.126.198.151/60/111460/18/check/dsvm-tempest-hp-lefthand/3ee1598/
>logs/etc/cinder/cinder.conf.txt.gz
>[DEFAULT]
>default_volume_type = dot
>
>-Original Message-
>From: Martin, Kurt Frederick (ESSN Storage MSDU)
>Sent: Thursday, August 14, 2014 2:00 PM
>To: OpenStack Development Mailing List (not for usage questions)
>Subject: Re: [openstack-dev] [Cinder] 3'rd party CI systems: Not
>Whitelisted Volume type with name * could not be found.
>
>Cinder.conf needs to have a default_volume_type  entry set under the
>[Default] group and a volume type that is valid, for example,
>default_volume_type=bronze. This allows for a volume to be created when a
>volume type is not selected, the default 'None'  type. This feature has
>been available for some time in cinder but recently enabled in devstack.
>~Kurt
>
>-Original Message-
>From: Asselin, Ramy
>Sent: Thursday, August 14, 2014 10:00 AM
>To: OpenStack Development Mailing List (not for usage questions)
>Subject: [openstack-dev] [Cinder] 3'rd party CI systems: Not Whitelisted
>Volume type with name * could not be found.
>
>Hi,
>
>Does anyone know how to configure cinder ci tests to not have these
>errors?
>
>12:32:11 *** Not Whitelisted *** 2014-08-14 12:23:56.179 18021 ERROR
>cinder.volume.volume_types [req-c9ec92ab-132b-4167-a467-15bd213659e8
>bd1f54a867ce47acb4097cd94149efa0 d15dcf4cb3c7495389799ec849e9036d - - -]
>Default volume type is not found, please check default_volume_type
>config: Volume type with name dot could not be found.
>
>17:43:28 *** Not Whitelisted *** 2014-08-11 17:31:20.343 2097 ERROR
>cinder.volume.volume_types [req-01e539ad-5357-4a0f-8d58-464b970114f9
>f72cd499be584d9d9585bc26ca71c603 d845ad2d7e6a47dfb84bdf0754f60384 - - -]
>Default volume type is not found, please check default_volume_type
>config: Volume type with name cat_1 could not be found.
>
>http://15.126.198.151/60/111460/18/check/dsvm-tempest-hp-lefthand/3ee1598/
>console.html
>http://publiclogs.emc.com/vnx_ostack/EMC_VNX_FC/212/console.html
>
>Thanks,
>Ramy
>
>___
>OpenStack-dev mailing list
>OpenStack-dev@lists.openstack.org
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>___
>OpenStack-dev mailing list
>OpenStack-dev@lists.openstack.org
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>___
>OpenStack-dev mailing list
>OpenStack-dev@lists.openstack.org
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>___
>OpenStack-dev mailing list
>OpenStack-dev@lists.openstack.org
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Devstack] [Cinder] Registering 3rd party driver as default

[Kerr, Andrew]

You either need to comment out the enabled_backends line, or you'll want
to put something similar to this in your cinder.conf file:

[DEFAULT]
...
enabled_backends = cloudbyte
...
[cloudbyte]
volume_driver = volume_driver =
cinder.volume.drivers.cloudbyte.cloudbyte.ElasticenterISCSIDriver
SAN_IP=20.10.22.245
CB_APIKEY=masQwghrmPOVIqbjyyWKQdg4z4bP2sNZ13fRQyUMwm453PUiYB-xyRSMBDoZeMj6R
0-XU9DCscxMbe3AhleDyQ
CB_ACCOUNT_NAME=acc1
TSM_NAME=openstacktsm



If you have enabled_backends set then it will only use those driver specs
and ignore all driver related details in the DEFAULT section.

You also probably want to comment (or remove) the default_volume_type
line, unless you plan to create that volume type after the services come
up.

Andrew Kerr
OpenStack QA
Cloud Solutions Group
NetApp


From:  Amit Das 
Reply-To:  "OpenStack Development Mailing List (not for usage questions)"

Date:  Thursday, August 14, 2014 at 5:37 AM
To:  OpenStack Development Mailing List 
Subject:  Re: [openstack-dev] [Devstack] [Cinder] Registering 3rd
party   driver as default


With further debugging, i find that none of the configuration options
present in /etc/cinder/cinder.conf are getting applied.


Regards,
AmitCloudByte Inc. 




On Thu, Aug 14, 2014 at 11:40 AM, Amit Das
 wrote:

Hi folks,

I have been trying to run devstack with my cinder driver as the default
volume_driver but with no luck.

Devstack seems to register the lvm driver as the default always.

I have tried below approaches:

1. directly modifying the /etc/cinder/cinder.conf file
2. creating a driver file @ ./devstack/lib/cinder_plugins/

1. ref - https://review.openstack.org/#/c/68726/






This is my localrc details:
http://paste.openstack.org/show/94822/


I run ./unstack.sh & then FORCE=yes ./stack.sh

This is the cinder.conf that is generated after running above stack.sh. I
comment out the [lvmdriver-1] section manually
(not sure if this section needs to be commented)

http://paste.openstack.org/show/94841/


These are portions of c-sch & c-vol logs after restarting them in their
respective screens.

http://paste.openstack.org/show/94842/



Regards,
AmitCloudByte Inc. 








___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder] 3rd party ci names for use by official cinder mandated tests

[Kerr, Andrew]

On 7/2/14, 11:00 AM, "Anita Kuno"  wrote:


>On 07/01/2014 01:13 PM, Asselin, Ramy wrote:
>> 3rd party ci names is currently becoming a bit controversial for what
>>we're trying to do in cinder: https://review.openstack.org/#/c/101013/
>> The motivation for the above change is to aid developers understand
>>what the 3rd party ci systems are testing in order to avoid confusion.
>> The goal is to aid developers reviewing cinder changes to understand
>>which 3rd party ci systems are running official cinder-mandated tests
>>and which are running unofficial/proprietary tests.
>> Since the use of "cinder" is proposed to be "reserved" (per change
>>under review above), I'd like to propose the following for Cinder
>>third-party names under the following conditions:
>> {Company-Name}-cinder-ci
>> * This CI account name is to be used strictly for official
>>cinder-defined dsvm-full-{driver} tests.
>> * No additional tests allowed on this account.
>> oA different account name will be used for unofficial / proprietary
>>tests.
>> * Account will only post reviews to cinder patches.
>> oA different account name will be used to post reviews in all other
>>projects.

I disagree with this approach.  It will mean that if we want to run tests
on multiple projects (specifically for NetApp we're planning at least
Cinder and eventually Manilla), then we'd have to needlessly maintain 2
service accounts. This is extra work for both us, and the infra team.  A
single account is perfectly capable of running different sets of tests on
different projects.  The name of the account can then be more generalized
out to {Company-Name}-ci


>> * Format of comments will be (as jgriffith commented in that
>>review):
>> 
>> {company name}-cinder-ci
>> 
>>dsvm-full-{driver-name}   pass/fail
>> 
>> 
>>dsvm-full-{other-driver-name} pass/fail
>> 
>> 
>>dsvm-full-{yet-another-driver-name}   pass/fail

I do like this format.  A single comment with each drivers' outcome on a
different line.  That will help cut down on email and comment spam.

>> 
>> 
>> Thoughts?
>> 
>> Ramy
>> 
>> 
>> 
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> 
>Thanks for starting this thread, Ramy.
>
>I too would like Cinder third party ci systems (and systems that might
>test Cinder now or in the future) to weigh in and share their thoughts.
>
>We do need to agree on a naming policy and whatever that policy is will
>frame future discussions with new accounts (and existing ones) so let's
>get some thoughts offered here so we all can live with the outcome.
>
>Thanks again, Ramy, I appreciate your help on this as we work toward a
>resolution.
>
>Thank you,
>Anita.
>
>___
>OpenStack-dev mailing list
>OpenStack-dev@lists.openstack.org
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [OpenStack-Dev][Cinder] Review days? (open to ANYBODY and EVERYBODY)

[Kerr, Andrew]

+1

Andrew Kerr


On 6/13/14, 10:30 AM, "Duncan Thomas"  wrote:

>Same as Jay, for much the same reasons. Having a fixed calendar time
>makes it easy for me to put up a 'do not disturb' sign.
>
>On 13 June 2014 05:10, Jay Bryant  wrote:
>> John,
>>
>> +2
>>
>> I am guilty of falling behind on reviews. Pulled in to a lot of other
>>stuff
>> since the summit ... and before.
>>
>> Having prescribed time on my calendar is a good idea.  Just put it on my
>> calendar.
>>
>> Jay
>>
>> On Jun 12, 2014 10:49 PM, "John Griffith" 
>> wrote:
>>>
>>> Hey Everyone,
>>>
>>> So I've been noticing some issues with regards to reviews in Cinder
>>> lately, namely we're not keeping up very well.  Most of this is a math
>>> problem (submitters >> reviewers).  We're up around 200+ patches in the
>>> queue, and a large number of them have no negative feedback but have
>>>just
>>> been waiting patiently (some > 2 months).
>>>
>>> Growth is good, new contributors are FANTASTIC... but stale
>>>submissions in
>>> the queue are BAD, and I hate for people interested in contributing to
>>> become discouraged and just go away (almost as much as I hate emails
>>>asking
>>> me to review patches).
>>>
>>> I'd like to propose we consider one or two review days a week for a
>>>while
>>> to try and work on our backlog.  I'd like to propose that on these
>>>days we
>>> make an attempt to NOT propose new code (or at least limit it to
>>>bug-fixes
>>> [real bugs, not features disguised as bugs]) and have an agreement from
>>> folks to focus on actually doing reviews and using IRC to collaborate
>>> together and knock some of these out.
>>>
>>> We did this sort of thing over a virtual meetup and it was really
>>> effective, I'd like to see if we can't do something for a brief
>>>duration
>>> over IRC.
>>>
>>> I'm thinking we give it a test run, set aside a few hours next Wed
>>>morning
>>> to start (coinciding with our Cinder weekly meeting since many folks
>>>around
>>> that morning across TZ's etc) where we all dedicate some time prior to
>>>the
>>> meeting to focus exclusively on helping each other get some reviews
>>>knocked
>>> out.  As a reminder Cinder weekly meeting is 16:00 UTC.
>>>
>>> Let me know what you all think, and keep in mind this is NOT limited to
>>> just current regular "Block-Heads" but anybody in the OpenStack
>>>community
>>> that's willing to help out and of course new reviewers are MORE than
>>> welcome.
>>>
>>> Thanks,
>>> John
>>>
>>> ___
>>> OpenStack-dev mailing list
>>> OpenStack-dev@lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>> ___
>> OpenStack-dev mailing list
>> OpenStack-dev@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
>
>-- 
>Duncan Thomas
>
>___
>OpenStack-dev mailing list
>OpenStack-dev@lists.openstack.org
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Efficient image cloning implementation in NetApp nfs drivers // make this part of base NFS driver

[Kerr, Andrew]

The NetApp driver uses NetApp specific API calls to implement the actual
cloning of the file.  You could probably generalize it by implementing the
keeping of a cached image file on the destination share for future copies,
and then implement a standard "copy file" method that could be overloaded
by individual drivers.  In implementing the cache image though you would
also need to implement a cache "scrubber" that will come around and clean
up the cached files once you reach (user) defined thresholds.

Andrew Kerr
OpenStack QA
Cloud Solutions Group
NetApp


From:  "Luohao   (brian)" 
Reply-To:  "OpenStack Development Mailing List (not for usage questions)"

Date:  Monday, April 14, 2014 at 2:17 AM
To:  "OpenStack Development Mailing List (not for usage questions)"

Subject:  Re: [openstack-dev] Efficient image cloning implementation in
NetApp nfs drivers // make this part of base NFS driver


Nice idea.
 
Actually, fast image cloning has been widely supported by most NAS
devices, and VMware VAAI also started to require this criteria many years
ago.
 
However, I am not quite sure what exactly need to put into the base NFS
driver, anyways, the fast cloning api will vary for specific vendors.
 
-Hao

 
From: Nilesh P Bhosale [mailto:nilesh.bhos...@in.ibm.com]

Sent: Monday, April 14, 2014 1:51 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] Efficient image cloning implementation in NetApp
nfs drivers // make this part of base NFS driver

 
Hi All,

I was going through the following blue print, NetApp proposed and
implemented in its driver (NetAppNFSDriver -
cinder/volume/drivers/netapp/nfs.py) a while back (change
):
https://blueprints.launchpad.net/cinder/+spec/netapp-cinder-nfs-image-cloni
ng

It looks quite an interesting and valuable feature for the end customers.
Can we make it part of the base NfsDriver (cinder/volume/drivers/nfs.py)?
so that the customers using the base NFS driver can benefit and also other
drivers inheriting from
 this base NFS driver (e.g. IBMNAS_NFSDriver, NexentaNfsDriver) can also
benefit.

Please let me know your valuable opinion.
I can start a blueprint for the Juno release.

Thanks,
Nilesh


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Cinder] XXXFSDriver: Query on usage of load_shares_config in ensure_shares_mounted

[Kerr, Andrew]

Hi Deepak,

I know that there are plans to completely change how NFS uses (or more
accurately, will not use) the shares.conf file in the future.  My guess is
that a lot of this code will be changed in the near future during that
rework.

Andrew Kerr
OpenStack QA
Cloud Solutions Group
NetApp


From:  Deepak Shetty 
Reply-To:  "OpenStack Development Mailing List (not for usage questions)"

Date:  Friday, April 11, 2014 at 7:54 AM
To:  "OpenStack Development Mailing List (not for usage questions)"

Subject:  [openstack-dev] [Cinder] XXXFSDriver: Query on usage
of  load_shares_config in ensure_shares_mounted


Hi,

   I am using the nfs and glusterfs driver as reference here.


I see that load_shares_config is called everytime via
_ensure_shares_mounted which I feel is incorrect mainly because
ensure_shares_mounted loads the config file again w/o restarting the
service


I think that the shares config file should only be loaded once (during
service startup) as part of do_setup and never again.

If someone changes something in the conf file, one needs to restart
service which calls do_setup again and the changes made in shares.conf is
taken effect.


In looking further.. the ensure_shares_mounted ends up calling
remotefsclient.mount() which does _Nothing_ if the share is already
mounted.. whcih is mostly the case. So even if someone changed something
in the shares file (like added -o options) it won't take
 effect as the share is already mounted & service already running.

In fact today, if you restart the service, even then the changes in share
won't take effect as the mount is not un-mounted, hence when the service
is started next, the mount is existing and ensures_shares_mounted just
returns w/o doing anything.


The only adv of calling load_shares_config in ensure_shares_mounted is if
someone changed the shares server IP while the service is running ... it
loads the new share usign the new server IP.. which again is wrong since
ideally the person should restart service
 for any shares.conf changes to take effect.

Hence i feel callign load_shares_config in ensure_shares_mounted is
Incorrect and should be removed

Thoughts ?

thanx,

deepak


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [cinder] Extend operation for NFS driver

[Kerr, Andrew]

Hi cinder,

Just noticed we have competing solutions to implement extend_volume in the
generic NFS driver [1] & [2].  I understand these are not targeted until
after RC1, but I also didn't want the duplicate effort lost in the
shuffle.  Are there any thoughts on which is the more appropriate
implementation?

[1] https://review.openstack.org/82020
[2] https://review.openstack.org/82100

Andrew Kerr


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev