date:20140815

On 08/15/2014 09:13 AM, Jay Pipes wrote:
 On 08/15/2014 04:21 AM, Roman Podoliaka wrote:
 Hi Oslo team,

 I propose that we add Mike Bayer (zzzeek) to the oslo.db core
 reviewers team.

 Mike is an author of SQLAlchemy, Alembic, Mako Templates and some
 other stuff we use in OpenStack. Mike has been working on OpenStack
 for a few months contributing a lot of good patches and code reviews
 to oslo.db [1]. He has also been revising the db patterns in our
 projects and prepared a plan how to solve some of the problems we have
 [2].

 I think, Mike would be a good addition to the team.
 
 Uhm, yeah... +10 :)

^2 :-)

-- 
Russell Bryant

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][all] switch from mysqldb to another eventlet aware mysql client

2014-08-15 Thread Ihar Hrachyshka

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Some updates on the matter:

- - oslo-spec was approved with narrowed scope which is now 'enabled
mysqlconnector as an alternative in gate' instead of 'switch the
default db driver to mysqlconnector'. We'll revisit the switch part
the next cycle once we have the new driver running in gate and real
benchmarking is heavy-lifted.

- - there are several patches that are needed to make devstack and
tempest passing deployment and testing. Those are collected under the
hood of: https://review.openstack.org/#/c/114207/ Not much of them.

- - we'll need a new oslo.db release to bump versions (this is needed to
set raise_on_warnings=False for the new driver, which was incorrectly
set to True in sqlalchemy till very recently). This is expected to be
released this month (as per Roman Podoliaka).

- - once the corresponding patch for sqlalchemy-migrate is merged, we'll
also need a new version released for this.

- - on PyPI side, no news for now. The last time I've heard from Geert
(the maintainer of MySQL Connector for Python), he was working on
this. I suspect there are some legal considerations running inside
Oracle. I'll update once I know more about that.

- - once all the relevant patches land in affected projects and
devstack, I'm going to introduce a separate gate job to run against
mysqlconnector.

Cheers,
/Ihar

On 22/07/14 15:03, Ihar Hrachyshka wrote:
 FYI: I've moved the spec to oslo space since the switch is not
 really limited to neutron, and most of coding is to be done in
 oslo.db (though not much anyway).
 
 New spec: https://review.openstack.org/#/c/108355/
 
 On 09/07/14 13:17, Ihar Hrachyshka wrote:
 Hi all,
 
 Multiple projects are suffering from db lock timeouts due to 
 deadlocks deep in mysqldb library that we use to interact with 
 mysql servers. In essence, the problem is due to missing
 eventlet support in mysqldb module, meaning when a db lock is
 encountered, the library does not yield to the next green thread,
 allowing other threads to eventually unlock the grabbed lock, and
 instead it just blocks the main thread, that eventually raises
 timeout exception (OperationalError).
 
 The failed operation is not retried, leaving failing request not
  served. In Nova, there is a special retry mechanism for
 deadlocks, though I think it's more a hack than a proper fix.
 
 Neutron is one of the projects that suffer from those timeout 
 errors a lot. Partly it's due to lack of discipline in how we do 
 nested calls in l3_db and ml2_plugin code, but that's not
 something to change in foreseeable future, so we need to find
 another solution that is applicable for Juno. Ideally, the
 solution should be applicable for Icehouse too to allow
 distributors to resolve existing deadlocks without waiting for
 Juno.
 
 We've had several discussions and attempts to introduce a
 solution to the problem. Thanks to oslo.db guys, we now have more
 or less clear view on the cause of the failures and how to easily
 fix them. The solution is to switch mysqldb to something eventlet
 aware. The best candidate is probably MySQL Connector module that
 is an official MySQL client for Python and that shows some
 (preliminary) good results in terms of performance.
 
 I've posted a Neutron spec for the switch to the new client in
 Juno at [1]. Ideally, switch is just a matter of several fixes
 to oslo.db that would enable full support for the new driver
 already supported by SQLAlchemy, plus 'connection' string
 modified in service configuration files, plus documentation
 updates to refer to the new official way to configure services
 for MySQL. The database code won't, ideally, require any major
 changes, though some adaptation for the new client library may be
 needed. That said, Neutron does not seem to require any changes,
 though it was revealed that there are some alembic migration
 rules in Keystone or Glance that need (trivial) modifications.
 
 You can see how trivial the switch can be achieved for a service 
 based on example for Neutron [2].
 
 While this is a Neutron specific proposal, there is an obvious
 wish to switch to the new library globally throughout all the
 projects, to reduce devops burden, among other things. My vision
 is that, ideally, we switch all projects to the new library in
 Juno, though we still may leave several projects for K in case
 any issues arise, similar to the way projects switched to
 oslo.messaging during two cycles instead of one. Though looking
 at how easy Neutron can be switched to the new library, I
 wouldn't expect any issues that would postpone the switch till
 K.
 
 It was mentioned in comments to the spec proposal that there
 were some discussions at the latest summit around possible switch
 in context of Nova that revealed some concerns, though they do
 not seem to be documented anywhere. So if you know anything about
 it, please comment.
 
 So, we'd like to hear from other projects what's your take on
 that move,

Re: [openstack-dev] [Sahara] Swift authentication and backwards compatibility

2014-08-15 Thread Trevor McKay

Thoughts, rapidfire :)

In short, I think we should plan on backward compat unless some stubborn
technical problem gets in our way 

I think backward compatibility is a good idea.  We can make the
user/pass inputs for data objects optional (they are required
currently), maybe even gray them out in the UI with a checkbox to turn
them on, or something like that.

Sahara can detect whether or not the proxy domain is there, and whether
or not it can be created.  If Sahara ends up in a situation where it
thinks user/pass are required, but the data objects don't have them,
we can return a meaningful error.

The job manager can key off of the values supplied for the data source
objects (no user/pass? must be proxy) and/or cluster configs (for
instance, a new cluster config could be added -- if it's absent we
assume old cluster and therefore old hadoop swfit plugin).  Workflow
can be generated accordingly.

The hadoop swift plugin can look at the config values provided, as you
noted yesterday, and get auth tokens in either manor.

Best,

Trev


On Thu, 2014-08-14 at 22:20 -0400, Michael McCune wrote:
 hello Sahara folks,
 
 I am working to get the revamped spec[1] finalized and I'd like to know the 
 group's thoughts on the idea of backward compatibility. It is possible to 
 implement the new authentication method and remain backward compatible, but 
 we will need to keep the username and password inputs in the Swift data forms.
 
 Having the backward compatibility would also give Sahara a way to react in 
 situations where the proxy domain is not available or the administrator 
 doesn't wish to use it. I'm not sure this is the behavior we want, but I 
 don't know if it is proper for Sahara to exit if no proxy domain can be found.
 
 If we choose not to remain backward compatible then we are requiring Sahara 
 operators to create the new proxy domain needed, and they must update all 
 virtual machine images.
 
 Thoughts?
 
 regards,
 mike
 
 [1]: https://review.openstack.org/#/c/113591/
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Concerns around the Extensible Resource Tracker design - revert maybe?

2014-08-15 Thread Andrew Laski



On 08/14/2014 03:21 AM, Nikola Đipanov wrote:

On 08/13/2014 06:05 PM, Sylvain Bauza wrote:

Le 13/08/2014 12:21, Sylvain Bauza a écrit :

Le 12/08/2014 22:06, Sylvain Bauza a écrit :

Le 12/08/2014 18:54, Nikola Đipanov a écrit :

On 08/12/2014 04:49 PM, Sylvain Bauza wrote:

(sorry for reposting, missed 2 links...)

Hi Nikola,

Le 12/08/2014 12:21, Nikola Đipanov a écrit :

Hey Nova-istas,

While I was hacking on [1] I was considering how to approach the fact
that we now need to track one more thing (NUMA node utilization)
in our
resources. I went with - I'll add it to compute nodes table
thinking
it's a fundamental enough property of a compute host that it
deserves to
be there, although I was considering  Extensible Resource Tracker
at one
point (ERT from now on - see [2]) but looking at the code - it did
not
seem to provide anything I desperately needed, so I went with
keeping it
simple.

So fast-forward a few days, and I caught myself solving a problem
that I
kept thinking ERT should have solved - but apparently hasn't, and I
think it is fundamentally a broken design without it - so I'd really
like to see it re-visited.

The problem can be described by the following lemma (if you take
'lemma'
to mean 'a sentence I came up with just now' :)):


Due to the way scheduling works in Nova (roughly: pick a host
based on
stale(ish) data, rely on claims to trigger a re-schedule), _same
exact_
information that scheduling service used when making a placement
decision, needs to be available to the compute service when
testing the
placement.


This is not the case right now, and the ERT does not propose any
way to
solve it - (see how I hacked around needing to be able to get
extra_specs when making claims in [3], without hammering the DB). The
result will be that any resource that we add and needs user supplied
info for scheduling an instance against it, will need a buggy
re-implementation of gathering all the bits from the request that
scheduler sees, to be able to work properly.

Well, ERT does provide a plugin mechanism for testing resources at the
claim level. This is the plugin responsibility to implement a test()
method [2.1] which will be called when test_claim() [2.2]

So, provided this method is implemented, a local host check can be
done
based on the host's view of resources.



Yes - the problem is there is no clear API to get all the needed
bits to
do so - especially the user supplied one from image and flavors.
On top of that, in current implementation we only pass a hand-wavy
'usage' blob in. This makes anyone wanting to use this in conjunction
with some of the user supplied bits roll their own
'extract_data_from_instance_metadata_flavor_image' or similar which is
horrible and also likely bad for performance.

I see your concern where there is no interface for user-facing
resources like flavor or image metadata.
I also think indeed that the big 'usage' blob is not a good choice
for long-term vision.

That said, I don't think as we say in French to throw the bath
water... ie. the problem is with the RT, not the ERT (apart the
mention of third-party API that you noted - I'll go to it later below)

This is obviously a bigger concern when we want to allow users to
pass
data (through image or flavor) that can affect scheduling, but
still a
huge concern IMHO.

And here is where I agree with you : at the moment, ResourceTracker
(and
consequently Extensible RT) only provides the view of the resources
the
host is knowing (see my point above) and possibly some other resources
are missing.
So, whatever your choice of going with or without ERT, your patch [3]
still deserves it if we want not to lookup DB each time a claim goes.



As I see that there are already BPs proposing to use this IMHO broken
ERT ([4] for example), which will surely add to the proliferation of
code that hacks around these design shortcomings in what is already a
messy, but also crucial (for perf as well as features) bit of Nova
code.

Two distinct implementations of that spec (ie. instances and flavors)
have been proposed [2.3] [2.4] so reviews are welcome. If you see the
test() method, it's no-op thing for both plugins. I'm open to comments
because I have the stated problem : how can we define a limit on
just a
counter of instances and flavors ?


Will look at these - but none of them seem to hit the issue I am
complaining about, and that is that it will need to consider other
request data for claims, not only data available for on instances.

Also - the fact that you don't implement test() in flavor ones tells me
that the implementation is indeed racy (but it is racy atm as well) and
two requests can indeed race for the same host, and since no claims are
done, both can succeed. This is I believe (at least in case of single
flavor hosts) unlikely to happen in practice, but you get the idea.

Agreed, these 2 patches probably require another iteration, in
particular how we make sure that it won't be racy. So I need another
run to think

Re: [openstack-dev] [Cinder] 3'rd party CI systems: Not Whitelisted Volume type with name * could not be found.

2014-08-15 Thread John Griffith

On Fri, Aug 15, 2014 at 6:39 AM, Kerr, Andrew andrew.k...@netapp.com
wrote:

 Can the default_volume_type be left empty and get the original None
 type, so we don't have to create the volume type prior to running tempest
 tests?

 Andrew Kerr
 OpenStack QA
 Cloud Solutions Group
 NetApp



 On 8/14/14, 5:23 PM, Martin, Kurt Frederick (ESSN Storage MSDU)
 kurt.f.mar...@hp.com wrote:

 They need to be manually added.
 
 -Original Message-
 From: Asselin, Ramy
 Sent: Thursday, August 14, 2014 2:17 PM
 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] [Cinder] 3'rd party CI systems: Not
 Whitelisted Volume type with name * could not be found.
 
 Both configurations have that set as you described. [1][2] Who actually
 creates that volume type? Is that supposed to be added manually to
 local.sh, or is this a bug in devstack?
 
 [1]
 
 http://publiclogs.emc.com/vnx_ostack/EMC_VNX_FC/212/logs/etc/cinder/cinder
 .conf.txt.gz
 [DEFAULT]
 default_volume_type = cat_1
 
 [2]
 
 http://15.126.198.151/60/111460/18/check/dsvm-tempest-hp-lefthand/3ee1598/
 logs/etc/cinder/cinder.conf.txt.gz
 [DEFAULT]
 default_volume_type = dot
 
 -Original Message-
 From: Martin, Kurt Frederick (ESSN Storage MSDU)
 Sent: Thursday, August 14, 2014 2:00 PM
 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] [Cinder] 3'rd party CI systems: Not
 Whitelisted Volume type with name * could not be found.
 
 Cinder.conf needs to have a default_volume_type  entry set under the
 [Default] group and a volume type that is valid, for example,
 default_volume_type=bronze. This allows for a volume to be created when a
 volume type is not selected, the default 'None'  type. This feature has
 been available for some time in cinder but recently enabled in devstack.
 ~Kurt
 
 -Original Message-
 From: Asselin, Ramy
 Sent: Thursday, August 14, 2014 10:00 AM
 To: OpenStack Development Mailing List (not for usage questions)
 Subject: [openstack-dev] [Cinder] 3'rd party CI systems: Not Whitelisted
 Volume type with name * could not be found.
 
 Hi,
 
 Does anyone know how to configure cinder ci tests to not have these
 errors?
 
 12:32:11 *** Not Whitelisted *** 2014-08-14 12:23:56.179 18021 ERROR
 cinder.volume.volume_types [req-c9ec92ab-132b-4167-a467-15bd213659e8
 bd1f54a867ce47acb4097cd94149efa0 d15dcf4cb3c7495389799ec849e9036d - - -]
 Default volume type is not found, please check default_volume_type
 config: Volume type with name dot could not be found.
 
 17:43:28 *** Not Whitelisted *** 2014-08-11 17:31:20.343 2097 ERROR
 cinder.volume.volume_types [req-01e539ad-5357-4a0f-8d58-464b970114f9
 f72cd499be584d9d9585bc26ca71c603 d845ad2d7e6a47dfb84bdf0754f60384 - - -]
 Default volume type is not found, please check default_volume_type
 config: Volume type with name cat_1 could not be found.
 
 
 http://15.126.198.151/60/111460/18/check/dsvm-tempest-hp-lefthand/3ee1598/
 console.html
 http://publiclogs.emc.com/vnx_ostack/EMC_VNX_FC/212/console.html
 
 Thanks,
 Ramy
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Update your devstack, it should be creating this for you.  There's a call
in stack.sh to create_volume_types (which is in lib/cinder).  If this isn't
working for you, double check your localrc setting.

John
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [heat] heat docker multi host scheduling support

2014-08-15 Thread Eric Windisch

On Thu, Aug 14, 2014 at 3:21 AM, Malawade, Abhijeet 
abhijeet.malaw...@nttdata.com wrote:

  Hi all,



 I am trying to use heat to create docker containers. I have configured
 heat-docker plugin.

 I am also able to create stack using heat successfully.



 To start container on different host we need to provide 'docker_endpoint'
 in template. For this we have to provide host address where container will
 run in template.



 Is there any way to schedule docker container on available hosts using
 heat-docker plugin without giving 'docker_endpoint' in template file.

 Is heat-docker plugin supports managing docker hosts cluster with
 scheduling logic.



 Please let me know your suggestions on the same.



Zane's responses were correct. Effort is underway, planning the development
of a containers service. This architecture of this service actually looks
quite a bit like the Docker+Heat+Nova story, but simplified through a
Nova-like API.  The spec for this is in progress:
https://review.openstack.org/#/c/114044/

As for scheduling with Docker+Heat, one option is to use swarmd/libswarm.
Unfortunately, this is an early-stage project, so it isn't quite an
out-of-the-box solution; It's not production-ready, but you're welcome to
investigate it... The usage model with Heat would be to spawn your Nova
instances not only with Docker, but with a connector to a libswarm server
(swarmd). That swarmd process would need to be running an listening
somewhere.  You'd need to load (and possibly write) plugins for
libswarm/swarmd to provide your scheduling. Your docker_endpoint would be
the IP/port of swarmd, and it would proxy those requests to the correct
backend Docker hosts.

-- 
Regards,
Eric Windisch
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo][db] Nominating Mike Bayer for the oslo.db core reviewers team

2014-08-15 Thread Ben Nemec

On 08/15/2014 08:20 AM, Russell Bryant wrote:
 On 08/15/2014 09:13 AM, Jay Pipes wrote:
 On 08/15/2014 04:21 AM, Roman Podoliaka wrote:
 Hi Oslo team,

 I propose that we add Mike Bayer (zzzeek) to the oslo.db core
 reviewers team.

 Mike is an author of SQLAlchemy, Alembic, Mako Templates and some
 other stuff we use in OpenStack. Mike has been working on OpenStack
 for a few months contributing a lot of good patches and code reviews
 to oslo.db [1]. He has also been revising the db patterns in our
 projects and prepared a plan how to solve some of the problems we have
 [2].

 I think, Mike would be a good addition to the team.

 Uhm, yeah... +10 :)
 
 ^2 :-)
 

What took us so long to do this? :-)

+1 obviously.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Sahara] Swift authentication and backwards compatibility

2014-08-15 Thread mike mccune


thanks for the thoughts Trevor,


On 08/15/2014 09:32 AM, Trevor McKay wrote:

I think backward compatibility is a good idea.  We can make the
user/pass inputs for data objects optional (they are required
currently), maybe even gray them out in the UI with a checkbox to turn
them on, or something like that.


This is similar to what I was thinking. We would allow the username and 
password inputs to accept a blank input.


I also like the idea of giving some sort of visual reference, like 
graying out the fields.



Sahara can detect whether or not the proxy domain is there, and whether
or not it can be created.  If Sahara ends up in a situation where it
thinks user/pass are required, but the data objects don't have them,
we can return a meaningful error.


I think it sounds like we are going to avoid having Sahara attempt to 
create a domain. It will be the duty of a stack administrator to create 
the domain and give it's name in the sahara.conf file.


Agreed about meaning errors.


The job manager can key off of the values supplied for the data source
objects (no user/pass? must be proxy) and/or cluster configs (for
instance, a new cluster config could be added -- if it's absent we
assume old cluster and therefore old hadoop swfit plugin).  Workflow
can be generated accordingly.


This sounds good. If there is some way to determine the version of the 
hadoop-swiftfs on the cluster that would be ideal.



The hadoop swift plugin can look at the config values provided, as you
noted yesterday, and get auth tokens in either manor.


exactly.



mike

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Python 3 support in os-*-config

2014-08-15 Thread Ben Nemec

On 08/15/2014 02:30 AM, Dougal Matthews wrote:
 Great stuff, really happy to see Python 3 support.

 - Original Message -
 From: Steve Kowalik ste...@wedontsleep.org
 To: openstack-dev@lists.openstack.org
 Sent: Friday, 15 August, 2014 4:11:04 AM
 Subject: [openstack-dev] [TripleO] Python 3 support in os-*-config

  * os-apply-config
This one looks to be the hardest one to port. Multiple test failures,
 hard coded use of '#!/usr/bin/env python' which is unhappy in a only
 Python 3 environment, map() changes, byte changes ...

Strange.  How is /usr/bin/env python causing issues?  I thought the
whole point of using env to look up python was to allow it to select the
correct one based on the system configuration.

 If I can help somehow, let me know.

  * os-cloud-config
I have a patch to fix current Python 3 issues up
 (https://review.openstack.org/111606), and then my plan is to add Python
 3 jobs to check *and* gate, so in-flight patches may need to watch out
 for that soon.

 LGTM.

  * os-collect-config
This needs six sprinkled throughout it due to URL parsing, and I need
 to work out what to do WRT absolute importing due to:
   File ./os_collect_config/collect.py, line 25, in module
 from openstack.common import log
 ImportError: No module named 'openstack'

 I think the import needs to be changed to this, as openstack is under the
 os_collect_config package.

 from os_collect_config.openstack.common import log

+1.  This is how all the other projects use openstack.common.

 Alternatively, `from .openstack.common import log` should work but that 
 is less favorable.

  * os-net-config
I know this isn't really a thing yet, but I also have yet to look
 into it's Python 3 support.

  * os-refresh-config
This has Python 3 jobs running in check and gate. Does anyone have a
 small fix to test them out, the last code change that landed in
 os-refresh-config was mid July.

 I don't have a change, but FWIW I just tried the tests locally on py33 
 and they passed fine. \o/

 Thanks,
 Dougal

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][core] Expectations of core reviewers

On 2014-08-14 09:33:20 -0400 (-0400), Russell Bryant wrote:
[...]
 Another issue is that some folks are just fundamentally opposed to
 using Google
[...]

I think that's a shallow depiction of the issue. I'm sure *some*
people really do just avoid Google specifically, but a bigger
concern should be around the statement our use of those services
presents to the rest of the World. By using commercial solutions
because the open alternatives aren't as useful/featureful/stable, we
rob those projects of a potential larger user community which could
help them achieve greater momentum and eventually dominate their
respective technologies.

Would we, as a community, rather see OpenStack used in production
and improved when it has bugs/lacks features? Or should users just
view it as a cheap lab platform instead, and then pay for solid
proprietary solutions to their production needs? I'm glad a lot of
people think using and improving OpenStack, even when there might
sometimes be easier/simpler closed alternatives, is worth the
long-term investment.

We, as visible leaders among the greater free software community,
should think very hard when making the choice not to extend this
same courtesy and consideration to other projects who may lack our
extraordinary base of resources. Through our example as a project,
we have the potential to improve things for open/free software
communities far beyond our own.

Of course I'm not a nova core reviewer and have never attended a
nova mid-cycle meetup, so I really have no say in how you conduct
your remote participation. But I do hope you'll consider,
collectively, that focusing on immediate convenience for your own
community can have negative longer-term consequences as you (whether
consciously or not) promote the use of proprietary solutions to your
needs rather than embracing less convenient free and open options
which may still require improvement.
-- 
Jeremy Stanley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] OS or os are not acronyms for OpenStack

2014-08-15 Thread Anita Kuno

OpenStack is OpenStack. The use of openstack is also acceptable in our
development conversations.

OS or os is operating system. I am starting to see some people us OS or
os to mean OpenStack. This is confusing and also incorrect[0].

No alterations: When using OpenStack Marks, you shall never vary the
spelling, hyphenation or spacing of the any portion of the marks.

Examples of Improper Display of an OpenStack Mark: Open-Stack; Open
Stack; OS Stack Examples of Proper Display of an OpenStack Mark:
OpenStack; OPENSTACK

While this comes from the OpenStack Trademark Policy, I think it is
important to remember this information and to implement it in our daily
use. I have had to change at least one wikipage so far, it is far easier
if folks simply employ the correct usage from the beginning.

My thanks,
Anita.

[0] http://www.openstack.org/brand/openstack-trademark-policy/

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Review metrics - what do we want to measure?

On 2014-08-13 19:51:52 -0500 (-0500), Ben Nemec wrote:
[...]
 make the check-tripleo job leave an actual vote rather than just a
 comment.
[...]

That, as previously discussed, will require some design work in
Zuul. Gerrit uses a single field per account for verify votes, which
means that if you want to have multiple concurrent votes for
different pipelines/job collections then we either need to use more
than one account for those or add additional columns for each.

There have already been discussions around how to implement this,
but in the TripleO case it might make more sense to revisit why we
have those additional pipelines and instead focus on resolving the
underlying issues which led to their use as a stop-gap.
-- 
Jeremy Stanley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] OS or os are not acronyms for OpenStack

2014-08-15 Thread Eric Windisch



 No alterations: When using OpenStack Marks, you shall never vary the
 spelling, hyphenation or spacing of the any portion of the marks.

 Examples of Improper Display of an OpenStack Mark: Open-Stack; Open
 Stack; OS Stack Examples of Proper Display of an OpenStack Mark:
 OpenStack; OPENSTACK


When used in marketing, sales, and other business contexts, I obviously
agree. This is proper use and license of a trademark.


 While this comes from the OpenStack Trademark Policy, I think it is
 important to remember this information and to implement it in our daily
 use. I have had to change at least one wikipage so far, it is far easier
 if folks simply employ the correct usage from the beginning.


On wiki pages and other published medium -- absolutely.

For our daily use in IRC and other casual discussion? Lets not get pedantic.

-- 
Regards,
Eric Windisch
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] OS or os are not acronyms for OpenStack

2014-08-15 Thread Steve Gordon

- Original Message -
 From: Anita Kuno ante...@anteaya.info
 To: openStack Development Mailing List openstack-dev@lists.openstack.org

 OpenStack is OpenStack. The use of openstack is also acceptable in our
 development conversations.

 OS or os is operating system. I am starting to see some people us OS or
 os to mean OpenStack. This is confusing and also incorrect[0].

 No alterations: When using OpenStack Marks, you shall never vary the
 spelling, hyphenation or spacing of the any portion of the marks.

 Examples of Improper Display of an OpenStack Mark: Open-Stack; Open
 Stack; OS Stack Examples of Proper Display of an OpenStack Mark:
 OpenStack; OPENSTACK

 While this comes from the OpenStack Trademark Policy, I think it is
 important to remember this information and to implement it in our daily
 use. I have had to change at least one wikipage so far, it is far easier
 if folks simply employ the correct usage from the beginning.

 My thanks,
 Anita.

 [0] http://www.openstack.org/brand/openstack-trademark-policy/

Hi Anita,

I noticed you raised this issue on this thread but didn't really understand why:

http://lists.openstack.org/pipermail/openstack-dev/2014-August/043087.html

Can you explain? The os types being referred to were the operating system types 
supported by VMware.

-Steve

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Which program for Rally

On 2014-08-13 19:28:48 -0700 (-0700), Joe Gordon wrote:
 We actually run out of nodes almost every day now (except
 weekends), we have about 800 nodes, and hit that quota most days
[...]

It's worth noting that a lot of the recent exhaustion has been due
to OpenStack bugs impacting the providers donating those resources
(instances perpetually stuck in error state when deleting because
nova calls to neutron timed out on port deletion, neutron floating IP
deletion through nova failing at random causing leaks eating into
our quota, et cetera). Not to say that there won't be new bugs
cropping up to take their place once those are solved/worked around,
but at least the current state is not entirely due to the volume and
duration of jobs we run.
-- 
Jeremy Stanley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] OS or os are not acronyms for OpenStack

2014-08-15 Thread Dan Smith

 OS or os is operating system. I am starting to see some people us OS or
 os to mean OpenStack. This is confusing and also incorrect[0].

Except in the nova API code, where 'os' means 'openstack'.

I too think that policing the use of abbreviations is silly. It's a
confusing abbreviation, but seriously...

--Dan



signature.asc
Description: OpenPGP digital signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] OS or os are not acronyms for OpenStack

2014-08-15 Thread Mike Spreitzer

Anita Kuno ante...@anteaya.info wrote on 08/15/2014 10:38:20 AM:

 OpenStack is OpenStack. The use of openstack is also acceptable in our
 development conversations.
 
 OS or os is operating system. I am starting to see some people us OS or
 os to mean OpenStack. This is confusing and also incorrect[0].
 
 ...

I have seen OS for OpenStack from the start.  Just look at the 
environment variables that the CLI reads.

Regards,
Mike

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [TripleO] fix poor tarball support in source-repositories

2014-08-15 Thread Brownell, Jonathan C (Corvallis)

The current DIB element support for downloading tarballs via 
source-repository allows an entry in the following form:

name tar targetdir url

Today, this feature is currently used only by the mysql DIB element. You can 
see how it's used here:
https://github.com/openstack/tripleo-image-elements/blob/master/elements/mysql/source-repository-mysql

However, the underlying diskimage-builder implementation of tarball handling is 
rather odd and inflexible. After downloading the file (or retrieving from 
cache) and unpacking into a tmp directory, it performs:

mv $tmp/*/* $targetdir

This does work as long as the tarball follows a structure where all its 
files/directories are contained within a single directory, but it fails if the 
tarball contains no subdirectories. (Even worse is when it contains some files 
and some subdirectories, in which case the files are lost and the contents of 
all subdirs get lumped together in the output folder.)

Since this tarball support is only used today by the mysql DIB element, I would 
love to fix this in both diskimage-builder and tripleo-image-element by 
changing to simply:

mv $tmp/* $targetdir

And then manually tweaking the directory structure of $targetdir from a new 
install.d script in the mysql element to restore the desired layout.

However, it's important to note that this will break backwards compatibility if 
tarball support is used in its current fashion by users with private DIB 
elements.

Personally, I consider the current behavior so egregious that it really needs 
to be fixed across the board rather than preserving backwards compatibility.

Do others agree? If not, do you have suggestions as to how to improve this 
mechanism cleanly without sacrificing backwards compatibility?

Thanks,

Jonathan
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Deprecating CONF.block_device_allocate_retries_interval

2014-08-15 Thread Andrew Laski



On 08/08/2014 08:42 AM, Nikola Đipanov wrote:

On 08/06/2014 07:54 PM, Jay Pipes wrote:

I bring this up on the mailing list because I think Liyi's patch offers
an interesting future direction to the way that we think about our retry
approach in Nova. Instead of having hard-coded or configurable interval
times, I think Liyi's approach of calculating the interval length based
on some input values is a good direction to take.


This indeed is a problem that we've seen bite us a number of times, and
I tried to solve it by proposing [1] but didn't get to work on it
further yet.

Having said that - after thinking about it more, I was not sure I like
my own approach in [1] on the grounds of it being too generic (and
overly elaborate) for the particular problem it is solving.

I was then thinking of something similar to what is proposed here, where
we would have a waiting time that is a function of a value that we could
query Cinder on. Allocation rate proposed here seems to fit this nicely,
but in my mind we would have a way to query cinder about it instead of
hardcoding it, however this can be done later and in cooperation with
the Cinder team.


I like this direction a lot, because the allocation time depends on 
Cinder and its backend more than anything.  Excepting perhaps image 
transfer speeds.



2) We should deprecate the CONF.block_device_allocate_retries_interval
option only, and keep the CONF.block_device_allocate_retries
configuration option as-is, changing the help text to read something
like Max number of retries. We calculate the interval of the retry
based on the size of the volume.


I'd go with this as the number of retries can still be useful as a tool
for easy workaround and troubleshooting, but I'd put a big disclaimer
that it is mostly meant for debugging/workaround purposes.


I disagree a bit with having a knob purely for debugging/workaround.  I 
think this is a place where it's very useful to have the knob though.  
The figures in the review seem sound, but this is probably not a place 
where one size is going to fit all.  Until/unless we can get some 
coordination between Nova and Cinder on this I think having a knob that 
is intended to be used is the right way to go.



N.

[1] https://review.openstack.org/#/c/87546/

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] OS or os are not acronyms for OpenStack

On 08/15/2014 11:00 AM, Mike Spreitzer wrote:
 Anita Kuno ante...@anteaya.info wrote on 08/15/2014 10:38:20 AM:
 
 OpenStack is OpenStack. The use of openstack is also acceptable in our
 development conversations.

 OS or os is operating system. I am starting to see some people us OS or
 os to mean OpenStack. This is confusing and also incorrect[0].

 ...
 
 I have seen OS for OpenStack from the start.  Just look at the
 environment variables that the CLI reads.

Yep, it's quite common and I think it's fine in the right context.

-- 
Russell Bryant

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] The future of the integrated release

2014-08-15 Thread Sandy Walsh

On 8/14/2014 6:42 PM, Doug Hellmann wrote:

On Aug 14, 2014, at 4:41 PM, Joe Gordon 
joe.gord...@gmail.commailto:joe.gord...@gmail.com wrote:




On Wed, Aug 13, 2014 at 12:24 PM, Doug Hellmann 
d...@doughellmann.commailto:d...@doughellmann.com wrote:

On Aug 13, 2014, at 3:05 PM, Eoghan Glynn 
egl...@redhat.commailto:egl...@redhat.com wrote:


 At the end of the day, that's probably going to mean saying No to more
 things. Everytime I turn around everyone wants the TC to say No to
 things, just not to their particular thing. :) Which is human nature.
 But I think if we don't start saying No to more things we're going to
 end up with a pile of mud that no one is happy with.

 That we're being so abstract about all of this is frustrating. I get
 that no-one wants to start a flamewar, but can someone be concrete about
 what they feel we should say 'no' to but are likely to say 'yes' to?


 I'll bite, but please note this is a strawman.

 No:
 * Accepting any more projects into incubation until we are comfortable with
 the state of things again
 * Marconi
 * Ceilometer

 Well -1 to that, obviously, from me.

 Ceilometer is on track to fully execute on the gap analysis coverage
 plan agreed with the TC at the outset of this cycle, and has an active
 plan in progress to address architectural debt.

Yes, there seems to be an attitude among several people in the community that 
the Ceilometer team denies that there are issues and refuses to work on them. 
Neither of those things is the case from our perspective.

Totally agree.


Can you be more specific about the shortcomings you see in the project that 
aren’t being addressed?


Once again, this is just a straw man.

You’re not the first person to propose ceilometer as a project to kick out of 
the release, though, and so I would like to be talking about specific reasons 
rather than vague frustrations.


I'm just not sure OpenStack has 'blessed' the best solution out there.

https://wiki.openstack.org/wiki/Ceilometer/Graduation#Why_we_think_we.27re_ready



  *   Successfully passed the challenge of being adopted by 3 related projects 
which have agreed to join or use ceilometer:
 *   Synaps
 *   Healthnmon
 *   
StackTachhttps://wiki.openstack.org/w/index.php?title=StackTachaction=editredlink=1

Stacktach seems to still be under active development 
(http://git.openstack.org/cgit/stackforge/stacktach/log/), is used by rackspace 
in production and from everything I hear is more mature then ceilometer.

Stacktach is older than ceilometer, but does not do all of the things 
ceilometer does now and aims to do in the future. It has been a while since I 
last looked at it, so the situation may have changed, but some of the reasons 
stacktach would not be a full replacement for ceilometer include: it only works 
with AMQP; it collects notification events, but doesn’t offer any metering 
ability per se (no tracking of values like CPU or bandwidth utilization); it 
only collects notifications from some projects, and doesn’t have a way to 
collect data from swift, which doesn’t emit notifications; and it does not 
integrate with Heat to trigger autoscaling alarms.

Well, that's my cue.

Yes, StackTach was started before the incubation process was established and it 
solves other problems. Specifically around usage, billing and performance 
monitoring, things I wouldn't use Ceilometer for. But, if someone asked me what 
they should use for metering today, I'd point them towards Monasca in a 
heartbeat. Another non-blessed project.

It is nice to see that Ceilometer is working to solve their problems, but there 
are other solutions operators should consider until that time comes. It would 
be nice to see the TC endorse those too. Solve the users need first.

We did work with a few of the Stacktach developers on bringing event collection 
into ceilometer, and that work is allowing us to modify the way we store the 
meter data that causes a lot of the performance issues we’ve seen. That work is 
going on now and will be continued into Kilo, when we expect to be adding 
drivers for time-series databases more appropriate for that type of data.


StackTach isn't actively contributing to Ceilometer any more. Square peg/round 
hole. We needed some room to experiment with alternative solutions and the 
rigidity of the process was a hindrance. Not a problem with the core team, just 
a problem with the dev process overall.

I recently suggested that the Ceilometer API (and integration tests) be 
separated from the implementation (two repos) so others might plug in a 
different implementation while maintaining compatibility, but that wasn't well 
received.

Personally, I'd like to see that model extended for all OpenStack projects. 
Keep compatible at the API level and welcome competing implementations.

We'll be moving StackTach.v3 [1] to StackForge soon and following that model. 
The API and integration tests are one repo (with a bare-bones implementation to 
make the

Re: [openstack-dev] [nova] Deprecating CONF.block_device_allocate_retries_interval

2014-08-15 Thread Matthew Booth

On 15/08/14 16:12, Andrew Laski wrote:
 
 On 08/08/2014 08:42 AM, Nikola Đipanov wrote:
 On 08/06/2014 07:54 PM, Jay Pipes wrote:
 I bring this up on the mailing list because I think Liyi's patch offers
 an interesting future direction to the way that we think about our retry
 approach in Nova. Instead of having hard-coded or configurable interval
 times, I think Liyi's approach of calculating the interval length based
 on some input values is a good direction to take.

 This indeed is a problem that we've seen bite us a number of times, and
 I tried to solve it by proposing [1] but didn't get to work on it
 further yet.

 Having said that - after thinking about it more, I was not sure I like
 my own approach in [1] on the grounds of it being too generic (and
 overly elaborate) for the particular problem it is solving.

 I was then thinking of something similar to what is proposed here, where
 we would have a waiting time that is a function of a value that we could
 query Cinder on. Allocation rate proposed here seems to fit this nicely,
 but in my mind we would have a way to query cinder about it instead of
 hardcoding it, however this can be done later and in cooperation with
 the Cinder team.
 
 I like this direction a lot, because the allocation time depends on
 Cinder and its backend more than anything.  Excepting perhaps image
 transfer speeds.
 
 2) We should deprecate the CONF.block_device_allocate_retries_interval
 option only, and keep the CONF.block_device_allocate_retries
 configuration option as-is, changing the help text to read something
 like Max number of retries. We calculate the interval of the retry
 based on the size of the volume.

 I'd go with this as the number of retries can still be useful as a tool
 for easy workaround and troubleshooting, but I'd put a big disclaimer
 that it is mostly meant for debugging/workaround purposes.
 
 I disagree a bit with having a knob purely for debugging/workaround.  I
 think this is a place where it's very useful to have the knob though. 
 The figures in the review seem sound, but this is probably not a place
 where one size is going to fit all.  Until/unless we can get some
 coordination between Nova and Cinder on this I think having a knob that
 is intended to be used is the right way to go.

FYI, this bug is admittedly only from code inspection, but I think many
interval config variables are currently broken in Nova:

https://bugs.launchpad.net/nova/+bug/1354403

If they've never been used, they're presumably of limited value.

Matt

 
 N.

 [1] https://review.openstack.org/#/c/87546/

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


-- 
Matthew Booth
Red Hat Engineering, Virtualisation Team

Phone: +442070094448 (UK)
GPG ID:  D33C3490
GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][ml2] Mech driver as out-of-tree add-on

2014-08-15 Thread Kyle Mestery

I think the review time alone is a huge issue. Even worse, for the
most part, core reviewers are reviewing code for which they themselves
can't test because it requires proprietary hardware or software,
making the situation brittle at best. Having a separate git repository
which is gated by stringent third-party CI requirements, with separate
(and possibly overlapping) core reviewers would help to alleviate this
problem. Another alternative is to move most intelligence out of the
plugins/drivers and into vendor owned packages which can live on pypi.
This is similar to what the PLUMgrid folks did for their plugin. This
allows vendor control over most of their bits, removes the constant
churn for simple bug fixes in the neutron tree, and adds the benefit
of being a part of the simultaneous release, which is the only thing
most vendors care about.

On Thu, Aug 14, 2014 at 10:34 PM, Kevin Benton blak...@gmail.com wrote:
I also feel like the drivers/plugins are currently BEYOND a tipping
 point, and are in fact dragging down velocity of the core project in
 many ways.

 Can you elaborate on the ways that they are slowing down the velocity? I
 know they take up reviewer time, but are there other ways that you think
 they slow down the project?


 On Thu, Aug 14, 2014 at 6:07 AM, Kyle Mestery mest...@mestery.com wrote:

 I also feel like the drivers/plugins are currently BEYOND a tipping
 point, and are in fact dragging down velocity of the core project in
 many ways. I'm working on a proposal for Kilo where we move all
 drivers/plugins out of the main Neutron tree and into a separate git
 repository under the networking program. We have way too many drivers,
 requiring way too man review cycles, for this to be a sustainable
 model going forward. Since the main reason plugin/driver authors want
 their code upstream is to be a part of the simultaneous release, and
 thus be packaged by distributions, having a separate repository for
 these will satisfy this requirement. I'm still working through the
 details around reviews of this repository, etc.

 Also, I feel as if the level of passion on the mailing list has died
 down a bit, so I thought I'd send something out to try and liven
 things up a bit. It's been somewhat non-emotional here for a day or
 so. :)

 Thanks,
 Kyle

 On Thu, Aug 14, 2014 at 5:09 AM, Salvatore Orlando sorla...@nicira.com
 wrote:
  I think there will soon be a discussion regarding what the appropriate
  location for plugin and drivers should be.
  My personal feeling is that Neutron has simply reached the tipping point
  where the high number of drivers and plugins is causing unnecessary load
  for
  the core team and frustration for the community.
 
  There I would totally support Luke's initiative about maintaining an
  out-of-tree ML2 driver. On the other hand, a plugin/driver diaspora
  might
  also have negative consequences such as frequent breakages such as those
  Bob
  was mentioning or confusion for users which might need to end up
  fetching
  drivers from disparate sources.
 
  As mentioned during the last Neutron IRC meeting this is another
  process
  aspect which will be discussed soon, with the aim of defining a plan
  for:
  - drastically reduce the number of plugins and drivers which must be
  maintained in the main source tree
  - enhance control of plugin/driver maintainers over their own code
  - preserve the ability of doing CI checks on gerrit as we do today
  - raise the CI bar (maybe finally set the smoketest as a minimum
  requirement?)
 
  Regards,
  Salvatore
 
 
 
  On 14 August 2014 11:47, loy wolfe loywo...@gmail.com wrote:
 
 
 
 
  On Thu, Aug 14, 2014 at 4:22 PM, Mathieu Rohon
  mathieu.ro...@gmail.com
  wrote:
 
  Hi,
 
  I would like to add that it would be harder for the community to help
  maintaining drivers.
  such a work [1] wouldn't have occured with an out of tree ODL driver.
 
 
  +1.
  It's better to move all MD for none built-in backend out of tree,
  maintaining these drivers shouldn't be the responsibility of community.
  Not
  only MD, but also plugin, agent should all obey this rule
 
 
 
  [1] https://review.openstack.org/#/c/96459/
 
  On Wed, Aug 13, 2014 at 1:09 PM, Robert Kukura
  kuk...@noironetworks.com
  wrote:
   One thing to keep in mind is that the ML2 driver API does sometimes
   change,
   requiring updates to drivers. Drivers that are in-tree get updated
   along
   with the driver API change. Drivers that are out-of-tree must be
   updated by
   the owner.
  
   -Bob
  
  
   On 8/13/14, 6:59 AM, ZZelle wrote:
  
   Hi,
  
  
   The important thing to understand is how to integrate with neutron
   through
   stevedore/entrypoints:
  
  
  
   https://github.com/dave-tucker/odl-neutron-drivers/blob/master/setup.cfg#L32-L34
  
  
   Cedric
  
  
   On Wed, Aug 13, 2014 at 12:17 PM, Dave Tucker d...@dtucker.co.uk
   wrote:
  
   I've been working on this for OpenDaylight
   https://github.com/dave-tucker/odl-neutron-drivers

Re: [openstack-dev] [all] The future of the integrated release

2014-08-15 Thread Eoghan Glynn


 But, if someone asked me what they should use for metering today,
 I'd point them towards Monasca in a heartbeat.

FWIW my view is that Monasca is an interesting emerging project, with a
team accreting around it that seems to be interested in collaboration.

We've had ongoing discussions with them about overlaps and differences
since the outset of this cycle, though of course our over-riding focus
for Juno has had to be on the TC gap analysis and on addressing
architectural debts.

But going forward into Kilo, I think there should be scope for possible
closer collaboration between the projects, figuring out the aspects that
are complementary and possible shared elements and/or converged APIs.

Cheers,
Eoghan

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Deprecating CONF.block_device_allocate_retries_interval

2014-08-15 Thread Andrew Laski



On 08/15/2014 11:43 AM, Matthew Booth wrote:

On 15/08/14 16:12, Andrew Laski wrote:

On 08/08/2014 08:42 AM, Nikola Đipanov wrote:

On 08/06/2014 07:54 PM, Jay Pipes wrote:

I bring this up on the mailing list because I think Liyi's patch offers
an interesting future direction to the way that we think about our retry
approach in Nova. Instead of having hard-coded or configurable interval
times, I think Liyi's approach of calculating the interval length based
on some input values is a good direction to take.


This indeed is a problem that we've seen bite us a number of times, and
I tried to solve it by proposing [1] but didn't get to work on it
further yet.

Having said that - after thinking about it more, I was not sure I like
my own approach in [1] on the grounds of it being too generic (and
overly elaborate) for the particular problem it is solving.

I was then thinking of something similar to what is proposed here, where
we would have a waiting time that is a function of a value that we could
query Cinder on. Allocation rate proposed here seems to fit this nicely,
but in my mind we would have a way to query cinder about it instead of
hardcoding it, however this can be done later and in cooperation with
the Cinder team.

I like this direction a lot, because the allocation time depends on
Cinder and its backend more than anything.  Excepting perhaps image
transfer speeds.


2) We should deprecate the CONF.block_device_allocate_retries_interval
option only, and keep the CONF.block_device_allocate_retries
configuration option as-is, changing the help text to read something
like Max number of retries. We calculate the interval of the retry
based on the size of the volume.


I'd go with this as the number of retries can still be useful as a tool
for easy workaround and troubleshooting, but I'd put a big disclaimer
that it is mostly meant for debugging/workaround purposes.

I disagree a bit with having a knob purely for debugging/workaround.  I
think this is a place where it's very useful to have the knob though.
The figures in the review seem sound, but this is probably not a place
where one size is going to fit all.  Until/unless we can get some
coordination between Nova and Cinder on this I think having a knob that
is intended to be used is the right way to go.

FYI, this bug is admittedly only from code inspection, but I think many
interval config variables are currently broken in Nova:

https://bugs.launchpad.net/nova/+bug/1354403

If they've never been used, they're presumably of limited value.


Fair point.  Though it's likely they'll be fixed as soon as someone 
starts tuning them and finds them broken.  And it'll be easier to fix 
them than to remove them and add them in later.  Which is not to say 
that all of them are useful.


But the two values in question here are ones I've been tuning recently 
and know that they work.




Matt


N.

[1] https://review.openstack.org/#/c/87546/

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev





___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][ml2] Mech driver as out-of-tree add-on

2014-08-15 Thread Daniel P. Berrange

On Thu, Aug 14, 2014 at 12:09:26PM +0200, Salvatore Orlando wrote:
 I think there will soon be a discussion regarding what the appropriate
 location for plugin and drivers should be.
 My personal feeling is that Neutron has simply reached the tipping point
 where the high number of drivers and plugins is causing unnecessary load
 for the core team and frustration for the community.
 
 There I would totally support Luke's initiative about maintaining an
 out-of-tree ML2 driver. On the other hand, a plugin/driver diaspora might
 also have negative consequences such as frequent breakages such as those
 Bob was mentioning or confusion for users which might need to end up
 fetching drivers from disparate sources.
 
 As mentioned during the last Neutron IRC meeting this is another process
 aspect which will be discussed soon, with the aim of defining a plan for:
 - drastically reduce the number of plugins and drivers which must be
 maintained in the main source tree
 - enhance control of plugin/driver maintainers over their own code
 - preserve the ability of doing CI checks on gerrit as we do today
 - raise the CI bar (maybe finally set the smoketest as a minimum
 requirement?)

Note also that there is a communications channel between Nova and Neutron
that has to pass port binding info when plugging VIFs, so that Nova can
configure things in the right way. 

The definition of the valid metadata in this API is not something that
I would consider external plugins to be permitted to extend/change as
it is a semi-public communications channel, not merely an internal
implementation detail. Also note that from Juno onwards Nova no longer
supports out of tree VIF driver implementations because we do not
consider the VIF driver classes to be a public extension point.

So if a Neutron plugin is to live out of tree, it needs to be able to
work with the VIF port binding metadata that is officially supported
in-tree by Neutron  Nova, and with an in-tree Nova VIF driver.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Cinder] 3'rd party CI systems: Not Whitelisted Volume type with name * could not be found.

2014-08-15 Thread Asselin, Ramy

John,

We’re running master on every patch set. What needs to be set in localrc?
Actually, I think I know what could be the issue (still need to confirm)


1.   devstack-gate/devstack-vm-gate-wrap.sh configures localrc

2.   We configuring the cinder backend in local.conf. We’re not setting 
CINDER_ENABLED_BACKENDS variable which stack.sh is looking at. [1]


Question: How do you configure devstack-gate/devstack-vm-gate-wrap.sh to set 
CINDER_ENABLED_BACKENDS in localrc?

Ramy

[1] 
http://15.126.198.151/09/114509/2/check/dsvm-tempest-hp-lefthand/b8e7c4d/logs/devstacklog.txt.gz

2014-08-15 14:10:57.619 | + create_2014-08-15 14:10:57.619 | + 
create_volume_types
2014-08-15 14:10:57.619 | + is_service_enabled c-api
2014-08-15 14:10:57.625 | + return 0
2014-08-15 14:10:57.625 | + [[ -n lvm:lvmdriver-1 ]]
2014-08-15 14:10:57.625 | + for be in '${CINDER_ENABLED_BACKENDS//,/ }'
2014-08-15 14:10:57.625 | + BE_TYPE=lvm
2014-08-15 14:10:57.626 | + BE_NAME=lvmdriver-1
2014-08-15 14:10:57.626 | + type configure_cinder_backend_lvm
2014-08-15 14:10:57.626 | + cinder type-create lvmdriver-1
2014-08-15 14:10:59.324 | +--+-+
2014-08-15 14:10:59.324 | |  ID  | Name|
2014-08-15 14:10:59.324 | +--+-+
2014-08-15 14:10:59.324 | | 5c154cdd-2fa3-4056-a669-f535b33d3105 | lvmdriver-1 |
2014-08-15 14:10:59.324 | +--+-+
2014-08-15 14:10:59.360 | + cinder type-key lvmdriver-1 set 
volume_backend_name=lvmdriver-1

From: John Griffith [mailto:john.griff...@solidfire.com]
Sent: Friday, August 15, 2014 6:48 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Cinder] 3'rd party CI systems: Not Whitelisted 
Volume type with name * could not be found.



On Fri, Aug 15, 2014 at 6:39 AM, Kerr, Andrew 
andrew.k...@netapp.commailto:andrew.k...@netapp.com wrote:
Can the default_volume_type be left empty and get the original None
type, so we don't have to create the volume type prior to running tempest
tests?

Andrew Kerr
OpenStack QA
Cloud Solutions Group
NetApp



On 8/14/14, 5:23 PM, Martin, Kurt Frederick (ESSN Storage MSDU)
kurt.f.mar...@hp.commailto:kurt.f.mar...@hp.com wrote:

They need to be manually added.

-Original Message-
From: Asselin, Ramy
Sent: Thursday, August 14, 2014 2:17 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Cinder] 3'rd party CI systems: Not
Whitelisted Volume type with name * could not be found.

Both configurations have that set as you described. [1][2] Who actually
creates that volume type? Is that supposed to be added manually to
local.sh, or is this a bug in devstack?

[1]
http://publiclogs.emc.com/vnx_ostack/EMC_VNX_FC/212/logs/etc/cinder/cinder
.conf.txt.gz
[DEFAULT]
default_volume_type = cat_1

[2]
http://15.126.198.151/60/111460/18/check/dsvm-tempest-hp-lefthand/3ee1598/
logs/etc/cinder/cinder.conf.txt.gz
[DEFAULT]
default_volume_type = dot

-Original Message-
From: Martin, Kurt Frederick (ESSN Storage MSDU)
Sent: Thursday, August 14, 2014 2:00 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Cinder] 3'rd party CI systems: Not
Whitelisted Volume type with name * could not be found.

Cinder.conf needs to have a default_volume_type  entry set under the
[Default] group and a volume type that is valid, for example,
default_volume_type=bronze. This allows for a volume to be created when a
volume type is not selected, the default 'None'  type. This feature has
been available for some time in cinder but recently enabled in devstack.
~Kurt

-Original Message-
From: Asselin, Ramy
Sent: Thursday, August 14, 2014 10:00 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [Cinder] 3'rd party CI systems: Not Whitelisted
Volume type with name * could not be found.

Hi,

Does anyone know how to configure cinder ci tests to not have these
errors?

12:32:11 *** Not Whitelisted *** 2014-08-14 12:23:56.179 18021 ERROR
cinder.volume.volume_types [req-c9ec92ab-132b-4167-a467-15bd213659e8
bd1f54a867ce47acb4097cd94149efa0 d15dcf4cb3c7495389799ec849e9036d - - -]
Default volume type is not found, please check default_volume_type
config: Volume type with name dot could not be found.

17:43:28 *** Not Whitelisted *** 2014-08-11 17:31:20.343 2097 ERROR
cinder.volume.volume_types [req-01e539ad-5357-4a0f-8d58-464b970114f9
f72cd499be584d9d9585bc26ca71c603 d845ad2d7e6a47dfb84bdf0754f60384 - - -]
Default volume type is not found, please check default_volume_type
config: Volume type with name cat_1 could not be found.

http://15.126.198.151/60/111460/18/check/dsvm-tempest-hp-lefthand/3ee1598/
console.html
http://publiclogs.emc.com/vnx_ostack/EMC_VNX_FC/212/console.html

Thanks,
Ramy

___

Re: [openstack-dev] [Cinder] 3'rd party CI systems: Not Whitelisted Volume type with name * could not be found.

2014-08-15 Thread John Griffith

On Fri, Aug 15, 2014 at 10:29 AM, Asselin, Ramy ramy.asse...@hp.com wrote:

  John,



 We’re running master on every patch set. What needs to be set in localrc?

 Actually, I think I know what could be the issue (still need to confirm)



 1.   devstack-gate/devstack-vm-gate-wrap.sh configures localrc

 2.   We configuring the cinder backend in local.conf. We’re not
 setting CINDER_ENABLED_BACKENDS variable which stack.sh is looking at. [1]

Correct, that's what you need.



 Question: How do you configure devstack-gate/devstack-vm-gate-wrap.sh to
 set CINDER_ENABLED_BACKENDS in localrc?

Recent change that honestly I never felt was really clear, but in your
localrc:
CINDER_ENABLED_BACKENDS={BACKEND_TYPE}:{BACKEND_NAME}, the work is done
here:
https://github.com/openstack-dev/devstack/blob/master/lib/cinder#L467




 Ramy



 [1]
 http://15.126.198.151/09/114509/2/check/dsvm-tempest-hp-lefthand/b8e7c4d/logs/devstacklog.txt.gz



 2014-08-15 14:10:57.619 | + create_2014-08-15 14:10:57.619 | +
 create_volume_types

 2014-08-15 14:10:57.619 | + is_service_enabled c-api

 2014-08-15 14:10:57.625 | + return 0

 2014-08-15 14:10:57.625 | + [[ -n lvm:lvmdriver-1 ]]

 2014-08-15 14:10:57.625 | + for be in '${CINDER_ENABLED_BACKENDS//,/ }'

 2014-08-15 14:10:57.625 | + BE_TYPE=lvm

 2014-08-15 14:10:57.626 | + BE_NAME=lvmdriver-1

 2014-08-15 14:10:57.626 | + type configure_cinder_backend_lvm

 2014-08-15 14:10:57.626 | + cinder type-create lvmdriver-1

 2014-08-15 14:10:59.324 |
 +--+-+

 2014-08-15 14:10:59.324 | |  ID  |
 Name|

 2014-08-15 14:10:59.324 |
 +--+-+

 2014-08-15 14:10:59.324 | | 5c154cdd-2fa3-4056-a669-f535b33d3105 |
 lvmdriver-1 |

 2014-08-15 14:10:59.324 |
 +--+-+

 2014-08-15 14:10:59.360 | + cinder type-key lvmdriver-1 set
 volume_backend_name=lvmdriver-1



 *From:* John Griffith [mailto:john.griff...@solidfire.com]
 *Sent:* Friday, August 15, 2014 6:48 AM

 *To:* OpenStack Development Mailing List (not for usage questions)
 *Subject:* Re: [openstack-dev] [Cinder] 3'rd party CI systems: Not
 Whitelisted Volume type with name * could not be found.







 On Fri, Aug 15, 2014 at 6:39 AM, Kerr, Andrew andrew.k...@netapp.com
 wrote:

 Can the default_volume_type be left empty and get the original None
 type, so we don't have to create the volume type prior to running tempest
 tests?

 Andrew Kerr
 OpenStack QA
 Cloud Solutions Group
 NetApp



 On 8/14/14, 5:23 PM, Martin, Kurt Frederick (ESSN Storage MSDU)

 kurt.f.mar...@hp.com wrote:

 They need to be manually added.
 
 -Original Message-
 From: Asselin, Ramy
 Sent: Thursday, August 14, 2014 2:17 PM
 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] [Cinder] 3'rd party CI systems: Not
 Whitelisted Volume type with name * could not be found.
 
 Both configurations have that set as you described. [1][2] Who actually
 creates that volume type? Is that supposed to be added manually to
 local.sh, or is this a bug in devstack?
 
 [1]
 
 http://publiclogs.emc.com/vnx_ostack/EMC_VNX_FC/212/logs/etc/cinder/cinder
 .conf.txt.gz
 [DEFAULT]
 default_volume_type = cat_1
 
 [2]
 
 http://15.126.198.151/60/111460/18/check/dsvm-tempest-hp-lefthand/3ee1598/
 logs/etc/cinder/cinder.conf.txt.gz
 [DEFAULT]
 default_volume_type = dot
 
 -Original Message-
 From: Martin, Kurt Frederick (ESSN Storage MSDU)
 Sent: Thursday, August 14, 2014 2:00 PM
 To: OpenStack Development Mailing List (not for usage questions)
 Subject: Re: [openstack-dev] [Cinder] 3'rd party CI systems: Not
 Whitelisted Volume type with name * could not be found.
 
 Cinder.conf needs to have a default_volume_type  entry set under the
 [Default] group and a volume type that is valid, for example,
 default_volume_type=bronze. This allows for a volume to be created when a
 volume type is not selected, the default 'None'  type. This feature has
 been available for some time in cinder but recently enabled in devstack.
 ~Kurt
 
 -Original Message-
 From: Asselin, Ramy
 Sent: Thursday, August 14, 2014 10:00 AM
 To: OpenStack Development Mailing List (not for usage questions)
 Subject: [openstack-dev] [Cinder] 3'rd party CI systems: Not Whitelisted
 Volume type with name * could not be found.
 
 Hi,
 
 Does anyone know how to configure cinder ci tests to not have these
 errors?
 
 12:32:11 *** Not Whitelisted *** 2014-08-14 12:23:56.179 18021 ERROR
 cinder.volume.volume_types [req-c9ec92ab-132b-4167-a467-15bd213659e8
 bd1f54a867ce47acb4097cd94149efa0 d15dcf4cb3c7495389799ec849e9036d - - -]
 Default volume type is not found, please check default_volume_type
 config: Volume type with name dot could not be found.
 
 17:43:28 *** Not Whitelisted *** 2014-08-11 17:31:20.343 2097 ERROR
 cinder.volume.volume_types

Re: [openstack-dev] OS or os are not acronyms for OpenStack

2014-08-15 Thread Anita Kuno

On 08/15/2014 09:12 AM, Russell Bryant wrote:
 On 08/15/2014 11:00 AM, Mike Spreitzer wrote:
 Anita Kuno ante...@anteaya.info wrote on 08/15/2014 10:38:20 AM:

 OpenStack is OpenStack. The use of openstack is also acceptable in our
 development conversations.

 OS or os is operating system. I am starting to see some people us OS or
 os to mean OpenStack. This is confusing and also incorrect[0].

 ...

 I have seen OS for OpenStack from the start.  Just look at the
 environment variables that the CLI reads.
 
 Yep, it's quite common and I think it's fine in the right context.
 
I think you hit the nail on the head here, Russell, it's fine in the
right context.

The definition of the right context however is somewhat elusive. I have
chosen (it is my own fault) to place myself in the area where the folks
I deal with struggle with understanding context. The newcomers to the
third party space and folks creating stackforge repos don't have the
benefit of the understanding that core reviewers have (would I be
accurate in saying that it is mostly nova reviewers who have responded
to my initial post thus far?).

If the people making the decisions about when to use which term were
clear on the context, I doubt highly that I would have been motivated to
post.

However I am seeing confusion in the ability to chose terms appropriate
to context and wanted to do something if I could (sometimes I can't and
sometimes I outright fail) before too much more
assumption-of-agreement-due-to-lack-of-stated-disagrement took place.

I'm open to suggestion as to how to create a situation in which everyone
understands the right context. Right now I am feeling there is a gap and
it is only getting wider.

Thanks for sharing your thoughts so far everyone, I'm glad to get a
chance to talk about this.
Anita.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] OS or os are not acronyms for OpenStack

2014-08-15 Thread Mike Spreitzer

Anita Kuno ante...@anteaya.info wrote on 08/15/2014 01:08:44 PM:

 ...
 I think you hit the nail on the head here, Russell, it's fine in the
 right context.
 
 The definition of the right context however is somewhat elusive. I have
 chosen (it is my own fault) to place myself in the area where the folks
 I deal with struggle with understanding context. The newcomers to the
 third party space and folks creating stackforge repos don't have the
 benefit of the understanding that core reviewers have (would I be
 accurate in saying that it is mostly nova reviewers who have responded
 to my initial post thus far?).

I suffered from an instance of this confusion myself when I was just 
getting started,
and have seen colleagues get confused too.  I suspect this problem hits 
many
newcomers.

Regards,
Mike

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] OS or os are not acronyms for OpenStack

On 08/15/2014 01:28 PM, Mike Spreitzer wrote:
 Anita Kuno ante...@anteaya.info wrote on 08/15/2014 01:08:44 PM:
 
 ...
 I think you hit the nail on the head here, Russell, it's fine in the
 right context.

 The definition of the right context however is somewhat elusive. I have
 chosen (it is my own fault) to place myself in the area where the folks
 I deal with struggle with understanding context. The newcomers to the
 third party space and folks creating stackforge repos don't have the
 benefit of the understanding that core reviewers have (would I be
 accurate in saying that it is mostly nova reviewers who have responded
 to my initial post thus far?).
 
 I suffered from an instance of this confusion myself when I was just
 getting started,
 and have seen colleagues get confused too.  I suspect this problem hits
 many
 newcomers.

but surely when it comes to learning OpenStack itself, the OpenStack
community, dev processes, tools, etc  this has got to be extremely
far down the list of barriers to entry.

-- 
Russell Bryant

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Solum] Juno-3 Items

2014-08-15 Thread Roshan Agrawal

Hi, following up from last week's IRC meeting, I updated the Solum roadmap wiki 
with items for consideration for Juno-3. Please mail back your 
comments/suggestions, as well as we can discuss more in the next IRC and 
finalize Juno release scope.

Wiki link: 
https://wiki.openstack.org/wiki/Solum/HighLevelRoadmap#Milestone_2014.2.3:_Juno-3

MILESTONE 2014.2.3: JUNO-3
==
. Jenkins Integration: as a customer already using Jenkins for my CI needs, I 
would be able to integrate my Jenkins CI instance with Solum and have access to 
Solum features (i.e. persist build output produced by Jenkins in glance via 
Solum, automated deployments, etc.)

. Trove Integration: as an application developer, I would be able to deploy 
apps that have ready access to trove MySQL service.

. Horizontal Scaleout: as an application developer, I would be able to 
horizontal scale out my apps via a load balanced set of app container instances

. LanguagePack-Java Tomcat 7: as an application developer, I can deploy my Java 
Tomcat 7 web app via Solum without having to deal with build scripts or setup 
of the language runtime

. LanguagePack-Node.js: as an application developer, I can deploy my Node.js 
web app via Solum without having to deal with setup of the language runtime

. Sample apps: as an application developer, I can build my apps without having 
to start from scratch by customizing sample apps provided by Solum. Sample Apps 
for Java Tomcat 7 and Node.js

Thanks  Regards,
Roshan Agrawal
Direct:    512.874.1278
Mobile:  512.354.5253
roshan.agra...@rackspace.com


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] OS or os are not acronyms for OpenStack

2014-08-15 Thread Mike Spreitzer

Russell Bryant rbry...@redhat.com wrote on 08/15/2014 01:49:40 PM:
 ... 
 but surely when it comes to learning OpenStack itself, the OpenStack
 community, dev processes, tools, etc  this has got to be extremely
 far down the list of barriers to entry.

No argument there.  I am spending decimal orders of magnitude more time 
looking for a working concrete example of using DevStack to install 
OpenStack with Neutron inside a VM so that the nested VMs can talk to the 
outside world.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] OS or os are not acronyms for OpenStack

2014-08-15 Thread Adam Lawson

I really think we have much bigger fish to fry than to start policing
community shorthand where nearly every meeting and communication is typed.
That's just my two cents for what it's worth.

Mahalo,
Adam


*Adam Lawson*
AQORN, Inc.
427 North Tatnall Street
Ste. 58461
Wilmington, Delaware 19801-2230
Toll-free: (844) 4-AQORN-NOW ext. 101
International: +1 302-387-4660
Direct: +1 916-246-2072



On Fri, Aug 15, 2014 at 10:49 AM, Russell Bryant rbry...@redhat.com wrote:

 On 08/15/2014 01:28 PM, Mike Spreitzer wrote:
  Anita Kuno ante...@anteaya.info wrote on 08/15/2014 01:08:44 PM:
 
  ...
  I think you hit the nail on the head here, Russell, it's fine in the
  right context.
 
  The definition of the right context however is somewhat elusive. I have
  chosen (it is my own fault) to place myself in the area where the folks
  I deal with struggle with understanding context. The newcomers to the
  third party space and folks creating stackforge repos don't have the
  benefit of the understanding that core reviewers have (would I be
  accurate in saying that it is mostly nova reviewers who have responded
  to my initial post thus far?).
 
  I suffered from an instance of this confusion myself when I was just
  getting started,
  and have seen colleagues get confused too.  I suspect this problem hits
  many
  newcomers.

 but surely when it comes to learning OpenStack itself, the OpenStack
 community, dev processes, tools, etc  this has got to be extremely
 far down the list of barriers to entry.

 --
 Russell Bryant

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Enabling silent Docker tests for Nova?

On 08/15/2014 02:45 PM, Eric Windisch wrote:
 I have proposed a _silent_ check for Nova for integration of the Docker
 driver:
 
 https://review.openstack.org/#/c/114547/
 
 It has been established that this code cannot move back into Nova until
 the tests are running and have a solid history of success. That cannot
 happen unless we're allowed to run the tests. Running a silent check on
 changes to Nova is the first step in establishing that history.
 
 Joe Gordon suggests we need a spec to bring the driver back into Nova.
 Besides the fact that specs are closed and there is no intention of
 reintegrating the driver for Juno, I'm uncertain of proposing a spec
 without first having solid history of successful testing, especially
 given the historical context of this driver's relationship with Nova.
 
 If we could enable silent checks, we could help minimize API skew and
 branch breakages, improving driver quality and reducing maintenance
 while we prepare for the Kilo spec + merge windows. Furthermore, by
 having a history of testing, we can seek faster inclusion into Kilo.
 
 Finally, I acknowledge that we may be entering a window of significant
 load on the CI servers and I'm sensitive to the needs of the
 infrastructure team to remain both focused and to conserve precious
 compute resources. If this is an issue, then I'd like to plot a
 timeline, however rough, with the infrastructure team. 

CI resources aside, I think enabling it sounds fine and useful.

Given resource concerns, maybe just adding it to the experimental
pipeline would be sufficient?  That doesn't run as often, but still
gives you the chance to run on demand against nova patches.  There are
other things in experimental for nova as well, so there will be other
people triggering runs.

-- 
Russell Bryant

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Enabling silent Docker tests for Nova?

On 08/15/2014 02:53 PM, Russell Bryant wrote:
 On 08/15/2014 02:45 PM, Eric Windisch wrote:
 I have proposed a _silent_ check for Nova for integration of the Docker
 driver:

 https://review.openstack.org/#/c/114547/

 It has been established that this code cannot move back into Nova until
 the tests are running and have a solid history of success. That cannot
 happen unless we're allowed to run the tests. Running a silent check on
 changes to Nova is the first step in establishing that history.

 Joe Gordon suggests we need a spec to bring the driver back into Nova.
 Besides the fact that specs are closed and there is no intention of
 reintegrating the driver for Juno, I'm uncertain of proposing a spec
 without first having solid history of successful testing, especially
 given the historical context of this driver's relationship with Nova.

 If we could enable silent checks, we could help minimize API skew and
 branch breakages, improving driver quality and reducing maintenance
 while we prepare for the Kilo spec + merge windows. Furthermore, by
 having a history of testing, we can seek faster inclusion into Kilo.

 Finally, I acknowledge that we may be entering a window of significant
 load on the CI servers and I'm sensitive to the needs of the
 infrastructure team to remain both focused and to conserve precious
 compute resources. If this is an issue, then I'd like to plot a
 timeline, however rough, with the infrastructure team. 
 
 CI resources aside, I think enabling it sounds fine and useful.
 
 Given resource concerns, maybe just adding it to the experimental
 pipeline would be sufficient?  That doesn't run as often, but still
 gives you the chance to run on demand against nova patches.  There are
 other things in experimental for nova as well, so there will be other
 people triggering runs.
 

And I missed that it's already in experimental.  Oops.

Feature freeze is only a few weeks away (Sept 4).  How about we just
leave it in experimental until after that big push?  That seems pretty
reasonable.

-- 
Russell Bryant

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Enabling silent Docker tests for Nova?

2014-08-15 Thread Eric Windisch


 Given resource concerns, maybe just adding it to the experimental
 pipeline would be sufficient?

For clarity, the discussed patch is to promote an existing experimental job
to silent.

Regards -Eric
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Enabling silent Docker tests for Nova?

2014-08-15 Thread Dan Smith

 Feature freeze is only a few weeks away (Sept 4).  How about we just
 leave it in experimental until after that big push?  That seems pretty
 reasonable.

Joe just proposed dropping a bunch of semi-useless largeops runs. Maybe
that leaves room to sneak this in? If the infra team is okay with it,
I'd say we should go ahead and do it, but regardless of that decision,
I'm +1 on getting this promoted so we can start the stability timer.

--Dan



signature.asc
Description: OpenPGP digital signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Enabling silent Docker tests for Nova?

On Fri, Aug 15, 2014 at 11:55 AM, Russell Bryant rbry...@redhat.com wrote:

 On 08/15/2014 02:53 PM, Russell Bryant wrote:
  On 08/15/2014 02:45 PM, Eric Windisch wrote:
  I have proposed a _silent_ check for Nova for integration of the Docker
  driver:
 
  https://review.openstack.org/#/c/114547/
 
  It has been established that this code cannot move back into Nova until
  the tests are running and have a solid history of success. That cannot
  happen unless we're allowed to run the tests. Running a silent check on
  changes to Nova is the first step in establishing that history.
 
  Joe Gordon suggests we need a spec to bring the driver back into Nova.
  Besides the fact that specs are closed and there is no intention of
  reintegrating the driver for Juno, I'm uncertain of proposing a spec
  without first having solid history of successful testing, especially
  given the historical context of this driver's relationship with Nova.
 
  If we could enable silent checks, we could help minimize API skew and
  branch breakages, improving driver quality and reducing maintenance
  while we prepare for the Kilo spec + merge windows. Furthermore, by
  having a history of testing, we can seek faster inclusion into Kilo.
 
  Finally, I acknowledge that we may be entering a window of significant
  load on the CI servers and I'm sensitive to the needs of the
  infrastructure team to remain both focused and to conserve precious
  compute resources. If this is an issue, then I'd like to plot a
  timeline, however rough, with the infrastructure team.
 
  CI resources aside, I think enabling it sounds fine and useful.
 
  Given resource concerns, maybe just adding it to the experimental
  pipeline would be sufficient?  That doesn't run as often, but still
  gives you the chance to run on demand against nova patches.  There are
  other things in experimental for nova as well, so there will be other
  people triggering runs.
 

 And I missed that it's already in experimental.  Oops.

 Feature freeze is only a few weeks away (Sept 4).  How about we just
 leave it in experimental until after that big push?  That seems pretty
 reasonable.


Sounds like a good idea to me too. Also I am unsure if we ant to have this
running as silent for a long period of time. Yes, as experimental job its
hard to get a large enough data set, so we need to generate a larger data
set, but how big is big enough? Will running this job on silent for 1 week
be enough? 2 weeks?



 --
 Russell Bryant

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [QA] Picking a Name for the Tempest Library

2014-08-15 Thread Matthew Treinish

Hi Everyone,

So as part of splitting out common functionality from tempest into a library [1]
we need to create a new repository. Which means we have the fun task of coming
up with something to name it. I'm personally thought we should call it:

 - mesocyclone

Which has the advantage of being a cloud/weather thing, and the name sort of
fits because it's a precursor to a tornado. Also, it's an available namespace on
both launchpad and pypi. But there has been expressed concern that both it is a
bit on the long side (which might have 80 char line length implications) and
it's unclear from the name what it does. 

During the last QA meeting some alternatives were also brought up:

 - tempest-lib / lib-tempest
 - tsepmet
 - blackstorm
 - calm
 - tempit
 - integration-test-lib

(although I'm not entirely sure I remember which ones were serious suggestions
or just jokes)

So as a first step I figured that I'd bring it up on the ML to see if anyone had
any other suggestions. (or maybe get a consensus around one choice) I'll take
the list, check if the namespaces are available, and make a survey so that
everyone can vote and hopefully we'll have a clear choice for a name from that.

-Matt Treinish

[1] http://specs.openstack.org/openstack/qa-specs/specs/tempest-library.html


pgpp3nT2M_RQQ.pgp
Description: PGP signature
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][core] Expectations of core reviewers

2014-08-15 Thread John Griffith

On Fri, Aug 15, 2014 at 8:20 AM, Jeremy Stanley fu...@yuggoth.org wrote:

 On 2014-08-14 09:33:20 -0400 (-0400), Russell Bryant wrote:
 [...]
  Another issue is that some folks are just fundamentally opposed to
  using Google
 [...]

 I think that's a shallow depiction of the issue. I'm sure *some*
 people really do just avoid Google specifically, but a bigger
 concern should be around the statement our use of those services
 presents to the rest of the World. By using commercial solutions
 because the open alternatives aren't as useful/featureful/stable, we
 rob those projects of a potential larger user community which could
 help them achieve greater momentum and eventually dominate their
 respective technologies.

 Would we, as a community, rather see OpenStack used in production
 and improved when it has bugs/lacks features? Or should users just
 view it as a cheap lab platform instead, and then pay for solid
 proprietary solutions to their production needs? I'm glad a lot of
 people think using and improving OpenStack, even when there might
 sometimes be easier/simpler closed alternatives, is worth the
 long-term investment.

 We, as visible leaders among the greater free software community,
 should think very hard when making the choice not to extend this
 same courtesy and consideration to other projects who may lack our
 extraordinary base of resources. Through our example as a project,
 we have the potential to improve things for open/free software
 communities far beyond our own.

 Of course I'm not a nova core reviewer and have never attended a
 nova mid-cycle meetup, so I really have no say in how you conduct
 your remote participation. But I do hope you'll consider,
 collectively, that focusing on immediate convenience for your own
 community can have negative longer-term consequences as you (whether
 consciously or not) promote the use of proprietary solutions to your
 needs rather than embracing less convenient free and open options
 which may still require improvement.
 --
 Jeremy Stanley

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Lots of stuff here, good points, bad points etc.

I just want to take the opportunity to state that I have ZERO intention of
ever pushing that if you're on core you must travel anywhere.  I think
it's completely alienating and unfair and frankly kinda surprised it was
actually raised.

For anybody that contributes, works hard and gives great reviews but can't
travel to the various Nova events around the world, feel free to check out
Cinder.  We don't care if you meet us in person and we love good
contributors regardless of where they live etc.

I could insert snarky comment about unable to work remotely using
technology and maybe should choose another project or career here but I
won't. 
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [QA] Picking a Name for the Tempest Library

2014-08-15 Thread Drew Fisher


What about 'teapot' (as in the idiom 'tempest in a teapot'[1])

-Drew

[1] http://en.wikipedia.org/wiki/Tempest_in_a_teapot


On 08/15/2014 01:14 PM, Matthew Treinish wrote:

Hi Everyone,

So as part of splitting out common functionality from tempest into a library [1]
we need to create a new repository. Which means we have the fun task of coming
up with something to name it. I'm personally thought we should call it:

  - mesocyclone

Which has the advantage of being a cloud/weather thing, and the name sort of
fits because it's a precursor to a tornado. Also, it's an available namespace on
both launchpad and pypi. But there has been expressed concern that both it is a
bit on the long side (which might have 80 char line length implications) and
it's unclear from the name what it does.

During the last QA meeting some alternatives were also brought up:

  - tempest-lib / lib-tempest
  - tsepmet
  - blackstorm
  - calm
  - tempit
  - integration-test-lib

(although I'm not entirely sure I remember which ones were serious suggestions
or just jokes)

So as a first step I figured that I'd bring it up on the ML to see if anyone had
any other suggestions. (or maybe get a consensus around one choice) I'll take
the list, check if the namespaces are available, and make a survey so that
everyone can vote and hopefully we'll have a clear choice for a name from that.

-Matt Treinish

[1] http://specs.openstack.org/openstack/qa-specs/specs/tempest-library.html



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][ml2] Mech driver as out-of-tree add-on

2014-08-15 Thread Kevin Benton

I definitely agree that reviewer time is wasted reviewing changes. However,
I don't think moving them to a different repo with different cores is going
to make them less brittle without some very strict guidelines about what a
driver/plugin is allowed to do.

For example, without neutron core reviewer oversight, what prevents a
plugin author from monkey patching parts of the neutron API? If nothing,
that will immediately break on any kind of API refactoring, module
renaming, etc.

That scenario also brings up another concern. Will changes to neutron that
break a vendor plugin even be blocked on the neutron side? If so, the
neutron repo will be held hostage by third-party code that isn't in Neutron
and lacks the quality control it would have in Neutron. If not, this will
immediately break the gate on the driver repo, forcing maintainers to
disable the gating job for that vendor plugin. Neither of these scenarios
seem less brittle to me.

What the PLUMgrid folks did works; however, IIUC it was at the sacrifice of
unit tests verifying any calls into the plumlib. There is just a fake
driver that accepts anything for the unit tests. [1] They could implement a
lot of mock logic to simulate the real driver, but then we are back to
square one and they might as well put the actual driver there.

I'm all for moving drivers/plugins out of Neutron, but we need to be really
careful here because we are going to lose a lot of quality control that
Neutron could end up taking the blame for since these drivers/plugins are
still in a public repo.

1.
https://github.com/openstack/neutron/blob/08529376f16837083c28b009411cc52e0e2a8d33/neutron/plugins/plumgrid/drivers/fake_plumlib.py

On Fri, Aug 15, 2014 at 8:50 AM, Kyle Mestery mest...@mestery.com wrote:

I think the review time alone is a huge issue. Even worse, for the
most part, core reviewers are reviewing code for which they themselves
can't test because it requires proprietary hardware or software,
making the situation brittle at best. Having a separate git repository
which is gated by stringent third-party CI requirements, with separate
(and possibly overlapping) core reviewers would help to alleviate this
problem. Another alternative is to move most intelligence out of the
plugins/drivers and into vendor owned packages which can live on pypi.
This is similar to what the PLUMgrid folks did for their plugin. This
allows vendor control over most of their bits, removes the constant
churn for simple bug fixes in the neutron tree, and adds the benefit
of being a part of the simultaneous release, which is the only thing
most vendors care about.

On Thu, Aug 14, 2014 at 10:34 PM, Kevin Benton blak...@gmail.com wrote:
I also feel like the drivers/plugins are currently BEYOND a tipping
point, and are in fact dragging down velocity of the core project in
many ways.

Can you elaborate on the ways that they are slowing down the velocity? I
know they take up reviewer time, but are there other ways that you think
they slow down the project?

On Thu, Aug 14, 2014 at 6:07 AM, Kyle Mestery mest...@mestery.com
wrote:

I also feel like the drivers/plugins are currently BEYOND a tipping
point, and are in fact dragging down velocity of the core project in
many ways. I'm working on a proposal for Kilo where we move all
drivers/plugins out of the main Neutron tree and into a separate git
repository under the networking program. We have way too many drivers,
requiring way too man review cycles, for this to be a sustainable
model going forward. Since the main reason plugin/driver authors want
their code upstream is to be a part of the simultaneous release, and
thus be packaged by distributions, having a separate repository for
these will satisfy this requirement. I'm still working through the
details around reviews of this repository, etc.

Also, I feel as if the level of passion on the mailing list has died
down a bit, so I thought I'd send something out to try and liven
things up a bit. It's been somewhat non-emotional here for a day or
so. :)

Thanks,
Kyle

On Thu, Aug 14, 2014 at 5:09 AM, Salvatore Orlando sorla...@nicira.com

wrote:
I think there will soon be a discussion regarding what the appropriate
location for plugin and drivers should be.
My personal feeling is that Neutron has simply reached the tipping
point
where the high number of drivers and plugins is causing unnecessary
load
for
the core team and frustration for the community.

There I would totally support Luke's initiative about maintaining an
out-of-tree ML2 driver. On the other hand, a plugin/driver diaspora
might
also have negative consequences such as frequent breakages such as
those
Bob
was mentioning or confusion for users which might need to end up
fetching
drivers from disparate sources.

As mentioned during the last Neutron IRC meeting this is another
process
aspect which will be

[openstack-dev] [Keystone] What to do with Specs after they are approved?

2014-08-15 Thread Adam Young

We are putting a lot of effort into the Specs process.  It seems like a 
pity to let them go to waste after the features they describe are 
implemented.  Ideally, they should be part of the official 
documentation.  Many are too detailed to be a release note.


Should they be part of the patch, and put under keystone[*]/docs? Is 
there a better place for them to live?


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [QA] Picking a Name for the Tempest Library

On 08/15/2014 03:26 PM, Drew Fisher wrote:
 What about 'teapot' (as in the idiom 'tempest in a teapot'[1])
 
 -Drew
 
 [1] http://en.wikipedia.org/wiki/Tempest_in_a_teapot

Though in this case it'd be teacup in tempest, I think?

There's also a TCup project [1] that uses tempest.  So, you have teapot
in tempest in tcup ... and that just gets confusing.  :-)

[1] https://wiki.openstack.org/wiki/RefStack/TCup

-- 
Russell Bryant

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] The future of the integrated release

On Fri, Aug 15, 2014 at 8:17 AM, Sandy Walsh sandy.wa...@rackspace.com
wrote:

  On 8/14/2014 6:42 PM, Doug Hellmann wrote:


  On Aug 14, 2014, at 4:41 PM, Joe Gordon joe.gord...@gmail.com wrote:




 On Wed, Aug 13, 2014 at 12:24 PM, Doug Hellmann d...@doughellmann.com
 wrote:


 On Aug 13, 2014, at 3:05 PM, Eoghan Glynn egl...@redhat.com wrote:

 
  At the end of the day, that's probably going to mean saying No to more
  things. Everytime I turn around everyone wants the TC to say No to
  things, just not to their particular thing. :) Which is human nature.
  But I think if we don't start saying No to more things we're going to
  end up with a pile of mud that no one is happy with.
 
  That we're being so abstract about all of this is frustrating. I get
  that no-one wants to start a flamewar, but can someone be concrete
 about
  what they feel we should say 'no' to but are likely to say 'yes' to?
 
 
  I'll bite, but please note this is a strawman.
 
  No:
  * Accepting any more projects into incubation until we are comfortable
 with
  the state of things again
  * Marconi
  * Ceilometer
 
  Well -1 to that, obviously, from me.
 
  Ceilometer is on track to fully execute on the gap analysis coverage
  plan agreed with the TC at the outset of this cycle, and has an active
  plan in progress to address architectural debt.

  Yes, there seems to be an attitude among several people in the community
 that the Ceilometer team denies that there are issues and refuses to work
 on them. Neither of those things is the case from our perspective.


  Totally agree.



 Can you be more specific about the shortcomings you see in the project
 that aren’t being addressed?



  Once again, this is just a straw man.


  You’re not the first person to propose ceilometer as a project to kick
 out of the release, though, and so I would like to be talking about
 specific reasons rather than vague frustrations.


  I'm just not sure OpenStack has 'blessed' the best solution out there.


 https://wiki.openstack.org/wiki/Ceilometer/Graduation#Why_we_think_we.27re_ready

  

- Successfully passed the challenge of being adopted by 3 related
projects which have agreed to join or use ceilometer:
   - Synaps
   - Healthnmon
   - StackTach
   
 https://wiki.openstack.org/w/index.php?title=StackTachaction=editredlink=1



  Stacktach seems to still be under active development (
 http://git.openstack.org/cgit/stackforge/stacktach/log/), is used by
 rackspace in production and from everything I hear is more mature then
 ceilometer.


  Stacktach is older than ceilometer, but does not do all of the things
 ceilometer does now and aims to do in the future. It has been a while since
 I last looked at it, so the situation may have changed, but some of the
 reasons stacktach would not be a full replacement for ceilometer include:
 it only works with AMQP; it collects notification events, but doesn’t offer
 any metering ability per se (no tracking of values like CPU or bandwidth
 utilization); it only collects notifications from some projects, and
 doesn’t have a way to collect data from swift, which doesn’t emit
 notifications; and it does not integrate with Heat to trigger autoscaling
 alarms.

   Well, that's my cue.

 Yes, StackTach was started before the incubation process was established
 and it solves other problems. Specifically around usage, billing and
 performance monitoring, things I wouldn't use Ceilometer for. But, if
 someone asked me what they should use for metering today, I'd point them
 towards Monasca in a heartbeat. Another non-blessed project.


I think this is the crux of the potential argument against ceilometer
today. There are several other viable competing open source candidates in
this space.  And an argument can be made that having OpenStack bless a
winner before one clearly emerges on its own doesn't help. If blessing a
winner resulted in the teams working together more and less duplicated
efforts that would be one thing, but that does not appear to be happening
in this space.



 It is nice to see that Ceilometer is working to solve their problems, but
 there are other solutions operators should consider until that time comes.
 It would be nice to see the TC endorse those too. Solve the users need
 first.


   We did work with a few of the Stacktach developers on bringing event
 collection into ceilometer, and that work is allowing us to modify the way
 we store the meter data that causes a lot of the performance issues we’ve
 seen. That work is going on now and will be continued into Kilo, when we
 expect to be adding drivers for time-series databases more appropriate for
 that type of data.


 StackTach isn't actively contributing to Ceilometer any more. Square
 peg/round hole. We needed some room to experiment with alternative
 solutions and the rigidity of the process was a hindrance. Not a problem
 with the core team, just a problem with the dev process overall.

 I recently

Re: [openstack-dev] [Keystone] What to do with Specs after they are approved?

2014-08-15 Thread Steve Martinelli

The are still alive - http://specs.openstack.org/openstack/keystone-specs/



Regards,

Steve Martinelli
Software Developer - OpenStack
Keystone Core Member





Phone:
1-905-413-2851
E-mail: steve...@ca.ibm.com

8200 Warden Ave
Markham, ON L6G 1C7
Canada




From:   
Adam Young ayo...@redhat.com
To:   
OpenStack Development
Mailing List openstack-dev@lists.openstack.org, 
Date:   
08/15/2014 03:50 PM
Subject:  
 [openstack-dev]
[Keystone] What to do with Specs after they areapproved?




We are putting a lot of effort into the Specs process.
It seems like a 
pity to let them go to waste after the features they describe are 
implemented. Ideally, they should be part of the official 
documentation. Many are too detailed to be a release note.

Should they be part of the patch, and put under keystone[*]/docs? Is 
there a better place for them to live?

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev





___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] fix poor tarball support in source-repositories

2014-08-15 Thread Clint Byrum

Excerpts from Brownell, Jonathan C (Corvallis)'s message of 2014-08-15 08:11:18 
-0700:
 The current DIB element support for downloading tarballs via 
 source-repository allows an entry in the following form:
 
 name tar targetdir url
 
 Today, this feature is currently used only by the mysql DIB element. You can 
 see how it's used here:
 https://github.com/openstack/tripleo-image-elements/blob/master/elements/mysql/source-repository-mysql
 
 However, the underlying diskimage-builder implementation of tarball handling 
 is rather odd and inflexible. After downloading the file (or retrieving from 
 cache) and unpacking into a tmp directory, it performs:
 
 mv $tmp/*/* $targetdir
 
 This does work as long as the tarball follows a structure where all its 
 files/directories are contained within a single directory, but it fails if 
 the tarball contains no subdirectories. (Even worse is when it contains some 
 files and some subdirectories, in which case the files are lost and the 
 contents of all subdirs get lumped together in the output folder.)
 
 Since this tarball support is only used today by the mysql DIB element, I 
 would love to fix this in both diskimage-builder and tripleo-image-element by 
 changing to simply:
 
 mv $tmp/* $targetdir
 
 And then manually tweaking the directory structure of $targetdir from a new 
 install.d script in the mysql element to restore the desired layout.
 
 However, it's important to note that this will break backwards compatibility 
 if tarball support is used in its current fashion by users with private DIB 
 elements.
 
 Personally, I consider the current behavior so egregious that it really needs 
 to be fixed across the board rather than preserving backwards compatibility.
 
 Do others agree? If not, do you have suggestions as to how to improve this 
 mechanism cleanly without sacrificing backwards compatibility?
 

How about we make a glob to use, so like this:

mysql tar /usr/local/mysql http://someplace/mysql.tar.gz mysql-5.*

That would result in

mv $tmp/mysql-5.*/* $targetdir

And then we would warn that assuming the glob will be '*' is deprecated,
to be changed in a later release.

Users who want your proposed behavior would use . until the default
changes. That would result in

mv $tmp/./* $targetdir

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] What to do with Specs after they are approved?

2014-08-15 Thread Anne Gentle

On Fri, Aug 15, 2014 at 2:47 PM, Adam Young ayo...@redhat.com wrote:

 We are putting a lot of effort into the Specs process.  It seems like a
 pity to let them go to waste after the features they describe are
 implemented.  Ideally, they should be part of the official documentation.
 Many are too detailed to be a release note.

 Should they be part of the patch, and put under keystone[*]/docs? Is there
 a better place for them to live?


They are very purposefully not docs - they are ideas that may or may not be
fully implemented any given release. That's why we stood up
specs.openstack.org.


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] The future of the integrated release

On Thu, Aug 14, 2014 at 4:02 PM, Eoghan Glynn egl...@redhat.com wrote:


   Additional cross-project resources can be ponied up by the large
   contributor companies, and existing cross-project resources are not
   necessarily divertable on command.
 
  Sure additional cross-project resources can and need to be ponied up,
 but I
  am doubtful that will be enough.

 OK, so what exactly do you suspect wouldn't be enough, for what
 exactly?


I am not sure what would be enough to get OpenStack back in a position
where more developers/users are happier with the current state of affairs.
Which is why I think we may want to try several things.



 Is it the likely number of such new resources, or the level of domain-
 expertise that they can be realistically be expected bring to the
 table, or the period of time to on-board them, or something else?


Yes, all of the above.


 And which cross-project concern do you think is most strained by the
 current set of projects in the integrated release? Is it:

  * QA
  * infra
  * release management
  * oslo
  * documentation
  * stable-maint

 or something else?


Good question.

IMHO QA, Infra and release management are probably the most strained. But I
also think there is something missing from this list. Many of the projects
are hitting similar issues and end up solving them in different ways, which
just leads to more confusion for the end user. Today we have a decent model
for rolling out cross-project libraries (Oslo) but we don't have a good way
of having broader cross project discussions such as: API standards (such as
discoverability of features), logging standards, aligning on concepts
(different projects have different terms and concepts for scaling and
isolating failure domains), and an overall better user experience. So I
think we have a whole class of cross project issues that we have not even
begun addressing.



 Each of those teams has quite different prerequisite skill-sets, and
 the on-ramp for someone jumping in seeking to make a positive impact
 will vary from team to team.

 Different approaches have been tried on different teams, ranging from
 dedicated project-liaisons (Oslo) to shared cores (Sahara/Infra) to
 newly assigned dedicated resources (QA/Infra). Which of these models
 might work in your opinion? Which are doomed to failure, and why?

 So can you be more specific here on why you think adding more cross-
 project resources won't be enough to address an identified shortage
 of cross-project resources, while de-integrating projects would be?

 And, please, can we put the proverbial strawman back in its box on
 this thread? It's all well and good as a polemic device, but doesn't
 really move the discussion forward in a constructive way, IMO.

 Thanks,
 Eoghan

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] gettext question about oslo.i18n library

2014-08-15 Thread Doug Hellmann

On Aug 15, 2014, at 3:18 AM, Peng Wu peng.e...@gmail.com wrote:

 Hi,
 
  Recently I just read the code of oslo.i18n library,
  The lazy translation idea is great!
 
  But I found a question about gettext contextual markers
  and plural form, such as pgettext and ungettext functions,
  see [3].
 
  It seems the two gettext functions are missing in the oslo.i18n
 library.
  Is it correct? or will support it?
 
 Thanks,
  Peng Wu

You’re right, those are not present.

We apparently haven’t used them anywhere, yet, because they weren’t exposed via 
the old gettextutils module in the incubator. We should add them. Are you 
interested in working on a blueprint for Kilo to do that?

Doug

 
 Refer URL:
 1. https://github.com/openstack/oslo.i18n
 2.
 http://lists.openstack.org/pipermail/openstack-dev/2014-July/039217.html
 3. https://wiki.openstack.org/wiki/I18n/TranslatableStrings
 
 
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] The future of the integrated release

On Thu, Aug 14, 2014 at 4:02 PM, Eoghan Glynn egl...@redhat.com wrote:


   Additional cross-project resources can be ponied up by the large
   contributor companies, and existing cross-project resources are not
   necessarily divertable on command.
 
  Sure additional cross-project resources can and need to be ponied up,
 but I
  am doubtful that will be enough.

 OK, so what exactly do you suspect wouldn't be enough, for what
 exactly?

 Is it the likely number of such new resources, or the level of domain-
 expertise that they can be realistically be expected bring to the
 table, or the period of time to on-board them, or something else?

 And which cross-project concern do you think is most strained by the
 current set of projects in the integrated release? Is it:

  * QA
  * infra
  * release management
  * oslo
  * documentation
  * stable-maint

 or something else?

 Each of those teams has quite different prerequisite skill-sets, and
 the on-ramp for someone jumping in seeking to make a positive impact
 will vary from team to team.

 Different approaches have been tried on different teams, ranging from
 dedicated project-liaisons (Oslo) to shared cores (Sahara/Infra) to
 newly assigned dedicated resources (QA/Infra). Which of these models
 might work in your opinion? Which are doomed to failure, and why?

 So can you be more specific here on why you think adding more cross-
 project resources won't be enough to address an identified shortage
 of cross-project resources, while de-integrating projects would be?

 And, please, can we put the proverbial strawman back in its box on
 this thread? It's all well and good as a polemic device, but doesn't
 really move the discussion forward in a constructive way, IMO.



/me puts his strawman back in the box



 Thanks,
 Eoghan

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][all] switch from mysqldb to another eventlet aware mysql client

2014-08-15 Thread Doug Hellmann


On Aug 15, 2014, at 9:29 AM, Ihar Hrachyshka ihrac...@redhat.com wrote:

 Signed PGP part
 Some updates on the matter:
 
 - oslo-spec was approved with narrowed scope which is now 'enabled
 mysqlconnector as an alternative in gate' instead of 'switch the
 default db driver to mysqlconnector'. We'll revisit the switch part
 the next cycle once we have the new driver running in gate and real
 benchmarking is heavy-lifted.
 
 - there are several patches that are needed to make devstack and
 tempest passing deployment and testing. Those are collected under the
 hood of: https://review.openstack.org/#/c/114207/ Not much of them.
 
 - we'll need a new oslo.db release to bump versions (this is needed to
 set raise_on_warnings=False for the new driver, which was incorrectly
 set to True in sqlalchemy till very recently). This is expected to be
 released this month (as per Roman Podoliaka).

This release is currently blocked on landing some changes in projects using the 
library so they don’t break when the new version starts using different 
exception classes. We’re tracking that work in 
https://etherpad.openstack.org/p/sqla_exceptions_caught

It looks like we’re down to 2 patches, one for cinder 
(https://review.openstack.org/#/c/111760/) and one for glance 
(https://review.openstack.org/#/c/109655). Roman, can you verify that those are 
the only two projects that need changes for the exception issue?

 
 - once the corresponding patch for sqlalchemy-migrate is merged, we'll
 also need a new version released for this.
 
 - on PyPI side, no news for now. The last time I've heard from Geert
 (the maintainer of MySQL Connector for Python), he was working on
 this. I suspect there are some legal considerations running inside
 Oracle. I'll update once I know more about that.

If we don’t have the new package on PyPI, how do we plan to include it in the 
gate? Are there options to allow an exception, or to make the mirroring 
software download it anyway?

Doug

 
 - once all the relevant patches land in affected projects and
 devstack, I'm going to introduce a separate gate job to run against
 mysqlconnector.
 
 Cheers,
 /Ihar
 
 On 22/07/14 15:03, Ihar Hrachyshka wrote:
  FYI: I've moved the spec to oslo space since the switch is not
  really limited to neutron, and most of coding is to be done in
  oslo.db (though not much anyway).
 
  New spec: https://review.openstack.org/#/c/108355/
 
  On 09/07/14 13:17, Ihar Hrachyshka wrote:
  Hi all,
 
  Multiple projects are suffering from db lock timeouts due to
  deadlocks deep in mysqldb library that we use to interact with
  mysql servers. In essence, the problem is due to missing
  eventlet support in mysqldb module, meaning when a db lock is
  encountered, the library does not yield to the next green thread,
  allowing other threads to eventually unlock the grabbed lock, and
  instead it just blocks the main thread, that eventually raises
  timeout exception (OperationalError).
 
  The failed operation is not retried, leaving failing request not
   served. In Nova, there is a special retry mechanism for
  deadlocks, though I think it's more a hack than a proper fix.
 
  Neutron is one of the projects that suffer from those timeout
  errors a lot. Partly it's due to lack of discipline in how we do
  nested calls in l3_db and ml2_plugin code, but that's not
  something to change in foreseeable future, so we need to find
  another solution that is applicable for Juno. Ideally, the
  solution should be applicable for Icehouse too to allow
  distributors to resolve existing deadlocks without waiting for
  Juno.
 
  We've had several discussions and attempts to introduce a
  solution to the problem. Thanks to oslo.db guys, we now have more
  or less clear view on the cause of the failures and how to easily
  fix them. The solution is to switch mysqldb to something eventlet
  aware. The best candidate is probably MySQL Connector module that
  is an official MySQL client for Python and that shows some
  (preliminary) good results in terms of performance.
 
  I've posted a Neutron spec for the switch to the new client in
  Juno at [1]. Ideally, switch is just a matter of several fixes
  to oslo.db that would enable full support for the new driver
  already supported by SQLAlchemy, plus 'connection' string
  modified in service configuration files, plus documentation
  updates to refer to the new official way to configure services
  for MySQL. The database code won't, ideally, require any major
  changes, though some adaptation for the new client library may be
  needed. That said, Neutron does not seem to require any changes,
  though it was revealed that there are some alembic migration
  rules in Keystone or Glance that need (trivial) modifications.
 
  You can see how trivial the switch can be achieved for a service
  based on example for Neutron [2].
 
  While this is a Neutron specific proposal, there is an obvious
  wish to switch to the new library globally throughout

Re: [openstack-dev] [Keystone] What to do with Specs after they are approved?

2014-08-15 Thread Dolph Mathews

Regarding the connection to release notes, it'd be useful to link from
official release note bullet points back to the corresponding spec on
specs.openstack.org to provide additional detail.


On Fri, Aug 15, 2014 at 2:58 PM, Anne Gentle a...@openstack.org wrote:




 On Fri, Aug 15, 2014 at 2:47 PM, Adam Young ayo...@redhat.com wrote:

 We are putting a lot of effort into the Specs process.  It seems like a
 pity to let them go to waste after the features they describe are
 implemented.  Ideally, they should be part of the official documentation.
 Many are too detailed to be a release note.

 Should they be part of the patch, and put under keystone[*]/docs? Is
 there a better place for them to live?


 They are very purposefully not docs - they are ideas that may or may not
 be fully implemented any given release. That's why we stood up
 specs.openstack.org.


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo][db] Nominating Mike Bayer for the oslo.db core reviewers team

2014-08-15 Thread Doug Hellmann


On Aug 15, 2014, at 10:00 AM, Ben Nemec openst...@nemebean.com wrote:

 On 08/15/2014 08:20 AM, Russell Bryant wrote:
 On 08/15/2014 09:13 AM, Jay Pipes wrote:
 On 08/15/2014 04:21 AM, Roman Podoliaka wrote:
 Hi Oslo team,
 
 I propose that we add Mike Bayer (zzzeek) to the oslo.db core
 reviewers team.
 
 Mike is an author of SQLAlchemy, Alembic, Mako Templates and some
 other stuff we use in OpenStack. Mike has been working on OpenStack
 for a few months contributing a lot of good patches and code reviews
 to oslo.db [1]. He has also been revising the db patterns in our
 projects and prepared a plan how to solve some of the problems we have
 [2].
 
 I think, Mike would be a good addition to the team.
 
 Uhm, yeah... +10 :)
 
 ^2 :-)
 
 
 What took us so long to do this? :-)
 
 +1 obviously.

I did think it would be a good idea to wait a *little* while and make sure we 
weren’t going to scare him off. ;-)

Seriously, Mike has been doing a great job collaborating with the existing team 
and helping us make oslo.db sane.

+1

Doug


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Keystone] What to do with Specs after they are approved?

2014-08-15 Thread Anne Gentle

On Fri, Aug 15, 2014 at 3:11 PM, Dolph Mathews dolph.math...@gmail.com
wrote:

 Regarding the connection to release notes, it'd be useful to link from
 official release note bullet points back to the corresponding spec on
 specs.openstack.org to provide additional detail.


I disagree, the people reading release notes are not the audience for
specs. Please point to deployer docs or user docs.



 On Fri, Aug 15, 2014 at 2:58 PM, Anne Gentle a...@openstack.org wrote:




 On Fri, Aug 15, 2014 at 2:47 PM, Adam Young ayo...@redhat.com wrote:

 We are putting a lot of effort into the Specs process.  It seems like a
 pity to let them go to waste after the features they describe are
 implemented.  Ideally, they should be part of the official documentation.
 Many are too detailed to be a release note.

 Should they be part of the patch, and put under keystone[*]/docs? Is
 there a better place for them to live?


 They are very purposefully not docs - they are ideas that may or may not
 be fully implemented any given release. That's why we stood up
 specs.openstack.org.


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo][db] Nominating Mike Bayer for the oslo.db core reviewers team

2014-08-15 Thread Morgan Fainberg

-Original Message-
From: Doug Hellmann d...@doughellmann.com
Reply: OpenStack Development Mailing List (not for usage questions) 
openstack-dev@lists.openstack.org
Date: August 15, 2014 at 13:29:15
To: Ben Nemec openst...@nemebean.com, OpenStack Development Mailing List 
(not for usage questions) openstack-dev@lists.openstack.org
Subject:  Re: [openstack-dev] [oslo][db] Nominating Mike Bayer for the oslo.db 
core reviewers team

 On Aug 15, 2014, at 10:00 AM, Ben Nemec wrote:

  On 08/15/2014 08:20 AM, Russell Bryant wrote:
  On 08/15/2014 09:13 AM, Jay Pipes wrote:
  On 08/15/2014 04:21 AM, Roman Podoliaka wrote:
  Hi Oslo team,

  I propose that we add Mike Bayer (zzzeek) to the oslo.db core
  reviewers team.

  Mike is an author of SQLAlchemy, Alembic, Mako Templates and some
  other stuff we use in OpenStack. Mike has been working on OpenStack
  for a few months contributing a lot of good patches and code reviews
  to oslo.db [1]. He has also been revising the db patterns in our
  projects and prepared a plan how to solve some of the problems we have
  [2].

  I think, Mike would be a good addition to the team.

  Uhm, yeah... +10 :)

  ^2 :-)

  What took us so long to do this? :-)

  +1 obviously.

 I did think it would be a good idea to wait a *little* while and make sure we 
 weren’t going  
 to scare him off. ;-)

 Seriously, Mike has been doing a great job collaborating with the existing 
 team and helping  
 us make oslo.db sane.

 +1

 Doug

Big +1 from me. Mike has been great across the board (I know I’m not oslo core)

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Octavia] Object Model and DB Structure

2014-08-15 Thread Eichberger, German

--Basically no shareable entities.
+1

That will make me insanely happy :-)

Regarding Listeners: I was assuming that a LoadBalancer would map to an haproxy 
instance - and a listener would be part of that haproxy. But I heard Stephen 
say that this so not so clear cut. So maybe listeners map to haproxy 
instances...

German

-Original Message-
From: Brandon Logan [mailto:brandon.lo...@rackspace.com] 
Sent: Thursday, August 14, 2014 10:17 PM
To: openstack-dev@lists.openstack.org
Subject: [openstack-dev] [Octavia] Object Model and DB Structure

So I've been assuming that the Octavia object model would be an exact copy of 
the neutron lbaas one with additional information for Octavia.
However, after thinking about it I'm not sure this is the right way to go 
because the object model in neutron lbaas may change in the future, and Octavia 
can't just change it's object model when neutron lbaas/openstack lbaas changes 
it's object model.  So if there are any lessons learned we would like to apply 
to Octavia's object model now is the time.

Entity name changes are also on the table if people don't really like some of 
the names.  Even adding new entities or removing entities if there are good 
reasons isn't out of the question.

Anyway here are a few of my suggestions.  Please add on to this if you want.  
Also, just flat out tell me I'm wrong on some of htese suggestions if you feel 
as such.

A few improvements I'd suggest (using the current entity names):
-A real root object that is the only top level object (loadbalancer).
--This would be 1:M relationship with Listeners, but Listeners would only be 
children of loadbalancers.
--Pools, Members, and Health Monitors would follow the same workflow.
--Basically no shareable entities.

-Provisioning status only on the root object (loadbalancer).
--PENDING_CREATE, PENDING_UPDATE, PENDING_DELETE, ACTIVE (No need for a 
DEFEERRED status! YAY!) --Also maybe a DELETED status.

-Operating status on other entities
--ACTIVE or ONLINE, DEGRADED, INACTIVE or OFFLINE --Pools and Members 
--Listeners have been mentioned but I'd like to hear more details on that.

-Adding status_description field in, or something similar.  Would only eixst on 
loadbalancer entity if loadbalancer is the only top level object.

Thanks,
Brandon
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron][IPv6]

2014-08-15 Thread Randy Tuttle

Hi Harm

It’s highly unlikely that it would work with a current version of Neutron 
(Icehouse was where the original effort went). I’ve just run out of cycles to 
work on this. I know Sean Collins continues to drive this effort, and the 
roadmap may have this BP (or a similar one) targeted for Kilo. Check with Sean.

Randy

On Aug 15, 2014, at 3:25 PM, Harm Weites h...@weites.com wrote:

 Hi Randy,
 
 I was reading throught your blueprint [1] and figured it was exactly
 what I'm after to get dualstack v4/v6 in my cloud. Sadly, the review has
 it's current state set to abandoned.
 
 Are there any plans to get back to it and get it into trunk?
 
 Furthermore, does it still apply (and work) on a recent checkout of
 Neutron, or should I just go ahead and give it a go?
 
 [1]
 https://blueprints.launchpad.net/neutron/+spec/allow-multiple-subnets-on-gateway-port
 [2] https://review.openstack.org/#/c/77471/
 
 Thanks,
 Harm


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [QA] Picking a Name for the Tempest Library

2014-08-15 Thread Valeriy Ponomaryov

OpenTest is a library for integrational and functional testing of
OpenStack cloud technology.

Sounds?


On Fri, Aug 15, 2014 at 10:48 PM, Russell Bryant rbry...@redhat.com wrote:

 On 08/15/2014 03:26 PM, Drew Fisher wrote:
  What about 'teapot' (as in the idiom 'tempest in a teapot'[1])
 
  -Drew
 
  [1] http://en.wikipedia.org/wiki/Tempest_in_a_teapot

 Though in this case it'd be teacup in tempest, I think?

 There's also a TCup project [1] that uses tempest.  So, you have teapot
 in tempest in tcup ... and that just gets confusing.  :-)

 [1] https://wiki.openstack.org/wiki/RefStack/TCup

 --
 Russell Bryant

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Kind Regards
Valeriy Ponomaryov
QA Engineer,
Mirantis, Kharkiv, Ukraine
www.mirantis.com
vponomar...@mirantis.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [QA] Picking a Name for the Tempest Library

2014-08-15 Thread Jay Pipes


On 08/15/2014 03:14 PM, Matthew Treinish wrote:

Hi Everyone,

So as part of splitting out common functionality from tempest into a library [1]
we need to create a new repository. Which means we have the fun task of coming
up with something to name it. I'm personally thought we should call it:

  - mesocyclone

Which has the advantage of being a cloud/weather thing, and the name sort of
fits because it's a precursor to a tornado. Also, it's an available namespace on
both launchpad and pypi. But there has been expressed concern that both it is a
bit on the long side (which might have 80 char line length implications) and
it's unclear from the name what it does.

During the last QA meeting some alternatives were also brought up:

  - tempest-lib / lib-tempest
  - tsepmet
  - blackstorm
  - calm
  - tempit
  - integration-test-lib

(although I'm not entirely sure I remember which ones were serious suggestions
or just jokes)

So as a first step I figured that I'd bring it up on the ML to see if anyone had
any other suggestions. (or maybe get a consensus around one choice) I'll take
the list, check if the namespaces are available, and make a survey so that
everyone can vote and hopefully we'll have a clear choice for a name from that.


I suggest that tempest should be the name of the import'able library, 
and that the integration tests themselves should be what is pulled out 
of the current Tempest repository, into their own repo called 
openstack-integration-tests or os-integration-tests.


Best,
-jay


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [QA] Picking a Name for the Tempest Library



 On Aug 15, 2014, at 5:39 PM, Jay Pipes jaypi...@gmail.com wrote:
 
 On 08/15/2014 03:14 PM, Matthew Treinish wrote:
 Hi Everyone,
 
 So as part of splitting out common functionality from tempest into a library 
 [1]
 we need to create a new repository. Which means we have the fun task of 
 coming
 up with something to name it. I'm personally thought we should call it:
 
  - mesocyclone
 
 Which has the advantage of being a cloud/weather thing, and the name sort of
 fits because it's a precursor to a tornado. Also, it's an available 
 namespace on
 both launchpad and pypi. But there has been expressed concern that both it 
 is a
 bit on the long side (which might have 80 char line length implications) and
 it's unclear from the name what it does.
 
 During the last QA meeting some alternatives were also brought up:
 
  - tempest-lib / lib-tempest
  - tsepmet
  - blackstorm
  - calm
  - tempit
  - integration-test-lib
 
 (although I'm not entirely sure I remember which ones were serious 
 suggestions
 or just jokes)
 
 So as a first step I figured that I'd bring it up on the ML to see if anyone 
 had
 any other suggestions. (or maybe get a consensus around one choice) I'll take
 the list, check if the namespaces are available, and make a survey so that
 everyone can vote and hopefully we'll have a clear choice for a name from 
 that.
 
 I suggest that tempest should be the name of the import'able library, and 
 that the integration tests themselves should be what is pulled out of the 
 current Tempest repository, into their own repo called 
 openstack-integration-tests or os-integration-tests.

Ooh, I like that idea!  +1

--
Russell Bryant
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [QA] Picking a Name for the Tempest Library

2014-08-15 Thread Dean Troyer

On Fri, Aug 15, 2014 at 4:31 PM, Jay Pipes jaypi...@gmail.com wrote:

 current Tempest repository, into their own repo called
 openstack-integration-tests or os-integration-tests.


I see what you did there...

+1 anyway

dt

Dean Troyer
dtro...@gmail.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Review priorities as we approach juno-3

2014-08-15 Thread Michael Still

On Fri, Aug 15, 2014 at 7:28 PM, Daniel P. Berrange berra...@redhat.com wrote:
 On Fri, Aug 15, 2014 at 06:53:41AM +1000, Michael Still wrote:
 On Fri, Aug 15, 2014 at 6:37 AM, Dan Smith d...@danplanet.com wrote:
  == Move Virt Drivers to use Objects (Juno Work) ==
 
  I couldn't actually find any code out for review for this one apart
  from https://review.openstack.org/#/c/94477/, is there more out there?
 
  This was an umbrella one to cover a bunch of virt driver objects work
  done early in the cycle. Much of that is done, I haven't gone looking
  for anything to see if there are any obvious things to include under
  this anymore, but I'll try to do that.

 Thanks, I'd appreciate that. If its all done, we should mark it implemented.

  == Add a virt driver for Ironic ==
 
  This one is in progress, but we need to keep going at it or we wont
  get it merged in time.
 
  * https://review.openstack.org/#/c/111223/ was approved, but a rebased
  ate it. Should be quick to re-approve.
  * https://review.openstack.org/#/c/111423/
  * https://review.openstack.org/#/c/111425/
  * ...there are more reviews in this series, but I'd be super happy to
  see even a few reviewed
 
  I've been reviewing this pretty heavy and I think that it's just taking
  a while to make changes given the roundabout way they're getting done
  first in Ironic. I'm pretty confident that this one will be okay.

 Yep, I appreciate your focus on this one -- as I am sure the ironic
 people do too. If another core was available to pair up with you on
 these we might be able to get them to land faster. I was doing that
 for a while, but I haven't had time in the last week or so.

 I've reviewed some parts of Ironic, but I see we've not broken it
 up into a number of parts, which I personally think is counter
 productive because we'll have more pain in waiting in the gate,
 but oh well. I'll continue looking at it.

I'm not sure I follow. The driver is broken up into a series of
reviews in a series. The ones yet to merge are:

https://review.openstack.org/#/c/111223/
https://review.openstack.org/#/c/111423/
https://review.openstack.org/#/c/111425/
https://review.openstack.org/#/c/111428/
https://review.openstack.org/#/c/111429/

Michael

-- 
Rackspace Australia

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Octavia] Object Model and DB Structure

2014-08-15 Thread Brandon Logan

Yeah, need details on that.  Maybe he's talking about having haproxy
listen on many ips and ports, each one being a separate front end
section and in the haproxy config with each mapped to its own
default_backend.

Even if that is the case, the load balancer + listener woudl still make
up one of those frontends so the mapping would still be correct.
Though, maybe a different structure would make more sense if that is the
case.

Also, I've created a WIP review of the initial database structure:
https://review.openstack.org/#/c/114671/

Added my own comments so everyone please look at that.  Stephen, if you
could comment on what German mentioned that'd be great.

Have a good weekend!

-Brandon

On Fri, 2014-08-15 at 20:34 +, Eichberger, German wrote:
 --Basically no shareable entities.
 +1
 
 That will make me insanely happy :-)
 
 Regarding Listeners: I was assuming that a LoadBalancer would map to an 
 haproxy instance - and a listener would be part of that haproxy. But I heard 
 Stephen say that this so not so clear cut. So maybe listeners map to haproxy 
 instances...
 
 German
 
 -Original Message-
 From: Brandon Logan [mailto:brandon.lo...@rackspace.com] 
 Sent: Thursday, August 14, 2014 10:17 PM
 To: openstack-dev@lists.openstack.org
 Subject: [openstack-dev] [Octavia] Object Model and DB Structure
 
 So I've been assuming that the Octavia object model would be an exact copy of 
 the neutron lbaas one with additional information for Octavia.
 However, after thinking about it I'm not sure this is the right way to go 
 because the object model in neutron lbaas may change in the future, and 
 Octavia can't just change it's object model when neutron lbaas/openstack 
 lbaas changes it's object model.  So if there are any lessons learned we 
 would like to apply to Octavia's object model now is the time.
 
 Entity name changes are also on the table if people don't really like some of 
 the names.  Even adding new entities or removing entities if there are good 
 reasons isn't out of the question.
 
 Anyway here are a few of my suggestions.  Please add on to this if you want.  
 Also, just flat out tell me I'm wrong on some of htese suggestions if you 
 feel as such.
 
 A few improvements I'd suggest (using the current entity names):
 -A real root object that is the only top level object (loadbalancer).
 --This would be 1:M relationship with Listeners, but Listeners would only be 
 children of loadbalancers.
 --Pools, Members, and Health Monitors would follow the same workflow.
 --Basically no shareable entities.
 
 -Provisioning status only on the root object (loadbalancer).
 --PENDING_CREATE, PENDING_UPDATE, PENDING_DELETE, ACTIVE (No need for a 
 DEFEERRED status! YAY!) --Also maybe a DELETED status.
 
 -Operating status on other entities
 --ACTIVE or ONLINE, DEGRADED, INACTIVE or OFFLINE --Pools and Members 
 --Listeners have been mentioned but I'd like to hear more details on that.
 
 -Adding status_description field in, or something similar.  Would only eixst 
 on loadbalancer entity if loadbalancer is the only top level object.
 
 Thanks,
 Brandon
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Enabling silent Docker tests for Nova?

2014-08-15 Thread Michael Still

Agreed. I think this should be in unless infra vetos it for load reasons.

Michael

On Sat, Aug 16, 2014 at 5:11 AM, Dan Smith d...@danplanet.com wrote:
 Feature freeze is only a few weeks away (Sept 4).  How about we just
 leave it in experimental until after that big push?  That seems pretty
 reasonable.

 Joe just proposed dropping a bunch of semi-useless largeops runs. Maybe
 that leaves room to sneak this in? If the infra team is okay with it,
 I'd say we should go ahead and do it, but regardless of that decision,
 I'm +1 on getting this promoted so we can start the stability timer.

 --Dan


 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




-- 
Rackspace Australia

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron][QA] Enabling full neutron Job

2014-08-15 Thread Salvatore Orlando

The neutron full job is finally voting, and the first patch [1] has already
passed it in gate checks!
I've collected a few data points before it was switched to voting, and we
should probably expect a failure rate around 4%. This is not bad, but
neither great, and everybody's contribution will be appreciated in
reporting and assessing the nature gate failures, which, needless to say,
are mostly races.

Note: we've also added the postgresql version of the same job, but that is
not voting yet as we never executed it before.

Salvatore

[1] https://review.openstack.org/#/c/105694/


On 12 August 2014 20:14, Salvatore Orlando sorla...@nicira.com wrote:

 And just when the patch was only missing a +A, another bug slipped in!
 The nova patch to fix it is available at [1]

 And while we're there, it won't be a bad idea to also push the neutron
 full job, as non-voting, into the integrated gate [2]

 Thanks in advance,
 (especially to the nova and infra cores who'll review these patches!)
 Salvatore

 [1] https://review.openstack.org/#/c/113554/
 [2] https://review.openstack.org/#/c/113562/


 On 7 August 2014 17:51, Salvatore Orlando sorla...@nicira.com wrote:

 Thanks Armando,

 The fix for the bug you pointed out was the reason of the failure we've
 been seeing.
 The follow-up patch merged and I've removed the wip status from the patch
 for the full job [1]

 Salvatore

 [1] https://review.openstack.org/#/c/88289/


 On 7 August 2014 16:50, Armando M. arma...@gmail.com wrote:

 Hi Salvatore,

 I did notice the issue and I flagged this bug report:

 https://bugs.launchpad.net/nova/+bug/1352141

 I'll follow up.

 Cheers,
 Armando


 On 7 August 2014 01:34, Salvatore Orlando sorla...@nicira.com wrote:

 I had to put the patch back on WIP because yesterday a bug causing a
 100% failure rate slipped in.
 It should be an easy fix, and I'm already working on it.
 Situations like this, exemplified by [1] are a bit frustrating for all
 the people working on improving neutron quality.
 Now, if you allow me a little rant, as Neutron is receiving a lot of
 attention for all the ongoing discussion regarding this group policy stuff,
 would it be possible for us to receive a bit of attention to ensure both
 the full job and the grenade one are switched to voting before the juno-3
 review crunch.

 We've already had the attention of the QA team, it would probably good
 if we could get the attention of the infra core team to ensure:
 1) the jobs are also deemed by them stable enough to be switched to
 voting
 2) the relevant patches for openstack-infra/config are reviewed

 Regards,
 Salvatore

 [1]
 http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwie3UnbWVzc2FnZSc6IHUnRmxvYXRpbmcgaXAgcG9vbCBub3QgZm91bmQuJywgdSdjb2RlJzogNDAwfVwiIEFORCBidWlsZF9uYW1lOlwiY2hlY2stdGVtcGVzdC1kc3ZtLW5ldXRyb24tZnVsbFwiIEFORCBidWlsZF9icmFuY2g6XCJtYXN0ZXJcIiIsImZpZWxkcyI6W10sIm9mZnNldCI6MCwidGltZWZyYW1lIjoiMTcyODAwIiwiZ3JhcGhtb2RlIjoiY291bnQiLCJ0aW1lIjp7InVzZXJfaW50ZXJ2YWwiOjB9LCJzdGFtcCI6MTQwNzQwMDExMDIwNywibW9kZSI6IiIsImFuYWx5emVfZmllbGQiOiIifQ==


 On 23 July 2014 14:59, Matthew Treinish mtrein...@kortar.org wrote:

 On Wed, Jul 23, 2014 at 02:40:02PM +0200, Salvatore Orlando wrote:
  Here I am again bothering you with the state of the full job for
 Neutron.
 
  The patch for fixing an issue in nova's server external events
 extension
  merged yesterday [1]
  We do not have yet enough data points to make a reliable assessment,
 but of
  out 37 runs since the patch merged, we had only 5 failures, which
 puts
  the failure rate at about 13%
 
  This is ugly compared with the current failure rate of the smoketest
 (3%).
  However, I think it is good enough to start making the full job
 voting at
  least for neutron patches.
  Once we'll be able to bring down failure rate to anything around 5%,
 we can
  then enable the job everywhere.

 I think that sounds like a good plan. I'm also curious how the failure
 rates
 compare to the other non-neutron jobs, that might be a useful
 comparison too
 for deciding when to flip the switch everywhere.

 
  As much as I hate asymmetric gating, I think this is a good
 compromise for
  avoiding developers working on other projects are badly affected by
 the
  higher failure rate in the neutron full job.

 So we discussed this during the project meeting a couple of weeks ago
 [3] and
 there was a general agreement that doing it asymmetrically at first
 would be
 better. Everyone should be wary of the potential harms with doing it
 asymmetrically and I think priority will be given to fixing issues
 that block
 the neutron gate should they arise.

  I will therefore resume work on [2] and remove the WIP status as
 soon as I
  can confirm a failure rate below 15% with more data points.
 

 Thanks for keeping on top of this Salvatore. It'll be good to finally
 be at
 least partially gating with a parallel job.

 -Matt Treinish

 
  [1] https://review.openstack.org/#/c/103865/
  [2]

Re: [openstack-dev] [Ceilometer] Way to get wrapped method's name/class using Pecan secure decorators?

2014-08-15 Thread Ryan Petrello

Eric,   
  

Doug's correct - this looks like a bug in pecan that occurs when you subclass
both rest.RestController and hooks.HookController.  I'm working on a bug fix as
we speak.  In the meantime, have you tried applying hooks at a global
application level?  This approach should still work.

On 08/14/14 04:38 PM, Pendergrass, Eric wrote:
 Sure, Doug.  We want the ability to selectively apply policies to certain
 Ceilometer
 API methods based on user/tenant roles.
 
 For example, we want to restrict the ability to execute Alarm deletes to
 admins and user/tenants who have a special role, say domainadmin.
 
 The policy file might look like this:
 {
 context_is_admin:  [[role:admin]],
 admin_and_matching_project_domain_id:  [[role:domainadmin]],
 admin_or_cloud_admin: [[rule:context_is_admin],
 [rule:admin_and_matching_project_domain_id]],
 telemetry:delete_alarms:  [[rule:admin_or_cloud_admin]]
 }
 
 The current acl.py and _query_to_kwargs access control setup either sets
 project_id scope to None (do everything) or to the project_id in the request
 header 'X-Project-Id'.  This allows for admin or project scope, but nothing
 in
 between.
 
 We tried hooks.  Unfortunately we can't seem to turn the API controllers
 into
 HookControllers just by adding HookController to the Controller class
 definition.  It causes infinite recursion on API startup.  For example, this
 doesn't work because ceilometer-api will not start with it:
 class MetersController(rest.RestController, HookController):
 
 If there was a way to use hooks with the v2. API controllers that might work
 really well.
 
 So we are left using the @secure decorator and deriving the method name from
 the request environ PATH_INFO and REQUEST_METHOD values.  This is how we
 determine the wrapped method within the class (REQUEST_METHOD + PATH_INFO =
 telemetry:delete_alarms with some munging).  We need the method name in
 order to
 selectively apply acces control to certain methods.
 
 Deriving the method this way isn't ideal but it's the only thing we've
 gotten working 
 between hooks, @secure, and regular decorators.
 
 I submitted a WIP BP here: https://review.openstack.org/#/c/112137/3.  It is
 slightly out of date but should give you a beter idea of our goals.
 
 Thanks
 
  Eric,
 
  If you can give us some more information about your end goal, independent
 of the implementation, maybe we can propose an alternate technique to
 achieve the same thing.
 
  Doug
 
  On Aug 12, 2014, at 6:21 PM, Ryan Petrello ryan.petre...@dreamhost.com
 wrote:
 
   Yep, you're right, this doesn't seem to work.  The issue is that
   security is enforced at routing time (while the controller is still
   actually being discovered).  In order to do this sort of thing with
   the `check_permissions`, we'd probably need to add a feature to pecan.
  
   On 08/12/14 06:38 PM, Pendergrass, Eric wrote:
   Sure, here's the decorated method from v2.py:
  
  class MetersController(rest.RestController):
  Works on meters.
  
  @pecan.expose()
  def _lookup(self, meter_name, *remainder):
  return MeterController(meter_name), remainder
  
  @wsme_pecan.wsexpose([Meter], [Query])
  @secure(RBACController.check_permissions)
  def get_all(self, q=None):
  
   and here's the decorator called by the secure tag:
  
  class RBACController(object):
  global _ENFORCER
  if not _ENFORCER:
  _ENFORCER = policy.Enforcer()
  
  
  @classmethod
  def check_permissions(cls):
  # do some stuff
  
   In check_permissions I'd like to know the class and method with the
 @secure tag that caused check_permissions to be invoked.  In this case, that
 would be MetersController.get_all.
  
   Thanks
  
  
   Can you share some code?  What do you mean by, is there a way for the
 decorator code to know it was called by MetersController.get_all
  
   On 08/12/14 04:46 PM, Pendergrass, Eric wrote:
   Thanks Ryan, but for some reason the controller attribute is None:
  
   (Pdb) from pecan.core import state
   (Pdb) state.__dict__
   {'hooks': [ceilometer.api.hooks.ConfigHook object at 0x31894d0,
   ceilometer.api.hooks.DBHook object at 0x3189650,
   ceilometer.api.hooks.PipelineHook object at 0x39871d0,
   ceilometer.api.hooks.TranslationHook object at 0x3aa5510], 'app':
   pecan.core.Pecan object at 0x2e76390, 'request': Request at
   0x3ed7390 GET http://localhost:8777/v2/meters, 'controller': None,
   'response': Response at 0x3ed74d0 200 OK}
  
   -Original Message-
   From: Ryan Petrello [mailto:ryan.petre...@dreamhost.com]
   Sent: Tuesday, August 12, 2014 10:34 AM
   To: OpenStack Development Mailing List (not for usage questions)
   Subject: Re: [openstack-dev] [Ceilometer] Way to get wrapped
 method's name/class using Pecan

Re: [openstack-dev] Enabling silent Docker tests for Nova?

On 2014-08-16 08:11:42 +1000 (+1000), Michael Still wrote:
 Agreed. I think this should be in unless infra vetos it for load
 reasons.

I don't think the Infra Team minds if you run reasonable jobs to
determine whether a candidate driver for a free-software backend is
suitable for inclusion into an official project. Pretty sure that
counts as one of the reasons we maintain that whole rig. ;)
-- 
Jeremy Stanley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [third-party] What tests are required to be run

2014-08-15 Thread Edgar Magana

Team,

I did a quick audit on the Neutron CI. Very sad results. Only few plugins
and drivers are running properly and testing all Neutron commits.
I created a report here:
https://wiki.openstack.org/wiki/Neutron_Plugins_and_Drivers#Existing_Plugin
_and_Drivers


We will discuss the actions to take on the next Neutron IRC meeting. So
please, reach me out to clarify what is the status of your CI.
I had two commits to quickly verify the CI reliability:

https://review.openstack.org/#/c/114393/

https://review.openstack.org/#/c/40296/


I would expect all plugins and drivers passing on the first one and
failing for the second but I got so many surprises.

Neutron code quality and reliability is a top priority, if you ignore this
report that plugin/driver will be candidate to be remove from Neutron tree.

Cheers,

Edgar

P.s. I hate to be the inquisitor hereŠ but someone has to do the dirty job!


On 8/14/14, 8:30 AM, Kyle Mestery mest...@mestery.com wrote:

Folks, I'm not sure if all CI accounts are running sufficient tests.
Per the requirements wiki page here [1], everyone needs to be running
more than just Tempest API tests, which I still see most neutron
third-party CI setups doing. I'd like to ask everyone who operates a
third-party CI account for Neutron to please look at the link below
and make sure you are running appropriate tests. If you have
questions, the weekly third-party meeting [2] is a great place to ask
questions.

Thanks,
Kyle

[1] https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
[2] https://wiki.openstack.org/wiki/Meetings/ThirdParty

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [QA] Picking a Name for the Tempest Library