date:20150622


On 06/22/2015 10:13 PM, Sajeesh Cimson Sasi wrote:




*From:* Adam Young [ayo...@redhat.com]
*Sent:* 23 June 2015 00:01:48
*To:* openstack-dev@lists.openstack.org
*Subject:* Re: [openstack-dev] [nova][keystone] Nova calls to Keystone

On 06/20/2015 02:46 PM, Sajeesh Cimson Sasi wrote:

Hi All,
   I need your advice for the implementation of the following 
blueprint. https://review.openstack.org/#/c/160605 
 .
   All the use cases mentioned in the blueprint have  been 
implemented and the complete code is up for review.

https://review.openstack.org/#/c/149828/
  However, we have an issue on which we need your input. In the nova 
quota api call, keystone calls are made to
  get the parent_id and the child project or sub project list. This 
is required because nova doesn't store any information
  regarding the hierarchy. Hierarchy Information is taken during run 
time,  from keystone. Since the keystone calls are
  made inside the api call, it is not possible to give any dummy or  
fake values while writing the unit tests. If the keystone
  call was made outside the api call, we could have given fake values 
in the test cases. However,  the keystone calls for

   parent_id and child projects are made inside the api call.
  Can anyone suggest an elegant solution to this problem? What is the 
proper way to implement this ?
Did anybody encounter and solve a  similar problem ? Many thanks 
for any suggestions!

 best regards
   sajeesh


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
If you are talking to a live Keystone server, make sure you are using 
valid data.


If you are not talking to a live keystone server in a unit test, use 
RequestsMock or equivalent (varied by project)  to handle the HTTP 
request and response.


A worst case approach is to monkey patch the Keystoneclient.  Please 
don't do that if you can avoid it; better to provide a mock alternative.



Hi Adam,
   Thanks a lot. I am not planning to talk to the live 
keystone server in the unit test.
   I don't think that I need to monkey patch the 
KeystoneClient. In the nova api code, there are two methods 
(get_parent_project and get_immediate_child_list),which uses 
keystoneclient.I can monkey patch those two methods to return fixed 
data according to a fake hierarchy. Am I right ?


Its not great, but not horrible.  It seems to match the scope of what 
you are testing.  However, you might want to consider doing a mock for 
the whole Keystoneclient call, as that really should beo utside of the 
unit test for the Nova code.




  best regards
   sajeesh


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][keystone] Nova calls to Keystone

2015-06-22 Thread Sajeesh Cimson Sasi

From: Adam Young [ayo...@redhat.com]
Sent: 23 June 2015 00:01:48
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [nova][keystone] Nova calls to Keystone

On 06/20/2015 02:46 PM, Sajeesh Cimson Sasi wrote:
Hi All,
   I need your advice for the implementation of the following blueprint. 
https://review.openstack.org/#/c/160605 .
   All the use cases mentioned in the blueprint have  been implemented and the 
complete code is up for review.
  https://review.openstack.org/#/c/149828/
  However, we have an issue on which we need your input. In the nova quota api 
call, keystone calls are made to
  get the parent_id and the child project or sub project list. This is required 
because nova doesn't store any information
  regarding the hierarchy. Hierarchy Information is  taken during run time,  
from keystone. Since the keystone calls are
  made inside the api call, it is not possible to give any dummy or  fake 
values while writing the unit tests. If the keystone
  call was made outside the api call, we could have given fake values in the 
test cases. However,  the keystone calls for
   parent_id and child projects are made inside the api call.
  Can anyone suggest an elegant solution to this problem? What is the proper 
way to implement this ?
Did anybody encounter and solve a  similar  problem ? Many thanks for any 
suggestions!
 best regards
   sajeesh

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

If you are talking to a live Keystone server, make sure you are using valid 
data.

If you are not talking to a live keystone server in a unit test, use 
RequestsMock or equivalent (varied by project)  to handle the HTTP request and 
response.

A worst case approach is to monkey patch the Keystoneclient.  Please don't do 
that if you can avoid it;  better to provide a mock alternative.

Hi Adam,
   Thanks a lot. I am not planning to talk to the live keystone 
server in the unit test.
   I don't think that I need to monkey patch the KeystoneClient. In 
the nova api code, there are two methods (get_parent_project and 
get_immediate_child_list),which uses keystoneclient.I can monkey patch those 
two methods to return fixed data according to a fake hierarchy. Am I right ?
  best regards
   sajeesh
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] M Naming Poll ended - results still need to clear legal


On 06/22/2015 01:30 PM, Clay Gerrard wrote:

http://civs.cs.cornell.edu/cgi-bin/results.pl?id=E_4983776e190c8dbc

how is the top pick not the author of the book of five rings [1]


I say we claim him as the mascot of the release anyway.

"This should be given careful thought and thorough reflection."







-Clay

1. https://en.wikipedia.org/wiki/The_Book_of_Five_Rings



On Mon, Jun 22, 2015 at 7:07 AM, Monty Taylor > wrote:


Hey all!

The M naming poll has concluded. I'd say "and the winner is ..."
except
we still need to get the winning choice(s) vetted for legal
entanglements.

Feel free to go and look at the results, they are publicly available -
but please DON'T start making t-shirts or having parties (ok, have as
many parties as you want) until we've gotten the alls-clear from the
trademark folks. I've sent the list to them already, so they're
working
on it furiously as we speak.

As soon as I get the word back, I will send out another announcement
with the official result.

Thanks!
Monty

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Cross-Project meeting, Tue June 23rd, 21:00 UTC

2015-06-22 Thread Nikhil Manchanda

Dear PTLs, cross-project liaisons and other interested parties:

We have a cross-project meeting tomorrow (June 23rd) at 21:00 UTC,
with the following agenda:

- Team announcements (horizontal, vertical, diagonal)
- Return request-id to caller (use thread local to store request-id)
  - Main use case is to return request-id to the user (tpatil)
- python-glanceclient [1]
- python-cinderclient [2] [3]
- python-neutronclient [4]
- Open discussion

[1]
https://blueprints.launchpad.net/python-glanceclient/+spec/expose-get-x-openstack-request-id
[2] https://blueprints.launchpad.net/python-cinderclient/+spec/return-req-id
[3]
http://eavesdrop.openstack.org/meetings/cinder/2015/cinder.2015-06-17-16.00.log.html
[4]
https://blueprints.launchpad.net/python-neutronclient/+spec/expose-get-x-openstack-request-id

See you all there!

For more details on this meeting, please see:
https://wiki.openstack.org/wiki/Meetings/CrossProjectMeeting
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ironic] Proposal to add a new repository

2015-06-22 Thread Devananda van der Veen

Oh - one more thing. If ahc-tools depends on data gathered by
enovance/hardware, then I'm not sure it makes sense to import one to
openstack/ without the other.

-Deva

On Mon, Jun 22, 2015 at 5:08 PM Devananda van der Veen <
devananda@gmail.com> wrote:

> I'm
> On Mon, Jun 22, 2015 at 8:19 AM John Trowbridge  wrote:
>
>>
>>
>> On 06/22/2015 10:40 AM, Dmitry Tantsur wrote:
>> > On 06/22/2015 04:19 PM, Devananda van der Veen wrote:
>> >> Hi John,
>> >>
>> >> Thanks for the excellent summary! I found it very helpful to get caught
>> >> up. I'd like to make sure I understand the direction ahc is going. A
>> >> couple questions...
>>
>> Thanks for your interest.
>>
>> >
>> > Let me add my $0.5 :)
>> >
>> >>
>> >> I see that ahc is storing its information in swift. That's clever, but
>> >> if Ironic provided a blob store for each node, would that be better?
>>
>> If the blob is large enough, this would be better. Originally we stored
>> the data in the extra column of the Ironic db, but that proved disastrous:
>>
>> https://bugs.launchpad.net/ironic-inspector/+bug/1461252
>>
>
>
>> >>
>> >> We discussed adding a search API to Ironic at the Vancouver summit,
>> >> though no work has been done on that yet, afaik. If ahc is going to
>> grow
>> >> a REST API for searching for nodes based on specific criteria that it
>> >> discovered, could/should we combine these within Ironic's API?
>> >
>> > I think John meant having API to replace scripts, so I guess search
>> > won't help. When we're talking about advanced matching, we're talking
>> > about the following:
>> > 1. We have a ramdisk tool (based on [8]) to get some insane of facts
>> > from withing the ramdisk (say, 1000 of them)
>> > 2. We have an Inspector plugin to put them all in Swift (or Ironic blob
>> > storage as above)
>> > 3. We have config files (aka rules) written in special JSON-alike DSL to
>> > do matching (one of the weak points is that these are files - I'd like
>> > API endpoint to accept these rules instead).
>> > 4. We have a script to run this DSL and get some output (match/not match
>> > + some matched variables - similar to what regexps do).
>> > As I understood it John want the latter to become an API endpoint,
>> > accepting rules (and maybe node UUIDs) and outputting some result.
>> >
>> > Not sure about benchmarking here, but again, it's probably an API
>> > endpoint that accepts some minimal expectations, and puts failed nodes
>> > to maintenance mode, if they fail to comply (again, that's how I
>> > understood it).
>> >
>> > It's not hard to make these API endpoints part of Inspector, but it's
>> > somewhat undesirable to have them optional...
>> >
>> >>
>> >>  From a service coupling perspective, I like the approach that ahc is
>> >> optional, and also that Ironic-inspector is optional, because this
>> keeps
>> >> the simple use-case for Ironic, well, simple! That said, this seems
>> more
>> >> like a configuration setting (should inspector do extra things?) than
>> an
>> >> entirely separate service, and separating them might be unnecessarily
>> >> complicated.
>> >
>> > We keep thinking about it as well. After all, right now it's just a
>> > couple of utilities. There are 2 more concerns that initially made me
>> > pull out this code:
>> > 1. ahc-tools currently depends on the library [8], which I wish would be
>> > developed much more openly
>>
>
> > 2. it's cool that inspector is pluggable, but it has its cost: there's a
>> > poor feedback loop from inspector processing plugins to a user - like
>> > with all highly asynchronous code
>> > 3. it's also not possible (at least for now) to request a set of
>> > processing plugins when starting introspection via inspector.
>> >
>> > We solved the latter 2 problems by moving code to scripts. So now
>> > Inspector only puts some data to Swift, and scripts can do everything
>> else.
>> >
>> > So now we've left with
>> > 1. dependency on "hardware" library
>> > 2. not very stable interface, much less stable than one of Inspector
>> >
>> > We still wonder how to solve these 2 without creating one more
>> > repository. Any ideas are welcome :)
>>
>>
> Oh - good point. There's some neat looking functionality in
> enovance/hardware repository, but yea, until this is not a
> single-vendor-controlled repository, I think you made the right call.
>
> How would folks feel about moving that into openstack/ ?
>
>
>> It is a goal of mine to solve issue 1 incrementally over time. Either by
>> improving the library (both in function and in openness), or by slowly
>> moving the implementation. That does not seem impossible to do within
>> the inspector tree.
>>
>
> Or that :) Either way, I agree with the direction -- moving the hardware
> inspection functionality into a common repository is good. If it makes
> sense to have two repositories (one for inspector, one for a hardware utils
> library) that's just fine with me.
>
>
> However, issue 2 is a fact. We currently have scripts, and we want to
>> ha

Re: [openstack-dev] [Ironic] Proposal to add a new repository

2015-06-22 Thread Devananda van der Veen

I'm
On Mon, Jun 22, 2015 at 8:19 AM John Trowbridge  wrote:

>
>
> On 06/22/2015 10:40 AM, Dmitry Tantsur wrote:
> > On 06/22/2015 04:19 PM, Devananda van der Veen wrote:
> >> Hi John,
> >>
> >> Thanks for the excellent summary! I found it very helpful to get caught
> >> up. I'd like to make sure I understand the direction ahc is going. A
> >> couple questions...
>
> Thanks for your interest.
>
> >
> > Let me add my $0.5 :)
> >
> >>
> >> I see that ahc is storing its information in swift. That's clever, but
> >> if Ironic provided a blob store for each node, would that be better?
>
> If the blob is large enough, this would be better. Originally we stored
> the data in the extra column of the Ironic db, but that proved disastrous:
>
> https://bugs.launchpad.net/ironic-inspector/+bug/1461252
>


> >>
> >> We discussed adding a search API to Ironic at the Vancouver summit,
> >> though no work has been done on that yet, afaik. If ahc is going to grow
> >> a REST API for searching for nodes based on specific criteria that it
> >> discovered, could/should we combine these within Ironic's API?
> >
> > I think John meant having API to replace scripts, so I guess search
> > won't help. When we're talking about advanced matching, we're talking
> > about the following:
> > 1. We have a ramdisk tool (based on [8]) to get some insane of facts
> > from withing the ramdisk (say, 1000 of them)
> > 2. We have an Inspector plugin to put them all in Swift (or Ironic blob
> > storage as above)
> > 3. We have config files (aka rules) written in special JSON-alike DSL to
> > do matching (one of the weak points is that these are files - I'd like
> > API endpoint to accept these rules instead).
> > 4. We have a script to run this DSL and get some output (match/not match
> > + some matched variables - similar to what regexps do).
> > As I understood it John want the latter to become an API endpoint,
> > accepting rules (and maybe node UUIDs) and outputting some result.
> >
> > Not sure about benchmarking here, but again, it's probably an API
> > endpoint that accepts some minimal expectations, and puts failed nodes
> > to maintenance mode, if they fail to comply (again, that's how I
> > understood it).
> >
> > It's not hard to make these API endpoints part of Inspector, but it's
> > somewhat undesirable to have them optional...
> >
> >>
> >>  From a service coupling perspective, I like the approach that ahc is
> >> optional, and also that Ironic-inspector is optional, because this keeps
> >> the simple use-case for Ironic, well, simple! That said, this seems more
> >> like a configuration setting (should inspector do extra things?) than an
> >> entirely separate service, and separating them might be unnecessarily
> >> complicated.
> >
> > We keep thinking about it as well. After all, right now it's just a
> > couple of utilities. There are 2 more concerns that initially made me
> > pull out this code:
> > 1. ahc-tools currently depends on the library [8], which I wish would be
> > developed much more openly
>

> 2. it's cool that inspector is pluggable, but it has its cost: there's a
> > poor feedback loop from inspector processing plugins to a user - like
> > with all highly asynchronous code
> > 3. it's also not possible (at least for now) to request a set of
> > processing plugins when starting introspection via inspector.
> >
> > We solved the latter 2 problems by moving code to scripts. So now
> > Inspector only puts some data to Swift, and scripts can do everything
> else.
> >
> > So now we've left with
> > 1. dependency on "hardware" library
> > 2. not very stable interface, much less stable than one of Inspector
> >
> > We still wonder how to solve these 2 without creating one more
> > repository. Any ideas are welcome :)
>
>
Oh - good point. There's some neat looking functionality in
enovance/hardware repository, but yea, until this is not a
single-vendor-controlled repository, I think you made the right call.

How would folks feel about moving that into openstack/ ?


> It is a goal of mine to solve issue 1 incrementally over time. Either by
> improving the library (both in function and in openness), or by slowly
> moving the implementation. That does not seem impossible to do within
> the inspector tree.
>

Or that :) Either way, I agree with the direction -- moving the hardware
inspection functionality into a common repository is good. If it makes
sense to have two repositories (one for inspector, one for a hardware utils
library) that's just fine with me.


However, issue 2 is a fact. We currently have scripts, and we want to
> have a REST API. I do not see a transition between the two that does not
> involve a large amount of churn.
>

>From a quick read of the code, it looks like ahc-tools merely analyzes data
already gathered by inspector & enovance/hardware, providing some more
advanced search/filtering/error-checking capabilities on the client side.
If that read is correct, then I would like to align that work wi

Re: [openstack-dev] [QA][Tempest] Proposing Jordan Pittier for Tempest Core

2015-06-22 Thread GHANSHYAM MANN

+1 :)

On Tue, Jun 23, 2015 at 5:23 AM, Matthew Treinish 
wrote:

>
>
> Hi Everyone,
>
> I'd like to propose we add Jordan Pittier (jordanP) to the tempest core
> team.
> Jordan has been a steady contributor and reviewer on tempest over the past
> few
> cycles and he's been actively engaged in the Tempest community. Jordan has
> had
> one of the higher review counts on Tempest for the past cycle, and he has
> consistently been providing reviews that show insight into both the project
> internals and it's future direction. I feel that Jordan will make an
> excellent
> addition to the core team.
>
> As per the usual, if the current Tempest core team members would please
> vote +1
> or -1(veto) to the nomination when you get a chance. We'll keep the polls
> open
> for 5 days or until everyone has voted.
>
> Thanks,
>
> Matt Treinish
>
> References:
>
>
> https://review.openstack.org/#/q/reviewer:jordan.pittier%2540scality.com+project:openstack/tempest+OR+project:openstack/tempest-lib,n,z
>
> http://stackalytics.com/?metric=marks&user_id=jordan-pittier
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>


-- 
Thanks & Regards
Ghanshyam Mann
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] can we get ceilometermiddleware to use a config file instead of transport_url?

2015-06-22 Thread gord chung




On 22/06/15 06:39 PM, Chris Dent wrote:

On Mon, 22 Jun 2015, Sean Dague wrote:


Could we deprecate the use of tranport_url in ceilometermiddleware and
move to an actual oslo.config file somewhere instead? That would bring
it in line with the rest of the RPC configuration for services, and
ensure that all connections in a cluster have access to all the same
options.


Replying on the top of the thread as it seems the responses have taken
us on a bit of a journey and I feel like instead of narrowing the goal
and figuring out what to do we've expanded scope and need to fix
everything. We can't fix everything so let's back up a bit and figure
out the original goal.

When you (Sean) and I first talked about this I understood the problem
to be that the use of just a URL was insufficient for covering the
configuration settings of the various messaging solutions and
additionally was not in keeping with convention.

I poked around in the code and discovered (as has been mentioned
elsewhere in the thread) the filter factory configuration settings are
merged into the oslo config.

So: is it appropriate, for now, to switch devstack:lib/swift to write
long form transport configuration in filter:ceilometer of swift
proxy's config (it's already writing plenty of other stuff) and kill
get_transport_url. ceilometermiddleware already calls get_transport
with a cfg.CONF that will be used for figuring out the transport url
if 'url' is None.


what's 'long form transport'?

it's not actually using cfg.CONF. to figure out transport url if not 
present. cfg.CONF passed in has nothing set and is basically just a 
bunch of defaults... the url obviously doesn't have a default so 
ceilometermiddleware will fail if you don't pass in a url.




This doesn't solve every extant issue but it does solve some of them.

The ceilometer middleware, whether the old one that was in the
ceilometer tree or this newer one, has always struggled to exist
correctly in that weird intersection between swift and the rest of the
openstack universe, but exist it does in its not quite correct way.
Whatever we can do to make it exist better, even if its just a little
bit better, is good.


--
gord


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Tracking optional requirements

2015-06-22 Thread Michael Still

bindep looks quite cool, and I think would do what I am after here.
That's especially true if we use its profiles support. I like the idea
that we can hand some form of documentation to distros that verifies
that they've noticed all the dependancies we've added in a given
release cycle.

Cheers,
Michael

On Tue, Jun 23, 2015 at 2:04 AM, Jeremy Stanley  wrote:
> On 2015-06-22 21:05:11 +1000 (+1000), Michael Still wrote:
>> Sure, except where the thing isn't in pip... I just learned tonight
>> that the libosinfo thing in the review at the start of this thread is
>> a c library and some gobject black magic. I think we want it, but it
>> will never work as a pip dependancy. So, I think something human
>> readable is still required?
>
> I'm hoping to solve this via bindep[1], and will propose a
> bindep-format requirements list to nova once I have the package
> caching in place for that on our workers and the current
> experimental job updated to make use of per-project bindep lists.
>
> [1] http://git.openstack.org/cgit/openstack-infra/bindep/
> --
> Jeremy Stanley
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Rackspace Australia

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [heat][tripleo]Recursive validation for easier composability

2015-06-22 Thread Steve Baker

On 23/06/15 06:49, Thomas Spatzier wrote:

From: Jay Dobies 
To: openstack-dev@lists.openstack.org
Date: 22/06/2015 19:22
Subject: Re: [openstack-dev] [heat][tripleo]Recursive validation for
easier composability

On 06/22/2015 12:19 PM, Steven Hardy wrote:

Hi all,

Lately I've been giving some thought to how we might enable easier
composability, and in particular how we can make it easier for folks to
plug in deeply nested optional extra logic, then pass data in via
parameter_defaults to that nested template.

Here's an example of the use-case I'm describing:

https://review.openstack.org/#/c/193143/5/environments/cinder-

netapp-config.yaml

Here, we want to allow someone to easily turn on an optional

configuration

or feature, in this case a netapp backend for cinder.

I think the actual desired goal is bigger than just optional
configuration. I think it revolves more around choosing a nested stack
implementation for a resource type and how to manage custom parameters
for that implementation. We're getting into the territory here of having
a parent stack defining an API that nested stacks can plug into. I'd
like to have some sort of way of deriving that information instead of
having it be completely relegated to outside documentation (but I'm
getting off topic; at the end I mention how I want to do a better write
up of the issues Tuskar has faced and I'll elaborate more there).

FWIW, adding a thought from my TOSCA background where we've been looking at
something similar, namely selecting a nested templates that declares to be
matching an interfaces consumed in a parent template (that's how I
understood Jay's words above). In TOSCA, there is a more type-safe kind of
template nesting, where nested templates do not just bring new resource
types into existence depending on what parameters they expose, but there is
a strict contract on the interface a nested template must fulfil - see [1],
and especially look for substitution_mapping.

Admittedly, this is not exactly the same as Steven's original problem, but
kind of related. And IIRC, some time back there was some discussion around
introduction of some kind of "interface" for HOT templates. So wanted to
bring this in and give it some thought whether something like this would
make sense for Heat.

[1]
http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-YAML/v1.0/csd03/TOSCA-Simple-Profile-YAML-v1.0-csd03.html#_Toc419746122

This sounds like the dormant blueprint interface-types [2]. I still 
think it would be appropriate to support this at the heat layer even 
though Murano ended up keeping their interface implementation at the 
Murano layer.

Interfaces is slightly different to how to set parameters on deeply 
nested stacks though.

[2] https://blueprints.launchpad.net/heat/+spec/interface-types

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] can we get ceilometermiddleware to use a config file instead of transport_url?

2015-06-22 Thread Chris Dent


On Mon, 22 Jun 2015, Sean Dague wrote:


Could we deprecate the use of tranport_url in ceilometermiddleware and
move to an actual oslo.config file somewhere instead? That would bring
it in line with the rest of the RPC configuration for services, and
ensure that all connections in a cluster have access to all the same
options.


Replying on the top of the thread as it seems the responses have taken
us on a bit of a journey and I feel like instead of narrowing the goal
and figuring out what to do we've expanded scope and need to fix
everything. We can't fix everything so let's back up a bit and figure
out the original goal.

When you (Sean) and I first talked about this I understood the problem
to be that the use of just a URL was insufficient for covering the
configuration settings of the various messaging solutions and
additionally was not in keeping with convention.

I poked around in the code and discovered (as has been mentioned
elsewhere in the thread) the filter factory configuration settings are
merged into the oslo config.

So: is it appropriate, for now, to switch devstack:lib/swift to write
long form transport configuration in filter:ceilometer of swift
proxy's config (it's already writing plenty of other stuff) and kill
get_transport_url. ceilometermiddleware already calls get_transport
with a cfg.CONF that will be used for figuring out the transport url
if 'url' is None.

This doesn't solve every extant issue but it does solve some of them.

The ceilometer middleware, whether the old one that was in the
ceilometer tree or this newer one, has always struggled to exist
correctly in that weird intersection between swift and the rest of the
openstack universe, but exist it does in its not quite correct way.
Whatever we can do to make it exist better, even if its just a little
bit better, is good.

--
Chris Dent tw:@anticdent freenode:cdent
https://tank.peermore.com/tanks/cdent

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][qa] Do we turn on voting for the tempest-dsvm-cells job?

2015-06-22 Thread Matt Riedemann




On 6/22/2015 4:38 PM, Matt Riedemann wrote:



On 6/22/2015 4:32 PM, Russell Bryant wrote:

On 06/22/2015 05:23 PM, Matt Riedemann wrote:

The check-tempest-dsvm-cells job has been in nova's check queue since
January as non-voting and has been stable for a couple of weeks now, so
before it's regressed melwitt proposed a change to making it voting and
gating on nova changes [1].

I raised a concern in that change that the tempest-dsvm-cells job is not
in the check queue for tempest or devstack changes, so if a change is
merged in tempest/devstack which breaks the cells job, it will block
nova changes from merging.

mtreinish noted that tempest already has around 30 jobs running against
it right now in the check queue, so he'd prefer that another one isn't
added since the nova API shouldn't be different in the case of cells,
but we know there are quirks.  That can be seen from the massive regex
of excluded tests for the tempest-dvsm-cells job [2].

If we could turn that regex list into tempest configurations, I think
that would make it possible to not have to run tempest changes through
the cells job in the check queue but also feel reasonably confident that
changes to tempest that use the config options properly won't break the
cells job (and block nova in the gate).

This is actually something we should do regardless of voting or not on
nova since new tests get added which might not fall in the regex and
break the cells job.  So I'm going to propose some changes so that the
regex will be moved to devstack-gate (regex exodus (tm)) and we'll work
on whittling down the regex there (and run those d-g changes against the
tempest-dsvm-cells job in the experimental queue).

The question for the nova team is, shall we make the tempest-dsvm-cells
job voting on nova changes knowing that the gate can be broken with a
change to tempest that isn't caught in the regex?  In my opinion I think
we should make it voting so we don't regress cells with changes to nova
that go unnoticed with the non-voting job today.  Cells v2 is a nova
priority for Liberty so we don't want setbacks now that it's stable.

If a change does land in tempest which breaks the cells job and blocks
nova, we (1) fix it or (2) modify the regex so it's excluded until fixed
as has been done up to this point.

I think we should probably mull this over in the ML and then vote on it
in this week's nova meeting.

[1] https://review.openstack.org/#/c/190894/
[2]
http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/devstack-gate.yaml#n1004





Regarding your "regex exodus", I recently added something for this.  In
another project, I'm setting the regex in a file I keep in the code repo
instead of project-config.

support for DEVSTACK_GATE_SETTINGS in devstack-gate:
https://review.openstack.org/190321

usage in a job definition: https://review.openstack.org/190325

a DEVSTACK_GATE_SETTINGS file that sets DEVSTACK_GATE_TEMPEST_REGEX:
https://review.openstack.org/186894

It all seems to be working for me, except I still need to tweak my regex
to get the job passing, but at least I can do that without updating
project-config now.



Awesome, that is way cleaner.  I'll go that route instead, thanks!



Here is the change that moves the regex into nova:

https://review.openstack.org/#/c/194411/

--

Thanks,

Matt Riedemann


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [qpid] [zmq] [RabbitMQ] [oslo] Pending deprecation of driver(s).

2015-06-22 Thread Ken Giusti

On Mon, Jun 22, 2015 at 2:27 PM Adam Young  wrote:

> On 06/20/2015 10:28 AM, Flavio Percoco wrote:
> >>
> >
> > As promissed: https://review.openstack.org/#/c/193804/
> >
> > Cheers,
> You can't deprecate a driver without providing a viable alternative.
>
> Right now, QPID is the only driver that supports  Kerberos.
>
> TO support Kerberos, tyou need support for the GSSAPI library, which is
> usually done via support for SASL.  Why is it so convoluted...historical...
>
> We've talked with both teams (I work with Ken) and I think Proton is
> likely going to be the first to have support.  The folks working on
> Rabbit have the double hurdle of getting SASL support into Erlang first,
> and then support for SASL into Rabbit. They've indicated a preference
> for getting it in to the AMQP 1.0 driver, and not bothering with the
> exisiting, but, check me on this, the Oso.Messaging  code only support
> the pre 1.0 Rabbit.
>
>
> So..until we have a viable alternative, please leave QPID alone. I've
> not been bothering people about it, as there seems to be work to get
> ahead, but until either Rabbit or  Proton support Kerberos, I need QPID
> as is.
>
>
Re: proton - Kerberos support is in progress upstream [0],[1], but as you
point out it's not yet available.  That blocks kerberos support for the
amqp1.0 driver.

Once proton does release that, and the amqp1.0 driver adopts it, you'll be
able to migrate to the amqp1.0 driver and continue to work with the QPID
broker (as long as the version of the QPID broker supports AMQP 1.0).

That doesn't help you now, but it is something to plan for.

[0]https://issues.apache.org/jira/browse/PROTON-334
[1]https://issues.apache.org/jira/browse/PROTON-911


> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [puppet] Clarification of 'Puppet Modules' Project Scope

2015-06-22 Thread Emilien Macchi

Hi Richard,

On 06/22/2015 01:05 PM, Richard Raseley wrote:
> I am currently hoping to build consensus (or seek clarity if I am the
> only one with this question) about the appropriate scope for our 'Puppet
> Modules' project.
> 
> The question in my mind is if we:
> 
> A) Only include those modules which represent a 1:1 mapping with other
> OpenStack projects.
> 
> B) Also include those modules which provide 'supporting' infrastructure
> to OpenStack components.
> 
> To be totally transparent, this came to mind for me because I am
> currently working with the folks at Midokura to publish a module which
> can be used to configure their open source Midonet SDN for Neutron and I
> was contemplating whether or not it would be reasonable to be part of
> the project.

I see two options here:

#1 you create the repository outside Stackforge & OpennStack, on your
own repo or in Midokura namespace.

#2 you create the repository on Stackforge (like puppet-ceph) but Puppet
OpenStack group won't officially support it for the simple reason it's
not an OpenStack project or a dependency to deploy it.

I would be against having puppet-midonet part of OpenStack namespace
though for the same reasons it could be on Stackforge.

I have a preference for #1 since IMHO it makes more sense for Midokura
to have their Puppet module close to their code but I would not be
against having it on Stackforge.

> FWIW, we have carried over the 'puppet-vswitch' repository over with us
> as part of the move (which would align with option B), but I didn't want
> to assume that was intended to be precedent setting.

If you look at contributors [1], the history shows that this module has
been written by people working on Puppet OpenStack modules and it made
sense to have this repository on Stackforge to benefit of OpenStack
integration.
Until recently, puppet-vswitch was a dependency to run puppet-neutron.
See [2].

[1] https://github.com/openstack/puppet-vswitch/graphs/contributors
[2]
https://github.com/openstack/puppet-neutron/tree/stable/juno/manifests/plugins/ovs

To be less specific, Puppet modules that reside in OpenStack namespace
are today:
* deploying an OpenStack project (neutron, horizon, etc)
* a dependency to deploy modules (openstacklib, vswitch)
* contain some code used by our community to help with CI,
documentation, consistency, etc (modulesync, cookiebutter, integration,
blueprints).

Any feedback is welcome,
Regards,
-- 
Emilien Macchi

signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] can we get ceilometermiddleware to use a config file instead of transport_url?

2015-06-22 Thread Brant Knudson

On Mon, Jun 22, 2015 at 2:43 PM, Doug Hellmann 
wrote:

> Excerpts from Sean Dague's message of 2015-06-22 13:30:44 -0400:
> > On 06/22/2015 01:15 PM, Michael Krotscheck wrote:
> > > Having _just_ done this, a couple of suggestions.
> > >
> > > - If the middleware is NOT optional - that is, it provides some kind of
> > > a fundamental component or specification of the API, like ETag caching,
> > > CORS, or DB Session management - then the middleware should be added
> > > during the app initialization routine, likely in the app factory. In
> > > this case, we have tight control over the initialization order, and can
> > > make sure that oslo.config is there first. Yay config.
> > >
> > > - If the middleware _is_ optional, then I really feel this problem is
> > > solved by pastedeploy's filter_factory pattern. It lets you define as
> > > many required parameters as you want, and spits back a middleware
> > > instance. That way you have the freedom to set whatever configuration
> > > options you want, or omit the middleware altogether.
> > >
> > > http://pythonpaste.org/deploy/#paste-filter-factory
> > >
> > > Ultimately, what you should want is to ask a factory method for a thing
> > > with some configuration options, and it spits an instance back at you.
> > > If your project doesn't support pastedeploy, that limits your options
> > > (ahem-ironic-ahem). Otherwise, it's really a team decision on whether
> > > something is a 'Fundamental feature' or something that's optional.
> > >
> > > In the case of Ceilometer? Sorry, but I think it's optional. You should
> > > be able to run any component of openstack without ceilometer. So I
> don't
> > > really think it should depend on oslo_config.
> >
> > Ok, here is where I'm confused. Ceilometer middleware *already* depends
> > on oslo_config.
> >
> >
> https://github.com/openstack/ceilometermiddleware/blob/master/ceilometermiddleware/swift.py#L42
> >
> > And it does stuff with it here:
> >
> >
> https://github.com/openstack/ceilometermiddleware/blob/master/ceilometermiddleware/swift.py#L107
> >
> > But regardless, it uses oslo_messaging, which uses oslo_config. So it
> > seems like the only issue at hand is middleware retriggering conf
> parsing.
> >
> > Because the current model of requiring simultaneous code lands in
> > oslo_messaging and ceilometermiddleware, and simultaneous config updates
> > in every installer and config management system to configure the same
> > thing in 2 different ways, does seem like a long term win. And honestly
> > just prohibits most of what people seem to want to do with new messaging
> > approaches.
>
> I explained this on IRC, but I'll repeat it here as a more permanent
> record.
>
> We should not have the ceilometer middleware (re)initializing
> oslo.config. That combines 2 concerns (setting up configuration and
> doing whatever ceilometermiddleware is already doing) in one piece
> of middleware, and that will make it more difficult to combine
> ceilometermiddleware with other middleware that also wants access
> to the configuration settings.
>
> If the application code isn't setting up oslo.config, then we should
> have another piece of middleware do just that one thing. It can take
> options from paste's config to tell it how to do that work, and then
> other middleware later in the pipeline can rely on oslo.config being set
> up and the config files being loaded.
>
> Doug
>
>
Whatever is decided for this, I assume it'll also apply to the auth_token
middleware, since that uses oslo.config, too. We already have a bug[1] and
a proposed change[2] that isn't passing jenkins.

[1] https://bugs.launchpad.net/keystonemiddleware/+bug/1406218
[2] https://review.openstack.org/#/c/143063/

Brant
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [QA][Tempest] Proposing Jordan Pittier for Tempest Core

2015-06-22 Thread Ken'ichi Ohmichi

+1 :-)

2015年6月23日(火) 5:24 Matthew Treinish :

>
>
> Hi Everyone,
>
> I'd like to propose we add Jordan Pittier (jordanP) to the tempest core
> team.
> Jordan has been a steady contributor and reviewer on tempest over the past
> few
> cycles and he's been actively engaged in the Tempest community. Jordan has
> had
> one of the higher review counts on Tempest for the past cycle, and he has
> consistently been providing reviews that show insight into both the project
> internals and it's future direction. I feel that Jordan will make an
> excellent
> addition to the core team.
>
> As per the usual, if the current Tempest core team members would please
> vote +1
> or -1(veto) to the nomination when you get a chance. We'll keep the polls
> open
> for 5 days or until everyone has voted.
>
> Thanks,
>
> Matt Treinish
>
> References:
>
>
> https://review.openstack.org/#/q/reviewer:jordan.pittier%2540scality.com+project:openstack/tempest+OR+project:openstack/tempest-lib,n,z
>
> http://stackalytics.com/?metric=marks&user_id=jordan-pittier
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][qa] Do we turn on voting for the tempest-dsvm-cells job?

2015-06-22 Thread Matt Riedemann




On 6/22/2015 4:32 PM, Russell Bryant wrote:

On 06/22/2015 05:23 PM, Matt Riedemann wrote:

The check-tempest-dsvm-cells job has been in nova's check queue since
January as non-voting and has been stable for a couple of weeks now, so
before it's regressed melwitt proposed a change to making it voting and
gating on nova changes [1].

I raised a concern in that change that the tempest-dsvm-cells job is not
in the check queue for tempest or devstack changes, so if a change is
merged in tempest/devstack which breaks the cells job, it will block
nova changes from merging.

mtreinish noted that tempest already has around 30 jobs running against
it right now in the check queue, so he'd prefer that another one isn't
added since the nova API shouldn't be different in the case of cells,
but we know there are quirks.  That can be seen from the massive regex
of excluded tests for the tempest-dvsm-cells job [2].

If we could turn that regex list into tempest configurations, I think
that would make it possible to not have to run tempest changes through
the cells job in the check queue but also feel reasonably confident that
changes to tempest that use the config options properly won't break the
cells job (and block nova in the gate).

This is actually something we should do regardless of voting or not on
nova since new tests get added which might not fall in the regex and
break the cells job.  So I'm going to propose some changes so that the
regex will be moved to devstack-gate (regex exodus (tm)) and we'll work
on whittling down the regex there (and run those d-g changes against the
tempest-dsvm-cells job in the experimental queue).

The question for the nova team is, shall we make the tempest-dsvm-cells
job voting on nova changes knowing that the gate can be broken with a
change to tempest that isn't caught in the regex?  In my opinion I think
we should make it voting so we don't regress cells with changes to nova
that go unnoticed with the non-voting job today.  Cells v2 is a nova
priority for Liberty so we don't want setbacks now that it's stable.

If a change does land in tempest which breaks the cells job and blocks
nova, we (1) fix it or (2) modify the regex so it's excluded until fixed
as has been done up to this point.

I think we should probably mull this over in the ML and then vote on it
in this week's nova meeting.

[1] https://review.openstack.org/#/c/190894/
[2]
http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/devstack-gate.yaml#n1004




Regarding your "regex exodus", I recently added something for this.  In
another project, I'm setting the regex in a file I keep in the code repo
instead of project-config.

support for DEVSTACK_GATE_SETTINGS in devstack-gate:
https://review.openstack.org/190321

usage in a job definition: https://review.openstack.org/190325

a DEVSTACK_GATE_SETTINGS file that sets DEVSTACK_GATE_TEMPEST_REGEX:
https://review.openstack.org/186894

It all seems to be working for me, except I still need to tweak my regex
to get the job passing, but at least I can do that without updating
project-config now.



Awesome, that is way cleaner.  I'll go that route instead, thanks!

--

Thanks,

Matt Riedemann


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][qa] Do we turn on voting for the tempest-dsvm-cells job?

2015-06-22 Thread Andrew Laski


On 06/22/15 at 04:23pm, Matt Riedemann wrote:
The check-tempest-dsvm-cells job has been in nova's check queue since 
January as non-voting and has been stable for a couple of weeks now, 
so before it's regressed melwitt proposed a change to making it 
voting and gating on nova changes [1].


I raised a concern in that change that the tempest-dsvm-cells job is 
not in the check queue for tempest or devstack changes, so if a 
change is merged in tempest/devstack which breaks the cells job, it 
will block nova changes from merging.


mtreinish noted that tempest already has around 30 jobs running 
against it right now in the check queue, so he'd prefer that another 
one isn't added since the nova API shouldn't be different in the case 
of cells, but we know there are quirks.  That can be seen from the 
massive regex of excluded tests for the tempest-dvsm-cells job [2].


If we could turn that regex list into tempest configurations, I think 
that would make it possible to not have to run tempest changes 
through the cells job in the check queue but also feel reasonably 
confident that changes to tempest that use the config options 
properly won't break the cells job (and block nova in the gate).


This is actually something we should do regardless of voting or not 
on nova since new tests get added which might not fall in the regex 
and break the cells job.  So I'm going to propose some changes so 
that the regex will be moved to devstack-gate (regex exodus (tm)) and 
we'll work on whittling down the regex there (and run those d-g 
changes against the tempest-dsvm-cells job in the experimental 
queue).


The question for the nova team is, shall we make the 
tempest-dsvm-cells job voting on nova changes knowing that the gate 
can be broken with a change to tempest that isn't caught in the 
regex?  In my opinion I think we should make it voting so we don't 
regress cells with changes to nova that go unnoticed with the 
non-voting job today.  Cells v2 is a nova priority for Liberty so we 
don't want setbacks now that it's stable.


I agree, though I'm pretty biased.



If a change does land in tempest which breaks the cells job and 
blocks nova, we (1) fix it or (2) modify the regex so it's excluded 
until fixed as has been done up to this point.


During our efforts last cycle to reduce the number of failing tests it 
seemed that we had regressions based on Tempest changes 2 times.  Once 
around security group logic being added to tests and again when the 
networking setup changed during tests that created an instance.


The security group change led to exclusions and the networking change we 
were able to fix with a devstack change, and a Tempest flag iirc.





I think we should probably mull this over in the ML and then vote on 
it in this week's nova meeting.


[1] https://review.openstack.org/#/c/190894/
[2] 
http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/devstack-gate.yaml#n1004

--

Thanks,

Matt Riedemann


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][qa] Do we turn on voting for the tempest-dsvm-cells job?

2015-06-22 Thread Russell Bryant

On 06/22/2015 05:23 PM, Matt Riedemann wrote:
> The check-tempest-dsvm-cells job has been in nova's check queue since
> January as non-voting and has been stable for a couple of weeks now, so
> before it's regressed melwitt proposed a change to making it voting and
> gating on nova changes [1].
> 
> I raised a concern in that change that the tempest-dsvm-cells job is not
> in the check queue for tempest or devstack changes, so if a change is
> merged in tempest/devstack which breaks the cells job, it will block
> nova changes from merging.
> 
> mtreinish noted that tempest already has around 30 jobs running against
> it right now in the check queue, so he'd prefer that another one isn't
> added since the nova API shouldn't be different in the case of cells,
> but we know there are quirks.  That can be seen from the massive regex
> of excluded tests for the tempest-dvsm-cells job [2].
> 
> If we could turn that regex list into tempest configurations, I think
> that would make it possible to not have to run tempest changes through
> the cells job in the check queue but also feel reasonably confident that
> changes to tempest that use the config options properly won't break the
> cells job (and block nova in the gate).
> 
> This is actually something we should do regardless of voting or not on
> nova since new tests get added which might not fall in the regex and
> break the cells job.  So I'm going to propose some changes so that the
> regex will be moved to devstack-gate (regex exodus (tm)) and we'll work
> on whittling down the regex there (and run those d-g changes against the
> tempest-dsvm-cells job in the experimental queue).
> 
> The question for the nova team is, shall we make the tempest-dsvm-cells
> job voting on nova changes knowing that the gate can be broken with a
> change to tempest that isn't caught in the regex?  In my opinion I think
> we should make it voting so we don't regress cells with changes to nova
> that go unnoticed with the non-voting job today.  Cells v2 is a nova
> priority for Liberty so we don't want setbacks now that it's stable.
> 
> If a change does land in tempest which breaks the cells job and blocks
> nova, we (1) fix it or (2) modify the regex so it's excluded until fixed
> as has been done up to this point.
> 
> I think we should probably mull this over in the ML and then vote on it
> in this week's nova meeting.
> 
> [1] https://review.openstack.org/#/c/190894/
> [2]
> http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/devstack-gate.yaml#n1004
> 
> 

Regarding your "regex exodus", I recently added something for this.  In
another project, I'm setting the regex in a file I keep in the code repo
instead of project-config.

support for DEVSTACK_GATE_SETTINGS in devstack-gate:
https://review.openstack.org/190321

usage in a job definition: https://review.openstack.org/190325

a DEVSTACK_GATE_SETTINGS file that sets DEVSTACK_GATE_TEMPEST_REGEX:
https://review.openstack.org/186894

It all seems to be working for me, except I still need to tweak my regex
to get the job passing, but at least I can do that without updating
project-config now.

-- 
Russell Bryant

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova][qa] Do we turn on voting for the tempest-dsvm-cells job?

2015-06-22 Thread Matt Riedemann

The check-tempest-dsvm-cells job has been in nova's check queue since 
January as non-voting and has been stable for a couple of weeks now, so 
before it's regressed melwitt proposed a change to making it voting and 
gating on nova changes [1].


I raised a concern in that change that the tempest-dsvm-cells job is not 
in the check queue for tempest or devstack changes, so if a change is 
merged in tempest/devstack which breaks the cells job, it will block 
nova changes from merging.


mtreinish noted that tempest already has around 30 jobs running against 
it right now in the check queue, so he'd prefer that another one isn't 
added since the nova API shouldn't be different in the case of cells, 
but we know there are quirks.  That can be seen from the massive regex 
of excluded tests for the tempest-dvsm-cells job [2].


If we could turn that regex list into tempest configurations, I think 
that would make it possible to not have to run tempest changes through 
the cells job in the check queue but also feel reasonably confident that 
changes to tempest that use the config options properly won't break the 
cells job (and block nova in the gate).


This is actually something we should do regardless of voting or not on 
nova since new tests get added which might not fall in the regex and 
break the cells job.  So I'm going to propose some changes so that the 
regex will be moved to devstack-gate (regex exodus (tm)) and we'll work 
on whittling down the regex there (and run those d-g changes against the 
tempest-dsvm-cells job in the experimental queue).


The question for the nova team is, shall we make the tempest-dsvm-cells 
job voting on nova changes knowing that the gate can be broken with a 
change to tempest that isn't caught in the regex?  In my opinion I think 
we should make it voting so we don't regress cells with changes to nova 
that go unnoticed with the non-voting job today.  Cells v2 is a nova 
priority for Liberty so we don't want setbacks now that it's stable.


If a change does land in tempest which breaks the cells job and blocks 
nova, we (1) fix it or (2) modify the regex so it's excluded until fixed 
as has been done up to this point.


I think we should probably mull this over in the ML and then vote on it 
in this week's nova meeting.


[1] https://review.openstack.org/#/c/190894/
[2] 
http://git.openstack.org/cgit/openstack-infra/project-config/tree/jenkins/jobs/devstack-gate.yaml#n1004


--

Thanks,

Matt Riedemann


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] M Naming Poll ended - results still need to clear legal

2015-06-22 Thread Salvatore Orlando

Anyway, if you want to print t-shirts once legal is cleared, here's a
vintage football idea [1].
Little and pointless trivia fact: Como calcio was sponsored for a few years
in the 80s by Mita copiers - now known as Kyocera.

Salvatore

[1]
http://www.calciocomo1907.it/images/news/thumbnails/Mattei%20Luca%20%201986-87_1000x700.JPG

On 22 June 2015 at 20:25, Clint Byrum  wrote:

> Excerpts from Clay Gerrard's message of 2015-06-22 10:30:49 -0700:
> > http://civs.cs.cornell.edu/cgi-bin/results.pl?id=E_4983776e190c8dbc
> >
> > how is the top pick not the author of the book of five rings [1]
> >
>
> Agreed. I don't think people fully appreciated the reputation of Mr.
> Musashi. ;)
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Issue with pymysql

2015-06-22 Thread Armando M.

Hi,

A brief update on the issue that sparked this thread:

A little over a week ago, bug [1] was filed. The gist of that was that the
switch to pymysql unveiled a number of latent race conditions that made
Neutron unstable.

To try and nip these in the bud, the Neutron team filed a number of patches
[2], to create an unstable configuration that would allow them to
troubleshoot and experiment a solution, by still keeping the stability in
check (a preliminary proposal for a fix has been available in [4]).

The latest failure rate trend is shown in [3]; as you can see, we're still
gathering data, but it seems that the instability gap between the two jobs
(stable vs unstable) has widened, and should give us plenty of data points
to devise a resolution strategy.

I have documented the most recurrent traces in the bug report [1].

Will update once we managed to get the two curves to kiss each other again
and close to a more acceptable failure rate.

Cheers,
Armando

[1] https://bugs.launchpad.net/neutron/+bug/1464612
[2] https://review.openstack.org/#/q/topic:neutron-unstable,n,z
[3] http://goo.gl/YM7gUC
[4] https://review.openstack.org/#/c/191540/


On 12 June 2015 at 11:13, Boris Pavlovic  wrote:

> Sean,
>
> Thanks for quick fix/revert https://review.openstack.org/#/c/191010/
> This unblocked Rally gates...
>
> Best regards,
> Boris Pavlovic
>
> On Fri, Jun 12, 2015 at 8:56 PM, Clint Byrum  wrote:
>
>> Excerpts from Mike Bayer's message of 2015-06-12 09:42:42 -0700:
>> >
>> > On 6/12/15 11:37 AM, Mike Bayer wrote:
>> > >
>> > >
>> > > On 6/11/15 9:32 PM, Eugene Nikanorov wrote:
>> > >> Hi neutrons,
>> > >>
>> > >> I'd like to draw your attention to an issue discovered by rally gate
>> job:
>> > >>
>> http://logs.openstack.org/96/190796/4/check/gate-rally-dsvm-neutron-rally/7a18e43/logs/screen-q-svc.txt.gz?level=TRACE
>> > >>
>> > >> I don't have bandwidth to take a deep look at it, but first
>> > >> impression is that it is some issue with nested transaction support
>> > >> either on sqlalchemy or pymysql side.
>> > >> Also, besides errors with nested transactions, there are a lot of
>> > >> Lock wait timeouts.
>> > >>
>> > >> I think it makes sense to start with reverting the patch that moves
>> > >> to pymysql.
>> > > My immediate reaction is that this is perhaps a concurrency-related
>> > > issue; because PyMySQL is pure python and allows for full blown
>> > > eventlet monkeypatching, I wonder if somehow the same PyMySQL
>> > > connection is being used in multiple contexts. E.g. one greenlet
>> > > starts up a savepoint, using identifier "_3" which is based on a
>> > > counter that is local to the SQLAlchemy Connection, but then another
>> > > greenlet shares that PyMySQL connection somehow with another
>> > > SQLAlchemy Connection that uses the same identifier.
>> >
>> > reading more of the log, it seems the main issue is just that there's a
>> > deadlock on inserting into the securitygroups table.  The deadlock on
>> > insert can be because of an index being locked.
>> >
>> >
>> > I'd be curious to know how many greenlets are running concurrently here,
>> > and what the overall transaction looks like within the operation that is
>> > failing here (e.g. does each transaction insert multiple rows into
>> > securitygroups?  that would make a deadlock seem more likely).
>>
>> This begs two questions:
>>
>> 1) Are we handling deadlocks with retries? It's important that we do
>> that to be defensive.
>>
>> 2) Are we being careful to sort the table order in any multi-table
>> transactions so that we minimize the chance of deadlocks happening
>> because of any cross table deadlocks?
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [QA][Tempest] Proposing Jordan Pittier for Tempest Core

2015-06-22 Thread gord chung

not a core myself, apologies if this is spamming, but +1. Jordan's been
a great help.

On 22/06/15 04:23 PM, Matthew Treinish wrote:

Hi Everyone,

I'd like to propose we add Jordan Pittier (jordanP) to the tempest core team.
Jordan has been a steady contributor and reviewer on tempest over the past few
cycles and he's been actively engaged in the Tempest community. Jordan has had
one of the higher review counts on Tempest for the past cycle, and he has
consistently been providing reviews that show insight into both the project
internals and it's future direction. I feel that Jordan will make an excellent
addition to the core team.

As per the usual, if the current Tempest core team members would please vote +1
or -1(veto) to the nomination when you get a chance. We'll keep the polls open
for 5 days or until everyone has voted.

Thanks,

Matt Treinish

References:

https://review.openstack.org/#/q/reviewer:jordan.pittier%2540scality.com+project:openstack/tempest+OR+project:openstack/tempest-lib,n,z

http://stackalytics.com/?metric=marks&user_id=jordan-pittier

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

--
gord

[openstack-dev] [mistral] Proposing Lingxian Kong as a core reviewer

2015-06-22 Thread W Chan

+1

Lingxian, keep up with the good work. :D
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [QA][Tempest] Proposing Jordan Pittier for Tempest Core

2015-06-22 Thread Masayuki Igawa

+1 :)
On 2015年6月23日(火) at 5:30 Matthew Treinish  wrote:

>
>
> Hi Everyone,
>
> I'd like to propose we add Jordan Pittier (jordanP) to the tempest core
> team.
> Jordan has been a steady contributor and reviewer on tempest over the past
> few
> cycles and he's been actively engaged in the Tempest community. Jordan has
> had
> one of the higher review counts on Tempest for the past cycle, and he has
> consistently been providing reviews that show insight into both the project
> internals and it's future direction. I feel that Jordan will make an
> excellent
> addition to the core team.
>
> As per the usual, if the current Tempest core team members would please
> vote +1
> or -1(veto) to the nomination when you get a chance. We'll keep the polls
> open
> for 5 days or until everyone has voted.
>
> Thanks,
>
> Matt Treinish
>
> References:
>
>
> https://review.openstack.org/#/q/reviewer:jordan.pittier%2540scality.com+project:openstack/tempest+OR+project:openstack/tempest-lib,n,z
>
> http://stackalytics.com/?metric=marks&user_id=jordan-pittier
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-- 
-- Masayuki on a tiny device
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [QA][Tempest] Proposing Jordan Pittier for Tempest Core

2015-06-22 Thread David Kranz


+1

On 06/22/2015 04:23 PM, Matthew Treinish wrote:


Hi Everyone,

I'd like to propose we add Jordan Pittier (jordanP) to the tempest core team.
Jordan has been a steady contributor and reviewer on tempest over the past few
cycles and he's been actively engaged in the Tempest community. Jordan has had
one of the higher review counts on Tempest for the past cycle, and he has
consistently been providing reviews that show insight into both the project
internals and it's future direction. I feel that Jordan will make an excellent
addition to the core team.

As per the usual, if the current Tempest core team members would please vote +1
or -1(veto) to the nomination when you get a chance. We'll keep the polls open
for 5 days or until everyone has voted.

Thanks,

Matt Treinish

References:

https://review.openstack.org/#/q/reviewer:jordan.pittier%2540scality.com+project:openstack/tempest+OR+project:openstack/tempest-lib,n,z

http://stackalytics.com/?metric=marks&user_id=jordan-pittier


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [QA][Tempest] Proposing Jordan Pittier for Tempest Core

2015-06-22 Thread Matthew Treinish



Hi Everyone,

I'd like to propose we add Jordan Pittier (jordanP) to the tempest core team.
Jordan has been a steady contributor and reviewer on tempest over the past few
cycles and he's been actively engaged in the Tempest community. Jordan has had
one of the higher review counts on Tempest for the past cycle, and he has
consistently been providing reviews that show insight into both the project
internals and it's future direction. I feel that Jordan will make an excellent
addition to the core team.

As per the usual, if the current Tempest core team members would please vote +1
or -1(veto) to the nomination when you get a chance. We'll keep the polls open
for 5 days or until everyone has voted.

Thanks,

Matt Treinish

References:

https://review.openstack.org/#/q/reviewer:jordan.pittier%2540scality.com+project:openstack/tempest+OR+project:openstack/tempest-lib,n,z

http://stackalytics.com/?metric=marks&user_id=jordan-pittier


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Oslo][Automaton] Proposal for new core (ruby loo)

+1

On Mon, Jun 22, 2015 at 2:55 PM, Doug Hellmann  wrote:
> Excerpts from Joshua Harlow's message of 2015-06-22 10:59:27 -0700:
>> Greetings all stackers,
>>
>> I propose that we add Ruby Loo[1] to the automaton-core team.
>>
>> Ruby has been actively contributing to automaton[2] for a while now,
>> both in helping make automaton better via reviews (she has been involved
>> from the start in helping sanity check that code) and helping get it
>> integrated into ironic. She has provided quality reviews and is doing an
>> awesome job with the various automaton concepts and helping make
>> automaton the best it can be!
>>
>> Overall I think she would make a great addition to the core review team.
>>
>> Please respond with +1/-1.
>>
>> Thanks much!
>>
>> [1] https://launchpad.net/~rloo
>> [2] https://github.com/openstack/automaton
>
> +1
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Davanum Srinivas :: https://twitter.com/dims

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][requirements] requirements reviews falling behind

Excerpts from Sean Dague's message of 2015-06-22 15:32:17 -0400:
> On 06/22/2015 03:27 PM, Monty Taylor wrote:
> > On 06/22/2015 02:49 PM, Doug Hellmann wrote:
> >> We have quite a long list of patches to the openstack/requirements 
> >> repository, many of which are straightforward changes to increase the 
> >> minimums for libraries we develop within the community (meaning we are 
> >> holding up projects from using new features, or we’re not being accurate 
> >> about our current requirements) or to add projects to the list where the 
> >> global requirements are synced.
> >>
> >> https://review.openstack.org/#/q/project:openstack%2Frequirements+is:open,n,z
> >>
> >> Patches like those should be approved quickly under normal circumstances. 
> >> Is there a reason to hold them up, or are we just short of reviewer time 
> >> for that repository?
> > 
> > Sorry - I'll take a quick pass through.
> 
> I just went through some of the easy ones, we should have a big flush
> shortly.

Thanks!


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] can we get ceilometermiddleware to use a config file instead of transport_url?

Excerpts from Sean Dague's message of 2015-06-22 13:30:44 -0400:
> On 06/22/2015 01:15 PM, Michael Krotscheck wrote:
> > Having _just_ done this, a couple of suggestions.
> > 
> > - If the middleware is NOT optional - that is, it provides some kind of
> > a fundamental component or specification of the API, like ETag caching,
> > CORS, or DB Session management - then the middleware should be added
> > during the app initialization routine, likely in the app factory. In
> > this case, we have tight control over the initialization order, and can
> > make sure that oslo.config is there first. Yay config.
> > 
> > - If the middleware _is_ optional, then I really feel this problem is
> > solved by pastedeploy's filter_factory pattern. It lets you define as
> > many required parameters as you want, and spits back a middleware
> > instance. That way you have the freedom to set whatever configuration
> > options you want, or omit the middleware altogether.
> > 
> > http://pythonpaste.org/deploy/#paste-filter-factory
> > 
> > Ultimately, what you should want is to ask a factory method for a thing
> > with some configuration options, and it spits an instance back at you.
> > If your project doesn't support pastedeploy, that limits your options
> > (ahem-ironic-ahem). Otherwise, it's really a team decision on whether
> > something is a 'Fundamental feature' or something that's optional.
> > 
> > In the case of Ceilometer? Sorry, but I think it's optional. You should
> > be able to run any component of openstack without ceilometer. So I don't
> > really think it should depend on oslo_config.
> 
> Ok, here is where I'm confused. Ceilometer middleware *already* depends
> on oslo_config.
> 
> https://github.com/openstack/ceilometermiddleware/blob/master/ceilometermiddleware/swift.py#L42
> 
> And it does stuff with it here:
> 
> https://github.com/openstack/ceilometermiddleware/blob/master/ceilometermiddleware/swift.py#L107
> 
> But regardless, it uses oslo_messaging, which uses oslo_config. So it
> seems like the only issue at hand is middleware retriggering conf parsing.
> 
> Because the current model of requiring simultaneous code lands in
> oslo_messaging and ceilometermiddleware, and simultaneous config updates
> in every installer and config management system to configure the same
> thing in 2 different ways, does seem like a long term win. And honestly
> just prohibits most of what people seem to want to do with new messaging
> approaches.

I explained this on IRC, but I'll repeat it here as a more permanent
record.

We should not have the ceilometer middleware (re)initializing
oslo.config. That combines 2 concerns (setting up configuration and
doing whatever ceilometermiddleware is already doing) in one piece
of middleware, and that will make it more difficult to combine
ceilometermiddleware with other middleware that also wants access
to the configuration settings.

If the application code isn't setting up oslo.config, then we should
have another piece of middleware do just that one thing. It can take
options from paste's config to tell it how to do that work, and then
other middleware later in the pipeline can rely on oslo.config being set
up and the config files being loaded.

Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [qpid] [zmq] [RabbitMQ] [oslo] Pending deprecation of driver(s).

2015-06-22 Thread Clint Byrum

Excerpts from Adam Young's message of 2015-06-22 11:26:54 -0700:
> On 06/20/2015 10:28 AM, Flavio Percoco wrote:
> >>
> >
> > As promissed: https://review.openstack.org/#/c/193804/
> >
> > Cheers, 
> You can't deprecate a driver without providing a viable alternative.
> 
> Right now, QPID is the only driver that supports  Kerberos.
> 
> TO support Kerberos, tyou need support for the GSSAPI library, which is 
> usually done via support for SASL.  Why is it so convoluted...historical...
> 
> We've talked with both teams (I work with Ken) and I think Proton is 
> likely going to be the first to have support.  The folks working on 
> Rabbit have the double hurdle of getting SASL support into Erlang first, 
> and then support for SASL into Rabbit. They've indicated a preference 
> for getting it in to the AMQP 1.0 driver, and not bothering with the 
> exisiting, but, check me on this, the Oso.Messaging  code only support 
> the pre 1.0 Rabbit.
> 
> 
> So..until we have a viable alternative, please leave QPID alone. I've 
> not been bothering people about it, as there seems to be work to get 
> ahead, but until either Rabbit or  Proton support Kerberos, I need QPID 
> as is.
> 

Adam that is all great information, thank you. However, the policy is
clear: commit resources for integration testing, or it needs to move
out of tree.

It's not a mountain of resources. Just an integration test that passes
reliably, and a couple of QPID+OpenStack experts who we can contact when
it breaks. If nobody is willing to put that much effort in, then it is
not really something we want in our official messaging library tree.

So please if you can carry that message up to those who want it to stay in
tree, that would be helpful and would put the stops on this deprecation.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][requirements] requirements reviews falling behind

On 06/22/2015 03:27 PM, Monty Taylor wrote:
> On 06/22/2015 02:49 PM, Doug Hellmann wrote:
>> We have quite a long list of patches to the openstack/requirements 
>> repository, many of which are straightforward changes to increase the 
>> minimums for libraries we develop within the community (meaning we are 
>> holding up projects from using new features, or we’re not being accurate 
>> about our current requirements) or to add projects to the list where the 
>> global requirements are synced.
>>
>> https://review.openstack.org/#/q/project:openstack%2Frequirements+is:open,n,z
>>
>> Patches like those should be approved quickly under normal circumstances. Is 
>> there a reason to hold them up, or are we just short of reviewer time for 
>> that repository?
> 
> Sorry - I'll take a quick pass through.

I just went through some of the easy ones, we should have a big flush
shortly.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][requirements] requirements reviews falling behind

2015-06-22 Thread Monty Taylor

On 06/22/2015 02:49 PM, Doug Hellmann wrote:
> We have quite a long list of patches to the openstack/requirements 
> repository, many of which are straightforward changes to increase the 
> minimums for libraries we develop within the community (meaning we are 
> holding up projects from using new features, or we’re not being accurate 
> about our current requirements) or to add projects to the list where the 
> global requirements are synced.
> 
> https://review.openstack.org/#/q/project:openstack%2Frequirements+is:open,n,z
> 
> Patches like those should be approved quickly under normal circumstances. Is 
> there a reason to hold them up, or are we just short of reviewer time for 
> that repository?

Sorry - I'll take a quick pass through.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo] Updates from the Summit

2015-06-22 Thread Joshua Harlow


Any progress on this?

Let me know and if not I can throw something together to (if you are 
busy or pre-occupied, or other) :)


-Josh

Joshua Harlow wrote:

Feel free to add it, or open a blueprint, or make a spec to :)

Any of the above are ok with me.

I did start prototyping something @
https://review.openstack.org/#/c/184663/ (external cancellation) but it
might be going in the wrong direction (or a different direction); so
feel free to comment on that review if so...

-Josh

Gorka Eguileor wrote:

On Wed, May 27, 2015 at 08:47:42AM -0400, Davanum Srinivas wrote:

Hi Team,

Here are the etherpads from the summit[1].


I remember that in Taskflow's Fishbowl session we discussed not only
pause/yield option but abort/cancel for long running tasks as well, but
reviewing the Etherpad now I don't see it there.

Should I just add it to "Ideas for Liberty" section or there's a
specific reason why it wasn't included?

Cheers,
Gorka.


Some highlights are as follows:
Oslo.messaging : Took status of the existing zmq driver, proposed a
new driver in parallel to the existing zmq driver. Also looked at
possibility of using Pika with RabbitMQ. Folks from pivotal promised
to help with our scenarios as well.
Oslo.rootwrap : Debated daemon vs a new privileged service. The Nova
change to add rootwrap as daemon is on hold pending progress on the
privsep proposal/activity.
Oslo.versionedobjects : We had a nice presentation from Dan about what
o.vo can do and a deepdive into what we could do in next release.
Taskflow : Josh and team came up with several new features and how to
improve usability

We will also have several new libraries in Liberty (oslo.cache,
oslo.service, oslo.reports, futurist, automaton etc). We talked about
our release processes, functional testing, deprecation strategies and
debated a but about how best to move to async models as well. Please
see etherpads for detailed information.

thanks,
dims

[1] https://wiki.openstack.org/wiki/Design_Summit/Liberty/Etherpads#Oslo

--
Davanum Srinivas :: https://twitter.com/dims

__

OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__

OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ironic][Horizon][Tuskar-ui] Making a dashboard for Ironic‏

2015-06-22 Thread Mario Villaplana

Hi! Just as an FYI, we use this for a dashboard:
https://github.com/rackerlabs/onmetal-dashboard

That might be useful when moving forward with using Horizon or the
ironic-dashboard project.

Mario

On Thu, Jun 18, 2015 at 7:43 PM, niuzhenguo  wrote:

>  Hi Thai Q Tran,
>
>
>
> Thanks for the links about the Angular Dashboard, I agree with starting
> with the new angular horizon, will begin to draft a init repo of the new
> ironic-dashboard.
>
> And maybe can work with Krotscheck together.
>
>
>
> And as Andreas Jaeger comments here [1], he suggested to push
> ironic-dashboard in the openstack namespace instead of stackforge, and have
> a separate core team,
>
> needs Ironicers chime in here.
>
>
>
> [1] https://review.openstack.org/#/c/191131/
>
>
>
> Regards
>
> -zhenguo
>
>
>
> *From:* Thai Q Tran [mailto:tqt...@us.ibm.com]
> *Sent:* Friday, June 19, 2015 6:36 AM
>
> *To:* OpenStack Development Mailing List (not for usage questions)
> *Subject:* Re: [openstack-dev] [Ironic][Horizon][Tuskar-ui] Making a
> dashboard for Ironic‏
>
>
>
> Hi Zhengou,
>
>
>
> I think it make sense to start with the angular version. It's true that we
> don't have an angular dashboard yet,
>
> but we have a pretty good idea of what needs to go into it. I'll link a
> few patches that will give you an idea
>
> of where we are headed. I think this will also save you some work in the
> long run.
>
>
>
> For creating a new dashboard: https://review.openstack.org/#/c/190852/
>
> For creating a new panel: https://review.openstack.org/#/c/190865/
>
> For demo patch: https://review.openstack.org/#/c/181253/
>
>
>
> The file and code structure I would say is pretty stable.
>
> There are still some infra stuff that needs to happen to make this easier
> to do.
>
> Things like translation in static HTML, auto discovery of static files,
> start dash for angular, etc...
>
>
> -niuzhenguo  wrote: -
>
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> From: niuzhenguo 
> Date: 06/17/2015 06:38PM
> Subject: Re: [openstack-dev] [Ironic][Horizon][Tuskar-ui] Making a
> dashboard for Ironic‏
>
> Hi Krotscheck,
>
>
>
> Sorry for not attending the last meeting due to TZ.
>
>
>
> Yes, Horizon is moving towards an Angular application, but for now there’s
> no any Angular Dashboard landed. I think it’s high time that we should make
> a standard for other projects which want to horizon compatible on whether
> they should based on Angular Dashboard or the current Horizon framework.
> This is important for the new Magnum and Ironic UI, personally, I’d prefer
> to use the current framework and  move to Angular Dashboard when it’s
> mature.
>
>
>
> And after a quick look at your JS project, I think it’s totally a
> standalone UI not based on Horizon Angular Dashboard (correct me if I
> missed something), and seems there’s no any update over a month, are you
> planning to push you repo to stackforge or openstack?
>
>
>
> Anyway, it’s clear that we should make an Ironic dashboard, it’s a good
> start.
>
>
>
>
>
> Regards
>
> -zhenguo
>
>
>
> *From:* Michael Krotscheck [mailto:krotsch...@gmail.com
> ]
> *Sent:* Wednesday, June 17, 2015 11:56 PM
> *To:* OpenStack Development Mailing List (not for usage questions)
> *Subject:* Re: [openstack-dev] [Ironic][Horizon][Tuskar-ui] Making a
> dashboard for Ironic‏
>
>
>
> Hey there!
>
> Yes, we are duplicating effort. I've spent quite a bit of effort over the
> past few months landing features inside openstack that will make it
> possible for a JavaScript client to be imported to horizon as a dependency.
> This includes CORS, configuration, caching, infra tooling, etc, with the
> end goal being a maximum amount of code reusability between the standalone
> UI and Horizon. While it may not appear that way, I _am_ actively working
> on this project, though I'm currently focused on javascript infrastructure
> tooling and oslo middleware than the ironic webclient itself.
>
>
>
> With Horizon also moving towards an angular application, I feel it makes
> far more sense to build components for the "new" Horizon than the old one.
>
>
>
> Michael
>
>
>
> On Tue, Jun 16, 2015 at 9:02 PM NiuZhenguo 
> wrote:
>
>  hi folks,
>
>
>
> I'm planning to propose a new horizon plugin ironic-dashboard to fill the
> gap that ironic doesn't have horizon support. I know there's a nodes panel
> on "infrastructure" dashboard handled by tuskar-ui, but it's specifically
> geared towards TripleO. Ironic needs a separate dashboard to present an
> interface for querying and managing ironic's resources (Drivers, Nodes, and
> Ports).
>
>
>
> After discussion with the ironic community, I pushed an ironic-dashboard
> project to stackforge [1].
>
>
>
> Also there's an existing JS UI for ironic in developing now [2], we may
> try to resolve the same goals, but as an integrated openstack project,
> there's clear needs to have horizon support.
>
>
>
> I'd like to get what's your suggestion, tha

Re: [openstack-dev] [Cinder] Implementation of ABC MetaClasses

2015-06-22 Thread Duncan Thomas

On 20 June 2015 at 02:59, John Griffith  wrote:

>
> Sure, I guess my question is what's the real value in this intermediate
> step.  The bulk of these are things that I'd argue shouldn't be optional
> anyway (snapshots, transfers, manage, extend, retype and even migrate).
> Snapshots in particular I find surprising to be considered as "optional".
> 
>

The ABC code has not made any new things optional, it has just highlighted
them. They are de-facto optional, since not every driver implements them.
The minute that is fixed, the relevant ABC can go away. I'll see if I can
make a start.

-- 
Duncan Thomas
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Oslo][Automaton] Proposal for new core (ruby loo)

Excerpts from Joshua Harlow's message of 2015-06-22 10:59:27 -0700:
> Greetings all stackers,
> 
> I propose that we add Ruby Loo[1] to the automaton-core team.
> 
> Ruby has been actively contributing to automaton[2] for a while now, 
> both in helping make automaton better via reviews (she has been involved 
> from the start in helping sanity check that code) and helping get it 
> integrated into ironic. She has provided quality reviews and is doing an 
> awesome job with the various automaton concepts and helping make 
> automaton the best it can be!
> 
> Overall I think she would make a great addition to the core review team.
> 
> Please respond with +1/-1.
> 
> Thanks much!
> 
> [1] https://launchpad.net/~rloo
> [2] https://github.com/openstack/automaton

+1

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [heat][tripleo]Recursive validation for easier composability

2015-06-22 Thread Thomas Spatzier

> From: Jay Dobies 
> To: openstack-dev@lists.openstack.org
> Date: 22/06/2015 19:22
> Subject: Re: [openstack-dev] [heat][tripleo]Recursive validation for
> easier composability
>
>
>
> On 06/22/2015 12:19 PM, Steven Hardy wrote:
> > Hi all,
> >
> > Lately I've been giving some thought to how we might enable easier
> > composability, and in particular how we can make it easier for folks to
> > plug in deeply nested optional extra logic, then pass data in via
> > parameter_defaults to that nested template.
> >
> > Here's an example of the use-case I'm describing:
> >
> > https://review.openstack.org/#/c/193143/5/environments/cinder-
> netapp-config.yaml
> >
> > Here, we want to allow someone to easily turn on an optional
configuration
> > or feature, in this case a netapp backend for cinder.
>
> I think the actual desired goal is bigger than just optional
> configuration. I think it revolves more around choosing a nested stack
> implementation for a resource type and how to manage custom parameters
> for that implementation. We're getting into the territory here of having
> a parent stack defining an API that nested stacks can plug into. I'd
> like to have some sort of way of deriving that information instead of
> having it be completely relegated to outside documentation (but I'm
> getting off topic; at the end I mention how I want to do a better write
> up of the issues Tuskar has faced and I'll elaborate more there).

FWIW, adding a thought from my TOSCA background where we've been looking at
something similar, namely selecting a nested templates that declares to be
matching an interfaces consumed in a parent template (that's how I
understood Jay's words above). In TOSCA, there is a more type-safe kind of
template nesting, where nested templates do not just bring new resource
types into existence depending on what parameters they expose, but there is
a strict contract on the interface a nested template must fulfil - see [1],
and especially look for substitution_mapping.

Admittedly, this is not exactly the same as Steven's original problem, but
kind of related. And IIRC, some time back there was some discussion around
introduction of some kind of "interface" for HOT templates. So wanted to
bring this in and give it some thought whether something like this would
make sense for Heat.

[1]
http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile-YAML/v1.0/csd03/TOSCA-Simple-Profile-YAML-v1.0-csd03.html#_Toc419746122

Regards,
Thomas

>
> > The parameters specific to this feature/configuration only exist in the
> > nested cinder-netapp-config.yaml template, then parameter_defaults are
used
> > to wire in the implementation specific data without having to pass the
> > values through every parent template (potentially multiple layers of
> > nesting).
> >
> > This approach is working out OK, but we're missing an interface which
makes
> > the schema for parameters over the whole tree available.
>  >
> > This is obviously
> > a problem, particularly for UI's, where you really need a clearly
defined
> > interface for what data is required, what type it is, and what valid
values
> > may be chosen.
>
> I think this is going to be an awesome addition to Heat. As you alluded
> to, we've struggled with this in TripleO. The parameter_defaults works
> to circumvent the parameter passing, but it's rough from a user
> experience point of view since getting the unified list of what's
> configurable is difficult.
>
> > I'm considering an optional additional flag to our template-validate
API
> > which allows recursive validation of a tree of templates, with the data
> > returned on success to include a tree of parameters, e.g:
> >
> > heat template-validate -f parent.yaml -e env.yaml --show-nested
> > {
> >"Description": "The Parent",
> >"Parameters": {
> >  "ParentConfig": {
> >"Default": [],
> >"Type": "Json",
> >"NoEcho": "false",
> >"Description": "",
> >"Label": "ExtraConfig"
> >  },
> >  "ControllerFlavor": {
> >"Type": "String",
> >"NoEcho": "false",
> >"Description": "",
> >"Label": "ControllerFlavor"
> >  }
> >}
> >   "NestedParameters": {
> >  "child.yaml": {
> >  "Parameters": {
> >"ChildConfig": {
> >"Default": [],
> >"Type": "Json",
> >"NoEcho": "false",
> >"Description": "",
> >"Label": "Child ExtraConfig"
> >}
> >  }
> >   }
> > }
>
> Are you intending on resolving parameters passed into a nested stack
> from the parent against what's defined in the nested stack's parameter
> list? I'd want NestedParameters to only list things that aren't already
> being specified to the parent.
>
> Specifically with regard to the TripleO Heat templates, there is still a
> lot of logic that needs to be applied to properly divide out parameters.
> For example, there are some things passed in from the parents to the
> nested st

[openstack-dev] [all][requirements] requirements reviews falling behind

We have quite a long list of patches to the openstack/requirements repository, 
many of which are straightforward changes to increase the minimums for 
libraries we develop within the community (meaning we are holding up projects 
from using new features, or we’re not being accurate about our current 
requirements) or to add projects to the list where the global requirements are 
synced.

https://review.openstack.org/#/q/project:openstack%2Frequirements+is:open,n,z

Patches like those should be approved quickly under normal circumstances. Is 
there a reason to hold them up, or are we just short of reviewer time for that 
repository?

Doug


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][keystone] Nova calls to Keystone


On 06/20/2015 02:46 PM, Sajeesh Cimson Sasi wrote:

Hi All,
   I need your advice for the implementation of the following 
blueprint. https://review.openstack.org/#/c/160605 
 .
   All the use cases mentioned in the blueprint have  been implemented 
and the complete code is up for review.

https://review.openstack.org/#/c/149828/
  However, we have an issue on which we need your input. In the nova 
quota api call, keystone calls are made to
  get the parent_id and the child project or sub project list. This is 
required because nova doesn't store any information
  regarding the hierarchy. Hierarchy Information is  taken during run 
time,  from keystone. Since the keystone calls are
  made inside the api call, it is not possible to give any dummy or  
fake values while writing the unit tests. If the keystone
  call was made outside the api call, we could have given fake values 
in the test cases. However,  the keystone calls for

   parent_id and child projects are made inside the api call.
  Can anyone suggest an elegant solution to this problem? What is the 
proper way to implement this ?
Did anybody encounter and solve a  similar  problem ? Many thanks 
for any suggestions!

 best regards
   sajeesh


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
If you are talking to a live Keystone server, make sure you are using 
valid data.


If you are not talking to a live keystone server in a unit test, use 
RequestsMock or equivalent (varied by project)  to handle the HTTP 
request and response.


A worst case approach is to monkey patch the Keystoneclient.  Please 
don't do that if you can avoid it;  better to provide a mock alternative.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Infra] Meeting Tuesday June 23rd at 19:00 UTC

2015-06-22 Thread Elizabeth K. Joseph

Hi everyone,

The OpenStack Infrastructure (Infra) team is having our next weekly
meeting on Tuesday June 23rd, at 19:00 UTC in #openstack-meeting

Meeting agenda available here:
https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting (anyone is
welcome to to add agenda items)

Everyone interested in infrastructure and process surrounding
automated testing and deployment is encouraged to attend.

In case you missed it or would like a refresher, the meeting minutes
and log from our last meeting are available here:

Minutes: 
http://eavesdrop.openstack.org/meetings/infra/2015/infra.2015-06-16-19.02.html
Minutes (text):
http://eavesdrop.openstack.org/meetings/infra/2015/infra.2015-06-16-19.02.txt
Log: 
http://eavesdrop.openstack.org/meetings/infra/2015/infra.2015-06-16-19.02.log.html

-- 
Elizabeth Krumbach Joseph || Lyz || pleia2

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [puppet] Installing dependent modules in functional testing

2015-06-22 Thread Colleen Murphy

On Thu, Jun 18, 2015 at 2:24 PM, Colleen Murphy  wrote:

> Now that we have the basic beaker-rspec framework set up in the modules
> and working in infra's CI, we need to start making our testing aware of
> Zuul dependencies. The infra team is facing similar challenges so it would
> be nice to work together on this. Discussions with jeblair and nibalizer
> have resulted in an etherpad[1] with some possible solutions, where Idea 1
> seems to be the most mutually beneficial. The idea is to have JJB prepare
> the node prior to testing, and to share an install script for when
> developers are running tests locally. This would install all of our modules
> and all their dependencies, not just the specific ones needed by one
> particular module, because this makes it easier to share a common thing
> between modules and frees us from worrying about implicit dependencies
> (modules needed to set up infrastructure that aren't listed explicitly in
> metadata.json) and transitive dependencies (dependencies of co-gated
> modules).
>
> I've written a possible implementation using r10k with a Puppetfile[2][3].
> R10k is generally promoted as the preferred way to manage puppet
> environments so it makes sense to use it in our tests. It also gives us the
> benefit of having a commonly defined Puppetfile that lays out versions of
> modules that are guaranteed to work together. This POC script is also using
> zuul-cloner to clone dependencies from Zuul if running in a CI environment.
> This part could be extracted out into the node prep stage later, which
> would be more in line with Idea 1 in the etherpad, but it's hard to test
> that functionality at this early stage.
>
> I'd like to create a new repo to hold this install script and Puppetfile.
> This repo could also eventually become a home for integration testing
> material, like a kind of devstack. I suggest we call it
> openstack/puppet-openstack-testing or
> openstack/puppet-openstack-integration. I would like to avoid calling it
> anything that could imply that it should be used as a composition module.
>
> Opening this up for thoughts on this implementation proposal and the repo
> name.
>
> Colleen
>
> [1] https://etherpad.openstack.org/p/puppet-git-dependencies
> [2] https://review.openstack.org/#/c/190839
> [3] https://github.com/cmurphy/puppet-openstack-dependencies
>

I proposed a patch to create a new repo for this, see
https://review.openstack.org/194287

Colleen
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [qpid] [zmq] [RabbitMQ] [oslo] Pending deprecation of driver(s).


On 06/20/2015 10:28 AM, Flavio Percoco wrote:




As promissed: https://review.openstack.org/#/c/193804/

Cheers, 

You can't deprecate a driver without providing a viable alternative.

Right now, QPID is the only driver that supports  Kerberos.

TO support Kerberos, tyou need support for the GSSAPI library, which is 
usually done via support for SASL.  Why is it so convoluted...historical...


We've talked with both teams (I work with Ken) and I think Proton is 
likely going to be the first to have support.  The folks working on 
Rabbit have the double hurdle of getting SASL support into Erlang first, 
and then support for SASL into Rabbit. They've indicated a preference 
for getting it in to the AMQP 1.0 driver, and not bothering with the 
exisiting, but, check me on this, the Oso.Messaging  code only support 
the pre 1.0 Rabbit.



So..until we have a viable alternative, please leave QPID alone. I've 
not been bothering people about it, as there seems to be work to get 
ahead, but until either Rabbit or  Proton support Kerberos, I need QPID 
as is.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] M Naming Poll ended - results still need to clear legal

2015-06-22 Thread Clint Byrum

Excerpts from Clay Gerrard's message of 2015-06-22 10:30:49 -0700:
> http://civs.cs.cornell.edu/cgi-bin/results.pl?id=E_4983776e190c8dbc
> 
> how is the top pick not the author of the book of five rings [1]
> 

Agreed. I don't think people fully appreciated the reputation of Mr.
Musashi. ;)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Oslo][Automaton] Proposal for new core (ruby loo)

2015-06-22 Thread Joshua Harlow


Greetings all stackers,

I propose that we add Ruby Loo[1] to the automaton-core team.

Ruby has been actively contributing to automaton[2] for a while now, 
both in helping make automaton better via reviews (she has been involved 
from the start in helping sanity check that code) and helping get it 
integrated into ironic. She has provided quality reviews and is doing an 
awesome job with the various automaton concepts and helping make 
automaton the best it can be!


Overall I think she would make a great addition to the core review team.

Please respond with +1/-1.

Thanks much!

[1] https://launchpad.net/~rloo
[2] https://github.com/openstack/automaton

--

Joshua Harlow

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] can we get ceilometermiddleware to use a config file instead of transport_url?

On 06/22/2015 01:15 PM, Michael Krotscheck wrote:
> Having _just_ done this, a couple of suggestions.
> 
> - If the middleware is NOT optional - that is, it provides some kind of
> a fundamental component or specification of the API, like ETag caching,
> CORS, or DB Session management - then the middleware should be added
> during the app initialization routine, likely in the app factory. In
> this case, we have tight control over the initialization order, and can
> make sure that oslo.config is there first. Yay config.
> 
> - If the middleware _is_ optional, then I really feel this problem is
> solved by pastedeploy's filter_factory pattern. It lets you define as
> many required parameters as you want, and spits back a middleware
> instance. That way you have the freedom to set whatever configuration
> options you want, or omit the middleware altogether.
> 
> http://pythonpaste.org/deploy/#paste-filter-factory
> 
> Ultimately, what you should want is to ask a factory method for a thing
> with some configuration options, and it spits an instance back at you.
> If your project doesn't support pastedeploy, that limits your options
> (ahem-ironic-ahem). Otherwise, it's really a team decision on whether
> something is a 'Fundamental feature' or something that's optional.
> 
> In the case of Ceilometer? Sorry, but I think it's optional. You should
> be able to run any component of openstack without ceilometer. So I don't
> really think it should depend on oslo_config.

Ok, here is where I'm confused. Ceilometer middleware *already* depends
on oslo_config.

https://github.com/openstack/ceilometermiddleware/blob/master/ceilometermiddleware/swift.py#L42

And it does stuff with it here:

https://github.com/openstack/ceilometermiddleware/blob/master/ceilometermiddleware/swift.py#L107

But regardless, it uses oslo_messaging, which uses oslo_config. So it
seems like the only issue at hand is middleware retriggering conf parsing.

Because the current model of requiring simultaneous code lands in
oslo_messaging and ceilometermiddleware, and simultaneous config updates
in every installer and config management system to configure the same
thing in 2 different ways, does seem like a long term win. And honestly
just prohibits most of what people seem to want to do with new messaging
approaches.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] M Naming Poll ended - results still need to clear legal

2015-06-22 Thread Clay Gerrard

http://civs.cs.cornell.edu/cgi-bin/results.pl?id=E_4983776e190c8dbc

how is the top pick not the author of the book of five rings [1]

-Clay

1. https://en.wikipedia.org/wiki/The_Book_of_Five_Rings



On Mon, Jun 22, 2015 at 7:07 AM, Monty Taylor  wrote:

> Hey all!
>
> The M naming poll has concluded. I'd say "and the winner is ..." except
> we still need to get the winning choice(s) vetted for legal entanglements.
>
> Feel free to go and look at the results, they are publicly available -
> but please DON'T start making t-shirts or having parties (ok, have as
> many parties as you want) until we've gotten the alls-clear from the
> trademark folks. I've sent the list to them already, so they're working
> on it furiously as we speak.
>
> As soon as I get the word back, I will send out another announcement
> with the official result.
>
> Thanks!
> Monty
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][ceilometer] proposal to send bulk hypervisor stats data in periodic notifications

2015-06-22 Thread Maish Saidel-Keesing


On 06/22/15 20:10, Daniel P. Berrange wrote:

On Mon, Jun 22, 2015 at 11:10:40AM -0500, Kevin L. Mitchell wrote:

On Sat, 2015-06-20 at 21:35 +0100, Daniel P. Berrange wrote:

In general I would say that is an unsupported deployment scenario to
have other random virt guests running on a nova compute node.

On the other hand, this is exactly how compute nodes themselves are
often deployed—a random guest on the hypervisor node…

In a devstack env maybe, but in production we expect the hypervisor to
be exclusively used by Nova, or things Nova uses.

Regards,
Daniel


+1 to that!!

I think it would be safe to say that in any production environment I 
would run, Nova would control the instances exclusively.


--
Best Regards,
Maish Saidel-Keesing

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [heat][tripleo]Recursive validation for easier composability

2015-06-22 Thread Jay Dobies




On 06/22/2015 12:19 PM, Steven Hardy wrote:

Hi all,

Lately I've been giving some thought to how we might enable easier
composability, and in particular how we can make it easier for folks to
plug in deeply nested optional extra logic, then pass data in via
parameter_defaults to that nested template.

Here's an example of the use-case I'm describing:

https://review.openstack.org/#/c/193143/5/environments/cinder-netapp-config.yaml

Here, we want to allow someone to easily turn on an optional configuration
or feature, in this case a netapp backend for cinder.


I think the actual desired goal is bigger than just optional 
configuration. I think it revolves more around choosing a nested stack 
implementation for a resource type and how to manage custom parameters 
for that implementation. We're getting into the territory here of having 
a parent stack defining an API that nested stacks can plug into. I'd 
like to have some sort of way of deriving that information instead of 
having it be completely relegated to outside documentation (but I'm 
getting off topic; at the end I mention how I want to do a better write 
up of the issues Tuskar has faced and I'll elaborate more there).



The parameters specific to this feature/configuration only exist in the
nested cinder-netapp-config.yaml template, then parameter_defaults are used
to wire in the implementation specific data without having to pass the
values through every parent template (potentially multiple layers of
nesting).

This approach is working out OK, but we're missing an interface which makes
the schema for parameters over the whole tree available.

>

This is obviously
a problem, particularly for UI's, where you really need a clearly defined
interface for what data is required, what type it is, and what valid values
may be chosen.


I think this is going to be an awesome addition to Heat. As you alluded 
to, we've struggled with this in TripleO. The parameter_defaults works 
to circumvent the parameter passing, but it's rough from a user 
experience point of view since getting the unified list of what's 
configurable is difficult.



I'm considering an optional additional flag to our template-validate API
which allows recursive validation of a tree of templates, with the data
returned on success to include a tree of parameters, e.g:

heat template-validate -f parent.yaml -e env.yaml --show-nested
{
   "Description": "The Parent",
   "Parameters": {
 "ParentConfig": {
   "Default": [],
   "Type": "Json",
   "NoEcho": "false",
   "Description": "",
   "Label": "ExtraConfig"
 },
 "ControllerFlavor": {
   "Type": "String",
   "NoEcho": "false",
   "Description": "",
   "Label": "ControllerFlavor"
 }
   }
  "NestedParameters": {
 "child.yaml": {
 "Parameters": {
   "ChildConfig": {
   "Default": [],
   "Type": "Json",
   "NoEcho": "false",
   "Description": "",
   "Label": "Child ExtraConfig"
   }
 }
  }
}


Are you intending on resolving parameters passed into a nested stack 
from the parent against what's defined in the nested stack's parameter 
list? I'd want NestedParameters to only list things that aren't already 
being specified to the parent.


Specifically with regard to the TripleO Heat templates, there is still a 
lot of logic that needs to be applied to properly divide out parameters. 
For example, there are some things passed in from the parents to the 
nested stacks that are kinda namespaced by convention, but its not a 
hard convention. So to try to group the parameters by service, we'd have 
to look at a particular NestedParameters section and then also add in 
anything from the parent that applies to that service. I don't believe 
we can use parameter groups to correlate them (we might be able to, or 
that might be its own improvement).


I realize that's less of a Heat issue and more of a THT issue, but I 
figured I'd bring it up anyway.



This implies that we also need to pass the "files" map to the validate API,
like we currently do for create (we already pass the environment, although
it's not really doing much beyond providing parameters for the parent stack
AFAICT, we completely skip validating TemplateResources because the files
aren't passed):

https://github.com/openstack/heat/blob/master/heat/engine/service.py#L873

Before I go ahead and spend time writing a spec/code for this, what do
folks think about enhancing validate like this?  Is there an alternative,
for example adding a parameters schema output to stack-preview?


For what it's worth, I'd rather see this as a spec before code. There 
are a lot of complications we hit in Tuskar in trying to make 
configuring the overcloud through THT user-friendly. This is one part of 
it, but there are others. I'd like to have them all talked out and see 
what the larger group of changes are.


For example, take the cinder-netapp-config example you ment

Re: [openstack-dev] [puppet] Clarification of 'Puppet Modules' Project Scope

2015-06-22 Thread Matt Fischer

I'm torn on this. Pedantically option A makes the most sense, but option B
gives us more control over the supporting modules. I like having OpenStack
CI run on vswitch and ceph rather than the typical github merge process.

On Mon, Jun 22, 2015 at 11:05 AM, Richard Raseley 
wrote:

> I am currently hoping to build consensus (or seek clarity if I am the only
> one with this question) about the appropriate scope for our 'Puppet
> Modules' project.
>
> The question in my mind is if we:
>
> A) Only include those modules which represent a 1:1 mapping with other
> OpenStack projects.
>
> B) Also include those modules which provide 'supporting' infrastructure to
> OpenStack components.
>
> To be totally transparent, this came to mind for me because I am currently
> working with the folks at Midokura to publish a module which can be used to
> configure their open source Midonet SDN for Neutron and I was contemplating
> whether or not it would be reasonable to be part of the project.
>
> FWIW, we have carried over the 'puppet-vswitch' repository over with us as
> part of the move (which would align with option B), but I didn't want to
> assume that was intended to be precedent setting.
>
> Regards,
>
> Richard
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][ceilometer] proposal to send bulk hypervisor stats data in periodic notifications

2015-06-22 Thread Kevin L. Mitchell

On Mon, 2015-06-22 at 18:10 +0100, Daniel P. Berrange wrote:
> On Mon, Jun 22, 2015 at 11:10:40AM -0500, Kevin L. Mitchell wrote:
> > On Sat, 2015-06-20 at 21:35 +0100, Daniel P. Berrange wrote:
> > > In general I would say that is an unsupported deployment scenario to
> > > have other random virt guests running on a nova compute node.
> > 
> > On the other hand, this is exactly how compute nodes themselves are
> > often deployed—a random guest on the hypervisor node…
> 
> In a devstack env maybe, but in production we expect the hypervisor to
> be exclusively used by Nova, or things Nova uses.

In production, at least in some Xen deployments, the hypervisor does not
run the nova-compute service; instead, we have a VM specifically set
aside for running the nova-compute.  This is done because the hypervisor
(in older Xen deployments, at least) has an old version of Python, and
because some operations are most conveniently done by a co-located
compute node.
-- 
Kevin L. Mitchell 
Rackspace


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] can we get ceilometermiddleware to use a config file instead of transport_url?

2015-06-22 Thread Michael Krotscheck

Having _just_ done this, a couple of suggestions.

- If the middleware is NOT optional - that is, it provides some kind of a
fundamental component or specification of the API, like ETag caching, CORS,
or DB Session management - then the middleware should be added during the
app initialization routine, likely in the app factory. In this case, we
have tight control over the initialization order, and can make sure that
oslo.config is there first. Yay config.

- If the middleware _is_ optional, then I really feel this problem is
solved by pastedeploy's filter_factory pattern. It lets you define as many
required parameters as you want, and spits back a middleware instance. That
way you have the freedom to set whatever configuration options you want, or
omit the middleware altogether.

http://pythonpaste.org/deploy/#paste-filter-factory

Ultimately, what you should want is to ask a factory method for a thing
with some configuration options, and it spits an instance back at you. If
your project doesn't support pastedeploy, that limits your options
(ahem-ironic-ahem). Otherwise, it's really a team decision on whether
something is a 'Fundamental feature' or something that's optional.

In the case of Ceilometer? Sorry, but I think it's optional. You should be
able to run any component of openstack without ceilometer. So I don't
really think it should depend on oslo_config.

Michael

On Mon, Jun 22, 2015 at 7:59 AM Doug Hellmann  wrote:

> Excerpts from Doug Hellmann's message of 2015-06-22 10:50:06 -0400:
> > Excerpts from Sean Dague's message of 2015-06-22 09:02:02 -0400:
> > > On 06/22/2015 08:58 AM, Doug Hellmann wrote:
> > > > Excerpts from Sean Dague's message of 2015-06-22 08:08:04 -0400:
> > > >> In extracting the contract for RPC backends in devstack (to have
> most of
> > > >> them live in plugins) one bit of an edge case was discovered.
> > > >>
> > > >>
> https://github.com/openstack-dev/devstack/blob/master/lib/swift#L388
> > > >>
> > > >> The connection to the RPC mechanism from ceilometermiddleware
> inside of
> > > >> swift uses a transport url instead of an oslo.messaging config
> block.
> > > >>
> > > >> ceilometermiddleware requires oslo.config and oslo.messaging, so it
> > > >> seems like it could use an oslo config block instead.
> > > >>
> > > >> One of the reasons why this seems like a better idea is that not
> all the
> > > >> properties of a messaging connection can be encoded in just a url
> today.
> > > >> For instance, Rabbit can specify heartbeating params -
> > > >>
> https://github.com/openstack-dev/devstack/blob/master/lib/rpc_backend#L282-L287
> ,
> > > >> and zmq needs matchmaker info -
> > > >>
> https://github.com/openstack-dev/devstack/blob/master/lib/rpc_backend#L282-L287
> > > >>
> > > >> (Note: zmq is not currently able to be configured for swift +
> ceilometer
> > > >> today
> > > >>
> https://github.com/openstack-dev/devstack/blob/master/lib/rpc_backend#L282-L287
> > > >> and given what it needs in it's config, it's not clear that it
> would be
> > > >> reasonable to do so.)
> > > >>
> > > >> Could we deprecate the use of tranport_url in ceilometermiddleware
> and
> > > >> move to an actual oslo.config file somewhere instead? That would
> bring
> > > >> it in line with the rest of the RPC configuration for services, and
> > > >> ensure that all connections in a cluster have access to all the same
> > > >> options.
> > > >
> > > > Swift doesn't use oslo.config, so we might need to make the
> middleware
> > > > support both configuration mechanisms.
> > >
> > > Right, but ceilometermiddleware does. Can't it load it's config from a
> > > side channel? Or is it only possible for it to load it's config from
> the
> > > same file as swift?
> >
> > Library code that uses oslo.config relies on having an initialized
> > configuration object, which means something has specified the files
> > to be loaded, handled command line argument parsing, etc. We wouldn't
> > want the middleware doing that, because doing it properly requires
> > knowing the name of the application it's running in and what some
> > of the application-specific defaults should be, and those are usually
> > best left to the application.
>
> We had a similar issue with the CORS middleware needing a config object.
> Perhaps we need a piece of middleware to set up the config using
> parameters coming in from the paste.ini (application, name, default
> config file, etc.). If that middleware sets up the global config
> instance, other middleware could be written to rely on it being earlier
> in the pipline.
>
> Doug
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage

Re: [openstack-dev] [nova][ceilometer] proposal to send bulk hypervisor stats data in periodic notifications

2015-06-22 Thread Daniel P. Berrange

On Mon, Jun 22, 2015 at 11:10:40AM -0500, Kevin L. Mitchell wrote:
> On Sat, 2015-06-20 at 21:35 +0100, Daniel P. Berrange wrote:
> > In general I would say that is an unsupported deployment scenario to
> > have other random virt guests running on a nova compute node.
> 
> On the other hand, this is exactly how compute nodes themselves are
> often deployed—a random guest on the hypervisor node…

In a devstack env maybe, but in production we expect the hypervisor to
be exclusively used by Nova, or things Nova uses.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [puppet] Clarification of 'Puppet Modules' Project Scope

2015-06-22 Thread Richard Raseley

I am currently hoping to build consensus (or seek clarity if I am the 
only one with this question) about the appropriate scope for our 'Puppet 
Modules' project.


The question in my mind is if we:

A) Only include those modules which represent a 1:1 mapping with other 
OpenStack projects.


B) Also include those modules which provide 'supporting' infrastructure 
to OpenStack components.


To be totally transparent, this came to mind for me because I am 
currently working with the folks at Midokura to publish a module which 
can be used to configure their open source Midonet SDN for Neutron and I 
was contemplating whether or not it would be reasonable to be part of 
the project.


FWIW, we have carried over the 'puppet-vswitch' repository over with us 
as part of the move (which would align with option B), but I didn't want 
to assume that was intended to be precedent setting.


Regards,

Richard

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [mistral] Team meeting minutes - 22/06/2015

2015-06-22 Thread Renat Akhmerov

Thanks everyone for participating our meeting today.

Meeting minutes: 
http://eavesdrop.openstack.org/meetings/mistral/2015/mistral.2015-06-22-16.00.html
 

Meeting log: 
http://eavesdrop.openstack.org/meetings/mistral/2015/mistral.2015-06-22-16.00.log.html
 


The next meeting will be on June 29 at the same time. Propose your topics at 
https://wiki.openstack.org/wiki/Meetings/MistralAgenda 
.

Renat Akhmerov
@ Mirantis Inc.



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] FYI - dropping non RabbitMQ support in devstack

Thomas,

This may be of interest to you as well:
http://specs.openstack.org/openstack/openstack-specs/specs/supported-messaging-drivers.html

On Mon, Jun 22, 2015 at 12:57 PM, Davanum Srinivas  wrote:
> Thomas,
>
> here's a review for qpid
> https://review.openstack.org/#/c/193804/
>
> -- dims
>
> On Mon, Jun 22, 2015 at 12:42 PM, Thomas Goirand  wrote:
>> On 06/16/2015 03:22 PM, Sean Dague wrote:
>>> FYI,
>>>
>>> One of the things that came out of the summit for Devstack plans going
>>> forward is to trim it back to something more opinionated and remove a
>>> bunch of low use optionality in the process.
>>>
>>> One of those branches to be trimmed is all the support for things beyond
>>> RabbitMQ in the rpc layer. RabbitMQ is what's used by 95%+ of our
>>> community, that's what the development environment should focus on.
>>>
>>> The patch to remove all of this is here -
>>> https://review.openstack.org/#/c/192154/. Expect this to merge by the
>>> end of the month. If people are interested in non RabbitMQ external
>>> plugins, now is the time to start writing them. The oslo.messaging team
>>> already moved their functional test installation for alternative
>>> platforms off of devstack, so this should impact a very small number of
>>> people.
>>>
>>>   -Sean
>>
>> I agree with this, if and only if, we also remove Qpid and such from
>> itoslo.messaging. It doesn't make sense if we only go half of the way,
>> and only remove it from Devstack.
>>
>> Cheers,
>>
>> Thomas Goirand (zigo)
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> --
> Davanum Srinivas :: https://twitter.com/dims



-- 
Davanum Srinivas :: https://twitter.com/dims

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] FYI - dropping non RabbitMQ support in devstack

Thomas,

here's a review for qpid
https://review.openstack.org/#/c/193804/

-- dims

On Mon, Jun 22, 2015 at 12:42 PM, Thomas Goirand  wrote:
> On 06/16/2015 03:22 PM, Sean Dague wrote:
>> FYI,
>>
>> One of the things that came out of the summit for Devstack plans going
>> forward is to trim it back to something more opinionated and remove a
>> bunch of low use optionality in the process.
>>
>> One of those branches to be trimmed is all the support for things beyond
>> RabbitMQ in the rpc layer. RabbitMQ is what's used by 95%+ of our
>> community, that's what the development environment should focus on.
>>
>> The patch to remove all of this is here -
>> https://review.openstack.org/#/c/192154/. Expect this to merge by the
>> end of the month. If people are interested in non RabbitMQ external
>> plugins, now is the time to start writing them. The oslo.messaging team
>> already moved their functional test installation for alternative
>> platforms off of devstack, so this should impact a very small number of
>> people.
>>
>>   -Sean
>
> I agree with this, if and only if, we also remove Qpid and such from
> itoslo.messaging. It doesn't make sense if we only go half of the way,
> and only remove it from Devstack.
>
> Cheers,
>
> Thomas Goirand (zigo)
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Davanum Srinivas :: https://twitter.com/dims

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Modular L2 Agent

2015-06-22 Thread Sean M. Collins

On Mon, Jun 22, 2015 at 10:47:39AM EDT, Salvatore Orlando wrote:
> I would probably start with something for enabling the L2 agent to process
> "features" such as QoS and security groups, working on the OVS agent, and
> then in a second step abstract a driver interface for communicating with
> the data plane. But I honestly do not know if this will keep the work too
> "OVS-centric" and therefore won't play well with the current efforts to put
> linux bridge on par with OVS in Neutron. For those question we should seek
> an answer from our glorious reference control plane lieutenant, and perhaps
> also from Sean Collins, who's coordinating efforts around linux bridge
> parity.

I think that what Salvatore has suggested is good. If we start creating
good API contracts, and well defined interfaces in the reference control
plane agents - this is a good first step. Even if we start off by doing
this just for the OVS agent, that'll be a good template for what we
would need to do for any agent-driven L2 implementation - and it could
easily be re-used by others.

To be honest, if you squint hard enough there really are very few
differences between the OVS agent and the Linux Bridge agent does -
the parts that handle control plane communication, processing
data updates, and so forth should all be very similar.

They only become different at the lower
levels where it's brctl/ip2 vs. ovs-vsctl/of-ofctl CLI calls - so why
maintain two separate agent implementations when quite a bit of what
they do is functionally identical?

-- 
Sean M. Collins

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] FYI - dropping non RabbitMQ support in devstack

2015-06-22 Thread Thomas Goirand

On 06/16/2015 03:22 PM, Sean Dague wrote:
> FYI,
> 
> One of the things that came out of the summit for Devstack plans going
> forward is to trim it back to something more opinionated and remove a
> bunch of low use optionality in the process.
> 
> One of those branches to be trimmed is all the support for things beyond
> RabbitMQ in the rpc layer. RabbitMQ is what's used by 95%+ of our
> community, that's what the development environment should focus on.
> 
> The patch to remove all of this is here -
> https://review.openstack.org/#/c/192154/. Expect this to merge by the
> end of the month. If people are interested in non RabbitMQ external
> plugins, now is the time to start writing them. The oslo.messaging team
> already moved their functional test installation for alternative
> platforms off of devstack, so this should impact a very small number of
> people.
> 
>   -Sean

I agree with this, if and only if, we also remove Qpid and such from
itoslo.messaging. It doesn't make sense if we only go half of the way,
and only remove it from Devstack.

Cheers,

Thomas Goirand (zigo)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Cinder] Implementation of ABC MetaClasses

2015-06-22 Thread Marc Koderer


Am 20.06.2015 um 01:59 schrieb John Griffith :
> * The BaseVD represents the functionality we require from all drivers.
> Yep
>  
> * The additional ABC classes represent features that are not required yet.
>   * These are represented by classes because some features require a
> bundle of methods for it to be fulfilled. Consistency group is an
> example. [2]
> 
> Sure, I suppose that's fine for things like CG and Replication.  Although I 
> would think that if somebody implemented something optional like CG's that 
> they should be able to figure out they need all of the "cg" methods, it's 
> kinda like that warning on ladders to not stand on the very top rung.  By the 
> way, maybe we should discuss the state of "optional API methods" at the 
> mid-cycle.
>  
>   * Any driver that wishes to mark their driver as supporting a
> non-required feature inherits this feature and fulfills the required
> methods.
> 
> Yeah... ok, I guess.
> 
> * After communication is done on said feature being required, there
> would be time for driver maintainers to begin supporting it.
>   * Eventually that feature would disappear from it's own class and be
> put in the BaseVD. Anybody not supporting it would have a broken
> driver, a broken CI, and eventually removed from the release.
> 
> Sure, I guess my question is what's the real value in this intermediate 
> step.  The bulk of these are things that I'd argue shouldn't be optional 
> anyway (snapshots, transfers, manage, extend, retype and even migrate).  
> Snapshots in particular I find surprising to be considered as "optional“.

Reducing the number of classes/optional functions sounds good to me.
I think it’s quite valuable to discuss what are the mandatory functions
of a cinder driver. Before ABC nobody really cared because all drivers simply 
raised an run-time exception :)

>  
> * Some people had thoughts of having a way generate a matrix with this
> information, which I dislike the idea of.
> 
> Yeah, back to the dreaded "Matrix", I can't imagine anything worse for 
> Cinder than a matrix of supported API calls on a driver per driver basis. 

I agree that such a matrix tool isn’t the finial goal.
But it helps to see which functions are already "common“ and which are 
"optional“.
Just try it [1] ;)

Regards
Marc

[1]: https://review.openstack.org/#/c/160346/
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [heat][tripleo]Recursive validation for easier composability

2015-06-22 Thread Steven Hardy

Hi all,

Lately I've been giving some thought to how we might enable easier
composability, and in particular how we can make it easier for folks to
plug in deeply nested optional extra logic, then pass data in via
parameter_defaults to that nested template.

Here's an example of the use-case I'm describing:

https://review.openstack.org/#/c/193143/5/environments/cinder-netapp-config.yaml

Here, we want to allow someone to easily turn on an optional configuration
or feature, in this case a netapp backend for cinder.

The parameters specific to this feature/configuration only exist in the
nested cinder-netapp-config.yaml template, then parameter_defaults are used
to wire in the implementation specific data without having to pass the
values through every parent template (potentially multiple layers of
nesting).

This approach is working out OK, but we're missing an interface which makes
the schema for parameters over the whole tree available.  This is obviously
a problem, particularly for UI's, where you really need a clearly defined
interface for what data is required, what type it is, and what valid values
may be chosen.

I'm considering an optional additional flag to our template-validate API
which allows recursive validation of a tree of templates, with the data
returned on success to include a tree of parameters, e.g:

heat template-validate -f parent.yaml -e env.yaml --show-nested
{
  "Description": "The Parent",
  "Parameters": {
"ParentConfig": {
  "Default": [],
  "Type": "Json",
  "NoEcho": "false",
  "Description": "",
  "Label": "ExtraConfig"
},
"ControllerFlavor": {
  "Type": "String",
  "NoEcho": "false",
  "Description": "",
  "Label": "ControllerFlavor"
}
  }
 "NestedParameters": {
"child.yaml": {
"Parameters": {
  "ChildConfig": {
  "Default": [],
  "Type": "Json",
  "NoEcho": "false",
  "Description": "",
  "Label": "Child ExtraConfig"
  }
}
 }
}

This implies that we also need to pass the "files" map to the validate API,
like we currently do for create (we already pass the environment, although
it's not really doing much beyond providing parameters for the parent stack
AFAICT, we completely skip validating TemplateResources because the files
aren't passed):

https://github.com/openstack/heat/blob/master/heat/engine/service.py#L873

Before I go ahead and spend time writing a spec/code for this, what do
folks think about enhancing validate like this?  Is there an alternative,
for example adding a parameters schema output to stack-preview?

Also interested in TripleO feedback on this, particularly to what extent do
we think this could potentially replace some of the template introspection
currently done via Tuskar?

Thanks,

Steve

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [requirements] re-ordered specifiers in requirements.txt

2015-06-22 Thread Matthew Thode

On 06/22/2015 05:07 AM, Sean Dague wrote:
> On 06/22/2015 04:33 AM, Robert Collins wrote:
>> You may have noticed the every repo has just been spammed with some
>> no-op changes where specifiers are re-ordered such as in
>> https://review.openstack.org/#/c/193973/1/test-requirements.txt
>>
>> where
>> sphinx>=1.1.2,!=1.2.0,!=1.3b1,<1.3
>> ->
>> sphinx!=1.2.0,!=1.3b1,<1.3,>=1.1.2
>>
>> Whats happening is that the requirements code now parses and
>> regenerates the specifiers as I mentioned last week. So this is
>> normal, but one-time.
> 
> So, while I get that it's all equivalent, it's pretty unhuman friendly
> to specify this in this order. Can we get >(=)* first, != next, and
> <(=)* last so that it reads as a human range?
> 
>   -Sean
> 
that would be awesome if changed like that.

-- 
Matthew Thode

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][ceilometer] proposal to send bulk hypervisor stats data in periodic notifications

2015-06-22 Thread Kevin L. Mitchell

On Sat, 2015-06-20 at 21:35 +0100, Daniel P. Berrange wrote:
> In general I would say that is an unsupported deployment scenario to
> have other random virt guests running on a nova compute node.

On the other hand, this is exactly how compute nodes themselves are
often deployed—a random guest on the hypervisor node…
-- 
Kevin L. Mitchell 
Rackspace


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [mistral] Proposing Lingxian Kong as a core reviewer

2015-06-22 Thread Nikolay Makhotkin

Hi,

+1 for Lingxian!

On Mon, Jun 22, 2015 at 7:08 PM, Nikolay Makhotkin 
wrote:

> Hi,
>
> +1 for Lingxian!
>
> --
> Best Regards,
> Nikolay
>



-- 
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Specify a domain in mapping rules

2015-06-22 Thread J . Pablo Martín Cobos

I'm sorry it does not work still with this commit:

https://review.openstack.org/#/c/181007/

I think this commit is for another feature, we does not use keystone like
idp. We use shibboleth like idp:

http://docs.openstack.org/developer/keystone/configure_federation.html

According to the next guide I could have a rule like this:

https://github.com/openstack/keystone-specs/blob/master/api/v3/identity-api-v3-os-federation-ext.rst#mappings


{
 "user": {
 "name": "username"
 "domain": {
 "name": "domain_name"
 }
 }
}

But it does not work,  :-(

Do I need to merge another commit? Could you help me? Is it a bug?

Thank you so much for your help!


Best regards,






2015-06-19 22:03 GMT+02:00 Brant Knudson :

>
> You might need these changes, which are proposed but not merged:
> https://review.openstack.org/#/q/status:open+project:openstack/keystone+branch:stable/kilo+topic:bug/1442787,n,z
>
> Salut, Brant
>
> On Thu, Jun 18, 2015 at 6:04 AM, J. Pablo Martín Cobos 
> wrote:
>
>> Hi all,
>>
>> I'm a Python/Django software developer [1].  We have to do an integration
>> of OpenStack and a Shibboleth IdP in my current project.
>>
>> This is not a easy feature to configure... but finally we got it :-) Now
>> we only need specify a domain for the user different to the "Federated"
>> default domain. This domain depends on an attribute from the IdP.
>>
>> Is it possible to get with stable/kilo branch? Is it a feature for the
>> next  release? [2] These are my rules:
>>
>> [
>> {
>> "local": [
>> {
>> "user": {
>> "name": "{0}",
>> "domain": {
>> "name": "{1}"
>> }
>> }
>> },
>> {
>> "group": {
>> "id": "0ff59ec2f97646eb9350fe75478f9600"
>> }
>> }
>> ],
>> "remote": [
>> {
>> "type": "identity"
>> },
>> {
>> "type": "domain"
>> }
>> ]
>> }
>> ]
>>
>> I have tested with a lot of rules with little changes:
>>
>> "domain": {
>> "name": "Default"
>> }
>>
>> or
>>
>> "domain": {
>> "id": "default"
>> }
>>
>> or
>>
>> "domain": {
>> "id": "14321243"
>> }
>>
>> etc... and this never works :-(
>>
>> Could you help me?
>>
>> REF's
>>
>> 1. https://github.com/goinnn
>> 2.
>> https://github.com/openstack/keystone-specs/blob/master/api/v3/identity-api-v3-os-federation-ext.rst
>>
>> Thanks a lot!!,
>>
>> --
>>
>> Pablo Martín
>> Software engineer
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Tracking optional requirements

2015-06-22 Thread Jeremy Stanley

On 2015-06-22 21:05:11 +1000 (+1000), Michael Still wrote:
> Sure, except where the thing isn't in pip... I just learned tonight
> that the libosinfo thing in the review at the start of this thread is
> a c library and some gobject black magic. I think we want it, but it
> will never work as a pip dependancy. So, I think something human
> readable is still required?

I'm hoping to solve this via bindep[1], and will propose a
bindep-format requirements list to nova once I have the package
caching in place for that on our workers and the current
experimental job updated to make use of per-project bindep lists.

[1] http://git.openstack.org/cgit/openstack-infra/bindep/
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [qa] Tempest bug triage questions

2015-06-22 Thread Matthew Treinish

On Mon, Jun 22, 2015 at 11:41:56AM -0400, David Kranz wrote:
> On 06/22/2015 10:15 AM, Yaroslav Lobankov wrote:
> >Hello everyone,
> >
> >I have some questions about the bug triage procedure for Tempest:
> >
> >1. Some bugs in Tempest have status "Fix committed". Should we move
> >statuses of these bugs to "Fix released"?
> Yes, tempest doesn't have the kind of releases where Fix committed makes
> sense.

Well, except for tempest-lib bugs. The tempest bug tracker is used for both
tempest and tempest-lib. So if there are bugs marked as fix committed against
tempest-lib we need to wait until they are included in a release before we mark
it as fix released. 

But, in general tempest bugs should go straight from in-progress to fix-released
on commit. There was a brief period where we didn't do that and you might be
catching leftovers from that time.

> >
> >2. Many bugs have statuses "In progress", but patches for these bugs have
> >-1 from someone (or their workflow is -1)
> >and it looks like these patches are abandoned, while statuses of such
> >patches are "Review in progress".
> >What should we do with such bugs?
> This is kind of tricky without the project having a "manager". I think we
> usually ping with a comment in the bug to see what the holdup is. If there
> is no response, we would set it back to Triaged of Confirmed.

Well if the patch is abandoned, I thought it does this automagically. But, if
the patch is abandoned and it's still in progress I'd move it back to a 
different
state to reflect the bug's real current state. -1 reviews are a different matter
though, that's ongoing work. (although maybe slowly)

> >
> >3. What should we do with bugs like this [1]? It says that
> >TimeoutException occurred, but the log of the test is unavailable
> >by the link already. I don't know how to reproduce the issue, besides
> >the log of the test is unavailable. What should I do with this bug?
> This bug is about how tempest should handle cases where a resource deletion
> fails. I think it is a legitimate issue though the answer is not clear given
> a gate that may be very slow at times.

I think the question was more about if the bug doesn't have a lot of details and
the gate links are dead, because of the log server expiration window. In this
case I normally mark the bug as incomplete and ask the submitter for more
details. Just a stack trace and/or a link to a gate failure often isn't enough
for a bug report long term. For short term fixes it's often just done as a quick
way to move forward so we track it for an e-r query and fix it. But, if it ends
up sitting for a while untriaged without an e-r query then it's not a good bug
report and it needs more details.

-Matt Treinish

> >[1] https://bugs.launchpad.net/tempest/+bug/1322011

pgpR8LI7SthYC.pgp
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [mistral] Proposing Lingxian Kong as a core reviewer

2015-06-22 Thread Renat Akhmerov

Team,

I’d like to propose Lingxian Kong (xylan_kong) to Mistral core team.

Over the last ~6 months Lingxian showed really strong technical skills, 
provided a series of very high quality reviews and has been very active in our 
IRC channel. He also has a solid vision about Mistral future. His energy is 
desire to be active is really hard to ignore :)

I’d like to ask you to support me in this proposal.

Thanks

Renat Akhmerov
@ Mirantis Inc.




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposal of nova-hyper driver

2015-06-22 Thread Peng Zhao

Thanks John.
I’m also not sure what the future would be, but I’d say that it would be nice to
have a hybrid OpenStack cluster of both VM/App-Container flavor. And yes, it is
more about a unified model between Nova and Magnum.
Best, Peng
- Hyper - Make VM run like 
Container

On Mon, Jun 22, 2015 at 5:10 PM, John Garbutt < j...@johngarbutt.com > wrote:
On 22 June 2015 at 09:18, Sahid Orentino Ferdjaoui
< sahid.ferdja...@redhat.com > wrote:
> On Sun, Jun 21, 2015 at 07:18:10PM +0300, Joe Gordon wrote:
>> On Fri, Jun 19, 2015 at 12:55 PM, Peng Zhao  wrote:
>>
>> > Hi, all,
>> >
>> > I would like to propose nova-hyper driver:
>> > https://blueprints.launchpad. net/nova/+spec/nova-hyper .
>> >
>> > - What is Hyper?
>> > Put simply, Hyper is a hypervisor-agnostic Docker runtime. It is
>> > similar to Intel’s ClearContainer, allowing to run a Docker image with any
>> > hypervisor.
>> >
>> > - Why Hyper driver?
>> > Given its hypervisor nature, Hyper makes it easy to integrate with
>> > OpenStack ecosystem, e.g. Nova, Cinder, Neutron
>> >
>> > - How to implement?
>> > Similar to nova-docker driver. Hyper has a daemon “hyperd” running on
>> > each physical box. hyperd exposed a set of REST APIs. Integrating Nova with
>> > the APIs would do the job.

For clarity, we are yet to accept the nova-docker driver into the Nova
project, due to various concerns about its potential future direction.
Hopefully we should get a more final answer on that soon.

>> > - Roadmap
>> > Integrate with Magnum & Ironic.
>> >
>> >
>> This sounds like a better fit for something on top of Nova such as Magnum
>> then as a Nova driver.

+1

On the surface, it feels like a possible Magnum driver.
Although I am far from certain that its an exact match.
But I think that would be a better starting point than Nova.

>> Nova only supports things that look like 'VMs'. That includes bare metal,
>> and containers, but it only includes a subset of container features.

+1

In your blueprint you mention:
"The difference between LXC and VM makes the driver hard to maintain a
unified model in Nova."

To be clear Nova has no intention of providing a unified model, in
part due to the truth behind your statement above. We provide things
that look like "servers". Please see:
http://docs.openstack.org/ developer/nova/project_scope. html#containers

I would recommending talking the container subgroup, in one of their
meetings, about how best to integrate with OpenStack:
https://wiki.openstack.org/ wiki/Meetings/Containers

>> Looking at the hyper CLI [0], there are many commands that nova would not
>> suppprt, such as:
>>
>> * The pod notion
>> * exec
>> * pull
>
> Then I guess you need to see if Hyper can implement mandatory features
> for Nova [1], [2].
>
> [1] http://docs.openstack.org/ developer/nova/support-matrix. html
> [2] https://wiki.openstack.org/ wiki/HypervisorSupportMatrix

We have no intention of expanding the scope of the Nova API to include
container operation. And the reverse is also true, we would want to
see an intention to support all the important existing APIs before
inclusion, and proving that be having tempest tests reliably passing.

Many thanks,
John

__ __ __
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists. openstack.org?subject: unsubscribe
http://lists.openstack.org/ cgi-bin/mailman/listinfo/ openstack-dev__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [qa] Tempest bug triage questions

2015-06-22 Thread David Kranz


On 06/22/2015 10:15 AM, Yaroslav Lobankov wrote:

Hello everyone,

I have some questions about the bug triage procedure for Tempest:

1. Some bugs in Tempest have status "Fix committed". Should we move 
statuses of these bugs to "Fix released"?
Yes, tempest doesn't have the kind of releases where Fix committed makes 
sense.


2. Many bugs have statuses "In progress", but patches for these bugs 
have -1 from someone (or their workflow is -1)
and it looks like these patches are abandoned, while statuses of 
such patches are "Review in progress".

What should we do with such bugs?
This is kind of tricky without the project having a "manager". I think 
we usually ping with a comment in the bug to see what the holdup is. If 
there is no response, we would set it back to Triaged of Confirmed.


3. What should we do with bugs like this [1]? It says that 
TimeoutException occurred, but the log of the test is unavailable
by the link already. I don't know how to reproduce the issue, 
besides the log of the test is unavailable. What should I do with this 
bug?
This bug is about how tempest should handle cases where a resource 
deletion fails. I think it is a legitimate issue though the answer is 
not clear given a gate that may be very slow at times.


 -David



Thank you!

[1] https://bugs.launchpad.net/tempest/+bug/1322011

Regards,
Yaroslav Lobankov.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [third-party] Third Party CI Working Group Meeting Tuesday June 23 1700UTC

2015-06-22 Thread Kurt Taylor

Hi all,

The Third Party CI Working Group is meeting at the new time this week -
1700UTC Tuesday, June 23rd in #openstack-meeting.

The agenda for the meeting is here:
https://wiki.openstack.org/wiki/Meetings/ThirdParty

We will be working on the CI systems monitoring dashboard hosting spec and
preparing for the common CI virtual sprint, among other things. Please try
to attend if at all possible.

As always, feel free to add items to the agenda and we will get to them as
time permits.

Thanks!
Kurt Taylor (krtaylor)
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ironic] Proposal to add a new repository

2015-06-22 Thread John Trowbridge



On 06/22/2015 10:40 AM, Dmitry Tantsur wrote:
> On 06/22/2015 04:19 PM, Devananda van der Veen wrote:
>> Hi John,
>>
>> Thanks for the excellent summary! I found it very helpful to get caught
>> up. I'd like to make sure I understand the direction ahc is going. A
>> couple questions...

Thanks for your interest.

> 
> Let me add my $0.5 :)
> 
>>
>> I see that ahc is storing its information in swift. That's clever, but
>> if Ironic provided a blob store for each node, would that be better?

If the blob is large enough, this would be better. Originally we stored
the data in the extra column of the Ironic db, but that proved disastrous:

https://bugs.launchpad.net/ironic-inspector/+bug/1461252

>>
>> We discussed adding a search API to Ironic at the Vancouver summit,
>> though no work has been done on that yet, afaik. If ahc is going to grow
>> a REST API for searching for nodes based on specific criteria that it
>> discovered, could/should we combine these within Ironic's API?
> 
> I think John meant having API to replace scripts, so I guess search
> won't help. When we're talking about advanced matching, we're talking
> about the following:
> 1. We have a ramdisk tool (based on [8]) to get some insane of facts
> from withing the ramdisk (say, 1000 of them)
> 2. We have an Inspector plugin to put them all in Swift (or Ironic blob
> storage as above)
> 3. We have config files (aka rules) written in special JSON-alike DSL to
> do matching (one of the weak points is that these are files - I'd like
> API endpoint to accept these rules instead).
> 4. We have a script to run this DSL and get some output (match/not match
> + some matched variables - similar to what regexps do).
> As I understood it John want the latter to become an API endpoint,
> accepting rules (and maybe node UUIDs) and outputting some result.
> 
> Not sure about benchmarking here, but again, it's probably an API
> endpoint that accepts some minimal expectations, and puts failed nodes
> to maintenance mode, if they fail to comply (again, that's how I
> understood it).
> 
> It's not hard to make these API endpoints part of Inspector, but it's
> somewhat undesirable to have them optional...
> 
>>
>>  From a service coupling perspective, I like the approach that ahc is
>> optional, and also that Ironic-inspector is optional, because this keeps
>> the simple use-case for Ironic, well, simple! That said, this seems more
>> like a configuration setting (should inspector do extra things?) than an
>> entirely separate service, and separating them might be unnecessarily
>> complicated.
> 
> We keep thinking about it as well. After all, right now it's just a
> couple of utilities. There are 2 more concerns that initially made me
> pull out this code:
> 1. ahc-tools currently depends on the library [8], which I wish would be
> developed much more openly
> 2. it's cool that inspector is pluggable, but it has its cost: there's a
> poor feedback loop from inspector processing plugins to a user - like
> with all highly asynchronous code
> 3. it's also not possible (at least for now) to request a set of
> processing plugins when starting introspection via inspector.
> 
> We solved the latter 2 problems by moving code to scripts. So now
> Inspector only puts some data to Swift, and scripts can do everything else.
> 
> So now we've left with
> 1. dependency on "hardware" library
> 2. not very stable interface, much less stable than one of Inspector
> 
> We still wonder how to solve these 2 without creating one more
> repository. Any ideas are welcome :)

It is a goal of mine to solve issue 1 incrementally over time. Either by
improving the library (both in function and in openness), or by slowly
moving the implementation. That does not seem impossible to do within
the inspector tree.

However, issue 2 is a fact. We currently have scripts, and we want to
have a REST API. I do not see a transition between the two that does not
involve a large amount of churn.

I am not sure how to solve issue 2 without a separate repository. I do
think there is a logical separation of concerns though, so we may not
need to completely merge the two in the future. Inspector collects data,
and ahc-tools (or whatever it is eventually named) is used to act on the
data.
> 
>>
>> It sounds like this is the direction you'd like to go, and you took the
>> current approach for expediency. If so, I'd like us to discuss a path to
>> merge the functionality as it matures, and decide whether a separate
>> repository is the right way to go long term.
>>
>> Thanks much,
>> Devananda
>>
>>
>> On Mon, Jun 22, 2015, 05:40 John Trowbridge > > wrote:
>>
>> This is a proposal to add a new repository governed by the ironic
>> inspector subteam. The current repository is named ahc-tools[1],
>> however
>> there is no attachment to this name. "ironic-inspector-extra"
>> would seem
>> to fit if this is moved under the Ironic umbrella.
>>
>> What is AHC?
>> -

Re: [openstack-dev] [cross-project] RBAC Policy Basics

On 06/22/2015 12:41 AM, Osanai, Hisashi wrote:

On Saturday, June 20, 2015 11:16 AM, Adam Young wrote:

What situations does a shared policy file require?
For example, there are policy files for Nova and Cinder and they have
same targets such as
"context_is_admin", "admin_or_owner" and "default".

A lot of these internal rules most likely should be removed. They do
conflict, with differenet interpretations between the proejcts. They are
also confusing two different things: scope and role./ I think we
should make it a point to keep them separate.

I don't understand why you think it as conflicts. They use same target name
such as "context_is_admin", "admin_or_owner" and "default" but they use them
on different processes. I might have mis-understanding here but for me there
is no conflict.

It is not an issue if you keep each of the policy files completely
separate, but it means that each service has its own meaning for the
same name, and that confuses operators; owner in Nova means "a user
that has a role on this project" where as "owner" in Keystone means
"Objects associated with a specific user".

http://lists.openstack.org/pipermail/openstack-dev/2015-May/063915.html
- HTTP_X_SERVICE_ROLES handling in _checks.py

I've missed there there was another push for "Service specif roles" out
there. We've been trying to make the concpet slighly more general by
saying that we were going to namespace roles, and that a Service would
be one potential namwspacing. Henry Nash had proposed Domain Specific
roles, in case you were wondering what else would need to be namespaced.

https://review.openstack.org/#/c/133855/

I like your thought " the concpet slighly more general" and it becomes a
solution for my issue.

Wow, I typoed this. Glad is was still comprehensible.

My concern now is:
* Service Tokens was implemented in Juno [1] but now we are not able to
Implement it with Oslo policy without extensions so far.
* I think to implement spec[2] needs more time.

[1]
https://github.com/openstack/keystone-specs/blob/master/specs/keystonemiddleware/implemented/service-tokens.rst
[2] https://review.openstack.org/#/c/133855/

Is there any way to support spec[1] in Oslo policy? Or
Should I wait for spec[2]?

I'm sorry, I am not sure what you are asking.

Thanks in advance,
Hisashi Osanai

[openstack-dev] [Murano] Watch new murano demos

2015-06-22 Thread Ekaterina Chernova

Hi all!

Recently murano team has recorded some demos describing features,
introduced in Kilo.
They can be found at murano wiki page [1].

Demo - is the easiest way to stay in touch with project news!

The list of updated demos:


   - Importing Murano Packages from Murano Repository
   
   - Plugins Opportunity 
   - New UI changes in Kilo 
   - Chef and Puppet executors in murano-agent
   
   - Steps of creation K8S applications with murano.
   


Follow murano channel on youtube [2] to stay tuned.

[1] - https://wiki.openstack.org/wiki/Murano/Screencasts
[2] -
https://www.youtube.com/playlist?list=PLdgU5edjetW_xCiO184uBsKkz6qmiIjZx
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Tracking optional requirements

Excerpts from Michael Still's message of 2015-06-22 21:05:11 +1000:
> Sure, except where the thing isn't in pip... I just learned tonight
> that the libosinfo thing in the review at the start of this thread is
> a c library and some gobject black magic. I think we want it, but it
> will never work as a pip dependancy. So, I think something human
> readable is still required?

I wonder if a simple list is going to cover that case, since the package
name might vary based on the underlying distro.

I've recently added some features to stevedore that let us inject
driver-specific documentation into sphinx-based manuals. So the
drivers could document their dependencies in their class docstrings,
and the results can show up in the nova docs in the short term, and
the install guide(s) in the longer term when that conversion work
is done. See http://docs.openstack.org/developer/stevedore/sphinxext.html
and http://docs.openstack.org/developer/oslo.messaging/drivers.html
for an (admittedly skimpy) example.

Doug

> 
> Cheers,
> Michael
> 
> On Mon, Jun 22, 2015 at 8:45 PM, Sean Dague  wrote:
> > On 06/22/2015 06:37 AM, Michael Still wrote:
> >> Hi,
> >>
> >> https://review.openstack.org/#/c/149625 made me think about optional
> >> requirements for nova. Some hypervisor drivers have requirements that
> >> are either only needed for their driver, or are optional to their
> >> driver. Examples include the ironic driver depending on the ironic
> >> client, and the review above which wants to add an optional dependancy
> >> on libosinfo.
> >>
> >> In general we keep these things out of requirements.txt because we
> >> don't want to force everyone in the world to install the ironic client
> >> in order to run their non-ironic hypervisor node.
> >>
> >> What I am pondering is that I think its a horrible user experience to
> >> not document these optional dependancies better. I feel like we need
> >> some sort of optional-requirements.txt file in the repo which has a
> >> (human readable) list of the optional dependancies of each driver and
> >> what those dependancies give you if installed.
> >>
> >> What would people think of such a thing? Is there a better way of
> >> expressing this?
> >
> > I believe that Robert and other oslo folks are working on a way to do
> > this with environment markers some how -
> > https://github.com/dstufft/pip/commit/e498d83db191ccd7fe7c1c2e4f9ec201787d1257
> >
> >
> > Because you don't really want optional-requirements.txt. You want:
> >
> >> pip install nova nova[libvirt] nova[mysql]
> >
> > (or something... hand wave on call out)
> >
> > -Sean
> >
> > --
> > Sean Dague
> > http://dague.net
> >
> > __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ironic][Horizon][Tuskar-ui] Making a dashboard for Ironic

2015-06-22 Thread NiuZhenguo

Hi Devananda,
> The git history appears to be squashed [1], and most files don't have an 
> attribution header [2], and > none of the headers refer to the company who 
> appears to be behind this (Huawei). What's the > rationale for these 
> inconsistencies, and who is actually behind the code?
No headers refer to Huawei because the files are based on the Angular Dashboard 
from Horizon codes and now is only an empty baremetal dashboard which can be 
combined with Ironic-webclient created by Krotscheck. I think pushing the 
initial horizon compatible dashboard to stackforge or openstack is the first 
step for collaboration, then we can build a team for that.
> Are you going to maintain this project personally, or is there a team at 
> Huawei (or somewhere else) > that is going to do that? Or are you expecting 
> Ironic's current developer teams to take ownership of > this code and 
> maintain it?
There is not a team working for the project, the purpose I want to create 
ironic-dashboard is that I contribute to both Horizon and Ironic, and 
considering I can help to fill the gaps that there's no Horizon support for 
Ironic. About who should maintain the project, I'm not sure whether Horizon or 
Ironic or even a separate team.

> Are you/they going to become part of Ironic's community, attend our weekly 
> meetings, and follow > our design process?

Sure, the dashboard should be geared towards Ironic.
> What is the vision / goal that is being working towards? What is the scope of 
> this dashboard? How > does it fit in with our existing plans?
The end goal for ironic-dashboard is that it can be configured as a standalone 
UI or a dashboard within Horizon. Currently the codes on github is an empty 
dashboard, the panels and features should be developed with the ironic 
community.

-zhenguo

From: devananda@gmail.com
Date: Fri, 19 Jun 2015 22:40:53 +
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [Ironic][Horizon][Tuskar-ui] Making a dashboard 
for Ironic

Hi Jim,

Your characterization of this is incomplete. These are not two equal projects 
proposing the same thing in different ways, and while I very much want to 
encourage collaboration, I value our community and feel that this was not done 
in the spirit of that community.
To be clear: ironic-webclient has been developed with the knowledge and support 
of the Ironic developer community, and was not moved into the openstack/ 
namespace on my request, because I have been holding projects to a certain 
level of maturity before including them in Ironic, roughly equivalent to the 
TC's bar for "big tent" inclusion.
On the other hand, ironic-dashboard was done without the knowledge of any 
Ironic cores, nor with even a "heads up" to the Ironic or Horizon PTLs. Rather 
than an open design process, this code was just dropped on github and Infra was 
asked to approve the project creation. I have not had the opportunity to talk 
with its author *at all* yet.
I'm glad that ya'll didn't just approve the project creation request without 
checking with me, and I'm glad we are now having this discussion.
Now that that is cleared up, let's move on.

Hi Zhenguo,
I have some questions about "ironic-dashboard" that I need answered before the 
Ironic Project Team accepts responsibility for it.
The git history appears to be squashed [1], and most files don't have an 
attribution header [2], and none of the headers refer to the company who 
appears to be behind this (Huawei). What's the rationale for these 
inconsistencies, and who is actually behind the code?
Are you going to maintain this project personally, or is there a team at Huawei 
(or somewhere else) that is going to do that? Or are you expecting Ironic's 
current developer teams to take ownership of this code and maintain it?
Are you/they going to become part of Ironic's community, attend our weekly 
meetings, and follow our design process?

What is the vision / goal that is being working towards? What is the scope of 
this dashboard? How does it fit in with our existing plans?

I'm not entirely opposed to having two separate UI projects for Ironic at the 
moment, but we should be very clear about the rationale if we go that route.
-Devananda

[1] 
https://github.com/niuzhenguo/ironic-dashboard/commit/4be73d19e54eb75aa31da3d1a38fa65c1287bc7b[2]
 https://github.com/niuzhenguo/ironic-dashboard/search?q=copyright

On Fri, Jun 19, 2015 at 12:00 PM James E. Blair  wrote:
Hi all,

I'm glad that by asking the ironic-dashboard creators to propose their

project to ironic initially (rather than stackforge) we have prompted

this conversation.  We now know that two independent groups of people

have created standalone ironic dashboards, neither of which is currently

part of an OpenStack project.

We have an opportunity for those teams to begin collaborating now.  I

would encourage them to do so, along with both the Ironic and Horizon

teams, on a path forward.  Let's en

Re: [openstack-dev] [Ironic] Proposal to add a new repository

2015-06-22 Thread Lucas Alvares Gomes

Hi,

> I see that ahc is storing its information in swift. That's clever, but if
> Ironic provided a blob store for each node, would that be better?
>

I can try to answer that. The initial implementation was doing that
but, AHC collect a fine-grained amount of data, e.g:

* it runs benchmark on all disks using different IO size. Read being
default but you can also test writing (disabled by default because
it's destructive)

* It tests memory performance with different blocks sizes (1K, 4K, 1M,
16M, 128M, 1G, 2G, ...)

* It stress test each physical CPU (and all at once)

So a machine with many disks, many CPUs and a lot of memory can
generate a really big amount of data. For that reason storing the blob
in Swift and just link the ID in Ironic is preferable.

Cheers,
Lucas

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] can we get ceilometermiddleware to use a config file instead of transport_url?

Excerpts from Doug Hellmann's message of 2015-06-22 10:50:06 -0400:
> Excerpts from Sean Dague's message of 2015-06-22 09:02:02 -0400:
> > On 06/22/2015 08:58 AM, Doug Hellmann wrote:
> > > Excerpts from Sean Dague's message of 2015-06-22 08:08:04 -0400:
> > >> In extracting the contract for RPC backends in devstack (to have most of
> > >> them live in plugins) one bit of an edge case was discovered.
> > >>
> > >> https://github.com/openstack-dev/devstack/blob/master/lib/swift#L388
> > >>
> > >> The connection to the RPC mechanism from ceilometermiddleware inside of
> > >> swift uses a transport url instead of an oslo.messaging config block.
> > >>
> > >> ceilometermiddleware requires oslo.config and oslo.messaging, so it
> > >> seems like it could use an oslo config block instead.
> > >>
> > >> One of the reasons why this seems like a better idea is that not all the
> > >> properties of a messaging connection can be encoded in just a url today.
> > >> For instance, Rabbit can specify heartbeating params -
> > >> https://github.com/openstack-dev/devstack/blob/master/lib/rpc_backend#L282-L287,
> > >> and zmq needs matchmaker info -
> > >> https://github.com/openstack-dev/devstack/blob/master/lib/rpc_backend#L282-L287
> > >>
> > >> (Note: zmq is not currently able to be configured for swift + ceilometer
> > >> today
> > >> https://github.com/openstack-dev/devstack/blob/master/lib/rpc_backend#L282-L287
> > >> and given what it needs in it's config, it's not clear that it would be
> > >> reasonable to do so.)
> > >>
> > >> Could we deprecate the use of tranport_url in ceilometermiddleware and
> > >> move to an actual oslo.config file somewhere instead? That would bring
> > >> it in line with the rest of the RPC configuration for services, and
> > >> ensure that all connections in a cluster have access to all the same
> > >> options.
> > > 
> > > Swift doesn't use oslo.config, so we might need to make the middleware
> > > support both configuration mechanisms.
> > 
> > Right, but ceilometermiddleware does. Can't it load it's config from a
> > side channel? Or is it only possible for it to load it's config from the
> > same file as swift?
> 
> Library code that uses oslo.config relies on having an initialized
> configuration object, which means something has specified the files
> to be loaded, handled command line argument parsing, etc. We wouldn't
> want the middleware doing that, because doing it properly requires
> knowing the name of the application it's running in and what some
> of the application-specific defaults should be, and those are usually
> best left to the application.

We had a similar issue with the CORS middleware needing a config object.
Perhaps we need a piece of middleware to set up the config using
parameters coming in from the paste.ini (application, name, default
config file, etc.). If that middleware sets up the global config
instance, other middleware could be written to rely on it being earlier
in the pipline.

Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Modular L2 Agent

2015-06-22 Thread Salvatore Orlando

I see Kyle's point that this is not something in-scope for liberty at this
stage.

However, on the other hand I would rather avoid having multiple agents on
the compute node performing various tasks in an un-coordinated way (well,
actually relying on neutron-server coordination).

QoS is an example, but what Miguel is doing for QoS applies, for instance,
to security groups and allowed address pairs processing. Even if probably
Mohammad has more in mind a "modular" agent that is able to talk to
different data planes using a well-defined driver interface,  a similar
framework could be used for "augmenting" the capabilities of an agent as
Miguel mentions.

I would probably start with something for enabling the L2 agent to process
"features" such as QoS and security groups, working on the OVS agent, and
then in a second step abstract a driver interface for communicating with
the data plane. But I honestly do not know if this will keep the work too
"OVS-centric" and therefore won't play well with the current efforts to put
linux bridge on par with OVS in Neutron. For those question we should seek
an answer from our glorious reference control plane lieutenant, and perhaps
also from Sean Collins, who's coordinating efforts around linux bridge
parity.

Salvatore

On 22 June 2015 at 16:30, Miguel Angel Ajo  wrote:

>
>
> In the context of Quality of Service, we need to extend the L2 agents
> (SR-IOV,
> OVS and LB), and we didn't want to simply hijack the processing loop of
> the agents,
> but take a moment, and put together a Modular L2 design
>
> https://review.openstack.org/#/c/189723/
>
> If you find it reasonable to do it in this context so it can be reused for
> neutron
> in general later, please join the reviews.
>
> I'm not sure if Irena was involved on previous Modular L2 Agent design
> sessions.
>
>
> Best regards,
> Miguel Ángel.
>
>
> Mohammad Banikazemi wrote:
>
>
>
> During the last couple of ML2 group meetings, the subject of Modular L2
> Agents has come up again and I was tasked to bring up the subject to the
> attention of the larger community.
> We are aware of the ongoing efforts to improve the L2 agent(s) and the
> patches which are currently under review and those that got merged
> recently. The question is whether the Neutron community thinks the effort
> started (and suspended) a while ago around creating a modular L2 agent is
> worth pursuing at all and if yes, whether this is a good cycle to get that
> work possibly restarted.
>
> Best,
>
> Mohammad
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>   Mohammad Banikazemi 
>  21 Jun 2015 18:54 via Postbox
> 
>
> During the last couple of ML2 group meetings, the subject of Modular L2
> Agents has come up again and I was tasked to bring up the subject to the
> attention of the larger community.
> We are aware of the ongoing efforts to improve the L2 agent(s) and the
> patches which are currently under review and those that got merged
> recently. The question is whether the Neutron community thinks the effort
> started (and suspended) a while ago around creating a modular L2 agent is
> worth pursuing at all and if yes, whether this is a good cycle to get that
> work possibly restarted.
>
> Best,
>
> Mohammad
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] can we get ceilometermiddleware to use a config file instead of transport_url?

2015-06-22 Thread gord chung




On 22/06/15 09:02 AM, Sean Dague wrote:

On 06/22/2015 08:58 AM, Doug Hellmann wrote:

Excerpts from Sean Dague's message of 2015-06-22 08:08:04 -0400:

In extracting the contract for RPC backends in devstack (to have most of
them live in plugins) one bit of an edge case was discovered.

https://github.com/openstack-dev/devstack/blob/master/lib/swift#L388

The connection to the RPC mechanism from ceilometermiddleware inside of
swift uses a transport url instead of an oslo.messaging config block.

ceilometermiddleware requires oslo.config and oslo.messaging, so it
seems like it could use an oslo config block instead.

One of the reasons why this seems like a better idea is that not all the
properties of a messaging connection can be encoded in just a url today.
For instance, Rabbit can specify heartbeating params -
https://github.com/openstack-dev/devstack/blob/master/lib/rpc_backend#L282-L287,
and zmq needs matchmaker info -
https://github.com/openstack-dev/devstack/blob/master/lib/rpc_backend#L282-L287

(Note: zmq is not currently able to be configured for swift + ceilometer
today
https://github.com/openstack-dev/devstack/blob/master/lib/rpc_backend#L282-L287
and given what it needs in it's config, it's not clear that it would be
reasonable to do so.)

Could we deprecate the use of tranport_url in ceilometermiddleware and
move to an actual oslo.config file somewhere instead? That would bring
it in line with the rest of the RPC configuration for services, and
ensure that all connections in a cluster have access to all the same
options.

Swift doesn't use oslo.config, so we might need to make the middleware
support both configuration mechanisms.

Right, but ceilometermiddleware does. Can't it load it's config from a
side channel? Or is it only possible for it to load it's config from the
same file as swift?
i coded it under the impression of the latter -- the "swift not using 
oslo" part made me have to make a few adjustments. i'd be happy to 
accommodate both options if it's possible.


--
gord


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Stuck on VIF plugin script?

2015-06-22 Thread Neil Jerram


On 22/06/15 15:11, Daniel P. Berrange wrote:

On Mon, Jun 22, 2015 at 11:28:35AM +0100, neil.jer...@metaswitch.com wrote:

‎It seems like we're possibly stuck now on the VIF plugin script spec [1]; 
there being core comments in apparently conflicting directions. I wonder if 
it's still feasible for a version of this to land during Liberty?

[1] https://review.openstack.org/#/c/162468/

I plan to reread everyone's comments and see if ‎I can propose a way forward - 
but thought I'd circulate this email first in case anyone wants to say that 
it's already too late...

If VIF plugin script doesn't happen this cycle, can we continue
processing VIF type additions/changes/deletions in the traditional way?


This is a possibility, but I really want us to do everything within our
power to avoid continuing with the status quo in Liberty, as it is not
really making anyone happy.  If we make a strong push, I think we can
get a better solution done for Liberty, even if it means we need a spec
freeze exception request.


Entirely agree.  Please don't hesitate to ask if I can help in any way.

Neil


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] can we get ceilometermiddleware to use a config file instead of transport_url?

Excerpts from Sean Dague's message of 2015-06-22 09:02:02 -0400:
> On 06/22/2015 08:58 AM, Doug Hellmann wrote:
> > Excerpts from Sean Dague's message of 2015-06-22 08:08:04 -0400:
> >> In extracting the contract for RPC backends in devstack (to have most of
> >> them live in plugins) one bit of an edge case was discovered.
> >>
> >> https://github.com/openstack-dev/devstack/blob/master/lib/swift#L388
> >>
> >> The connection to the RPC mechanism from ceilometermiddleware inside of
> >> swift uses a transport url instead of an oslo.messaging config block.
> >>
> >> ceilometermiddleware requires oslo.config and oslo.messaging, so it
> >> seems like it could use an oslo config block instead.
> >>
> >> One of the reasons why this seems like a better idea is that not all the
> >> properties of a messaging connection can be encoded in just a url today.
> >> For instance, Rabbit can specify heartbeating params -
> >> https://github.com/openstack-dev/devstack/blob/master/lib/rpc_backend#L282-L287,
> >> and zmq needs matchmaker info -
> >> https://github.com/openstack-dev/devstack/blob/master/lib/rpc_backend#L282-L287
> >>
> >> (Note: zmq is not currently able to be configured for swift + ceilometer
> >> today
> >> https://github.com/openstack-dev/devstack/blob/master/lib/rpc_backend#L282-L287
> >> and given what it needs in it's config, it's not clear that it would be
> >> reasonable to do so.)
> >>
> >> Could we deprecate the use of tranport_url in ceilometermiddleware and
> >> move to an actual oslo.config file somewhere instead? That would bring
> >> it in line with the rest of the RPC configuration for services, and
> >> ensure that all connections in a cluster have access to all the same
> >> options.
> > 
> > Swift doesn't use oslo.config, so we might need to make the middleware
> > support both configuration mechanisms.
> 
> Right, but ceilometermiddleware does. Can't it load it's config from a
> side channel? Or is it only possible for it to load it's config from the
> same file as swift?

Library code that uses oslo.config relies on having an initialized
configuration object, which means something has specified the files
to be loaded, handled command line argument parsing, etc. We wouldn't
want the middleware doing that, because doing it properly requires
knowing the name of the application it's running in and what some
of the application-specific defaults should be, and those are usually
best left to the application.

Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] devstack plugin writers guide

On 06/22/2015 10:36 AM, Davanum Srinivas wrote:
> Sean,
> 
> Can we please add a bit about what things in devstack can they assume
> to be available when their script is running (example: can they use
> stuff from functions-common without actually sourcing it?)

Sure, the answer is that functions-common is *not* stable interface,
those are available, but subject to change without notice. i.e. use them
if you must, but realize you might need to react quickly chasing those
changes.

Functions in devstack/inc/ are considered a medium strong interface,
those include config file manipulation, python package installation,
rootwrap setup. We'll expand the set of contract functions over time,
but all those adds need to come with in tree unit tests so that we can
prevent accidental regressions on the contract.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ironic] Proposal to add a new repository

2015-06-22 Thread Dmitry Tantsur


On 06/22/2015 04:19 PM, Devananda van der Veen wrote:

Hi John,

Thanks for the excellent summary! I found it very helpful to get caught
up. I'd like to make sure I understand the direction ahc is going. A
couple questions...


Let me add my $0.5 :)



I see that ahc is storing its information in swift. That's clever, but
if Ironic provided a blob store for each node, would that be better?

We discussed adding a search API to Ironic at the Vancouver summit,
though no work has been done on that yet, afaik. If ahc is going to grow
a REST API for searching for nodes based on specific criteria that it
discovered, could/should we combine these within Ironic's API?


I think John meant having API to replace scripts, so I guess search 
won't help. When we're talking about advanced matching, we're talking 
about the following:
1. We have a ramdisk tool (based on [8]) to get some insane of facts 
from withing the ramdisk (say, 1000 of them)
2. We have an Inspector plugin to put them all in Swift (or Ironic blob 
storage as above)
3. We have config files (aka rules) written in special JSON-alike DSL to 
do matching (one of the weak points is that these are files - I'd like 
API endpoint to accept these rules instead).
4. We have a script to run this DSL and get some output (match/not match 
+ some matched variables - similar to what regexps do).
As I understood it John want the latter to become an API endpoint, 
accepting rules (and maybe node UUIDs) and outputting some result.


Not sure about benchmarking here, but again, it's probably an API 
endpoint that accepts some minimal expectations, and puts failed nodes 
to maintenance mode, if they fail to comply (again, that's how I 
understood it).


It's not hard to make these API endpoints part of Inspector, but it's 
somewhat undesirable to have them optional...




 From a service coupling perspective, I like the approach that ahc is
optional, and also that Ironic-inspector is optional, because this keeps
the simple use-case for Ironic, well, simple! That said, this seems more
like a configuration setting (should inspector do extra things?) than an
entirely separate service, and separating them might be unnecessarily
complicated.


We keep thinking about it as well. After all, right now it's just a 
couple of utilities. There are 2 more concerns that initially made me 
pull out this code:
1. ahc-tools currently depends on the library [8], which I wish would be 
developed much more openly
2. it's cool that inspector is pluggable, but it has its cost: there's a 
poor feedback loop from inspector processing plugins to a user - like 
with all highly asynchronous code
3. it's also not possible (at least for now) to request a set of 
processing plugins when starting introspection via inspector.


We solved the latter 2 problems by moving code to scripts. So now 
Inspector only puts some data to Swift, and scripts can do everything else.


So now we've left with
1. dependency on "hardware" library
2. not very stable interface, much less stable than one of Inspector

We still wonder how to solve these 2 without creating one more 
repository. Any ideas are welcome :)




It sounds like this is the direction you'd like to go, and you took the
current approach for expediency. If so, I'd like us to discuss a path to
merge the functionality as it matures, and decide whether a separate
repository is the right way to go long term.

Thanks much,
Devananda


On Mon, Jun 22, 2015, 05:40 John Trowbridge mailto:tr...@redhat.com>> wrote:

This is a proposal to add a new repository governed by the ironic
inspector subteam. The current repository is named ahc-tools[1], however
there is no attachment to this name. "ironic-inspector-extra" would seem
to fit if this is moved under the Ironic umbrella.

What is AHC?

* AHC as a term comes from the enovance edeploy installation method[2].
* The general concept is that we want to have a very granular picture of
the physical hardware being used in a deployment in order to be able to
match specific hardware to specific roles, as well as the ability to
find poor performing outliers before we attempt to deploy.
* For example: As a cloud operator, I want to make sure all logical
disks have random read IOPs within 15% variance of each other.
* The huge benefit of this tooling over current inspection is the number
of facts collected (~1000 depending on the hardware), all of which can
be used for matching.
* Another example: As an end user, I would like to request a bare metal
machine with a specific model GPU.

What is ahc-tools?
--
* We first tried to place all of this logic into a plugin in
inspector[3] (discoverd at the time). [4]
* This worked fine for just collecting some of the simple facts, however
we now had a coupling between booting a ramdisk, and matching against
the collected data.
* ahc-tools started as a

Re: [openstack-dev] [Neutron] Proposing Rossella Sblendido for the Control Plane core team

2015-06-22 Thread Miguel Angel Ajo


I missed this one!, congratulations Rosella :-)


Anna Kamyshnikova 
20 Jun 2015 11:55via Postbox 


Congratulations Rossella!



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Kevin Benton 
20 Jun 2015 00:50via Postbox 


It's been a week with no negative feedback. Welcome to the team Rossella!

On Mon, Jun 15, 2015 at 2:53 AM, Oleg Bondarev 


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Oleg Bondarev 
15 Jun 2015 11:53via Postbox 


+1

On Mon, Jun 15, 2015 at 12:14 PM, Ihar Hrachyshka 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Ihar Hrachyshka 
15 Jun 2015 11:14via Postbox 


-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

No doubt +1.

On 06/12/2015 09:44 PM, Kevin Benton wrote:

Hello!

As the Lieutenant of the built-in control plane[1], I would like
Rossella Sblendido to be a member of the control plane core
reviewer team.

Her review stats are in line with other cores[2] and her feedback
on patches related to the agents has been great. Additionally, she
has been working quite a bit on the blueprint to restructure the L2
agent code so she is very familiar with the agent code and the APIs
it leverages.

Existing cores that have spent time working on the reference
implementation (agents and AMQP code), please vote +1/-1 for her
addition to the team. Aaron, Gary, Assaf, Maru, Kyle, Armando, Carl
and Oleg; you have all been reviewing things in these areas
recently so I would like to hear from you specifically.

1.
http://docs.openstack.org/developer/neutron/policies/core-reviewers.ht

ml#core-review-hierarchy
2. http://stackalytics.com/report/contribution/neutron-group/30

Cheers -- Kevin Benton


__


OpenStack Development Mailing List (not for usage questions)

Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


-BEGIN PGP SIGNATURE-
Version: GnuPG v2

iQEcBAEBCAAGBQJVfpdmAAoJEC5aWaUY1u574skIAOm15mNKujH3X3XSpwtmy+V4
cWNvjYZy0lCG2lbyAwGwgQD0Ybs88/RKOZPPpu9gugAuWcFQRHMSMcsahaTLcH5B
6imK0tEUUn2HdCd9522kEWtf7Hkpux6p5TLQQo2tucvIBQohJuU42EidfKs9Peat
ed/2kiXm+TGRSnZHAYwzJIrttux3ntTLQeTvElo+4vUQEQJNwm8eguXiAPENEjr9
7Ou3TgnYeLXsfVopHXNV+jtkHDr92giB4joODqwc8RNDKE2+dhaqDxCrXq6ksYq7
RFiyVy1qve394ebP0Ta0dq6LKmAYZCnRC9i928GSK5RvQBvcnJRiyFGIgtn4Qu0=
=59av
-END PGP SIGNATURE-

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Kevin Benton 
12 Jun 2015 21:44via Postbox 


Hello!

As the Lieutenant of the built-in control plane[1], I would like Rossella
Sblendido to be a member of the control plane core reviewer team.

Her review stats are in line with other cores[2] and her feedback on
patches related to the agents has been great. Additionally, she has been
working quite a bit on the blueprint to restructure the L2 agent code so
she is very familiar with the agent code and the APIs it leverages.

Existing cores that have spent time working on the reference 
implementation

(agents and AMQP code), please vote +1/-1 for her addition to the team.
Aaron, Gary, Assaf, Maru, Kyle, Armando, Carl and Oleg; you have all been
reviewing things in these areas recently so I would like to hear from you
specifically.

1.
http://docs.openstack.org/developer/neutron/policies/core-reviewers.html#core-review-hierarchy
2. http://stackalytics.com/report/contribution/neutron-group/30

Cheers
__
OpenStack Development Mailing L

Re: [openstack-dev] [all] devstack plugin writers guide

Sean,

Can we please add a bit about what things in devstack can they assume
to be available when their script is running (example: can they use
stuff from functions-common without actually sourcing it?)

-- dims

On Mon, Jun 22, 2015 at 10:12 AM, Sean Dague  wrote:
> There has been a bit of confusion about writing plugins for devstack,
> especially as we moved the primary model from in tree to out of tree
> plugins. To help clarify I did a big restructure of the DevStack Plugin
> Guide on Friday - http://docs.openstack.org/developer/devstack/plugins.html
>
> For new projects that want to integrate with DevStack, the plugin model
> is the way to do that. I'd be happy to field any follow on questions in
> this thread to clarify things that aren't straight forward there (which
> would make it into future versions of this document). It's always hard
> to write documentation without audience participation, because being too
> close to the technology, I forget what other people know or don't know
> about it.
>
> Thanks,
>
> -Sean
>
> --
> Sean Dague
> http://dague.net
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Davanum Srinivas :: https://twitter.com/dims

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Modular L2 Agent

2015-06-22 Thread Miguel Angel Ajo




In the context of Quality of Service, we need to extend the L2 agents 
(SR-IOV,
OVS and LB), and we didn't want to simply hijack the processing loop of 
the agents,

but take a moment, and put together a Modular L2 design

https://review.openstack.org/#/c/189723/

If you find it reasonable to do it in this context so it can be reused 
for neutron

in general later, please join the reviews.

I'm not sure if Irena was involved on previous Modular L2 Agent design 
sessions.



Best regards,
Miguel Ángel.


Mohammad Banikazemi wrote:



During the last couple of ML2 group meetings, the subject of Modular L2
Agents has come up again and I was tasked to bring up the subject to the
attention of the larger community.
We are aware of the ongoing efforts to improve the L2 agent(s) and the
patches which are currently under review and those that got merged
recently. The question is whether the Neutron community thinks the effort
started (and suspended) a while ago around creating a modular L2 agent is
worth pursuing at all and if yes, whether this is a good cycle to get that
work possibly restarted.

Best,

Mohammad
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




Mohammad Banikazemi 
21 Jun 2015 18:54via Postbox 



During the last couple of ML2 group meetings, the subject of Modular L2
Agents has come up again and I was tasked to bring up the subject to the
attention of the larger community.
We are aware of the ongoing efforts to improve the L2 agent(s) and the
patches which are currently under review and those that got merged
recently. The question is whether the Neutron community thinks the effort
started (and suspended) a while ago around creating a modular L2 agent is
worth pursuing at all and if yes, whether this is a good cycle to get that
work possibly restarted.

Best,

Mohammad
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ironic] Proposal to add a new repository

2015-06-22 Thread Devananda van der Veen

Hi John,

Thanks for the excellent summary! I found it very helpful to get caught up.
I'd like to make sure I understand the direction ahc is going. A couple
questions...

I see that ahc is storing its information in swift. That's clever, but if
Ironic provided a blob store for each node, would that be better?

We discussed adding a search API to Ironic at the Vancouver summit, though
no work has been done on that yet, afaik. If ahc is going to grow a REST
API for searching for nodes based on specific criteria that it discovered,
could/should we combine these within Ironic's API?

>From a service coupling perspective, I like the approach that ahc is
optional, and also that Ironic-inspector is optional, because this keeps
the simple use-case for Ironic, well, simple! That said, this seems more
like a configuration setting (should inspector do extra things?) than an
entirely separate service, and separating them might be unnecessarily
complicated.

It sounds like this is the direction you'd like to go, and you took the
current approach for expediency. If so, I'd like us to discuss a path to
merge the functionality as it matures, and decide whether a separate
repository is the right way to go long term.

Thanks much,
Devananda

On Mon, Jun 22, 2015, 05:40 John Trowbridge  wrote:

> This is a proposal to add a new repository governed by the ironic
> inspector subteam. The current repository is named ahc-tools[1], however
> there is no attachment to this name. "ironic-inspector-extra" would seem
> to fit if this is moved under the Ironic umbrella.
>
> What is AHC?
> 
> * AHC as a term comes from the enovance edeploy installation method[2].
> * The general concept is that we want to have a very granular picture of
> the physical hardware being used in a deployment in order to be able to
> match specific hardware to specific roles, as well as the ability to
> find poor performing outliers before we attempt to deploy.
> * For example: As a cloud operator, I want to make sure all logical
> disks have random read IOPs within 15% variance of each other.
> * The huge benefit of this tooling over current inspection is the number
> of facts collected (~1000 depending on the hardware), all of which can
> be used for matching.
> * Another example: As an end user, I would like to request a bare metal
> machine with a specific model GPU.
>
> What is ahc-tools?
> --
> * We first tried to place all of this logic into a plugin in
> inspector[3] (discoverd at the time). [4]
> * This worked fine for just collecting some of the simple facts, however
> we now had a coupling between booting a ramdisk, and matching against
> the collected data.
> * ahc-tools started as a way to uncouple these two steps[5].
> * We also added a wrapper around the enovance report tooling[6], as it
> already had the ability to generate reports based on the collected data,
> but was designed to read in the data from the filesystem.
> * The report tool has two functions.
> * First, it can group the systems by category (NICs, Firmware,
> Processors, etc.).
> * Second, it can use statistical analysis to find performance outliers.
>
> Why is ahc-tools useful to Ironic?
> --
> * If we run benchmarks on hardware whenever it is turned back in by a
> tenant, we can easily put nodes into maintenance if the hardware is
> performing below some set threshold. This would allow us to have better
> certainty that the end user is getting what we promised them.
> * The advanced matching could also prove very useful. For VMs, I think
> the pets vs cattle analogy holds up very well, however many use cases
> for having cloud based bare metal involve access to specific hardware
> capabilities. I think advanced matching could help bridge this gap.
>
> Why not just put this code directly into inspector?
> ---
> * Clearly this code is 100% dependent on inspector. However, inspector
> is quite stable, and works great without any of this extra tooling.
> * ahc-tools is very immature, and will need many breaking changes to get
> to the same stability level of inspector.
>
> Why aren't you following the downstream->stackforge->openstack path?
> 
> * This was the initial plan[7], however we were told that under the new
> "big tent", that the openstack namespace is no longer meant to signify
> maturity of a project.
> * Instead, we were told we should propose the project directly to
> Ironic, or make a new separate project.
>
> What is the plan to make ahc-tools better?
> --
> * The first major overhaul we would like to do is to put the reporting
> and matching functionality behind a REST API.
> * Reporting in particular will require significant work, as the current
> wrapper script wraps code that was never designed to be a library (Its
> output is just a series of print

[openstack-dev] [qa] Tempest bug triage questions

2015-06-22 Thread Yaroslav Lobankov

Hello everyone,

I have some questions about the bug triage procedure for Tempest:

1. Some bugs in Tempest have status "Fix committed". Should we move
statuses of these bugs to "Fix released"?

2. Many bugs have statuses "In progress", but patches for these bugs have
-1 from someone (or their workflow is -1)
and it looks like these patches are abandoned, while statuses of such
patches are "Review in progress".
What should we do with such bugs?

3. What should we do with bugs like this [1]? It says that
TimeoutException occurred,
but the log of the test is unavailable
by the link already. I don't know how to reproduce the issue, besides
the log of the test is unavailable. What should I do with this bug?

Thank you!

[1] https://bugs.launchpad.net/tempest/+bug/1322011

Regards,
Yaroslav Lobankov.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Stuck on VIF plugin script?

2015-06-22 Thread Daniel P. Berrange

On Mon, Jun 22, 2015 at 11:28:35AM +0100, neil.jer...@metaswitch.com wrote:
> ‎It seems like we're possibly stuck now on the VIF plugin script spec [1]; 
> there being core comments in apparently conflicting directions. I wonder if 
> it's still feasible for a version of this to land during Liberty? 
> 
> [1] https://review.openstack.org/#/c/162468/
> 
> I plan to reread everyone's comments and see if ‎I can propose a way forward 
> - but thought I'd circulate this email first in case anyone wants to say that 
> it's already too late...
> 
> If VIF plugin script doesn't happen this cycle, can we continue
> processing VIF type additions/changes/deletions in the traditional way?

This is a possibility, but I really want us to do everything within our
power to avoid continuing with the status quo in Liberty, as it is not
really making anyone happy.  If we make a strong push, I think we can
get a better solution done for Liberty, even if it means we need a spec
freeze exception request.

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [all] devstack plugin writers guide

There has been a bit of confusion about writing plugins for devstack,
especially as we moved the primary model from in tree to out of tree
plugins. To help clarify I did a big restructure of the DevStack Plugin
Guide on Friday - http://docs.openstack.org/developer/devstack/plugins.html

For new projects that want to integrate with DevStack, the plugin model
is the way to do that. I'd be happy to field any follow on questions in
this thread to clarify things that aren't straight forward there (which
would make it into future versions of this document). It's always hard
to write documentation without audience participation, because being too
close to the technology, I forget what other people know or don't know
about it.

Thanks,

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] M Naming Poll ended - results still need to clear legal

2015-06-22 Thread Monty Taylor

Hey all!

The M naming poll has concluded. I'd say "and the winner is ..." except
we still need to get the winning choice(s) vetted for legal entanglements.

Feel free to go and look at the results, they are publicly available -
but please DON'T start making t-shirts or having parties (ok, have as
many parties as you want) until we've gotten the alls-clear from the
trademark folks. I've sent the list to them already, so they're working
on it furiously as we speak.

As soon as I get the word back, I will send out another announcement
with the official result.

Thanks!
Monty

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [puppet][keystone] How to uniquely name Keystone v3 resources in puppet?

2015-06-22 Thread Rich Megginson

The problem with puppet-keystone and Keystone v3 domains is naming of 
puppet resources contained within domains - users, groups, projects.


Suppose you have an admin user in domain "dom1" and an admin user in 
domain "dom2".  How do you declare these in puppet?  You can't do


keystone_user { 'admin': domain => 'dom1'}
and
keystone_user { 'admin': domain => 'dom2'}

AFAIK, resource names in puppet have to be unique.  I have introduced 
the string "::" as the domain delimiter in puppet code. This allows you 
to "fully qualify" resource names with the domain in all domain scoped 
resources.


keystone_user { 'admin::dom1': }
and
keystone_user { 'admin': domain => 'dom2'}

This is a valid puppet manifest - the resource names are unique.

How do I refer to the "admin" user?  For example, I want to have a role 
assignment:


keystone_user_role { 'admin::dom2@project::somedomain': roles => 
['adminrole'] }


How do I have an autorequire for the user?  That is, I can't know, in 
the autorequire method in the keystone_user_role.rb type code, if the 
resource name is going to be 'admin' or 'admin::dom2'.  If I just do 
autorequire for ['admin'], that will cause problems if a) there is no 
'admin' user because all 'admin' users are fully qualified b) the user 
named 'admin' is the one in dom1


I could avoid using autorequire and force the use of explicit requires 
=> 'name' in every keystone_user_role, but that would break existing 
manifests.


I could use autorequire, but force manifest writers to be consistent 
with fully qualified resource naming.  That is, the manifest writer 
would have to know that the resource name of the admin user in dom1 is 
always 'admin::dom1' and the resource name of the admin user in dom2 is 
always 'admin', and they must be used that way, consistently, 
everywhere, even in deeply nested classes/defines.


Another problem is with the puppet provider self.instances method. This 
method queries all of the external resources of a given type and 
constructs something like a named puppet resource representing the 
external resource.  For example: the keystone_user self.instances does 
'openstack user list --long' and creates named puppet resources.


What happens if openstack user list --long returns admin in dom1 and 
admin in dom2?  How does self.instances name these uniquely?  If it 
always names these "fully qualified" with ::domain, this will cause 
problems.  This could be handled with some extra logic:
1) If name is unique (i.e. there is only one user named 'admin' in all 
domains), then just create the resource with the name 'admin' - no 
::domain.  This would require self.instances to keep track of every 
returned resource e.g. collect all of the list results in a hash, and 
only instantiate the resources once it is known that the name is unique.
2) If the resource is in the designated "default domain" (i.e. the 
domain id matches [identity] default_domain_id, or 'default' if 
[identity] default_domain_id not configured), then use the name without 
the domain


2 seems like the intuitive way to go - works well with existing 
manifests which are not yet domain aware, and things like puppet 
resource keystone_user [name] will almost always work as expected. But 
this means that manifest writers have to know that all puppet-keystone 
manifests and code need to know what the domain name corresponding to 
the default_domain_id is, and know that the resource in that domain 
should always be referred to as 'name', not 'name::domain'.  For 
example, if you have user 'someuser' in domain 'Default', and domain 
'Default' is the default domain with an id of 'default', all resources 
should should use keystone_user { 'name': } and keystone_user_role { 
'name@project': }


self.instances could return the fully qualified name _and_ the short 
name, where the short name is the resource in the default domain. This 
might cause confusion with puppet resource 'resource_name', when the 
user sees both 'admin' and 'admin::dom1' but at least the user will see 
one or the other, and both "puppet resource keystone_user admin" and 
"puppet resource keystone_user admin::dom1" will work.  This would 
require some minor adjustment to the currently proposed patches for the 
self.instances - self.instances would need to be aware of the default 
domain and construct short names for those resources, and self.prefetch 
would also need to know to use the shortnamed resource for the default 
domain.  In this case, the short name would be an "alias" for the fully 
qualified name, sort of like a dns query where both the shortnames and 
fqdns are returned.





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Stuck on VIF plugin script?

2015-06-22 Thread Jay Pipes


On 06/22/2015 06:59 AM, Neil Jerram wrote:

On 22/06/15 11:28, neil.jer...@metaswitch.com wrote:

‎It seems like we're possibly stuck now on the VIF plugin script spec
[1]; there being core comments in apparently conflicting directions. I
wonder if it's still feasible for a version of this to land during
Liberty?

[1] https://review.openstack.org/#/c/162468/

I plan to reread everyone's comments and see if ‎I can propose a way
forward - but thought I'd circulate this email first in case anyone
wants to say that it's already too late...


I'm sorry, actually Dan PB has already done this with [2], so I'll
review that.

[2] https://review.openstack.org/#/c/193668/


Yes, and actually this weekend, I pushed a start of an os_vif library to 
GitHub:


https://github.com/jaypipes/os_vif

I still need to port/reconcile the unit tests and add functional tests, 
but all the basics have been ported from the Nova libvirt/vif.py file, 
along with supporting code from nova/network/linux_net.py.


The proposed library above adds object models for the VIF, Network, and 
Subnet objects, using oslo.versionedobjects, that can be used to 
construct the VIF requests.


Each vendor/VIF type has its own plugin, loaded via Stevedore and 
standard Python setuptools entry points, making it trivial to add new 
plugins and use this library for coding instead of the Nova codebase 
itself, which should solve the problem of development velocity.


The plugins need only implement plug() and unplug(). I left the libvirt 
XML configuration pieces in Nova's libvirt/vif.py file for a couple reasons:


a) libvirt knows best how to configure this stuff

b) os_vif should really be a resource for more than just libvirt

Anyway, it's at least something that we can use as a starting point for 
the discussion on Dan B's spec above...


Best,
-jay


(For some reason Gerrit has stopped sending me new comments on [1]. Does
anyone know why that happens?)

Regards,
 Neil

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api] [all] To changes-since or not to changes-since

2015-06-22 Thread Brant Knudson

Keystone also has a resource that provides changes since[1], the query
parameter used is "since".

[1]
http://specs.openstack.org/openstack/keystone-specs/api/v3/identity-api-v3-os-revoke-ext.html#list-revocation-events

Ciao, Brant



On Fri, Jun 19, 2015 at 3:25 PM, Sean Dague  wrote:

> On 06/19/2015 05:07 AM, Chris Dent wrote:
> >
> > There's an open question in the API-WG on whether to formalize or
> > otherwise enshrine the concept of a "changes-since" query parameter
> > on collection oriented resources across the projects. The original
> > source of this concept is from Nova's API:
> >
> >
> >
> http://docs.openstack.org/developer/nova/v2/polling_changes-since_parameter.html
> >
> >
> > There are arguments for and against but we've been unable to reach a
> > consensus so the agreed next step was to bring the issue to the
> > mailing list so more people can hash it out and provide their input.
> > The hope is that concerns or constraints that those in the group
> > are not aware of will be revealed and a better decision will be
> > reached.
> >
> > The basic idea of "changes-since" is that it can be used as a way to
> > signal that the requestor is doing some polling and would like to
> > ask "Give me stuff that has changed since the last time I checked".
> > As I understand it, for the current implementations (in Nova and
> > Glance) this means "including stuff that has been deleted". Repeated
> > requests to the resource with a "changes-since" parameter gives a
> > running report on the evolving state of the resource. This is intended
> > to allow "efficient polling"[0].
> >
> > The pro camp on this likes it because this is very useful and quite
> > humane: The requestor doesn't need to know the details of how the
> > query is is implemented under the hood. That is, if there are
> > several timestamps associated with the singular resources in the
> > collection which of those are used for time comparison and which
> > attributes (such as "state" with a value of "deleted") are used to
> > determine if a singular resource is included. The service gets to
> > decide these things and in order for "efficient polling" to actually
> > be achieved it needs to do in order to make effective queries of the
> > data store.
> >
> > The con camp doesn't like it because it introduces magic, ambiguity
> > and inconsistency into the API (when viewed from a cross-project
> > perspective) and one of the defining goals of the working group is
> > to slowly guide things to some measure of consistency. The
> > alternate approach is to formulate a fairly rigorous query system
> > for both filtering[1] and sorting[2] and use that to specify
> > explicit queries that state "I want resources that are newer than time
> > X in timestamp attribute 'updated_at' _and_ have attribute 'state'
> > that is one of 'foo', 'bar' or 'baz'".
> >
> > (I hope I have represented the two camps properly here and not
> > introduced any bias. Myself I'm completely on the fence. If you
> > think I've misrepresented the state of things please post a
> > clarifying response.)
> >
> > The questions come down to:
> >
> > * Are there additional relevant pros and cons for the two proposals?
> > * Are there additional proposals which can address the shortcomings
> >   in either?
> >
> > Thanks for your input.
> >
> > [0] Please try to refrain from responses on the line of "ha!
> > efficiency! that's hilarious!" and "ZOMG, polling, that's so
> > last century". Everybody knows this already and it's not
> > germane to the immediate concerns. We'll get to a fully message
> > driven architecture next week. This week we're still working
> > with what we've got.
> > [1] filtering guideline proposal
> > https://review.openstack.org/#/c/177468/
> > [2] sorting guideline proposal
> > https://review.openstack.org/#/c/145579/
>
> I think that is a reasonable summation. My personal feeling is that if
> 'changes-since' is strictly defined as the AND of 2 other standard
> filters, it's not feeling very relevant to recommend it existing. It's
> now just a 2nd way to do exactly the same thing, and all we can do is
> screw it up ("A man with one watch always knows what time it is. A man
> with two watches is never quite sure.").
>
> If it's left a little more vague of "for resources that you expect to be
> regularly polled, this can provide an interface to only show the
> consumer relevent resources. If you choose to implement this, you must
> document what criteria is used to return resources from the url." And
> allow the possibility that their might be different sensible definitions
> depending on the resource, I'm happier about it.
>
> -Sean
>
> --
> Sean Dague
> http://dague.net
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bi

Re: [openstack-dev] Proposal of nova-hyper driver

2015-06-22 Thread Peng Zhao

Hyper is using hypervisor to run Docker image, therefore it can support most
features in the matrix, both mandatory and optional/choice.
- Hyper - Make VM run like 
Container


On Mon, Jun 22, 2015 at 4:18 PM, Sahid Orentino Ferdjaoui < 
sahid.ferdja...@redhat.com > wrote:
On Sun, Jun 21, 2015 at 07:18:10PM +0300, Joe Gordon wrote:
> On Fri, Jun 19, 2015 at 12:55 PM, Peng Zhao  wrote:
>
> > Hi, all,
> >
> > I would like to propose nova-hyper driver:
> > https://blueprints.launchpad. net/nova/+spec/nova-hyper .
> >
> > - What is Hyper?
> > Put simply, Hyper is a hypervisor-agnostic Docker runtime. It is
> > similar to Intel’s ClearContainer, allowing to run a Docker image with any
> > hypervisor.
> >
> >
> > - Why Hyper driver?
> > Given its hypervisor nature, Hyper makes it easy to integrate with
> > OpenStack ecosystem, e.g. Nova, Cinder, Neutron
> >
> > - How to implement?
> > Similar to nova-docker driver. Hyper has a daemon “hyperd” running on
> > each physical box. hyperd exposed a set of REST APIs. Integrating Nova with
> > the APIs would do the job.
> >
> > - Roadmap
> > Integrate with Magnum & Ironic.
> >
> >
> This sounds like a better fit for something on top of Nova such as Magnum
> then as a Nova driver.
>
> Nova only supports things that look like 'VMs'. That includes bare metal,
> and containers, but it only includes a subset of container features.
>
> Looking at the hyper CLI [0], there are many commands that nova would not
> suppprt, such as:
>
> * The pod notion
> * exec
> * pull

Then I guess you need to see if Hyper can implement mandatory features
for Nova [1], [2].

[1] http://docs.openstack.org/ developer/nova/support-matrix. html
[2] https://wiki.openstack.org/ wiki/HypervisorSupportMatrix

> [0] https://docs.hyper.sh/ reference/cli.html
>
>
>
> > Appreciate for comments and inputs!
> > Thanks,Peng
> >
> > -- ---
> > Hyper - Make VM run like Container
> >
> >
> > __ __ __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: OpenStack-dev-request@lists. openstack.org?subject: unsubscribe
> > http://lists.openstack.org/ cgi-bin/mailman/listinfo/ openstack-dev
> >
> >

> __ __ __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-request@lists. openstack.org?subject: unsubscribe
> http://lists.openstack.org/ cgi-bin/mailman/listinfo/ openstack-dev


__ __ __
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-request@lists. openstack.org?subject: unsubscribe
http://lists.openstack.org/ cgi-bin/mailman/listinfo/ openstack-dev__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ceilometer] can we get ceilometermiddleware to use a config file instead of transport_url?