Re: [openstack-dev] [Nova] Some thoughts on API microversions

[Ghanshyam Mann]

> -Original Message-
> From: Andrew Laski [mailto:and...@lascii.com]
> Sent: 04 August 2016 22:18
> To: openstack-dev@lists.openstack.org
> Subject: Re: [openstack-dev] [Nova] Some thoughts on API microversions
> 
> 
> 
> On Thu, Aug 4, 2016, at 08:20 AM, Sean Dague wrote:
> > On 08/03/2016 08:54 PM, Andrew Laski wrote:
> > > I've brought some of these thoughts up a few times in conversations
> > > where the Nova team is trying to decide if a particular change
> > > warrants a microversion. I'm sure I've annoyed some people by this
> > > point because it wasn't germane to those discussions. So I'll lay
> > > this out in it's own thread.
> > >
> > > I am a fan of microversions. I think they work wonderfully to
> > > express when a resource representation changes, or when different
> > > data is required in a request. This allows clients to make the same
> > > request across multiple clouds and expect the exact same response
> > > format, assuming those clouds support that particular microversion.
> > > I also think they work well to express that a new resource is
> > > available. However I do think think they have some shortcomings in
> > > expressing that a resource has been removed. But in short I think
> > > microversions work great for expressing that there have been changes
> > > to the structure and format of the API.
> > >
> > > I think microversions are being overused as a signal for other types
> > > of changes in the API because they are the only tool we have
> > > available. The most recent example is a proposal to allow the
> > > revert_resize API call to work when a resizing instance ends up in
> > > an error state. I consider microversions to be problematic for
> > > changes like that because we end up in one of two situations:
> > >
> > > 1. The microversion is a signal that the API now supports this
> > > action, but users can perform the action at any microversion. What
> > > this really indicates is that the deployment being queried has
> > > upgraded to a certain point and has a new capability. The structure
> > > and format of the API have not changed so an API microversion is the
> > > wrong tool here. And the expected use of a microversion, in my
> > > opinion, is to demarcate that the API is now different at this particular
> point.
> > >
> > > 2. The microversion is a signal that the API now supports this
> > > action, and users are restricted to using it only on or after that
> microversion.
> > > In many cases this is an artificial constraint placed just to
> > > satisfy the expectation that the API does not change before the
> microversion.
> > > But the reality is that if the API change was exposed to every
> > > microversion it does not affect the ability I lauded above of a
> > > client being able to send the same request and receive the same
> > > response from disparate clouds. In other words exposing the new
> > > action for all microversions does not affect the interoperability
> > > story of Nova which is the real use case for microversions. I do
> > > recognize that the situation may be more nuanced and constraining
> > > the action to specific microversions may be necessary, but that's not
> always true.
> > >
> > > In case 1 above I think we could find a better way to do this. And I
> > > don't think we should do case 2, though there may be special cases
> > > that warrant it.
> > >
> > > As possible alternate signalling methods I would like to propose the
> > > following for consideration:
> > >
> > > Exposing capabilities that a user is allowed to use. This has been
> > > discussed before and there is general agreement that this is
> > > something we would like in Nova. Capabilities will programatically
> > > inform users that a new action has been added or an existing action
> > > can be performed in more cases, like revert_resize. With that in
> > > place we can avoid the ambiguous use of microversions to do that. In
> > > the meantime I would like the team to consider not using
> > > microversions for this case. We have enough of them being added that
> > > I think for now we could just wait for the next microversion after a
> > > capability is added and document the new capability there.
> >
> > The problem with this approach is that the capability add isn't on a
> > microversion boundary, as long as we continue to believe that we want
> > to support CD deployments this means people can deploy code with the
> > behavior change, that's not documented or signaled any way.
> 
> The fact that the capability add isn't on a microversion boundary is exactly 
> my
> point. There's no need for it to be in many cases. But it would only apply for
> capability adds which don't affect the interoperability of multiple
> deployments.
> 
> The signaling would come from the ability to query the capabilities listing. A
> change in what that listing returns indicates a behavior change.
> 
> Another reason I like the above mechanism is that it handles differences in
> policy 

Re: [openstack-dev] [tempest][swift][radosgw] Can we please merge the fix for the RFC 7230 violation issue?

[John Dickinson]

On 8 Aug 2016, at 9:50, Jay Pipes wrote:

> Tempest devs,
>
> Let me please draw your attention to a LP bug that may not seem particularly 
> high priority, but I believe could be resolved easily with a patch already 
> proposed.
>
> LP bug 1536251 [1] accurately states that Tempest is actively verifying that 
> an OpenStack API call violates RFC 7230.
>
> When a 204 No Content is received, the Content-Length header MUST NOT be 
> present.
>
> However, Swift returns a Content-Length header and also an HTTP response code 
> of 204 for a request to list containers of a new user (that has no 
> containers).
>
> Tempest has been validating this behaviour even though it is a violation of 
> RFC 7230:
>
> https://github.com/openstack/tempest/blob/master/tempest/api/object_storage/test_account_services.py#L81
>
> RadosGW provides a proxy API that attempts to match the OpenStack Object 
> Storage API, backed by Ceph object storage. In order for RadosGW to pass 
> RefStack's burden of compatibility, it must pass the Tempest OpenStack Object 
> Storage API tests. It currently cannot do so because RadosGW does not violate 
> RFC 7230.
>
> The RadosGW developer community does not wish to argue about whether or not 
> to make Swift's API comply with RFC 7230. At the same time, they do not want 
> to add a configuration option to RadosGW to force the proxy service to 
> violate RFC 7230 just to satisfy the RefStack/Tempest API tests.

I think tempest should merge the proposed patch (or one like it) so that 
content-length isn't checked on a 204 response. That solves the discrepancy 
when deployers are running Swift behind some other web server or caching system 
that is stripping the header. On the Swift side, we've got to consider the 
question of risk of breaking clients vs violating updated RFCs. My gut says 
we'll drip the header, but that's something that we'll discuss. But either way, 
it seems silly to me to be blocked here by tempest.


>
> Therefore, Radoslaw (cc'd) has proposed a patch to Tempest that would allow 
> RadosGW's proxy API to meet the RefStack compatibility tests while also not 
> violating RFC 7230 and not requiring any change of Swift:
>
> https://review.openstack.org/#/c/272062
>
> I ask Tempest devs to re-review the above patch and consider merging it for 
> the sake of collaboration between the OpenStack and Ceph developer 
> communities.
>
> Thanks very much!
> -jay
>
> [1] https://bugs.launchpad.net/tempest/+bug/1536251
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Network Template Generator

[Dan Prince]

On Mon, 2016-08-08 at 15:42 -0500, Ben Nemec wrote:
> This is something that has existed for a while, but I had been
> hesitant
> to evangelize it until it was a little more proven.  At this point
> I've
> used it to generate templates for a number of different environments,
> and it has worked well.  I decided it was time to record another demo
> and throw it out there for the broader community to look at.  See
> details on my blog:
> http://blog.nemebean.com/content/tripleo-network-isolation-template-g
> enerator
> 
> Most of what you need to know is either there or in the video itself.
> Let me know what you think.

Very cool. For those that don't like "hand cutting" their own network
configuration templates this is a good CLI based generator.

Like you mention it would be nice to eventually converge this tool
somehow into both the UI and CLI but given that it works with older
releases as well it makes sense that it is CLI only for now.

Dan

> 
> Thanks.
> 
> -Ben
> 
> _
> _
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubs
> cribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Infra] Meeting Tuesday August 9th at 19:00 UTC

[Elizabeth K. Joseph]

Hi everyone,

The OpenStack Infrastructure (Infra) team is having our next weekly
meeting on Tuesday August 9th, at 19:00 UTC in #openstack-meeting

Meeting agenda available here:
https://wiki.openstack.org/wiki/Meetings/InfraTeamMeeting#Agenda_for_next_meeting

Anyone is welcome to to add agenda items and everyone interested in
the project infrastructure and process surrounding automated testing
and deployment is encouraged to attend.

In case you missed it or would like a refresher, the meeting minutes
and full logs from our last meeting are available:

Minutes: 
http://eavesdrop.openstack.org/meetings/infra/2016/infra.2016-08-02-19.04.html
Minutes (text):
http://eavesdrop.openstack.org/meetings/infra/2016/infra.2016-08-02-19.04.txt
Log: 
http://eavesdrop.openstack.org/meetings/infra/2016/infra.2016-08-02-19.04.log.html

-- 
Elizabeth Krumbach Joseph || Lyz || pleia2

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tacker] Weekly meeting agenda

[Sridhar Ramaswamy]

Hi Dharmendra,

Sure, will add it to the agenda. However keep in mind, Newton release
related items are higher priority though.

- Sridhar

On Mon, Aug 8, 2016 at 5:55 AM, Dharmendra Kushwaha
 wrote:
> Dear Sridhar,
>
> Can we also discuss the NSD?
>
> Thanks & Regards
> Dharmendra Kushwaha
> 
> From: Sridhar Ramaswamy [sric...@gmail.com]
> Sent: Monday, August 08, 2016 8:49 AM
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: [openstack-dev] [tacker] Weekly meeting agenda
>
> Tackers,
>
> Here is the agenda for this week's irc meeting,
>
> https://wiki.openstack.org/wiki/Meetings/Tacker
>
> - Announcements
> - Newton release deadlines
> - Newton priorities - feature / RFE go/no-go check
> - Mistral workflow BP
> - Open Discussion
>
> Let me know if you've anything else to discuss.
>
> thanks,
> Sridhar
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra][neutron] - best way to load 8021q kernel module into cirros

[Ian Wienand]

On 08/09/2016 02:10 AM, Jeremy Stanley wrote:

I haven't personally tested the CirrOS build instructions, but have
a feeling writing a diskimage-builder element wrapper for that
wouldn't be particularly challenging.


I'm not exactly sure it fits that well into dib; it seems like
"bundle" has it mostly figured out.  I say that based on [1] where we
are discussing a similar thing for cirros images with watchdog
support.

As mentioned we can easily build these and store them, and put them on
mirrors if we need them closer to nodes.  What I mentioned in [1] and
didn't particularly like is if the build of these images is totally
removed from where they're actually used (e.g. a custom script inside
a job in project-config, where basically anyone outside infra can't
easily replicate the build for a local test).  But if several projects
are building slightly different cusomised cirros images, it might be
worth consolidating.

-i

[1] https://review.openstack.org/#/c/338167/2/tools/build-watchdog-images.sh


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [magnum][heat] 2 million requests / sec, 100s of nodes

[Zane Bitter]

On 08/08/16 17:09, Ricardo Rocha wrote:

* trying the convergence_engine: as far as i could see this is already
there, just not enabled by default. We can give it a try and let you
know how it goes if there's no obvious drawback. Would it just work
with the current schema? We're running heat mitaka


There's been a bunch of bug fixes (including performance fixes) during 
Newton, which is why it wasn't enabled by default in Mitaka (and 
hopefully will be in Newton - it is enabled on master right now). We 
also haven't been backporting convergence bugfixes to the stable branch. 
So while you can just turn it on and it should work modulo bugs, I 
wouldn't recommend it.


cheers,
Zane.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [scheduler] Use ResourceProviderTags instead of ResourceClass?

[Chris Dent]

On Mon, 8 Aug 2016, Ed Leafe wrote:


Me too. I think that this is one case where thinking in SQL messes you
up. Sure, you can probably make it work by hacking in the concept of
infinity into the code, but there will still be a conceptual
disconnect. And later, when we inevitably need to enhance resource
providers and their capabilities, we will end up creating another hack
to work around what is an actual inventory and what is this new
infinite inventory thing.


/me shrugs

This has been an interesting exercise because it does get people to
express their ideas and their concerns a bit more. Which even if it
doesn't change this round of stuff that we create helps for the next
round.

In my case my interest in exploring the model I've been describing is
largely driven by wanting to _not_ think in SQL but in a way that is
more simple math but with a very constrained grammar.

Part of the goal here is to make "inevitable enhancement" which
involves a different conceptual model something that is constrained
to such an extent that it might be impossible and thus has to happen
in a separate service; so that we can avoid monoliths

This constrained grammar thing is really important.


Oh, and think of the person coming in new to the placement engine, and
having to explain what an infinite inventory is. Imagine their face.


Weird, I wonder what experience I had that makes that so natural for me.

I conceded a long time ago, but you guys keep saying things that
make me want to keep talking about it.

--
Chris Dent   ┬─┬ノ( º _ ºノ) http://anticdent.org/
freenode: cdent tw: @anticdent__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Cinder] [stable] [all] Changing stable policy for drivers

[Ihar Hrachyshka]

Duncan Thomas  wrote:


On 8 August 2016 at 21:12, Matthew Treinish  wrote:
Ignoring all that, this is also contrary to how we perform testing in  
OpenStack.

We don't turn off entire classes of testing we have so we can land patches,
that's just a recipe for disaster.

But is it more of a disaster (for the consumers) than zero testing, zero  
review, scattered around the internet if-you're-lucky-with-a-good-wind  
you'll maybe get the right patch set? Because that's where we are right  
now, and vendors, distributors and the cinder core team are all saying  
it's a disaster.


If consumers rely on upstream releases, then they are expected to migrate  
to newer releases after EOL, not switch to a random branch on the internet.  
If they rely on some commercial product, then they usually have an extended  
period of support and certification for their drivers, so it’s not a  
problem for them.


Ihar

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tempest][swift][radosgw] Can we please merge the fix for the RFC 7230 violation issue?

[Dave Walker]

On 8 August 2016 at 17:50, Jay Pipes  wrote:

> Tempest devs,
>
> Let me please draw your attention to a LP bug that may not seem
> particularly high priority, but I believe could be resolved easily with a
> patch already proposed.
>
> LP bug 1536251 [1] accurately states that Tempest is actively verifying
> that an OpenStack API call violates RFC 7230.
>
> When a 204 No Content is received, the Content-Length header MUST NOT be
> present.
>
> However, Swift returns a Content-Length header and also an HTTP response
> code of 204 for a request to list containers of a new user (that has no
> containers).
>
> Tempest has been validating this behaviour even though it is a violation
> of RFC 7230:
>
> https://github.com/openstack/tempest/blob/master/tempest/api
> /object_storage/test_account_services.py#L81
>
> RadosGW provides a proxy API that attempts to match the OpenStack Object
> Storage API, backed by Ceph object storage. In order for RadosGW to pass
> RefStack's burden of compatibility, it must pass the Tempest OpenStack
> Object Storage API tests. It currently cannot do so because RadosGW does
> not violate RFC 7230.
>
> The RadosGW developer community does not wish to argue about whether or
> not to make Swift's API comply with RFC 7230. At the same time, they do not
> want to add a configuration option to RadosGW to force the proxy service to
> violate RFC 7230 just to satisfy the RefStack/Tempest API tests.
>
> Therefore, Radoslaw (cc'd) has proposed a patch to Tempest that would
> allow RadosGW's proxy API to meet the RefStack compatibility tests while
> also not violating RFC 7230 and not requiring any change of Swift:
>
> https://review.openstack.org/#/c/272062
>
> I ask Tempest devs to re-review the above patch and consider merging it
> for the sake of collaboration between the OpenStack and Ceph developer
> communities.
>
> Thanks very much!
> -jay
>
> [1] https://bugs.launchpad.net/tempest/+bug/1536251



These sorts of issues aren't just theoretical and following policy for the
sake of it..  Glance had 3 x areas where 200 responses that also included a
Location header (against RFC-2616 §14.30) which totally broke glance when
deployed behind apache+fcgid+flup (the presence of Location, that stack
rewrote it to a 302).

Fun bug btw:
https://bugs.launchpad.net/glance/+bug/1299095

--
Kind Regards,
Dave Walker
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Cinder] [stable] [all] Changing stable policy for drivers

[Duncan Thomas]

On 8 August 2016 at 21:12, Matthew Treinish  wrote:

> Ignoring all that, this is also contrary to how we perform testing in
> OpenStack.
> We don't turn off entire classes of testing we have so we can land patches,
> that's just a recipe for disaster.
>

But is it more of a disaster (for the consumers) than zero testing, zero
review, scattered around the internet if-you're-lucky-with-a-good-wind
you'll maybe get the right patch set? Because that's where we are right
now, and vendors, distributors and the cinder core team are all saying it's
a disaster.



-- 
Duncan Thomas
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Nova mascot

[Clint Byrum]

Excerpts from Matt Riedemann's message of 2016-08-08 14:35:12 -0500:
> Not to be a major curmudgeon but I think we'd basically decided at the 
> midcycle (actually weeks before) that Nova wasn't doing the mascot thing.
> 

Could you maybe summarize the reason for this decision?

Seems like everybody else is taking this moment to look inward and
think about how they want to be seen. Why wouldn't Nova want to take an
opportunity to do the same?

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [scheduler] Use ResourceProviderTags instead of ResourceClass?

[Ed Leafe]

On Aug 8, 2016, at 9:42 AM, Jay Pipes  wrote:

>> When some ssd-ness is consumed, all of it (infinity) is still left
>> over.
>> 
>> For me it is easier: a resource provider has just one way of
>> describing what it can do: classes of inventory (that provide
>> gigabytes of disk that are ssd). When we add tags/capabilities
>> we have another mode of description.
> 
> I could not disagree more. :)

Me too. I think that this is one case where thinking in SQL messes you up. 
Sure, you can probably make it work by hacking in the concept of infinity into 
the code, but there will still be a conceptual disconnect. And later, when we 
inevitably need to enhance resource providers and their capabilities, we will 
end up creating another hack to work around what is an actual inventory and 
what is this new infinite inventory thing.

Oh, and think of the person coming in new to the placement engine, and having 
to explain what an infinite inventory is. Imagine their face.


-- Ed Leafe






__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [magnum][heat] 2 million requests / sec, 100s of nodes

[Ricardo Rocha]

Hi.

On Mon, Aug 8, 2016 at 6:17 PM, Zane Bitter  wrote:
> On 05/08/16 12:01, Hongbin Lu wrote:
>>
>> Add [heat] to the title to get more feedback.
>>
>>
>>
>> Best regards,
>>
>> Hongbin
>>
>>
>>
>> *From:*Ricardo Rocha [mailto:rocha.po...@gmail.com]
>> *Sent:* August-05-16 5:48 AM
>> *To:* OpenStack Development Mailing List (not for usage questions)
>> *Subject:* Re: [openstack-dev] [magnum] 2 million requests / sec, 100s
>> of nodes
>>
>>
>>
>> Hi.
>>
>>
>>
>> Quick update is 1000 nodes and 7 million reqs/sec :) - and the number of
>> requests should be higher but we had some internal issues. We have a
>> submission for barcelona to provide a lot more details.
>>
>>
>>
>> But a couple questions came during the exercise:
>>
>>
>>
>> 1. Do we really need a volume in the VMs? On large clusters this is a
>> burden, and local storage only should be enough?
>>
>>
>>
>> 2. We observe a significant delay (~10min, which is half the total time
>> to deploy the cluster) on heat when it seems to be crunching the
>> kube_minions nested stacks. Once it's done, it still adds new stacks
>> gradually, so it doesn't look like it precomputed all the info in advance
>>
>>
>>
>> Anyone tried to scale Heat to stacks this size? We end up with a stack
>> with:
>>
>> * 1000 nested stacks (depth 2)
>>
>> * 22000 resources
>>
>> * 47008 events
>
>
> Wow, that's a big stack :) TripleO has certainly been pushing the boundaries
> of how big a stack Heat can handle, but this sounds like another step up
> even from there.
>
>> And already changed most of the timeout/retrial values for rpc to get
>> this working.
>>
>>
>>
>> This delay is already visible in clusters of 512 nodes, but 40% of the
>> time in 1000 nodes seems like something we could improve. Any hints on
>> Heat configuration optimizations for large stacks very welcome.
>
>
> Y'all were right to set max_resources_per_stack to -1, because actually
> checking the number of resources in a tree of stacks is sloow. (Not as
> slow as it used to be when it was O(n^2), but still pretty slow.)
>
> We're actively working on trying to make Heat more horizontally scalable
> (even at the cost of some performance penalty) so that if you need to handle
> this kind of scale then you'll be able to reach it by adding more
> heat-engines. Another big step forward on this front is coming with Newton,
> as (barring major bugs) the convergence_engine architecture will be enabled
> by default.
>
> RPC timeouts are caused by the synchronous work that Heat does before
> returning a result to the caller. Most of this is validation of the data
> provided by the user. We've talked about trying to reduce the amount of
> validation done synchronously to a minimum (just enough to guarantee that we
> can store and retrieve the data from the DB) and push the rest into the
> asynchronous part of the stack operation alongside the actual create/update.
> (FWIW, TripleO typically uses a 600s RPC timeout.)
>
> The "QueuePool limit of size ... overflow ... reached" sounds like we're
> pulling messages off the queue even when we don't have threads available in
> the pool to pass them to. If you have a fix for this it would be much
> appreciated. However, I don't think there's any guarantee that just leaving
> messages on the queue can't lead to deadlocks. The problem with very large
> trees of nested stacks is not so much that it's a lot of stacks (Heat
> doesn't have _too_ much trouble with that) but that they all have to be
> processed simultaneously. e.g. to validate the top level stack you also need
> to validate all of the lower level stacks before returning the result. If
> higher-level stacks consume all of the thread pools then you'll get a
> deadlock as you'll be unable to validate any lower-level stacks. At this
> point you'd have maxed out the capacity of your Heat engines to process
> stacks simultaneously and you'd need to scale out to more Heat engines. The
> solution is probably to try limit the number of nested stack validations we
> send out concurrently.
>
> Improving performance at scale is a priority area of focus for the Heat team
> at the moment. That's been mostly driven by TripleO and Sahara, but we'd be
> very keen to hear about the kind of loads that Magnum is putting on Heat and
> working with folks across the community to figure out how to improve things
> for those use cases.

Thanks for the detailed reply, especially regarding the handling of
the nested stacks by the engines, much clearer now.

Seems like there's a couple of things we can try already:
* scaling the heat engines (we're currently running 3 nodes with 5
engines each, can check if more help, though it seems with >1000
nested stacks it will be hard to avoid starvation)
* trying the convergence_engine: as far as i could see this is already
there, just not enabled by default. We can give it a try and let you
know how it goes if there's no obvious drawback. Would it just work
with the current 

[openstack-dev] [ironic] weekly subteam status report

[Loo, Ruby]

Hi,

We are zealous to present this week's subteam report for Ironic. As usual, this 
is pulled directly from the Ironic whiteboard[0] and formatted.

Bugs (dtantsur)
===
- no update - dtantsur is out sick today

Network isolation (Neutron/Ironic work) (jroll, TheJulia, devananda)

* trello: 
https://trello.com/c/HWVHhxOj/1-multi-tenant-networking-network-isolation
- main code landed, non-voting job in the gate. Please pay attention to it!
- work still to do: portgroups, security groups, inspector integration

Gate improvements (jlvillal, lucasagomes, dtantsur)
===
- python-ironicclient now has a voting functional test job
- ironic now has an experimental job for multi-node testing. This is a step in 
our work to get to a multi-node grenade job

Multiple compute hosts (jroll, devananda)
=
* trello: https://trello.com/c/OXYBHStp/7-multiple-compute-hosts
- nova code has one +2
- https://review.openstack.org/#/c/348443/

Generic boot-from-volume (TheJulia, dtantsur, lucasagomes)
==
* trello: https://trello.com/c/UttNjDB7/13-generic-boot-from-volume
- Specification needs reviews: https://review.openstack.org/#/c/294995/
- Volume connector revisions are available for review, and start at 
https://review.openstack.org/#/c/200983/

Agent top-level API promotion (dtantsur)

* trello: 
https://trello.com/c/37YuKIB8/28-promote-agent-vendor-passthru-to-core-api
- agent vendor passthru API deprecation is done, last patch is waiting for gate 
fixes, should land today

Driver composition (dtantsur)
=
* trello: https://trello.com/c/fTya14y6/14-driver-composition
- (unknown - dtantsur is out today)

OpenStackClient plugin for ironic (thrash, dtantsur, rloo)
==
* trello: https://trello.com/c/ckqtq3kG/16-openstackclient-plugin-for-ironic
- patches ready for review: 
https://review.openstack.org/#/q/status:open+project:openstack/python-ironicclient+branch:master+topic:bug/1526479

Notifications (mariojv)
===
* trello: https://trello.com/c/MD8HNcwJ/17-notifications
- Responded to comments and rebased, code needs re-reviews
- https://review.openstack.org/#/c/298461
- https://review.openstack.org/#/c/321865/
- CRUD and provision state notification spec needs reviews 
https://review.openstack.org/#/c/347242/

Keystone policy support (JayF, devananda)
=
* trello: https://trello.com/c/P5q3Es2z/15-keystone-policy-support
- most code landed, need to finish follow-ups and docs

Software metrics (JayF, alineb)
===
* trello: https://trello.com/c/XtPGyHcP/18-software-metrics
- Metrics coverage:
- Full API coverage
- About 50% metrics coverage in various drivers (Should be up to 75% by EOW)
- IPA metrics code needs reviews -> https://review.openstack.org/#/c/119981/

Active node creation (TheJulia)
===
* trello: https://trello.com/c/BwA91osf/22-active-node-creation
- No update, reviews for tempest test addition are still pending.  
https://review.openstack.org/#/q/status:open+project:openstack/ironic+branch:master+topic:bug/1526315

Serial console (yossy, hshiina, yuikotakadamori)

* trello: https://trello.com/c/nm3I8djr/20-serial-console
- console_utils: merged
- IPMITool driver interface: merged
- follow-on patches:
- https://review.openstack.org/#/c/335378/: need review (all primary 
contacts +1d)
- https://review.openstack.org/#/c/349400/: merged
- install guide: needs review (all primary contacts +1d)  
https://review.openstack.org/#/c/293872/
- nova patch: needs review (-2 was removed)  
https://review.openstack.org/#/c/328157/

Enhanced root device hints (lucasagomes)

* trello: https://trello.com/c/f9DTEvDB/21-enhanced-root-device-hints
- Ironic-lib patch extending the parse_root_device_hints() method to work with 
operators (need more tests): https://review.openstack.org/#/c/348953/

Rescue mode (JayF)
==
* trello: https://trello.com/c/PwH1pexJ/23-rescue-mode
- Spec needs reviews  https://review.openstack.org/#/c/171878/ (1x+2, no 
negative feedback)
- Draft code already partially up, should be 75%+ up by EOW

Inspector (dtansur)
===
- HA effort: introspection state in database; patch up for review: 
https://review.openstack.org/#/c/348943/

Drivers:

OneView (gabriel-bezerra/thiagop)
-
- Dynamic allocation for Oneview drivers code was merged
- Docs included (thanks JayF and NobodyCam)

.

Until next week,
--ruby

[0] https://etherpad.openstack.org/p/IronicWhiteBoard

[openstack-dev] [TripleO] Network Template Generator

[Ben Nemec]

This is something that has existed for a while, but I had been hesitant
to evangelize it until it was a little more proven.  At this point I've
used it to generate templates for a number of different environments,
and it has worked well.  I decided it was time to record another demo
and throw it out there for the broader community to look at.  See
details on my blog:
http://blog.nemebean.com/content/tripleo-network-isolation-template-generator

Most of what you need to know is either there or in the video itself.
Let me know what you think.

Thanks.

-Ben

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Nova mascot

[Matt Riedemann]

On 8/8/2016 3:36 PM, Chris Friesen wrote:

On 08/08/2016 01:35 PM, Matt Riedemann wrote:


Not to be a major curmudgeon but I think we'd basically decided at the
midcycle
(actually weeks before) that Nova wasn't doing the mascot thing.


What will the Foundation do if nova doesn't propose a mascot?  Leave
nova out of the rebranding exercise?  Or will they just pick a mascot on
their own?

Chris


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



I asked the same thing when this was rolled out:

http://lists.openstack.org/pipermail/openstack-dev/2016-July/099065.html

"If you don’t select a mascot, don’t worry—we won’t pick one for you. 
We’ll be happy to create a text-only treatment of just your project 
name. Our intent is to use the logos across openstack.org, Summit 
branding, and other marketing materials to promote your project. You’ll 
also get the image file and you can use it in your presentations or to 
create more swag if you like. "


--

Thanks,

Matt Riedemann


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Nova mascot

[Augustina Ragwitz]

> What will the Foundation do if nova doesn't propose a mascot?

Nothing, I believe.



-- 
Augustina Ragwitz
Señora Software Engineer
---
Ask me about contributing to OpenStack Nova!
https://wiki.openstack.org/wiki/Nova/Mentoring
---
email: aragwitz+n...@pobox.com
irc: auggy

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Nova mascot

[Chris Friesen]

On 08/08/2016 01:35 PM, Matt Riedemann wrote:


Not to be a major curmudgeon but I think we'd basically decided at the midcycle
(actually weeks before) that Nova wasn't doing the mascot thing.


What will the Foundation do if nova doesn't propose a mascot?  Leave nova out of 
the rebranding exercise?  Or will they just pick a mascot on their own?


Chris


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Nova mascot

[Matt Riedemann]

On 8/8/2016 6:07 AM, Alexis Lee wrote:

I propose the ant, known for discovering probabilistic graph traversal
algorithms.

My other thought, the bee, has already been taken by refstack.


lxsli

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



Not to be a major curmudgeon but I think we'd basically decided at the 
midcycle (actually weeks before) that Nova wasn't doing the mascot thing.


--

Thanks,

Matt Riedemann


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [puppet] weekly meeting #89

[Emilien Macchi]

If you have any topic for our weekly meeting, please add it here:
https://etherpad.openstack.org/p/puppet-openstack-weekly-meeting-20160809

See you tomorrow,

On Tue, Aug 2, 2016 at 11:06 AM, Emilien Macchi  wrote:
> no item in our agenda, we cancelled the meeting, see you next week!
>
> On Mon, Aug 1, 2016 at 3:31 PM, Emilien Macchi  wrote:
>> Hi Puppeteers!
>>
>> We'll have our weekly meeting tomorrow at 3pm UTC on #openstack-meeting-4.
>>
>> Here's a first agenda:
>> https://etherpad.openstack.org/p/puppet-openstack-weekly-meeting-20160802
>>
>> Feel free to add topics, and any outstanding bug and patch.
>>
>> See you tomorrow!
>> Thanks,
>> --
>> Emilien Macchi
>
>
>
> --
> Emilien Macchi



-- 
Emilien Macchi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Nova mascot

[Augustina Ragwitz]

I second the supernova concept. I'm just not a huge fan of an animal.
Unless it's a Tardigrade.

I found some cartoonish things that show a supernova could be
graphically stylized:
* https://heasarc.gsfc.nasa.gov/Images/objects/snrs/SNR_cartoon2.gif
*
http://coffeeshopphysics.com/articles/2011-10/12_standing_on_a_crunchy_star/kapow.png
* http://usscouts.org/advance/Images/Nova.gif




-- 
Augustina Ragwitz
Señora Software Engineer
---
Ask me about contributing to OpenStack Nova!
https://wiki.openstack.org/wiki/Nova/Mentoring
---
email: aragwitz+n...@pobox.com
irc: auggy

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [networking-sfc] Newton release plans?

[Cathy Zhang]

Hi James,

Networking-sfc will release Newton version after Neutron pull the stable/Newton 
branch (around late Oct. and Early Nov time frame).
Basically after the branch is pulled, we will make sure all unit, functional, 
tempest tests including multi-node end-to-end tests against stable/Newton 
branch are passed, and then will do the release.

Thanks,
Cathy

From: James Page [mailto:james.p...@canonical.com]
Sent: Monday, August 08, 2016 2:08 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [networking-sfc] Newton release plans?

Hi Networking SFC team

I'm trying to get a view on a few Neutron related projects with the objective 
of lining up a release in Ubuntu and Debian alongside OpenStack Newton of 
vmware-nsx, networking-l2gw, networking-sfc and tap-as-a-service.

What are your plans for Newton? I can push snapshots into Debian/Ubuntu during 
development, but it would be nice to know at what point in time networking-sfc 
will release a version for Newton so we don't end up with a dangling snapshot 
of networking-sfc after the release of Ubuntu 16.10 and OpenStack Newton.

Cheers

James
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] devstack changing to neutron by default RSN - current issues with OVH

[Sean M. Collins]

+1 on this plan too, if the +2's I've been slinging haven't made it
obvious :)
-- 
Sean M. Collins

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Cinder] [stable] [all] Changing stable policy for drivers

[Matthew Treinish]

On Mon, Aug 08, 2016 at 07:40:56PM +0300, Duncan Thomas wrote:
> On 8 August 2016 at 18:31, Matthew Treinish  wrote:
> 
> >
> > This argument comes up at least once a cycle and there is a reason we
> > don't do
> > this. When we EOL a branch all of the infrastructure for running any ci
> > against
> > it goes away. This means devstack support, job definitions, tempest skip
> > checks,
> > etc. Leaving the branch around advertises that you can still submit
> > patches to
> > it which you can't anymore. As a community we've very clearly said that we
> > don't
> > land any code without ensuring it passes tests first, and we do not
> > maintain any
> > of the infrastructure for doing that after an EOL.
> >
> >
> Ok, to turn the question around, we (the cinder team) have recognised a
> definite and strong need to have somewhere for vendors to share patches on
> versions of Cinder older than the stable branch policy allows.
> 
> Given this need, what are our options?
> 
> 1. We could do all this outside Openstack infrastructure. There are
> significant downsides to doing so from organisational, maintenance, cost
> etc points of view. Also means that the place vendors go for these patches
> is not obvious, and the process for getting patches in is not standard.

This is probably your only viable option. As a community we've hit this boundary
many times. Everyone claims to want longer support windows but when it comes
down to it there is very little activity in making things work on stable
branches. Our support window is at it's maximum viable length now, and that's
unlikely to change anytime soon. We had a discussion on this exact topic at
summit:

https://etherpad.openstack.org/p/r.ddabf5c865d6f77740bcfbc112ed391c

> 
> 2. We could have something not named 'stable' that has looser rules than
> stable branches,, maybe just pep8 / unit / cinder in-tree tests. No
> devstack.

This is not an option, as I said before this isn't feasible. All the
infrastructure for running jobs on the old branches goes away. It's much more
than you realize is actually there. Including things like global requirements,
job definitions, and old node types. A lot of work goes into keeping all of
this running, and it's all interconnected. There is a reason we EOL a branch,
it's not to be vindictive, it's because keeping it running is too much work for
the small number of people who fix things. (and to a lesser extent an increased
burden our CI resources)

Ignoring all that, this is also contrary to how we perform testing in OpenStack.
We don't turn off entire classes of testing we have so we can land patches,
that's just a recipe for disaster.

-Matt Treinish

> 
> 3. We go with the Neutron model and take drivers out of tree. This is not
> something the cinder core team are in favour of - we see significant value
> in the code review that drivers currently get - the code quality
> improvements between when a driver is submitted and when it is merged are
> sometimes very significant. Also, taking the code out of tree makes it
> difficult to get all the drivers checked out in one place to analyse e.g.
> how a certain driver call is implemented across all the drivers, when
> reasoning or making changes to core code.
> 
> Given we've identified a clear need, and have repeated rejected one
> solution (take drivers out of tree - it has been discussed at every summit
> and midcycle for 3+ cycles), what positive suggestions can people make?
> 


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Cinder] [stable] [all] Changing stable policy for drivers

[Ben Swartzlander]

On 08/08/2016 12:36 PM, Jeremy Stanley wrote:

On 2016-08-08 13:03:51 +0200 (+0200), Ihar Hrachyshka wrote:

Sean McGinnis  wrote:

[...]

The suggestion was to just change our stable policy in regards to driver
bugfix backports. No need to create and maintain more branches. No need
to set up gate jobs and things like that.


Unless you manage to get it approved for the global policy

[...]

That was the gist of my suggestion to Sean as far as bringing this
discussion to the ML as a first option. Basically, if lots of
projects see their driver maintainers and downstream distros forking
their stable branches to add driver updates for support of newer
hardware, then see if the current OpenStack Stable Branch policy
should be amended to say that bug fixes and newer hardware support
specifically in driver source code (as long as it doesn't touch the
core service in that repo) are acceptable.

As far as the tangent this thread has taken on changing when we
delete stable branches, I feel like the only solution there is
working with the Stable Branch team to find ways to properly extend
support for the branches in question (including keeping them
properly tested). There have been ongoing efforts to make stable
branch testing less problematic, so it's possible over time we'll be
able to increase the support duration for them. Stating that it's
okay to keep them open for changes with no testing sets a terrible
precedent.


The proposal isn't "no testing". The proposal is that the gate tests 
would be minimal. We would rely heavily on the 3rd party CI system to 
actually test the patch and tell us that nothing is broken. If the 3rd 
party CI systems can't be relied on for this purpose then they're 
useless IMO.


Yes a human would have to recognize that the patch affects a particular 
vendor and know which CI system to look at before putting his +2 on. 
This is an unfortunate effect of not having 3rd party CI vote.


-Ben Swartzlander


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Cinder] [stable] [all] Changing stable policy for drivers

[Ben Swartzlander]

On 08/08/2016 12:40 PM, Duncan Thomas wrote:

On 8 August 2016 at 18:31, Matthew Treinish > wrote:


This argument comes up at least once a cycle and there is a reason
we don't do
this. When we EOL a branch all of the infrastructure for running any
ci against
it goes away. This means devstack support, job definitions, tempest
skip checks,
etc. Leaving the branch around advertises that you can still submit
patches to
it which you can't anymore. As a community we've very clearly said
that we don't
land any code without ensuring it passes tests first, and we do not
maintain any
of the infrastructure for doing that after an EOL.


Ok, to turn the question around, we (the cinder team) have recognised a
definite and strong need to have somewhere for vendors to share patches
on versions of Cinder older than the stable branch policy allows.

Given this need, what are our options?

1. We could do all this outside Openstack infrastructure. There are
significant downsides to doing so from organisational, maintenance, cost
etc points of view. Also means that the place vendors go for these
patches is not obvious, and the process for getting patches in is not
standard.

2. We could have something not named 'stable' that has looser rules than
stable branches,, maybe just pep8 / unit / cinder in-tree tests. No
devstack.


This. (2) is what I thought we were proposing from the beginning. Add a 
requirement for 3rd party CI from the affected vendor to pass and I 
think it works and benefits everyone.


-Ben Swartzlander


3. We go with the Neutron model and take drivers out of tree. This is
not something the cinder core team are in favour of - we see significant
value in the code review that drivers currently get - the code quality
improvements between when a driver is submitted and when it is merged
are sometimes very significant. Also, taking the code out of tree makes
it difficult to get all the drivers checked out in one place to analyse
e.g. how a certain driver call is implemented across all the drivers,
when reasoning or making changes to core code.

Given we've identified a clear need, and have repeated rejected one
solution (take drivers out of tree - it has been discussed at every
summit and midcycle for 3+ cycles), what positive suggestions can people
make?

--
Duncan Thomas


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra][tripleo] status of tripleo-test-cloud-rh1

[James Slagle]

On Mon, Aug 8, 2016 at 1:06 PM, Jeremy Stanley  wrote:
> On 2016-08-08 11:47:56 -0400 (-0400), James Slagle wrote:
> [...]
>> I suppose it's also possible that we might be pushing too strongly
>> down the multinode path? Is the general concensus in infra that they'd
>> like to help enable project teams to eventually add 3 and 4 (and maybe
>> more) node multinode jobs?
> [...]
>
> We've not outright rejected the idea, but do want to make sure that
> there's been suitable due diligence done explaining how the things
> you'll be able to test with >2 job nodes effectively can't be done
> with <=2.

Our current 2 node job uses the first node as the undercloud which
deploys an AIO Overcloud on the 2nd node. TripleO traditionally has
also been able to deploy standalone Compute, Cinder, Swift, and Ceph
nodes. Additionally in this cycle, a lot of work has gone into making
it fully customizable what services are deployed on which roles. You
can deploy nodes that are just API services, or just a DB server, or
rabbitmq, etc. In order to test the composability feature we need to
deploy to more than one node.

Also, we'd need at least 3 Overcloud nodes to successfully test that
we can deploy a Pacemaker managed cluster successfully.

> Also we want to be sure that projects who are interested
> in multi-node jobs start with just 2 job nodes and get some initial
> tests performing well and returning stable results before trying to
> push past 2.

I think that the 2 node job that we've added has been stable. We've
worked a few issues out that we've hit depending on which cloud
provider we land on, but generally speaking it has been very stable.

We make use of the ovs_vxlan_bridge function from devstack-gate to
configure the private networking among the nodes. I think this was a
good first step since that has been a proven way in the devstack
multinode jobs. I'd like to move to using TripleO's os-net-config in
the future though, since that is the tool used in TripleO. The end
result of the network configuration would be the same (using ovs vxlan
bridges), we'd just use a different tool to get there.

-- 
-- James Slagle
--

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Cinder] [stable] [all] Changing stable policy for drivers

[Philipp Marek]

> Ok, to turn the question around, we (the cinder team) have recognised a
> definite and strong need to have somewhere for vendors to share patches on
> versions of Cinder older than the stable branch policy allows.
> 
> Given this need, what are our options?
> 
> 1. We could do all this outside Openstack infrastructure. There are
> significant downsides to doing so from organisational, maintenance, cost
> etc points of view. Also means that the place vendors go for these patches
> is not obvious, and the process for getting patches in is not standard.
And if some people have 2 or more different storages, they might need to 
run separate Cinder processes, because the vendors' stable tree diverge and 
cannot easily be merged again.

> 2. We could have something not named 'stable' that has looser rules than
> stable branches,, maybe just pep8 / unit / cinder in-tree tests. No
> devstack.
+1 from me.
Name it "long-term-driver-only-updates" or so ;)


> 3. We go with the Neutron model and take drivers out of tree. This is not
> something the cinder core team are in favour of - we see significant value
> in the code review that drivers currently get - the code quality
> improvements between when a driver is submitted and when it is merged are
> sometimes very significant. Also, taking the code out of tree makes it
> difficult to get all the drivers checked out in one place to analyse e.g.
> how a certain driver call is implemented across all the drivers, when
> reasoning or making changes to core code.
-1

> Given we've identified a clear need, and have repeated rejected one
> solution (take drivers out of tree - it has been discussed at every summit
> and midcycle for 3+ cycles), what positive suggestions can people make?
Number 2 - a centralized branch (ie. in openstack.org) that *only* takes 
driver updates (and doesn't need CI).
If a driver is broken, too bad for that vendor - must have been the last 
driver update, as the Cinder code didn't change since EOL...


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] devstack changing to neutron by default RSN - current issues with OVH

[Armando M.]

On 8 August 2016 at 08:01, Sean Dague  wrote:

> In summary, it turns out we learned a few things:
>
> 1) neutron guests in our gate runs don't have the ability to route
> outwards. For instance, if they tried to do a package update, it would
> fail.
>
> 2) adding the ability for them to route outwards (as would be expected
> for things like package updates) was deemed table stakes for the
> devstack default.
>
> 3) doing so fails one tempest test on OVH, because they seem to be
> reflecting network traffic? We see connectivity between guests when it's
> not expected.
>
>
> My proposed path forward:
>
> 1) merge https://review.openstack.org/#/c/350750/ - devstack default
> change
> 2) merge https://review.openstack.org/#/c/352463/ - skip of tempest test
> that will fail on OVH (which turns into a 10% fail rate for neutron)
> 3) look at moving something like
> https://review.openstack.org/#/c/351876/3 into devstack-gate to handle
> OVH special casing. This is going to take time, especially given that we
> get maybe 2 iterations a day due to the gate being overloaded.
> 4) revert https://review.openstack.org/#/c/352463/
>
> If we don't have the devstack default change merged by the middle of the
> week, we probably need to abandon merging in this cycle at all, because
> we need breathing space to address any possible fallout from the merge.
>

I am good with this plan, thanks for the update.


> -Sean
>
> --
> Sean Dague
> http://dague.net
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] OpenStack Developer Mailing List Digest July 23 to August 5

[Mike Perez]

http://www.openstack.org/blog/2016/08/openstack-developer-mailing-list-digest-20160805/

-- 
Mike Perez

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [charms] nominating thedac for charms-release team

[Ryan Beisner]

Greetings,

I would like to nominate David Ames  for addition to the
charms-release team, as he has played a valuable role in the charm release
processes.  This change will grant privileges such as new stable branch
creation, among other things necessary to facilitate the charm release
process.

Cheers,

Ryan
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra][tripleo] status of tripleo-test-cloud-rh1

[Jeremy Stanley]

On 2016-08-08 11:47:56 -0400 (-0400), James Slagle wrote:
[...]
> I suppose it's also possible that we might be pushing too strongly
> down the multinode path? Is the general concensus in infra that they'd
> like to help enable project teams to eventually add 3 and 4 (and maybe
> more) node multinode jobs?
[...]

We've not outright rejected the idea, but do want to make sure that
there's been suitable due diligence done explaining how the things
you'll be able to test with >2 job nodes effectively can't be done
with <=2. Also we want to be sure that projects who are interested
in multi-node jobs start with just 2 job nodes and get some initial
tests performing well and returning stable results before trying to
push past 2.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Security] Proposing Luke Hinds (lhinds) for Security Core

[Jeremy Stanley]

On 2016-08-08 09:33:20 -0700 (-0700), Rob C wrote:
> I'd like to nominate Luke for a CoreSec position as part of the Security
> Project.
[...]

Sounds good to me. Luke's feedback on security bugs so far has been
quite helpful, so I'm all for encouraging more of the same.

This would bring the "OSSG CoreSec (ossg-coresec)" group in
Launchpad up to 5 active members, which seems like a good number to
better distribute the work and diversity of viewpoints for embargoed
bugs that require operational security consultation.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [tempest][swift][radosgw] Can we please merge the fix for the RFC 7230 violation issue?

[Jay Pipes]

Tempest devs,

Let me please draw your attention to a LP bug that may not seem 
particularly high priority, but I believe could be resolved easily with 
a patch already proposed.


LP bug 1536251 [1] accurately states that Tempest is actively verifying 
that an OpenStack API call violates RFC 7230.


When a 204 No Content is received, the Content-Length header MUST NOT be 
present.


However, Swift returns a Content-Length header and also an HTTP response 
code of 204 for a request to list containers of a new user (that has no 
containers).


Tempest has been validating this behaviour even though it is a violation 
of RFC 7230:


https://github.com/openstack/tempest/blob/master/tempest/api/object_storage/test_account_services.py#L81

RadosGW provides a proxy API that attempts to match the OpenStack Object 
Storage API, backed by Ceph object storage. In order for RadosGW to pass 
RefStack's burden of compatibility, it must pass the Tempest OpenStack 
Object Storage API tests. It currently cannot do so because RadosGW does 
not violate RFC 7230.


The RadosGW developer community does not wish to argue about whether or 
not to make Swift's API comply with RFC 7230. At the same time, they do 
not want to add a configuration option to RadosGW to force the proxy 
service to violate RFC 7230 just to satisfy the RefStack/Tempest API tests.


Therefore, Radoslaw (cc'd) has proposed a patch to Tempest that would 
allow RadosGW's proxy API to meet the RefStack compatibility tests while 
also not violating RFC 7230 and not requiring any change of Swift:


https://review.openstack.org/#/c/272062

I ask Tempest devs to re-review the above patch and consider merging it 
for the sake of collaboration between the OpenStack and Ceph developer 
communities.


Thanks very much!
-jay

[1] https://bugs.launchpad.net/tempest/+bug/1536251

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Nova mascot

[Jeremy Stanley]

On 2016-08-08 14:26:18 +0100 (+0100), Alexis Lee wrote:
> Markus Zoeller said on Mon, Aug 08, 2016 at 01:29:49PM +0200:
> > On 08.08.2016 13:07, Alexis Lee wrote:
> > > I propose the ant, known for discovering probabilistic graph traversal
> > > algorithms.
> > > 
> > > My other thought, the bee, has already been taken by refstack.
> > 
> > AFAIK #infra is going to chose the ant.
> 
> I suppose getting in there quick would be considered unsporting?
[...]

We already submitted the ranked list from our CIVS poll to Heidi Joy
a week ago. That said, I didn't sense much enthusiasm over it (or
really over mascot selection in general) so if Nova *really* wants
to be an ant there might be room for negotiation.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [glare][kolla][odm] timing of release of glare and request for technical interview on IRC wherever the glare team wants to have it

[Steven Dake (stdake)]

Mike was kind enough to drop by the IRC channel today to help the
operational deployment managers (ODMs) understand the impact of Glare for
technical planning purposes.

The log of the discussion is here:
http://eavesdrop.openstack.org/irclogs/%23openstack-kolla/%23openstack-koll
a.2016-08-08.log.html#t2016-08-08T16:01:50


Thanks again Mike.

Regards,
-steve

On 8/8/16, 8:01 AM, "Jay Pipes"  wrote:

>On 08/08/2016 07:48 AM, Steven Dake (stdake) wrote:
>> Cool thanks for the response.  Appreciate it.  I think the big take away
>> is all the ODMs are free from churn in Newton and have a full cycle to
>> adapt to the changes which is great news!
>
>Yes, that is absolutely the case.
>
>Best,
>-jay
>
>__
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Cinder] [stable] [all] Changing stable policy for drivers

[Duncan Thomas]

On 8 August 2016 at 18:31, Matthew Treinish  wrote:

>
> This argument comes up at least once a cycle and there is a reason we
> don't do
> this. When we EOL a branch all of the infrastructure for running any ci
> against
> it goes away. This means devstack support, job definitions, tempest skip
> checks,
> etc. Leaving the branch around advertises that you can still submit
> patches to
> it which you can't anymore. As a community we've very clearly said that we
> don't
> land any code without ensuring it passes tests first, and we do not
> maintain any
> of the infrastructure for doing that after an EOL.
>
>
Ok, to turn the question around, we (the cinder team) have recognised a
definite and strong need to have somewhere for vendors to share patches on
versions of Cinder older than the stable branch policy allows.

Given this need, what are our options?

1. We could do all this outside Openstack infrastructure. There are
significant downsides to doing so from organisational, maintenance, cost
etc points of view. Also means that the place vendors go for these patches
is not obvious, and the process for getting patches in is not standard.

2. We could have something not named 'stable' that has looser rules than
stable branches,, maybe just pep8 / unit / cinder in-tree tests. No
devstack.

3. We go with the Neutron model and take drivers out of tree. This is not
something the cinder core team are in favour of - we see significant value
in the code review that drivers currently get - the code quality
improvements between when a driver is submitted and when it is merged are
sometimes very significant. Also, taking the code out of tree makes it
difficult to get all the drivers checked out in one place to analyse e.g.
how a certain driver call is implemented across all the drivers, when
reasoning or making changes to core code.

Given we've identified a clear need, and have repeated rejected one
solution (take drivers out of tree - it has been discussed at every summit
and midcycle for 3+ cycles), what positive suggestions can people make?

-- 
Duncan Thomas
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Security] Proposing Luke Hinds (lhinds) for Security Core

[Rob C]

I'd like to nominate Luke for a CoreSec position as part of the Security
Project.

CoreSec team members support the VMT with extended consultation on
externally reported vulnerabilities.

Luke has been an active member of the Security project for quite some time.
He's done significant recent work on OSSNs and been an avid contributor to
our work in general. Luke has experience of vulnerability management from
his dayjob and will be a valuable member of the team.

-Rob
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Glance][Glare] External locations design

[Fox, Kevin M]

Has the use case been considered where the compute nodes have no nat to the 
internet?

The glare api server might have to pass the data through to the requesting 
client in that case or download/cache an internal copy.

Thanks,
Kevin

From: Mikhail Fedosin [mfedo...@mirantis.com]
Sent: Monday, August 08, 2016 4:08 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Glance][Glare] External locations design

Thanks for you comments Stuart!

I also have some thoughts on this matter: first of all, I never liked how 
locations are implemented in glance v2, and not just because of inconsistencies 
in multiple locations per one image, but for the reason that there are no 
differences between internal and external locations.
So, I think we're on right track with glare's implementation. But I suppose we 
can improve it even more:  in my understanding, the service generally does not 
work with external data. I will try to explain with an example:

Let's assume that we have a link to external ubuntu image ( 
http://releases.ubuntu.com/16.04.1/ubuntu-16.04.1-server-amd64.iso ). When user 
adds this link to his image artifact, Glare doesn't validate it, it just 
creates a blob instance in DB (optionally user may specify a checksum).
When user requests 'show' on his image, the blob url will look like this: 
{"url": "http://releases.ubuntu.com/16.04.1/ubuntu-16.04.1-server-amd64.iso"}, 
and if user wants to download it, the image won't be proxied with glare - it 
will be a direct download from ubuntu.com from the client.

In case of internal locations their urls are {"url": 
"/artifacts/images//file"} and these downloads are done with Glare as 
a proxy between the client and internal cloud storage, like Swift.

For compatibility, internal urls should work with external blobs as well, but 
there will be a redirect to the external url: 
"/artifacts/images//file" -> 
"http://releases.ubuntu.com/16.04.1/ubuntu-16.04.1-server-amd64.iso;.

We can discuss it on the artifact meeting at 1730 UTC on Mondays at 
#openstack-meeting-alt

Best,
Mike

On Tue, Aug 2, 2016 at 1:16 PM, 
> wrote:
Hi Kairat,

I think it's great to try and tease through the various issues here.
I added some comments to the etherpad.

-Stuart

Hello all,

I would like to start to describe some design decisions we made in Glare
code (https://review.openstack.org/#/q/topic:bp/glare-api+status:open).  If
you are not familiar with Glare I suggest you to read the following spec:

https://github.com/openstack/glance-specs/blob/master/specs/newton/approved/glance/glare-api.rst

I hope it will help other folks to understand Glare approach and provide
some constructive feedback for Glare. I think that we can also use Glare
solution for Glance in near future to address some drawbacks we have in
Glance.

Glare locations

Glance and Glare have possibility to set some external url as
image(artifact) location. This feature is quite useful for users who would
like to refer to some external image or artifact (for example, Fedora image
on official Fedora site) and not to store this image or artifact in the
cloud.

External locations in Glance have several specialities:

  1.

  It is possible to setup multiple locations for an image. Glance uses
  special location strategy to define which location to use. This strategy
  defined in glance codebase and can be configured in glance conf.
  2.

  Glance doesn?t differ image locations specified by url and image
  locations uploaded to Glance backend. Glance has some restrictions about
  which urls to use for locations (see Glance docs for more info).


Glare external locations designed in different way to address some
drawbacks we have in Glance. So the approach is the following:

  1.

  Glare doesn?t support multiple locations, you can specify dict of blobs
  in artifact type and add url for each blob in dict. User must define a
  name(f.e. region name or priority) for blob in dict and this name can be
  used to retrieve this blob from artifact. So decision about which location
  to use will be outside of Glare.
  2.

  Glare adds a special flag to database for external locations. So they
  will be treated differently in Glare when delete artifact. If blob value is
  external url then we don?t need to pass this url to backend and just delete
  the record in DB. For now, Glare allows only http(s) locations set but it
  may be extended in future but the idea still the same: external location
  are just records in DB.
  3.

  Glare saves blob size and checksum when specifying external url. When
  user specified url Glare downloads the blob by url, calculates its size and
  checksum. Of course, it leads to some performance degradation but we can
  ensure that the external blob is immutable. We made this because security
  seems more important for Glare than performance. Also there are 

Re: [openstack-dev] [Cinder] [stable] [all] Changing stable policy for drivers

[Jeremy Stanley]

On 2016-08-08 13:03:51 +0200 (+0200), Ihar Hrachyshka wrote:
> Sean McGinnis  wrote:
[...]
> > The suggestion was to just change our stable policy in regards to driver
> > bugfix backports. No need to create and maintain more branches. No need
> > to set up gate jobs and things like that.
> 
> Unless you manage to get it approved for the global policy
[...]

That was the gist of my suggestion to Sean as far as bringing this
discussion to the ML as a first option. Basically, if lots of
projects see their driver maintainers and downstream distros forking
their stable branches to add driver updates for support of newer
hardware, then see if the current OpenStack Stable Branch policy
should be amended to say that bug fixes and newer hardware support
specifically in driver source code (as long as it doesn't touch the
core service in that repo) are acceptable.

As far as the tangent this thread has taken on changing when we
delete stable branches, I feel like the only solution there is
working with the Stable Branch team to find ways to properly extend
support for the branches in question (including keeping them
properly tested). There have been ongoing efforts to make stable
branch testing less problematic, so it's possible over time we'll be
able to increase the support duration for them. Stating that it's
okay to keep them open for changes with no testing sets a terrible
precedent.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra] Project tailored VM image for Jenkins jobs

[Jeremy Stanley]

On 2016-08-08 06:00:14 -0500 (-0500), e...@itsonlyme.name wrote:
[...]
> One solution we were thinking of is having a VM image that
> contains the Docker image we need (e.g. saved using docker save),
> thus bypassing the need to do "docker pull". Is that possible? I
> realise that so far probably devstack is the only example making a
> special tailored image.

It's less that we have a specially tailored image for DevStack, and
more that our "standard" images pre-cache any downloads that
DevStack claims it may use... basically we cache all of the
$IMAGE_URLS list built up in
http://git.openstack.org/cgit/openstack-dev/devstack/tree/stackrc as
returned by the
http://git.openstack.org/cgit/openstack-dev/devstack/tree/tools/image_list.sh
script.

Keeping the set of images we cache on our nodes trimmed to a limited
number and only caching reasonably small images is important,
however, since our base images are already quite large (in the
neighborhood of 5GiB as compressed qcow2) and we don't want to bloat
them further if we can help it.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra][tripleo] status of tripleo-test-cloud-rh1

[Emilien Macchi]

On Mon, Aug 8, 2016 at 11:47 AM, James Slagle  wrote:
> On Sat, Aug 6, 2016 at 8:17 PM, Paul Belanger  wrote:
>> Greetings,
>>
>> 5 months ago fungi posted:
>>
>>   [tripleo] becoming third party CI (was: enabling third party CI)[1]
>>
>> About having the discussion whether the existing TripleO CI should itself 
>> follow
>> our third-party integration model instead of the current implementation 
>> relying
>> on our main community Zuul/Nodepool/Jenkins servers.
>>
>> The result of the thread had some pros and cons, which I encourge people to
>> re-read.
>>
>> At the Austin summit we continued the topic of moving tripleo-ci into 3rd 
>> party
>> CI. Again, consensus could not be reached however we made some progress.  I
>> would take on the responsibility to help bring tripleo-test-cloud-rh1 more
>> inline with openstack-infra tooling.
>>
>> That includes, but is not limited to:
>>
>>   - Initial support for centos-7 jenkins slave (tripleo-ci)
>> https://review.openstack.org/#/c/312725/
>>   - Add centos-7 to tripleo cloud (project-config)
>> https://review.openstack.org/#/c/311721/
>>   - Revert "Revert "Migrate tripleo to centos-7"" (project-config)
>> https://review.openstack.org/#/c/327425/
>>   - Revert "Disable tripleo-test-cloud-rh1 until we have AFS mirrors" 
>> (project-config)
>> https://review.openstack.org/#/c/349659/
>>   - Add tripleo-test-cloud grafana dashboard
>> https://review.openstack.org/#/c/351251/
>>
>> And various other reviews adding AFS mirrors for centos / epel. Updates to
>> tripleo-ci using our openstack-infra AFS mirrors along with providing general
>> support for both tripleo-test-cloud-rh1 and tripleo-test-cloud-rh2.
>>
>> In a short amount of time, we've made great progress with
>> tripleo-test-cloud-rh1, helping bring it more inline with openstack-infra
>> tooling.  While we are not finished, there is still some private 
>> infrastrucuture
>> that tripleo-ci is depending on. I am confident in the next 3 months we 
>> should
>> have that all replaced and using openstack community infrastruture.
>>
>> However on Friday[2], we started talking about tripleo-test-cloud-rh1 again 
>> in
>> #openstack-infra and found ourselves revisiting the original email. It is all
>> driven from the current effort from tripleo to start using move community 
>> clouds
>> for running tripleo-ci jobs.  Today, 3 different type of tripleo-ci jobs are 
>> now
>> run across all our clouds, for example there is a centos-7-2-node jobs. 
>> However,
>> tripleo-test-cloud-rh1 is only today setup to accept only tripleo-ci jobs. 
>> This
>> job does not run on tripleo-test-cloud-rh1.
>>
>> jeblair posted the following statement:
>>
>>   It feels like the tripleo cloud has been grandfathered in its current state
>>   for a while.  I'd just like to make sure we're being fair to everyone.  So 
>> if
>>   tripleo wants to run tripleo jobs, then i think we should move it to 3rd 
>> party
>>   ci.  I think that's a fine choice and we can continue to work together
>>   (please!) but with better division of reponsibilities.  Or, if we want to
>>   revise the idea of a multi-provider hardware platform that's available for 
>> all
>>   openstack projects, i'm game for that.  It would be great, but more work.
>>
>> Should we continue the push to move tripleo-test-cloud-rh1 to 3rd party CI
>> (removing from nodepool.o.o) or do we start enabling more jobs on
>> tripleo-test-cloud-rh1 bringing the cloud even more into openstack-infra?
>>
>> My personal thoughts, as somebody who's been working on it for the last 4
>> months, I still feel tripleo-test-cloud-rh1 should move to 3rd party CI.
>> However, with the work done in the last 4 months, I believe
>> tripleo-test-cloud-rh1 _could_ start running additional jobs based on the 
>> work
>> above.
>>
>> [1] http://lists.openstack.org/pipermail/openstack-dev/2016-March/088988.html
>> [2] 
>> http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2016-08-05.log.html#t2016-08-05T23:07:35
>
> I'd like to provide some additional context about where we'd like to
> go with the CI jobs running on all cloud providers. We've added
> centos7 2-node multinode jobs, and we have some single node jobs as
> well. What I'd like clarification around is there is
> hesitation/concern around TripleO building out additional mutlinode
> jobs that run on any cloud.
>
> I'd like to continue down this path of adding more CI jobs that can
> run on any nodepool managed cloud provider. To do so, we'd need to
> have 3+ node jobs. A 3 node job would let us test with an undercloud
> and a 2 node overcloud, while a 4 node job would let us test with an
> undercloud and a 3 node overcloud so we could test HA. 4 node HA jobs
> seem to work fine in my testing that I've done on
> tripleo-test-cloud-rh2, using the same network setup and infra images
> that we do for the 2 node job.
>
> tripleo-ci can't move entirely to 

Re: [openstack-dev] [magnum][heat] 2 million requests / sec, 100s of nodes

[Zane Bitter]

On 05/08/16 12:01, Hongbin Lu wrote:

Add [heat] to the title to get more feedback.



Best regards,

Hongbin



*From:*Ricardo Rocha [mailto:rocha.po...@gmail.com]
*Sent:* August-05-16 5:48 AM
*To:* OpenStack Development Mailing List (not for usage questions)
*Subject:* Re: [openstack-dev] [magnum] 2 million requests / sec, 100s
of nodes



Hi.



Quick update is 1000 nodes and 7 million reqs/sec :) - and the number of
requests should be higher but we had some internal issues. We have a
submission for barcelona to provide a lot more details.



But a couple questions came during the exercise:



1. Do we really need a volume in the VMs? On large clusters this is a
burden, and local storage only should be enough?



2. We observe a significant delay (~10min, which is half the total time
to deploy the cluster) on heat when it seems to be crunching the
kube_minions nested stacks. Once it's done, it still adds new stacks
gradually, so it doesn't look like it precomputed all the info in advance



Anyone tried to scale Heat to stacks this size? We end up with a stack with:

* 1000 nested stacks (depth 2)

* 22000 resources

* 47008 events


Wow, that's a big stack :) TripleO has certainly been pushing the 
boundaries of how big a stack Heat can handle, but this sounds like 
another step up even from there.



And already changed most of the timeout/retrial values for rpc to get
this working.



This delay is already visible in clusters of 512 nodes, but 40% of the
time in 1000 nodes seems like something we could improve. Any hints on
Heat configuration optimizations for large stacks very welcome.


Y'all were right to set max_resources_per_stack to -1, because actually 
checking the number of resources in a tree of stacks is sloow. (Not 
as slow as it used to be when it was O(n^2), but still pretty slow.)


We're actively working on trying to make Heat more horizontally scalable 
(even at the cost of some performance penalty) so that if you need to 
handle this kind of scale then you'll be able to reach it by adding more 
heat-engines. Another big step forward on this front is coming with 
Newton, as (barring major bugs) the convergence_engine architecture will 
be enabled by default.


RPC timeouts are caused by the synchronous work that Heat does before 
returning a result to the caller. Most of this is validation of the data 
provided by the user. We've talked about trying to reduce the amount of 
validation done synchronously to a minimum (just enough to guarantee 
that we can store and retrieve the data from the DB) and push the rest 
into the asynchronous part of the stack operation alongside the actual 
create/update. (FWIW, TripleO typically uses a 600s RPC timeout.)


The "QueuePool limit of size ... overflow ... reached" sounds like we're 
pulling messages off the queue even when we don't have threads available 
in the pool to pass them to. If you have a fix for this it would be much 
appreciated. However, I don't think there's any guarantee that just 
leaving messages on the queue can't lead to deadlocks. The problem with 
very large trees of nested stacks is not so much that it's a lot of 
stacks (Heat doesn't have _too_ much trouble with that) but that they 
all have to be processed simultaneously. e.g. to validate the top level 
stack you also need to validate all of the lower level stacks before 
returning the result. If higher-level stacks consume all of the thread 
pools then you'll get a deadlock as you'll be unable to validate any 
lower-level stacks. At this point you'd have maxed out the capacity of 
your Heat engines to process stacks simultaneously and you'd need to 
scale out to more Heat engines. The solution is probably to try limit 
the number of nested stack validations we send out concurrently.


Improving performance at scale is a priority area of focus for the Heat 
team at the moment. That's been mostly driven by TripleO and Sahara, but 
we'd be very keen to hear about the kind of loads that Magnum is putting 
on Heat and working with folks across the community to figure out how to 
improve things for those use cases.


cheers,
Zane.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra][neutron] - best way to load 8021q kernel module into cirros

[Jeremy Stanley]

On 2016-08-08 11:20:54 +0200 (+0200), Miguel Angel Ajo Pelayo wrote:
> The problem with the other projects image builds is that they are
> based for bigger systems, while cirros is an embedded-device-like
> image which boots in a couple of seconds.

Yep, smaller is certainly better when it comes to trying to run Nova
in a virtual machine.

> Couldn't we contribute to cirros to have such module load by default [1]?

It's worth chatting with them, for sure, and see what they say about
it.

> Or may be it's time for Openstack to build their own "cirros-like"
> image with all the capabilities we may be missing for general tempest
> testing? (ipv6, vlan, etc..? )

I haven't personally tested the CirrOS build instructions, but have
a feeling writing a diskimage-builder element wrapper for that
wouldn't be particularly challenging.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Manila] Nominate Tom Barron for core reviewer team

[Victoria Martínez de la Cruz]

On my brief participation on Manila team I've find Tom contributions to be
very helpful. I think he will be a great addition!

2016-08-04 11:16 GMT-03:00 Sean McGinnis :

> I have zero say here, but just want to say Tom is always a great guy to
> have around and knows his stuff. +1 from me that he should be core.
>
> On Wed, Aug 03, 2016 at 02:42:04PM -0400, Ben Swartzlander wrote:
> > Tom (tbarron on IRC) has been working on OpenStack (both cinder and
> > manila) for more than 2 years and has spent a great deal of time on
> > Manila reviews in the last release. Tom brings another
> > package/distro point of view to the community as well as former
> > storage vendor experience.
> >
> > -Ben Swartzlander
> > Manila PTL
> >
> > 
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:
> unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python-bileanclient] [infra] Duplicate entries in test-requirement.txt

[Jeremy Stanley]

On 2016-08-08 17:09:33 +0800 (+0800), 吕冬兵 wrote:
> I uploaded a new project to gerrit, but test-requirement.txt had
> duplicate entries by mistake. That makes jenkins fail whatever I
> commit
> (http://logs.openstack.org/58/351458/7/check/gate-python-bileanclient-requirements/017cd3a/console.html).
> Anybody have some idea to fix that?

This looks like a corner case recently broken in our script for that
job. I have proposed https://review.openstack.org/352490 as a fix
for it.
-- 
Jeremy Stanley

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [octavia]redirection and barbican config

[Michael Johnson]

Hi Akshay,

Currently we are only allowing one VIP per amphora.

You can log into the amphora if you specify a ssh keypair loaded into
nova in the octavia.conf file.  When that is specified you can log
into the amphora via SSH on the management network interface.

Michael

On Mon, Aug 1, 2016 at 5:15 PM, Akshay Kumar Sanghai
 wrote:
> Hi Michael,
> Thanks. I have few more queries:
> - Is it possible to create multiple VIPs on one amphora?
>
> -I created a LB 2 days back. I created all the objects loadbalancer,
> listener, pool and members. The curl was successful for the vip. Today I
> added one more listener listening on port 443 (Terminated https) and added
> pool for it and members for the pool. I have barbican installed and I have
> tried ssl offloading with barbican with haproxy namespace driver.  The curl
> for http and https were giving me code 503, but when I did a curl to the
> member, it was working 200 ok. I tried to figure out where its going wrong,
> but could not. I could not find any errors in octavia-api.log and
> octavia-worker.log. So, I deleted everything and recreated again. Now it was
> working. But for a similar future scenario, how should i figure out where
> things went wrong or where the packet is dropped. Is it possible to login to
> the amphora vm?
>
> Thanks
> Akshay
>
> On Sat, Jul 30, 2016 at 11:45 PM, Michael Johnson 
> wrote:
>>
>> Hi Akshay,
>>
>> For 80 to 443 redirection, you can accomplish this using the new L7
>> rules capability.  You would setup a listener on port 80 that has a
>> redirect rule to the 443 URL.
>>
>> On the barbican question, if you are using the octavia driver, you
>> will need to set the required settings in the octavia.conf file for
>> proper barbican access.
>> Those settings are called out here:
>>
>> http://docs.openstack.org/developer/octavia/config-reference/octavia-config-table.html
>>
>> Michael
>>
>>
>> On Thu, Jul 28, 2016 at 1:02 PM, Akshay Kumar Sanghai
>>  wrote:
>> > Hi,
>> > I have a couple on questions on octavia. Please answer or redirect me to
>> > relevant documentation:
>> > - Assume listener is listening on 443 and client hits the vip on port
>> > 80,
>> > the connection will be refused.  Is it possible to configure http to
>> > https
>> > direction?
>> >
>> > - For the barbican config, the only config item i can find is
>> > cert_manager_type in neutron_lbaas.conf. How do we configure the
>> > barbican
>> > access for lbaas. I assume since we do the access config for nova and
>> > keystone in neutron.conf, there should be some config file where we
>> > define
>> > the barbican access(username, password, auth_url).
>> >
>> > The community has been very helpful to me. Thanks a lot Guys. Appreciate
>> > your efforts.
>> >
>> > Thanks
>> > Akshay Sanghai
>> >
>> >
>> > __
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe:
>> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [octavia] Multi-node controller testing

[Michael Johnson]

Miguel,

Thank you for your work here.  I would support an effort to setup a
multi-node gate job.

Michael


On Mon, Aug 8, 2016 at 5:04 AM, Miguel Angel Ajo Pelayo
 wrote:
> Recently, I sent a series of patches [1] to make it easier for
> developers to deploy a multi node octavia controller with
> n_controllers x [api, cw, hm, hk] with an haproxy in front of the API.
>
> Since this is the way the service is designed to work (with horizontal
> scalability in mind), and we want to have a good guarantee that any
> bug related to such configuration is found early, and addressed, I was
> thinking that an extra job that runs a two node controller deployment
> could be beneficial for the project.
>
>
> If we all believe it makes sense, I would be willing to take on this
> work but I'd probably need some pointers and light help, since I've
> never dealt with setting up or modifying existing jobs.
>
> How does this sound?
>
>
> [1] 
> https://review.openstack.org/#/q/status:merged+project:openstack/octavia+branch:master+topic:multinode-devstack
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [infra][tripleo] status of tripleo-test-cloud-rh1

[James Slagle]

On Sat, Aug 6, 2016 at 8:17 PM, Paul Belanger  wrote:
> Greetings,
>
> 5 months ago fungi posted:
>
>   [tripleo] becoming third party CI (was: enabling third party CI)[1]
>
> About having the discussion whether the existing TripleO CI should itself 
> follow
> our third-party integration model instead of the current implementation 
> relying
> on our main community Zuul/Nodepool/Jenkins servers.
>
> The result of the thread had some pros and cons, which I encourge people to
> re-read.
>
> At the Austin summit we continued the topic of moving tripleo-ci into 3rd 
> party
> CI. Again, consensus could not be reached however we made some progress.  I
> would take on the responsibility to help bring tripleo-test-cloud-rh1 more
> inline with openstack-infra tooling.
>
> That includes, but is not limited to:
>
>   - Initial support for centos-7 jenkins slave (tripleo-ci)
> https://review.openstack.org/#/c/312725/
>   - Add centos-7 to tripleo cloud (project-config)
> https://review.openstack.org/#/c/311721/
>   - Revert "Revert "Migrate tripleo to centos-7"" (project-config)
> https://review.openstack.org/#/c/327425/
>   - Revert "Disable tripleo-test-cloud-rh1 until we have AFS mirrors" 
> (project-config)
> https://review.openstack.org/#/c/349659/
>   - Add tripleo-test-cloud grafana dashboard
> https://review.openstack.org/#/c/351251/
>
> And various other reviews adding AFS mirrors for centos / epel. Updates to
> tripleo-ci using our openstack-infra AFS mirrors along with providing general
> support for both tripleo-test-cloud-rh1 and tripleo-test-cloud-rh2.
>
> In a short amount of time, we've made great progress with
> tripleo-test-cloud-rh1, helping bring it more inline with openstack-infra
> tooling.  While we are not finished, there is still some private 
> infrastrucuture
> that tripleo-ci is depending on. I am confident in the next 3 months we should
> have that all replaced and using openstack community infrastruture.
>
> However on Friday[2], we started talking about tripleo-test-cloud-rh1 again in
> #openstack-infra and found ourselves revisiting the original email. It is all
> driven from the current effort from tripleo to start using move community 
> clouds
> for running tripleo-ci jobs.  Today, 3 different type of tripleo-ci jobs are 
> now
> run across all our clouds, for example there is a centos-7-2-node jobs. 
> However,
> tripleo-test-cloud-rh1 is only today setup to accept only tripleo-ci jobs. 
> This
> job does not run on tripleo-test-cloud-rh1.
>
> jeblair posted the following statement:
>
>   It feels like the tripleo cloud has been grandfathered in its current state
>   for a while.  I'd just like to make sure we're being fair to everyone.  So 
> if
>   tripleo wants to run tripleo jobs, then i think we should move it to 3rd 
> party
>   ci.  I think that's a fine choice and we can continue to work together
>   (please!) but with better division of reponsibilities.  Or, if we want to
>   revise the idea of a multi-provider hardware platform that's available for 
> all
>   openstack projects, i'm game for that.  It would be great, but more work.
>
> Should we continue the push to move tripleo-test-cloud-rh1 to 3rd party CI
> (removing from nodepool.o.o) or do we start enabling more jobs on
> tripleo-test-cloud-rh1 bringing the cloud even more into openstack-infra?
>
> My personal thoughts, as somebody who's been working on it for the last 4
> months, I still feel tripleo-test-cloud-rh1 should move to 3rd party CI.
> However, with the work done in the last 4 months, I believe
> tripleo-test-cloud-rh1 _could_ start running additional jobs based on the work
> above.
>
> [1] http://lists.openstack.org/pipermail/openstack-dev/2016-March/088988.html
> [2] 
> http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2016-08-05.log.html#t2016-08-05T23:07:35

I'd like to provide some additional context about where we'd like to
go with the CI jobs running on all cloud providers. We've added
centos7 2-node multinode jobs, and we have some single node jobs as
well. What I'd like clarification around is there is
hesitation/concern around TripleO building out additional mutlinode
jobs that run on any cloud.

I'd like to continue down this path of adding more CI jobs that can
run on any nodepool managed cloud provider. To do so, we'd need to
have 3+ node jobs. A 3 node job would let us test with an undercloud
and a 2 node overcloud, while a 4 node job would let us test with an
undercloud and a 3 node overcloud so we could test HA. 4 node HA jobs
seem to work fine in my testing that I've done on
tripleo-test-cloud-rh2, using the same network setup and infra images
that we do for the 2 node job.

tripleo-ci can't move entirely to using these types of multinode jobs
though, because they do not exercise the nova/ironic deployment during
the tests. So, there will always be some subset of tripleo-ci jobs
that still need 

Re: [openstack-dev] [infra][tripleo] status of tripleo-test-cloud-rh1

[Ben Nemec]

On 08/06/2016 07:17 PM, Paul Belanger wrote:
> Greetings,
> 
> 5 months ago fungi posted:
> 
>   [tripleo] becoming third party CI (was: enabling third party CI)[1]
> 
> About having the discussion whether the existing TripleO CI should itself 
> follow
> our third-party integration model instead of the current implementation 
> relying
> on our main community Zuul/Nodepool/Jenkins servers.
> 
> The result of the thread had some pros and cons, which I encourge people to
> re-read.
> 
> At the Austin summit we continued the topic of moving tripleo-ci into 3rd 
> party
> CI. Again, consensus could not be reached however we made some progress.  I
> would take on the responsibility to help bring tripleo-test-cloud-rh1 more
> inline with openstack-infra tooling.
> 
> That includes, but is not limited to:
> 
>   - Initial support for centos-7 jenkins slave (tripleo-ci)
> https://review.openstack.org/#/c/312725/
>   - Add centos-7 to tripleo cloud (project-config)
> https://review.openstack.org/#/c/311721/
>   - Revert "Revert "Migrate tripleo to centos-7"" (project-config)
> https://review.openstack.org/#/c/327425/
>   - Revert "Disable tripleo-test-cloud-rh1 until we have AFS mirrors" 
> (project-config)
> https://review.openstack.org/#/c/349659/
>   - Add tripleo-test-cloud grafana dashboard
> https://review.openstack.org/#/c/351251/
> 
> And various other reviews adding AFS mirrors for centos / epel. Updates to
> tripleo-ci using our openstack-infra AFS mirrors along with providing general
> support for both tripleo-test-cloud-rh1 and tripleo-test-cloud-rh2.
> 
> In a short amount of time, we've made great progress with
> tripleo-test-cloud-rh1, helping bring it more inline with openstack-infra
> tooling.  While we are not finished, there is still some private 
> infrastrucuture
> that tripleo-ci is depending on. I am confident in the next 3 months we should
> have that all replaced and using openstack community infrastruture.
> 
> However on Friday[2], we started talking about tripleo-test-cloud-rh1 again in
> #openstack-infra and found ourselves revisiting the original email. It is all
> driven from the current effort from tripleo to start using move community 
> clouds
> for running tripleo-ci jobs.  Today, 3 different type of tripleo-ci jobs are 
> now
> run across all our clouds, for example there is a centos-7-2-node jobs. 
> However,
> tripleo-test-cloud-rh1 is only today setup to accept only tripleo-ci jobs. 
> This
> job does not run on tripleo-test-cloud-rh1.
> 
> jeblair posted the following statement:
> 
>   It feels like the tripleo cloud has been grandfathered in its current state
>   for a while.  I'd just like to make sure we're being fair to everyone.  So 
> if
>   tripleo wants to run tripleo jobs, then i think we should move it to 3rd 
> party
>   ci.  I think that's a fine choice and we can continue to work together
>   (please!) but with better division of reponsibilities.  Or, if we want to
>   revise the idea of a multi-provider hardware platform that's available for 
> all
>   openstack projects, i'm game for that.  It would be great, but more work.
> 
> Should we continue the push to move tripleo-test-cloud-rh1 to 3rd party CI
> (removing from nodepool.o.o) or do we start enabling more jobs on
> tripleo-test-cloud-rh1 bringing the cloud even more into openstack-infra?
> 
> My personal thoughts, as somebody who's been working on it for the last 4
> months, I still feel tripleo-test-cloud-rh1 should move to 3rd party CI.
> However, with the work done in the last 4 months, I believe
> tripleo-test-cloud-rh1 _could_ start running additional jobs based on the work
> above.

I was always in favor of going third-party, so +1 to that option.  If we
still insist on staying integrated then I think it's totally fair to say
we need to start running other jobs too though.

Long term we'd still like to get away from needing to run on a custom
cloud, but until OVB support is available in more of the regular infra
clouds I think we're stuck with our custom one.

> 
> [1] http://lists.openstack.org/pipermail/openstack-dev/2016-March/088988.html
> [2] 
> http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2016-08-05.log.html#t2016-08-05T23:07:35
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Cinder] [stable] [all] Changing stable policy for drivers

[Matthew Treinish]

On Mon, Aug 08, 2016 at 09:47:53AM -0500, Sean McGinnis wrote:
> > 
> > Unless you manage to get it approved for the global policy, I think
> > you will effectively make your stable:follows-policy tag obsolete,
> > and then it should be removed from your project. Read the
> > requirements:
> > 
> > https://governance.openstack.org/reference/tags/stable_follows-policy.html#requirements
> > 
> > Support phases are part of the stable policy, and so if you don’t
> > mostly adhere to their definitions, you should not carry the tag.
> > Which is fine with me, it’s up to Cinder team to decide whether it’s
> > worth it.
> 
> I think "currently active stable branches" is key there. These branches
> would no longer be "currently active". They would get an EOL tag when it
> reaches the end of the support phases. We just wouldn't delete the
> branch.

This argument comes up at least once a cycle and there is a reason we don't do
this. When we EOL a branch all of the infrastructure for running any ci against
it goes away. This means devstack support, job definitions, tempest skip checks,
etc. Leaving the branch around advertises that you can still submit patches to
it which you can't anymore. As a community we've very clearly said that we don't
land any code without ensuring it passes tests first, and we do not maintain any
of the infrastructure for doing that after an EOL. 

> 
> Again, this is only for driver code. We would not allow backports to the
> core Cinder codebase.

This distinction does not actually matter, you're still trying to backport code
without the ability to run tests in the gate. The fact that it's part of a
driver doesn't really change anything.

-Matt Treinish


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone][tripleo] Federation, mod_mellon, and HA Proxy

[John Dennis]

On 08/07/2016 11:16 PM, Adam Young wrote:

On 08/06/2016 08:44 AM, John Dennis wrote:

On 08/05/2016 06:06 PM, Adam Young wrote:

Ah...just noticed the redirect is to :5000, not port :13000 which is
the HA Proxy port.


OK, this is due to the SAML request:


https://identity.ayoung-dell-t1700.test/auth/realms/openstack/protocol/saml;

Consent="urn:oasis:names:tc:SAML:2.0:consent:current-implicit"
ForceAuthn="false"
IsPassive="false"

AssertionConsumerServiceURL="https://openstack.ayoung-dell-t1700.test:5000/v3/mellon/postResponse;
>

https://openstack.ayoung-dell-t1700.test:5000/v3/mellon/metadata




My guess is HA proxy is not passing on the proper, and the
mod_auth_mellon does not know to rewrite it from 5000 to 13000


You can't change the contents of a SAML AuthnRequest, often they are
signed. Also, the AssertionConsumerServiceURL's and other URL's in
SAML messages are validated to assure they match the metadata
associated with EntityID (issuer). The addresses used inbound and
outbound have to be correctly handled by the proxy configuration
without modifying the content of the message being passed on the
transport.


Got a a little further by twerking HA proxy settings.  Added in

  redirect scheme https code 301 if { hdr(host) -i 10.0.0.4 } !{ ssl_fc }
  rsprep ^Location:\ http://(.*) Location:\ https://\1

whicxh tells HA proxy to translate Location headers (used in redirects)
from http to https.


As of now, it looks good up until the response comes back from the IdP
and mod mellon rejects it.  I think this is due to Mellon issuing a
request for http://:  but it gets translated through the
proxy as https://:.


mod_auth_mellon is failing the following check in auth_mellon_handler.c


  url = am_reconstruct_url(r);

  ...

  if (response->parent.Destination) {

if (strcmp(response->parent.Destination, url)) {
ap_log_rerror(APLOG_MARK, APLOG_ERR, 0, r,
  "Invalid Destination on Response. Should be: %s",
  url);
lasso_login_destroy(login);
return HTTP_BAD_REQUEST;
}
}

It does not spit out the parent.Destination value, but considering I am
seeing http and not https in the error message, I assume that at least
the protocol does not match.  Full error message at the bottom.

Assuming the problem is just that the URL is http and not https,   I
have an approach that should work.  I need to test it out, but want to
record it here, and also get feedback:

I can clone the current 10-keystone_wsgi_main.conf which listens for
straight http on port 5000.  If I make a file
11-keystone_wsgi_main.conf  that listens on port 13000 (not on the
external VIP)  but that enables SSL, I should be able to make HA proxy
talk to that port and re-encrypt traffic, maintaining the 'https://'
protocol.


However, I am not certain that Destination means the SP URL.  It seems
like it should mean the IdP.  Further on in auth_mellon_handler.c

  destination_url = lasso_provider_get_metadata_one(
provider, "SingleSignOnService HTTP-Redirect");
if (destination_url == NULL) {
/* HTTP-Redirect unsupported - try HTTP-POST. */
http_method = LASSO_HTTP_METHOD_POST;
destination_url = lasso_provider_get_metadata_one(
provider, "SingleSignOnService HTTP-POST");
}

Looking in the metadata, it seems that this value should be:

 https://identity.ayoung-dell-t1700.test/auth/realms/openstack/protocol/saml;
/>

So maybe something has rewritten the value used as the url ?


Here is the full error message


Invalid Destination on Response. Should be:
http://openstack.ayoung-dell-t1700.test:5000/v3/mellon/postResponse,
referer:
https://identity.ayoung-dell-t1700.test/auth/realms/openstack/protocol/saml?SAMLRequest=nZJba%2BMwEEb%2FitG7I%2BXi1Igk4OYCge5S0m4f%2BlKEM2lFLcmrGWc3%2F35HDu22D22hIDCMZ%2FTpHGmGxjWtrjp68jv43QFS9tc1HnX%2FYy666HUwaFF74wA11fqm%2BnGlRwOl2xgo1KERb0Y%2BnzCIEMkGL7Ltai4e1LoYq%2FFoXapJWU2GhSouN5vhelpNyqIcX2xEdgcRuX8ueJyHEDvYeiTjiUtqOM1VmavprRppXkVxL7IVM1hvqJ96ImpRS2n34MnSaWBOofOP%2BR6aJqfhhVID4n5pWICMYBqHMrSQEupn%2BQIoE5nIlsEjpODPEOtzk667GPmbW9c2trYksk2INfSm5%2BJgGoTEc81K7BFeK9WLoRTWOYg3EI%2B2hl%2B7q%2F80ryf8AEcXSil5HEvH9eBlG5B2gG06mljMEo3uVcbFd7d0QGZvyMzk291m5%2Bf0k61sV9eBwU8J25kvpKWK3eeHvlVTNB4ty2MdHPZnyRdDrIhiB0IuzpHvH%2B3iHw%3D%3D=http%3A%2F%2Fopenstack.ayoung-dell-t1700.test%3A5000%2Fv3%2Fauth%2FOS-FEDERATION%2Fwebsso%2Fsaml2%3Forigin%3Dhttp%3A%2F%2Fopenstack.ayoung-dell-t1700.test%2Fdashboard%2Fauth%2Fwebsso%2F=http%3A%2F%2Fwww.w3.org%2F2000%2F09%2Fxmldsig%23rsa-sha1=oJzAwE7ma3m0gZtO%2FvPQKCnk18u4OsjKcRQ3wiDu7txUGiPr4Cc9XIzKIGwzSGPSaWi8j1qbN76XwdNICOk!

Re: [openstack-dev] [nova] [infra] Intel NFV CI voting permission

[Ptacek, MichalX]

Change deployed,
Thanks for your comments,

Michal

-Original Message-
From: Markus Zoeller [mailto:mzoel...@linux.vnet.ibm.com] 
Sent: Monday, August 08, 2016 3:15 PM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [nova] [infra] Intel NFV CI voting permission

On 03.08.2016 22:02, Znoinski, Waldemar wrote:
> Thanks
> Much appreciated
> 
> Making use of the opportunity here... what's the next big thing a CI 
> (like one testing NFV) should be doing? (multinode or there's 
> something else more important?)

Two tiny nits on the comment the CI is giving a change:

"Build failed (check pipeline). To recheck leave a comment with
intel-nfv-ci recheck. For more info go to
https://wiki.openstack.org/wiki/ThirdPartySystems/Intel-NFV-CI.;

1) Could you put the recheck command in quotation? Like:
   To recheck leave a comment with 'intel-nfv-ci recheck'.
2) The link to the wiki seems to be wrong (hyphen vs. underscore):
   https://wiki.openstack.org/wiki/ThirdPartySystems/Intel_NFV_CI

--
Regards, Markus Zoeller (markus_z)

>  >-Original Message-
>  >From: Matt Riedemann [mailto:mrie...@linux.vnet.ibm.com]
>  >Sent: Wednesday, August 3, 2016 8:28 PM
>  >To: openstack-dev@lists.openstack.org
>  >Subject: Re: [openstack-dev] [nova] [infra] Intel NFV CI voting 
> permission  >  >On 8/3/2016 8:18 AM, Znoinski, Waldemar wrote:
>  >> [WZ] Hi Matt, do we need, a Gerrit-style, +2 on that from someone?
>  >> Infra Team doesn't keep these settings in their puppet/config 
> files on git -  >all the Gerrit changes are done via Gerrit GUI so 
> they rely on Cores to add CIs  >to the appropriate ci group, nova-ci in this 
> case.
>  >>
>  >>
>  >
>  >Sorry, I wasn't sure what the next step here was, I guess it was the 
> nova-ci  >membership change, which is done now:
>  >
>  >https://review.openstack.org/#/admin/groups/511,members
>  >
>  >--
>  >
>  >Thanks,
>  >
>  >Matt Riedemann
>  >
>  >
>  >__
>  >
>  >OpenStack Development Mailing List (not for usage questions)
>  >Unsubscribe: OpenStack-dev-
>  >requ...@lists.openstack.org?subject:unsubscribe
>  >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> --
> Intel Research and Development Ireland Limited Registered in Ireland 
> Registered Office: Collinstown Industrial Park, Leixlip, County 
> Kildare Registered Number: 308263
> 
> 
> This e-mail and any attachments may contain confidential material for 
> the sole use of the intended recipient(s). Any review or distribution 
> by others is strictly prohibited. If you are not the intended 
> recipient, please contact the sender and delete all copies.
> 
> 
> __
>  OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: 
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
--
Intel Research and Development Ireland Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263


This e-mail and any attachments may contain confidential material for the sole
use of the intended recipient(s). Any review or distribution by others is
strictly prohibited. If you are not the intended recipient, please contact the
sender and delete all copies.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [new][horizon] django_openstack_auth 2.4.0 release (newton)

[no-reply]

We are enthusiastic to announce the release of:

django_openstack_auth 2.4.0: Django authentication backend for use
with OpenStack Identity

This release is part of the newton release series.

With source available at:

http://git.openstack.org/cgit/openstack/django_openstack_auth/

With package available at:

https://pypi.python.org/pypi/django_openstack_auth

Please report issues through launchpad:

https://bugs.launchpad.net/django-openstack-auth

For more details, please see below.

Changes in django_openstack_auth 2.3.0..2.4.0
-

6c8a1d4 Fix django 1.10 tox env
90822c4 Updated from global requirements
946321d Updated from global requirements
e46f0dc Fix Django 1.10 tox env
cdccf59 [Django 1.10] Define TEMPLATES
9ad524b Add 'is_admin_project' attribute in token
a8c273f Not authorized when logout and creating instance
6e58b9f Updated from global requirements
ac1ff60 Adding tenant_id to policy default
0afe75f Add Django 1.10 tox env
9c73ff2 Updated from global requirements
1ad9525 Updated from global requirements
c40b265 Make fix_auth_url_version() delegate emitting the warning up the stack
4a81cdd Updated from global requirements
962076b Imported Translations from Zanata
d2b9c5a Imported Translations from Zanata
4d07804 Updated from global requirements
2a3fb85 Updated from global requirements


Diffstat (except docs and test files)
-

openstack_auth/backend.py | 14 +++-
openstack_auth/locale/cs/LC_MESSAGES/django.po| 21 +++---
openstack_auth/locale/en_GB/LC_MESSAGES/django.po | 12 ++--
openstack_auth/policy.py  |  3 +
openstack_auth/user.py|  2 +
openstack_auth/utils.py   | 87 +++
openstack_auth/views.py   |  6 +-
requirements.txt  |  6 +-
test-requirements.txt |  4 +-
tox.ini   |  4 ++
12 files changed, 139 insertions(+), 72 deletions(-)


Requirements updates


diff --git a/requirements.txt b/requirements.txt
index 6d19807..59bbca7 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -6,2 +6,2 @@ Django<1.9,>=1.8 # BSD
-oslo.config>=3.9.0 # Apache-2.0
-oslo.policy>=0.5.0 # Apache-2.0
+oslo.config>=3.14.0 # Apache-2.0
+oslo.policy>=1.9.0 # Apache-2.0
@@ -9 +9 @@ python-keystoneclient!=1.8.0,!=2.1.0,>=1.7.0 # Apache-2.0
-keystoneauth1>=2.1.0 # Apache-2.0
+keystoneauth1>=2.10.0 # Apache-2.0
diff --git a/test-requirements.txt b/test-requirements.txt
index 282368c..6610270 100644
--- a/test-requirements.txt
+++ b/test-requirements.txt
@@ -7 +7 @@ coverage>=3.6 # Apache-2.0
-mock>=1.2 # BSD
+mock>=2.0 # BSD
@@ -9 +9 @@ mox3>=0.7.0 # Apache-2.0
-sphinx!=1.2.0,!=1.3b1,<1.3,>=1.1.2 # BSD
+sphinx!=1.3b1,<1.3,>=1.2.1 # BSD



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [glare][kolla] timing of release of glare and request for technical interview on IRC wherever the glare team wants to have it

[Jay Pipes]

On 08/08/2016 07:48 AM, Steven Dake (stdake) wrote:

Cool thanks for the response.  Appreciate it.  I think the big take away
is all the ODMs are free from churn in Newton and have a full cycle to
adapt to the changes which is great news!


Yes, that is absolutely the case.

Best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] devstack changing to neutron by default RSN - current issues with OVH

[Sean Dague]

In summary, it turns out we learned a few things:

1) neutron guests in our gate runs don't have the ability to route
outwards. For instance, if they tried to do a package update, it would fail.

2) adding the ability for them to route outwards (as would be expected
for things like package updates) was deemed table stakes for the
devstack default.

3) doing so fails one tempest test on OVH, because they seem to be
reflecting network traffic? We see connectivity between guests when it's
not expected.


My proposed path forward:

1) merge https://review.openstack.org/#/c/350750/ - devstack default change
2) merge https://review.openstack.org/#/c/352463/ - skip of tempest test
that will fail on OVH (which turns into a 10% fail rate for neutron)
3) look at moving something like
https://review.openstack.org/#/c/351876/3 into devstack-gate to handle
OVH special casing. This is going to take time, especially given that we
get maybe 2 iterations a day due to the gate being overloaded.
4) revert https://review.openstack.org/#/c/352463/

If we don't have the devstack default change merged by the middle of the
week, we probably need to abandon merging in this cycle at all, because
we need breathing space to address any possible fallout from the merge.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [octavia] Multi-node controller testing

[Kosnik, Lubosz]

Great work with that multi-node setup Miguel.
About that multinode Infra is supporting two nodes setup used currently by 
grenade jobs but in my opinion we don’t have any tests which can cover that 
type of testing. We’re still struggling with selecting proper tool to test 
Octavia from integration/functional perspective so probably it’s too early to 
make it happen.
Maybe it’s great start to finally make some decision about testing tools and 
there will be a lot of work for you after that also with setting up an infra 
multi-node job for that.

Cheers,
Lubosz Kosnik
Cloud Software Engineer OSIC
lubosz.kos...@intel.com

> On Aug 8, 2016, at 7:04 AM, Miguel Angel Ajo Pelayo  
> wrote:
> 
> Recently, I sent a series of patches [1] to make it easier for
> developers to deploy a multi node octavia controller with
> n_controllers x [api, cw, hm, hk] with an haproxy in front of the API.
> 
> Since this is the way the service is designed to work (with horizontal
> scalability in mind), and we want to have a good guarantee that any
> bug related to such configuration is found early, and addressed, I was
> thinking that an extra job that runs a two node controller deployment
> could be beneficial for the project.
> 
> 
> If we all believe it makes sense, I would be willing to take on this
> work but I'd probably need some pointers and light help, since I've
> never dealt with setting up or modifying existing jobs.
> 
> How does this sound?
> 
> 
> [1] 
> https://review.openstack.org/#/q/status:merged+project:openstack/octavia+branch:master+topic:multinode-devstack
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Cinder] [stable] [all] Changing stable policy for drivers

[Sean McGinnis]

> 
> Unless you manage to get it approved for the global policy, I think
> you will effectively make your stable:follows-policy tag obsolete,
> and then it should be removed from your project. Read the
> requirements:
> 
> https://governance.openstack.org/reference/tags/stable_follows-policy.html#requirements
> 
> Support phases are part of the stable policy, and so if you don’t
> mostly adhere to their definitions, you should not carry the tag.
> Which is fine with me, it’s up to Cinder team to decide whether it’s
> worth it.

I think "currently active stable branches" is key there. These branches
would no longer be "currently active". They would get an EOL tag when it
reaches the end of the support phases. We just wouldn't delete the
branch.

Again, this is only for driver code. We would not allow backports to the
core Cinder codebase.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][grafana][infra] how to read grafana

[Ihar Hrachyshka]

Matthew Treinish  wrote:


On Mon, Aug 08, 2016 at 02:40:31PM +0200, Ihar Hrachyshka wrote:

Hi,

I was looking at grafana today, and spotted another weirdness.

See the periodic jobs dashboard:

http://grafana.openstack.org/dashboard/db/neutron-failure-rate?panelId=4

Currently it shows for me 100% failure rate for py34/oslo-master job,
starting from ~Aug 3. But when I go to openstack-health, I don’t see those
runs at all:

http://status.openstack.org/openstack-health/#/job/periodic-neutron-py34-with-neutron-lib-master

(^ The last run is July 31.)

But then when I drill down into files, I can see more recent runs, like:

http://logs.openstack.org/periodic/periodic-neutron-py34-with-neutron-lib-master/?C=M;O=A
http://logs.openstack.org/periodic/periodic-neutron-py34-with-neutron-lib-master/faa24e0/testr_results.html.gz

The last link points to a run from yesterday. And as you can see it is
passing.


That run isn't actually from yesterday, it's from July 30th. The  
directory shows
a recent date, but the last modified dates for the individual files is  
older:


http://logs.openstack.org/periodic/periodic-neutron-py34-with-neutron-lib-master/faa24e0/

The openstack-health data goes up until the job started failing, this is  
likely
because the failures occur early enough in the test run that no subunit  
output

is generated for the run.


So, what’s wrong with the grafana dashboard? And why doesn’t
openstack-health show the latest runs?


On the openstack-health side it looks like you're running into an issue  
with
using subunit2sql as the primary data source there. If you look at an  
example

output from what's not in openstack-health, like:

http://logs.openstack.org/periodic/periodic-neutron-py34-with-neutron-lib-master/37cd5eb/console.html.gz


Nice! I guess you just picked one of those that is not present on Health  
dashboard? Or you did something more elaborate to come up with the link?




You'll see that the failure is occuring before any subunit output is  
generated.
(during the discovery phase of testr) If there is no subunit file in the  
log
output for the run, then there is nothing to populate the subunit2sql DB  
with.

The grafana/graphite data doesn't share this limitation because it gets
populated directly by zuul.

This is a known limitation with openstack-health right, and the plan to  
solve it
is to add a zuul sql data store that we can query like subunit2sql for  
job level
information, and then use subunit2sql for more fine grained details. The  
work on
that currently depends on: https://review.openstack.org/#/c/22/ which  
adds
the datastore to zuul. Once that lands we can work on the  
openstack-health side

consume that data in conjunction with subunit2sql.

-Matt Treinish


Just want to say a huge thank you for the reply. It both pointed me to the  
immediate problem to solve as well as gave wider perspective on the  
mechanics that I should be aware of. It’s great to work in a community of  
individuals that so often go an extra mile for their fellow.


Ihar


signature.asc
Description: Message signed with OpenPGP using GPGMail
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [mistral] Porting security-related utils, Context and dependencies to Mistral-Lib

[Ryan Brady]

In accordance with the spec[1], I started a patch[2] to port security
related items from mistral to mistral-lib.  This may not be the right way
to approach this task and I'm hoping the patch provides a means to
illustrate the problem and starts a discussion on the right solution.

A custom action that creates a client that requires keystone auth will need
to get an endpoint for a given project to create a client object, so I
ported over the utility class[3] that deals with keystone.  That utility
class requires the mistral.context.

I started looking at the context requirements from two separate points of
view:
 - create a security context in mistral lib that could be an attribute in
the mistral context
 - port entire mistral context over

When I looked at the attributes[4] currently in the mistral.context, most
if not all of them seem to be security related anyway.  I chose to port the
entire context over, but that also required dependencies on 4 threading
utility methods[5] and mistral.exceptions[6], so those were also ported
over.

I would appreciate any feedback or discussion on the current proposed
design.

Thanks,

Ryan


[1]
https://specs.openstack.org/openstack/mistral-specs/specs/newton/approved/mistral-custom-actions-api.html

[2] https://review.openstack.org/#/c/352435/

[3]
https://github.com/openstack/mistral/blob/master/mistral/utils/openstack/keystone.py

[4]
https://github.com/openstack/mistral/blob/master/mistral/context.py#L76-L87

[5]
https://github.com/openstack/mistral/blob/master/mistral/utils/__init__.py#L49-L94

[6] https://github.com/openstack/mistral/blob/master/mistral/exceptions.py

-- 
Ryan Brady
Cloud Engineering
rbr...@redhat.com
919.890.8925
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [E] [TripleO] scripts to do post deployment analysis of an overcloud

[Hugh Brock]

On Mon, Aug 8, 2016 at 4:02 PM, Tomas Sedovic  wrote:
> On 08/08/2016 11:05 AM, Hugh Brock wrote:
>>
>> On Wed, Aug 3, 2016 at 4:49 PM, Joe Talerico  wrote:
>>>
>>> On Wed, Jul 27, 2016 at 2:04 AM, Hugh Brock  wrote:

 On Jul 26, 2016 8:08 PM, "Gordon, Kent"
 
 wrote:
>
>
>
>
>
>
>
>> -Original Message-
>> From: Gonéri Le Bouder [mailto:gon...@lebouder.net]
>> Sent: Tuesday, July 26, 2016 12:24 PM
>> To: openstack-dev@lists.openstack.org
>> Subject: [E] [openstack-dev] [TripleO] scripts to do post deployment
>> analysis
>> of an overcloud
>>
>> Hi all,
>>
>> For the Distributed-CI[0] project, we did two scripts[1] that we use
>> to
>> extract
>
>
> Links not included in message
>
>> information from an overcloud.
>> We use this information to improve the readability of the deployment
>> logs.
>> I attached an example to show how we use the extracted stack
>> information.
>>
>> Now my question, do you know some other tools that we can use to do
>> this
>> kind of anaylsis?
>> --
>> Gonéri Le Bouder
>
>
> 
>>
>>
>> Tomaš, this seems like it might be useful for the deploy validations,
>> and also for improving the quality of error reporting -- would you
>> mind taking a look?
>
>
> Yeah we may end up using the heat stack dump tool for some validations since
> it puts all the Heat data in one place.
>
> However, this seems like a great thing to be included in the openstack
> overcloud deploy command and/or the mistral workflow.
>
> I.e. after each deployment, we could run list_nodes_status to verify that
> the overcloud nodes are indeed running and accessible (before tempest) and
> dump the o-c-c logs to a known directory.

Cool!

Also don't forget about sbaker's amazing new error reporting tool
"openstack stack failures list overcloud" -- might be useful...

--Hugh


-- 
  ,   ,| Hugh Brock, hbr...@redhat.com
  )-_"""_-(| Director of Engineering, OpenStack Management
 ./ o\ /o \.   | TripleO: Install, configure, and scale OpenStack.
. \__/ \__/ .  | http://rdoproject.org, http://tripleo.org
...   V   ...  |
... - - - ...  | "I know that you believe you understand what you
 .   - -   .   | think I said, but I'm not sure you realize that what
  `-.-´| you heard is not what I meant." --Robert McCloskey
 "TripleOwl"

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [scheduler] Use ResourceProviderTags instead of ResourceClass?

[Jay Pipes]

On 08/08/2016 06:14 AM, Chris Dent wrote:

On Mon, 8 Aug 2016, Alex Xu wrote:


Chris, thanks for the blog to explain your idea! It helps me understand
your idea better.


Thanks for reading it. As I think I've mentioned a few times I'm not
really trying to sell the idea, just make sure it is clear enough to
be evaluated.


I agree the goal for API interface design in your blog. But one point I
guess you also agree, that is "The interface is easy to understand for
API
user". So look at the example of API request flow with gabbi,  it is
pretty
clear for me even I didn't spend any time to learn the gabbi. That means:
gabbi is cool and the interface is clear! But the only confuse is "total:
∞". And the related ResourceClass is "ssd", does it mean disk size is
infinite? For a user, he is learning our API, he needs to search the
document, due to he want to know "what is this special usage way means
to".
If user can understand our API without any document, so that is prefect.


I think the main source of confusion is that where I find it pretty
easy to think of qualitative characteristics as being an "infinity
of a quantity" that's not an easy concept in general.

In the "ssd" example what it means is not that disk size is infinite
but taht the ssd-ness of the resource provider is infinite. So, for
example a resource provider which provides disk space that is hosted
on ssd has (at least) two resource classes:

   DISK_GB: 
   SSD: 

When some ssd-ness is consumed, all of it (infinity) is still left
over.

For me it is easier: a resource provider has just one way of
describing what it can do: classes of inventory (that provide
gigabytes of disk that are ssd). When we add tags/capabilities
we have another mode of description.


I could not disagree more. :)

You don't have an "inventory" of SSD. You have an inventory of DISK_GB. 
Whether the resource provider of that DISK_GB resource uses SSD or HDD 
disks is an *adjective* -- i.e. a capability -- that describes the provider.


Saying capabilities are just resources with infinite inventory just 
makes the API more confusing IMHO.


It's like saying "hey, give me two apples and an infinite amount of 
red." Just doesn't make sense.


My two pesos,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Cinder] [stable] [all] Changing stable policy for drivers

[Sean McGinnis]

On Sat, Aug 06, 2016 at 08:00:06PM -0400, Ben Swartzlander wrote:
> On 08/06/2016 06:11 PM, Jeremy Stanley wrote:
> >On 2016-08-06 17:51:02 -0400 (-0400), Ben Swartzlander wrote:
> >[...]
> >>when it's no longer to run dsvm jobs on them (because those jobs
> >>WILL eventually break as infra stops maintaining support for very
> >>old releases) then we simply remove those jobs and rely on vendor
> >>CI + minimal upstream tests (pep8, unit tests).
> >
> >This suggestion has been resisted in the past as it's not up to our
> >community's QA standards, and implying there is "support" when we
> >can no longer test that changes don't cause breakage is effectively
> >dishonest. In the past we've held that if a branch is no longer
> >testable, then there's not much reason to collaborate on code
> >reviewing proposed backports in the first place. If we're reducing
> >these branches to merely a holding place for "fixes" that "might
> >work" it doesn't sound particularly beneficial.
> 
> Well this was the whole point, and the reason I suggested using a
> different branch other than stable/release. Keeping the branches
> open for driver bugfix backports is only valuable if we can go 5
> releases back.

We can debate how far back they should go, but I think initially we can
limit it to less that 5. I haven't done anywhere near a thorough survey,
but my impression is most distros so far have tried to stay with three
releases of the current one.

> 
> I agree the level of QA we can do gets less as releases get older,
> and nobody expects the Infra team to keep devstack-gate running on
> such old releases. However vendors and distros DO support such old
> releases and the proposal to create these branches is largely to
> simplify the distributions of bugfixes from vendors to customers and
> distros.
> 
> Compare this proposal to the status quo, which is that several
> vendors effectively maintain forks of Cinder on github or other
> public repos just to have a place to distribute bugfixes on old
> releases. Distros either need to know about these repos or do the
> backports from master themselves when taking bugfixes into old
> releases.

I think this is a key point. The need for this is to have a common and
known place for these driver fixes to be backported and made available
to whomever needs them. This isn't necessarily saying we are going to
thoroughly test and guarantee that these backports are 100% OK.

>From my understanding, part of what has failed with keeping stable
branches around longer has been the lack of involvement from distros and
vendors to keep those branches up to date. After the EOL date, these
backported driver fixes would be 100% on the vendors to have tested
their drivers and made sure they work. We as a community would just be
provider a place for those updates to go and a known place for those
that need the fixes to find them.

> 
> -Ben
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][grafana][infra] how to read grafana

[Matthew Treinish]

On Mon, Aug 08, 2016 at 02:40:31PM +0200, Ihar Hrachyshka wrote:
> Hi,
> 
> I was looking at grafana today, and spotted another weirdness.
> 
> See the periodic jobs dashboard:
> 
> http://grafana.openstack.org/dashboard/db/neutron-failure-rate?panelId=4
> 
> Currently it shows for me 100% failure rate for py34/oslo-master job,
> starting from ~Aug 3. But when I go to openstack-health, I don’t see those
> runs at all:
> 
> http://status.openstack.org/openstack-health/#/job/periodic-neutron-py34-with-neutron-lib-master
> 
> (^ The last run is July 31.)
> 
> But then when I drill down into files, I can see more recent runs, like:
> 
> http://logs.openstack.org/periodic/periodic-neutron-py34-with-neutron-lib-master/?C=M;O=A
> http://logs.openstack.org/periodic/periodic-neutron-py34-with-neutron-lib-master/faa24e0/testr_results.html.gz
> 
> The last link points to a run from yesterday. And as you can see it is
> passing.

That run isn't actually from yesterday, it's from July 30th. The directory shows
a recent date, but the last modified dates for the individual files is older:

http://logs.openstack.org/periodic/periodic-neutron-py34-with-neutron-lib-master/faa24e0/

The openstack-health data goes up until the job started failing, this is likely
because the failures occur early enough in the test run that no subunit output
is generated for the run.

> 
> So, what’s wrong with the grafana dashboard? And why doesn’t
> openstack-health show the latest runs?
> 

On the openstack-health side it looks like you're running into an issue with
using subunit2sql as the primary data source there. If you look at an example
output from what's not in openstack-health, like:

http://logs.openstack.org/periodic/periodic-neutron-py34-with-neutron-lib-master/37cd5eb/console.html.gz

You'll see that the failure is occuring before any subunit output is generated.
(during the discovery phase of testr) If there is no subunit file in the log
output for the run, then there is nothing to populate the subunit2sql DB with.
The grafana/graphite data doesn't share this limitation because it gets
populated directly by zuul.

This is a known limitation with openstack-health right, and the plan to solve it
is to add a zuul sql data store that we can query like subunit2sql for job level
information, and then use subunit2sql for more fine grained details. The work on
that currently depends on: https://review.openstack.org/#/c/22/ which adds
the datastore to zuul. Once that lands we can work on the openstack-health side
consume that data in conjunction with subunit2sql.

-Matt Treinish


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [E] [TripleO] scripts to do post deployment analysis of an overcloud

[Tomas Sedovic]

On 08/08/2016 11:05 AM, Hugh Brock wrote:

On Wed, Aug 3, 2016 at 4:49 PM, Joe Talerico  wrote:

On Wed, Jul 27, 2016 at 2:04 AM, Hugh Brock  wrote:

On Jul 26, 2016 8:08 PM, "Gordon, Kent" 
wrote:








-Original Message-
From: Gonéri Le Bouder [mailto:gon...@lebouder.net]
Sent: Tuesday, July 26, 2016 12:24 PM
To: openstack-dev@lists.openstack.org
Subject: [E] [openstack-dev] [TripleO] scripts to do post deployment
analysis
of an overcloud

Hi all,

For the Distributed-CI[0] project, we did two scripts[1] that we use to
extract


Links not included in message


information from an overcloud.
We use this information to improve the readability of the deployment
logs.
I attached an example to show how we use the extracted stack
information.

Now my question, do you know some other tools that we can use to do this
kind of anaylsis?
--
Gonéri Le Bouder






Tomaš, this seems like it might be useful for the deploy validations,
and also for improving the quality of error reporting -- would you
mind taking a look?


Yeah we may end up using the heat stack dump tool for some validations 
since it puts all the Heat data in one place.


However, this seems like a great thing to be included in the openstack 
overcloud deploy command and/or the mistral workflow.


I.e. after each deployment, we could run list_nodes_status to verify 
that the overcloud nodes are indeed running and accessible (before 
tempest) and dump the o-c-c logs to a known directory.




--Hugh





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Nova mascot

[Chris Friesen]

On 08/08/2016 05:07 AM, Alexis Lee wrote:

I propose the ant, known for discovering probabilistic graph traversal
algorithms.

My other thought, the bee, has already been taken by refstack.


How about the octopus, since nova has tentacles reaching into many parts of the 
system...


Chris


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Nova mascot

[Alexis Lee]

Markus Zoeller said on Mon, Aug 08, 2016 at 01:29:49PM +0200:
> On 08.08.2016 13:07, Alexis Lee wrote:
> > I propose the ant, known for discovering probabilistic graph traversal
> > algorithms.
> > 
> > My other thought, the bee, has already been taken by refstack.
> 
> AFAIK #infra is going to chose the ant.

I suppose getting in there quick would be considered unsporting?

At least this illustrates why we should hurry up.

> I propose the "supernova":
> https://de.wikipedia.org/wiki/Supernova#/media/File:SN1994D.jpg
> 
> https://en.wikipedia.org/wiki/Nova#/media/File:GKPersei-MiniSuperNova-20150316.jpg

IMHO it fails the "distinct in stylized cartoon illustrations"
criterion, it just looks like a star or explosion.

The gorilla is taken but we could have a monkey. I like the Golden Lion
Tamarin. Monkeys are vaguely smart and its ruff is not unreminiscent of
a supernova, or at least a firework explosion.

Alternatively, we could have a mountain*. Massively scalable. Very
traditional symbol, easy to draw.

Other ideas, mostly with issues - adder (CPU); snowshoe hare, star-nosed
mole, lynx (Nova Scotia); dolphin, open brain coral (brain); wellspring,
mitosis (novelty); hummingbird, tiger (generic); sheepdog (herding).


Alexis (lxsli)

* It's tempting to suggest a volcano (mountain that's liable to explode
taking out everyone in a several-mile radius).
-- 
Nova developer, Hewlett-Packard Limited.
Registered Office: Cain Road, Bracknell, Berkshire RG12 1HN.
Registered Number: 00690597 England
VAT number: GB 314 1496 79

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [infra] Intel NFV CI voting permission

[Markus Zoeller]

On 03.08.2016 22:02, Znoinski, Waldemar wrote:
> Thanks
> Much appreciated
> 
> Making use of the opportunity here... what's the next big thing a CI (like 
> one testing NFV) should be doing? (multinode or there's something else more 
> important?)

Two tiny nits on the comment the CI is giving a change:

"Build failed (check pipeline). To recheck leave a comment with
intel-nfv-ci recheck. For more info go to
https://wiki.openstack.org/wiki/ThirdPartySystems/Intel-NFV-CI.;

1) Could you put the recheck command in quotation? Like:
   To recheck leave a comment with 'intel-nfv-ci recheck'.
2) The link to the wiki seems to be wrong (hyphen vs. underscore):
   https://wiki.openstack.org/wiki/ThirdPartySystems/Intel_NFV_CI

-- 
Regards, Markus Zoeller (markus_z)

>  >-Original Message-
>  >From: Matt Riedemann [mailto:mrie...@linux.vnet.ibm.com]
>  >Sent: Wednesday, August 3, 2016 8:28 PM
>  >To: openstack-dev@lists.openstack.org
>  >Subject: Re: [openstack-dev] [nova] [infra] Intel NFV CI voting permission
>  >
>  >On 8/3/2016 8:18 AM, Znoinski, Waldemar wrote:
>  >> [WZ] Hi Matt, do we need, a Gerrit-style, +2 on that from someone?
>  >> Infra Team doesn't keep these settings in their puppet/config files on 
> git -
>  >all the Gerrit changes are done via Gerrit GUI so they rely on Cores to add 
> CIs
>  >to the appropriate ci group, nova-ci in this case.
>  >>
>  >>
>  >
>  >Sorry, I wasn't sure what the next step here was, I guess it was the nova-ci
>  >membership change, which is done now:
>  >
>  >https://review.openstack.org/#/admin/groups/511,members
>  >
>  >--
>  >
>  >Thanks,
>  >
>  >Matt Riedemann
>  >
>  >
>  >__
>  >
>  >OpenStack Development Mailing List (not for usage questions)
>  >Unsubscribe: OpenStack-dev-
>  >requ...@lists.openstack.org?subject:unsubscribe
>  >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> --
> Intel Research and Development Ireland Limited
> Registered in Ireland
> Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
> Registered Number: 308263
> 
> 
> This e-mail and any attachments may contain confidential material for the sole
> use of the intended recipient(s). Any review or distribution by others is
> strictly prohibited. If you are not the intended recipient, please contact the
> sender and delete all copies.
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [mistral] Team meeting reminder - 08/08/2016

[Renat Akhmerov]

Hi,

This is a reminder that we’ll have a team meeting today at #openstack-meeting 
at 16.00 UTC.

Agenda:
Review action items
Current status (progress, issues, roadblocks, further plans)
Open discussion


Renat Akhmerov
@Nokia

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][nova][SR-IOV] deprecation of supported_pci_vendor_devs

[Beliveau, Ludovic]

+1

This validation in neutron is redundant.

/ludovic

-Original Message-
From: Moshe Levi [mailto:mosh...@mellanox.com] 
Sent: August-08-16 8:43 AM
To: OpenStack Development Mailing List (not for usage questions) 
(openstack-dev@lists.openstack.org)
Subject: [openstack-dev] [neutron][nova][SR-IOV] deprecation of 
supported_pci_vendor_devs

Hi all,

To reduce complexity in configuring SR-IOV I want to deprecate the 
supported_pci_vendor_devs option [1] in the neutron-server ml2 config.
This option is doing extra validation that pci vendor id and product id 
provided by nova in the neutron port binding profile is matching to the vendor 
id and product id  in supported_pci_vendor_devs. 

In my opinion this is redundant, nova-scheduler is the point to do validation 
and select a suitable hypervisor. 
The compute node is already validating this through the 
pci_passthrough_whitelist option in nova.conf [2].

I don't see a reason why the neutron-server should validate the pci vendor_id 
and product_id again from the neutron port binding profile. 

If there is good reason to keep it please let me know, otherwise I will 
deprecate it.

[1] - supported_pci_vendor_devs = ['15b3:1004', '8086:10ca'] [2] - 
pci_passthrough_whitelist = 
{"address":"*:06:00.*","physical_network":"physnet1"} 


Thanks,
Moshe

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api][os-api-ref] openstackdocstheme integration

[Sean Dague]

On 08/08/2016 08:44 AM, Doug Hellmann wrote:
> Excerpts from Hayes, Graham's message of 2016-08-08 11:28:35 +:
>> On 05/08/2016 19:15, Doug Hellmann wrote:
>>> Excerpts from Hayes, Graham's message of 2016-08-05 17:04:35 +:
 Hey,

 We look like we are getting close to merging the os-api-ref integration
 with openstackdocstheme.

 Unfortunately, there is no "phased" approach available - the version
 released with compatibility for openstackdocstheme will not work
 with oslo.sphinx.
>>>
>>> In what way doesn't it work? Is one of the themes missing something?
>>>
>>> Doug
>>
>> Both themes are laid out differently. One uses bootstrap and the other
>> doesn't, one has a different view on what should be hidden, and where
>> TOCs belong.
>>
>> The end result was that for the oslosphinx integration we included extra
>> CSS / JS, but that code can cause conflicts with openstackdocstheme.
> 
> Would putting that extra stuff into oslosphinx, as an optional part of
> the them, make the transition any easier?

It's actually somewhat the inverse problem (IIRC).

oslosphinx is written as an appropriate sphinx extension / theme, it
plays nice with others. You can tell the author(s) were familiar with
sphinx.

openstackdocstheme was done as a bootstrap UX, then grafted into sphinx
builds in a way that just barely works, as long as you don't include any
other sphinx extensions. The moment you do, things get really funky.
Given that it does things like carry it's own jquery (needed by
bootstrap), instead of doing the standard scripts include in sphinx.
This was clearly written by folks that were familiar with boostrap, and
not really with sphinx.

When we hacked together os-api-ref the incompatibilities with
openstackdocstheme were getting in the way, so it was done with
oslosphinx in mind. There were definitely styling elements we needed
differently, and instead of negotiating changing the style on everything
else, that styling was done in os-api-ref.

os-api-ref also needs some dynamic elements. For instance, section
expand / collapse, and sensible bookmarking. In a perfect world that
probably ends up in the theme layer, which means doing it in both
oslosphinx and openstackdocstheme, the extension only creating base markup.

However, the goal isn't to support both. It's to support
openstackdocstheme which is the strategic UX for all openstack docs
(even though it's actually not very sphinxy, which is a whole other
issue that will probably hurt us other places).

So, we could do a lot of work to smooth the transition, which would get
thrown away shortly after, or just create a flag day and have docs
broken for a bit until people get across it.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api][os-api-ref] openstackdocstheme integration

[Hayes, Graham]

On 08/08/2016 13:47, Doug Hellmann wrote:
> Excerpts from Hayes, Graham's message of 2016-08-08 11:28:35 +:
>> On 05/08/2016 19:15, Doug Hellmann wrote:
>>> Excerpts from Hayes, Graham's message of 2016-08-05 17:04:35 +:
 Hey,

 We look like we are getting close to merging the os-api-ref integration
 with openstackdocstheme.

 Unfortunately, there is no "phased" approach available - the version
 released with compatibility for openstackdocstheme will not work
 with oslo.sphinx.
>>>
>>> In what way doesn't it work? Is one of the themes missing something?
>>>
>>> Doug
>>
>> Both themes are laid out differently. One uses bootstrap and the other
>> doesn't, one has a different view on what should be hidden, and where
>> TOCs belong.
>>
>> The end result was that for the oslosphinx integration we included extra
>> CSS / JS, but that code can cause conflicts with openstackdocstheme.
>
> Would putting that extra stuff into oslosphinx, as an optional part of
> the them, make the transition any easier?
>
> Doug

I don't think so - with the changes to the structure of the HTML things
will be broken anyway.

It is unfortunate, but I think we have a better chance of doing the cut
over now, before more projects start using the library.

If everyone agrees with the patch for the phased roll over, I can submit
the patches to all the required repos, and help them get merged.

Graham

>>
>> As one theme already uses bootstrap, the css (and the classes applied
>> to the HTML elements) has to be modified, and is incompatible with the
>> old theme, as it was only using the sideloaded bootstrap css in a
>> section of the page.
>>
>> The review for the integration is here:
>>
>> https://review.openstack.org/#/c/322430/
>>
>> - Graham
>>
>>>
 So, we need a way to use oslosphinx until it is released, and the new
 theme after it is released.


 I suggest we put a temporary section of code in the `conf.py` of each
 project using os-api-ref - I have a WIP preview for designate up for
 review [0]

 Can I get some feedback, if people think this is a good way forward?

 The list of repos I have using os-api-ref is (from [1]:

 openstack/networking-sfc
 openstack/ceilometer
 openstack/glance
 openstack/heat
 openstack/ironic
 openstack/keystone
 openstack/manila
 openstack/designate
 openstack/neutron-lib
 openstack/nova
 openstack/sahara
 openstack/searchlight
 openstack/senlin
 openstack/swift
 openstack/zaqar

 Thanks,

 Graham

 0 - https://review.openstack.org/#/c/351800/
 1 -
 http://codesearch.openstack.org/?q=os_api_ref=nope=api-ref%2Fsource%2Fconf.py=

>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tacker] Weekly meeting agenda

[Dharmendra Kushwaha]

Dear Sridhar,

Can we also discuss the NSD?

Thanks & Regards
Dharmendra Kushwaha

From: Sridhar Ramaswamy [sric...@gmail.com]
Sent: Monday, August 08, 2016 8:49 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: [openstack-dev] [tacker] Weekly meeting agenda

Tackers,

Here is the agenda for this week's irc meeting,

https://wiki.openstack.org/wiki/Meetings/Tacker

- Announcements
- Newton release deadlines
- Newton priorities - feature / RFE go/no-go check
- Mistral workflow BP
- Open Discussion

Let me know if you've anything else to discuss.

thanks,
Sridhar

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ironic] Driver composition defaults call

[Julia Kreger]

Thank you for sending the corrected link Mathieu!  I thought I fixed it
before I sent the email, but... *shrug*

Anyway!  Looking at the doodle, the mutually available time is 4 PM UTC on
this Wednesday (8/10/16).  If there are no objections, I guess we will hear
those seeking to discuss defaults on conferencing[0] bridge number  at
that time.

-Julia

[0] https://wiki.openstack.org/wiki/Infrastructure/Conferencing
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api][os-api-ref] openstackdocstheme integration

[Doug Hellmann]

Excerpts from Hayes, Graham's message of 2016-08-08 11:28:35 +:
> On 05/08/2016 19:15, Doug Hellmann wrote:
> > Excerpts from Hayes, Graham's message of 2016-08-05 17:04:35 +:
> >> Hey,
> >>
> >> We look like we are getting close to merging the os-api-ref integration
> >> with openstackdocstheme.
> >>
> >> Unfortunately, there is no "phased" approach available - the version
> >> released with compatibility for openstackdocstheme will not work
> >> with oslo.sphinx.
> >
> > In what way doesn't it work? Is one of the themes missing something?
> >
> > Doug
> 
> Both themes are laid out differently. One uses bootstrap and the other
> doesn't, one has a different view on what should be hidden, and where
> TOCs belong.
> 
> The end result was that for the oslosphinx integration we included extra
> CSS / JS, but that code can cause conflicts with openstackdocstheme.

Would putting that extra stuff into oslosphinx, as an optional part of
the them, make the transition any easier?

Doug

> 
> As one theme already uses bootstrap, the css (and the classes applied
> to the HTML elements) has to be modified, and is incompatible with the
> old theme, as it was only using the sideloaded bootstrap css in a
> section of the page.
> 
> The review for the integration is here:
> 
> https://review.openstack.org/#/c/322430/
> 
> - Graham
> 
> >
> >> So, we need a way to use oslosphinx until it is released, and the new
> >> theme after it is released.
> >>
> >>
> >> I suggest we put a temporary section of code in the `conf.py` of each
> >> project using os-api-ref - I have a WIP preview for designate up for
> >> review [0]
> >>
> >> Can I get some feedback, if people think this is a good way forward?
> >>
> >> The list of repos I have using os-api-ref is (from [1]:
> >>
> >> openstack/networking-sfc
> >> openstack/ceilometer
> >> openstack/glance
> >> openstack/heat
> >> openstack/ironic
> >> openstack/keystone
> >> openstack/manila
> >> openstack/designate
> >> openstack/neutron-lib
> >> openstack/nova
> >> openstack/sahara
> >> openstack/searchlight
> >> openstack/senlin
> >> openstack/swift
> >> openstack/zaqar
> >>
> >> Thanks,
> >>
> >> Graham
> >>
> >> 0 - https://review.openstack.org/#/c/351800/
> >> 1 -
> >> http://codesearch.openstack.org/?q=os_api_ref=nope=api-ref%2Fsource%2Fconf.py=
> >>
> >
> > __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
> 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron][nova][SR-IOV] deprecation of supported_pci_vendor_devs

[Moshe Levi]

Hi all,

To reduce complexity in configuring SR-IOV I want to deprecate the 
supported_pci_vendor_devs option [1] in the neutron-server ml2 config.
This option is doing extra validation that pci vendor id and product id 
provided by nova in the neutron port binding profile is matching to the vendor 
id and product id  in supported_pci_vendor_devs. 

In my opinion this is redundant, nova-scheduler is the point to do validation 
and select a suitable hypervisor. 
The compute node is already validating this through the 
pci_passthrough_whitelist option in nova.conf [2].

I don't see a reason why the neutron-server should validate the pci vendor_id 
and product_id again from the neutron port binding profile. 

If there is good reason to keep it please let me know, otherwise I will 
deprecate it.

[1] - supported_pci_vendor_devs = ['15b3:1004', '8086:10ca']
[2] - pci_passthrough_whitelist = 
{"address":"*:06:00.*","physical_network":"physnet1"} 


Thanks,
Moshe

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron][grafana][infra] how to read grafana

[Ihar Hrachyshka]

Hi,

I was looking at grafana today, and spotted another weirdness.

See the periodic jobs dashboard:

http://grafana.openstack.org/dashboard/db/neutron-failure-rate?panelId=4

Currently it shows for me 100% failure rate for py34/oslo-master job,  
starting from ~Aug 3. But when I go to openstack-health, I don’t see those  
runs at all:


http://status.openstack.org/openstack-health/#/job/periodic-neutron-py34-with-neutron-lib-master

(^ The last run is July 31.)

But then when I drill down into files, I can see more recent runs, like:

http://logs.openstack.org/periodic/periodic-neutron-py34-with-neutron-lib-master/?C=M;O=A
http://logs.openstack.org/periodic/periodic-neutron-py34-with-neutron-lib-master/faa24e0/testr_results.html.gz

The last link points to a run from yesterday. And as you can see it is  
passing.


So, what’s wrong with the grafana dashboard? And why doesn’t  
openstack-health show the latest runs?


Ihar

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Some thoughts on API microversions

[Chris Dent]

On Wed, 3 Aug 2016, Andrew Laski wrote:


I am a fan of microversions. I think they work wonderfully to express
when a resource representation changes, or when different data is
required in a request. This allows clients to make the same request
across multiple clouds and expect the exact same response format,
assuming those clouds support that particular microversion. I also think
they work well to express that a new resource is available. However I do
think think they have some shortcomings in expressing that a resource
has been removed. But in short I think microversions work great for
expressing that there have been changes to the structure and format of
the API.


Yeah.

microversions are effectively a locally created stand in for the pre-
existing concept of content negotiation (of versioned media-types) in
HTTP with some overloaded functionality and semantics. Where the
microversions functionality is aligned with content negotiation
(expressing differences in resource representations in requests and
responses) it works well enough as a control mechanism. Where it tries
to operate as a stand in for other aspects of HTTP it doesn't work as
well. For example, using a microversion to indicate the removal of a
resource is redundant. HTTP provides 404 and 410 (and sometimes things
in the 3xx range) for that. If client code is relaying that response to
the user agent, that's a pretty effective post hoc _signal_. There's no
good way to a priori signal the missing resources. Some initial action
is required by the client, whether that's pulling in some
capabilities[1] or simply reading some docs.

In the context of a removed resource all a microversion provides is a
mode of control (I want the old version so I can get at the old
thing), separate from any server->client signalling.

I think we can do a better job of treating and respecting users (end and
middle) as humans by having strong expectations on each side of the
equation that there will be human and humane communication about the
state of the API[2]. We persist in the belief that microversions are a
signal that gets straight to the actual user but it isn't really all
that clear that they are anywhere near as visible as that statement
would suggest. Nor, really, should they be. The behavior of the API
is the most visible thing.

As Jay says, we should encourage and vigorously require them to upgrade
(by raising the minimum version) and more importantly we should make it
worth their while. At the moment it seems like we make either make it
too easy for users and deployers to stay in the past[3] of
insufficiently desirable for them to upgrade.

[1] Mind you, capabilities feels like a bit like a locally created stand
in for hypermedia[2].

[2] Hypermedia may or may not be a myth.

[3] Of course at some point it needs to be true that an API is done
and other functionality should happen elsewhere.

--
Chris Dent   ┬─┬ノ( º _ ºノ) http://anticdent.org/
freenode: cdent tw: @anticdent__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [vitrage] Nova host list api - performance

[Weyl, Alexey (Nokia - IL)]

I need to check if the data transferred is compressed, but in either way I 
don’t want to get more than 99% (that the manager object takes) of the data 
sent for the instance / host / availablility zone, so I would like to get the 
data without that property.

From: Yujun Zhang [mailto:zhangyujun+...@gmail.com]
Sent: Monday, August 08, 2016 3:21 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [nova] [vitrage] Nova host list api - performance

Is the data transfer compressed?

If there are lots of repeated pattern in the payload, compressing the content 
may result in great improvement in performance.
--
Yujun

On Mon, Aug 8, 2016 at 3:19 PM Weyl, Alexey (Nokia - IL) 
> wrote:
Hi,

I am running the "client.hosts.list()" command in Vitrage to get all the hosts 
in the system (we also run similar commands for availability zones and 
instances). Vitrage does this to gain a holistic view of system resources[1].

As part of our performance checks we noticed that when calling 
client.hosts.list(), the info on each host includes the "HostManager" object, 
which weighs about 840KB on the oslo messaging bus, which is quite heavy.

We would like to reduce that amount of data transferred, to improve 
performance. Is it possible to pass some parameter in "client.hosts.list()" so 
the manager property won't be sent on the bus?

Thanks in advance,
Alexey

[1] https://wiki.openstack.org/wiki/Vitrage


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [vitrage] Nova host list api - performance

[Yujun Zhang]

Is the data transfer compressed?

If there are lots of repeated pattern in the payload, compressing the
content may result in great improvement in performance.
--
Yujun

On Mon, Aug 8, 2016 at 3:19 PM Weyl, Alexey (Nokia - IL) <
alexey.w...@nokia.com> wrote:

> Hi,
>
> I am running the "client.hosts.list()" command in Vitrage to get all the
> hosts in the system (we also run similar commands for availability zones
> and instances). Vitrage does this to gain a holistic view of system
> resources[1].
>
> As part of our performance checks we noticed that when calling
> client.hosts.list(), the info on each host includes the "HostManager"
> object, which weighs about 840KB on the oslo messaging bus, which is quite
> heavy.
>
> We would like to reduce that amount of data transferred, to improve
> performance. Is it possible to pass some parameter in "client.hosts.list()"
> so the manager property won't be sent on the bus?
>
> Thanks in advance,
> Alexey
>
> [1] https://wiki.openstack.org/wiki/Vitrage
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova] next notification subteam meeting

[Balázs Gibizer]

Hi, 

The next notification subteam meeting will be held on 2016.08.09 17:00 UTC [1] 
on #openstack-meeting-4.

Cheers,
Gibi

[1] https://www.timeanddate.com/worldclock/fixedtime.html?iso=20160809T17

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [octavia] Multi-node controller testing

[Miguel Angel Ajo Pelayo]

Recently, I sent a series of patches [1] to make it easier for
developers to deploy a multi node octavia controller with
n_controllers x [api, cw, hm, hk] with an haproxy in front of the API.

Since this is the way the service is designed to work (with horizontal
scalability in mind), and we want to have a good guarantee that any
bug related to such configuration is found early, and addressed, I was
thinking that an extra job that runs a two node controller deployment
could be beneficial for the project.


If we all believe it makes sense, I would be willing to take on this
work but I'd probably need some pointers and light help, since I've
never dealt with setting up or modifying existing jobs.

How does this sound?


[1] 
https://review.openstack.org/#/q/status:merged+project:openstack/octavia+branch:master+topic:multinode-devstack

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [glare][kolla] timing of release of glare and request for technical interview on IRC wherever the glare team wants to have it

[Steven Dake (stdake)]

Cool thanks for the response.  Appreciate it.  I think the big take away is all 
the ODMs are free from churn in Newton and have a full cycle to adapt to the 
changes which is great news!

Thanks!
-steve


From: Mikhail Fedosin >
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
>
Date: Monday, August 8, 2016 at 3:09 AM
To: "OpenStack Development Mailing List (not for usage questions)" 
>
Subject: Re: [openstack-dev] [glare][kolla] timing of release of glare and 
request for technical interview on IRC wherever the glare team wants to have it

Hello Steven!
Our plans for Glare in Newton are: 1. Implement beta version of App-Catalog, 
based on Glare v1; 2. Develop an experimental support (POC) for Murano, Heat 
and probably Tacker. It means that all big updates will be in Ocata, and 
despite the fact that the code will be ready in this cycle, I do not think we 
need an immediate adoption in Kolla.

And for sure, I'll be glad to answer all your questions, 1600 UTC today works 
for me. So, let's meet at this time in your channel.

Best,
Mike

On Sat, Aug 6, 2016 at 2:21 PM, Steven Dake (stdake) 
> wrote:
Hey folks,

I guess the split of glare and glance have been in the works for awhile.  It is 
challenging for operational deployment managers (ODMs) such as Kolla to keep up 
with the internal goings-on of every big tent project (or projects that shave 
off from big-tent projects).  Just to be clear up front, the Kolla community 
doesn't care that glare split the work out.  The Kolla development team adapts 
very quickly to upstream changes.  What we do care about is that we present an 
accurate deployment for Newton that represents the best that OpenStack has to 
offer and offer a seamless operational experience - within Kolla's capacity 
constraints.

 I need information on when the code base will be ready to consume (from a 
tarball on tarballs.oo).  Is this planned for milestone 3 - or post Newton?  
I'd recommend post-Newton for the split to be consumable - it would ease the 
difficulty of adoption if the various ODMs (and Operators) had more then 3 
weeks to work with on what is clearly a project required by every deployment 
based upon the threads I read.

I have some other technical questions related to how the glance registry 
disappears (I believe this point was mentioned in another thread by Jay) as 
well as the upgrade mechanism (if any is needed) that would best be served by a 
high bandwidth conversation on IRC (and those conversations are recorded on 
eavesdrop for others to benefit).

Would one of the technical cats from glare team drop by #opentack-kolla so we 
can get a quick (30 minutes) technical interview on the work to understand how 
this change impacts OpenStack in the short term (prior to Newton) and the long  
term layout of the two projects so we can make a determination a to how to 
proceed technically?  I don't necessarily need to be there - any of our core 
reviewer team can handle the Q - but would like to be there if possible.

If that doesn't work for the glare team, could we get 30 minutes of agenda time 
in what I'm sure is a busy glare team meeting to have the same technical 
discussion?

If that doesn't work for the glare team, we can host the topic in the kolla 
team meeting (UTC1600 on Wednesdays) if a glare core reviewer or the glare PTL 
can stop by.

Please let me know how you wish to proceed.

TIA
-steve


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Nova mascot

[Markus Zoeller]

On 08.08.2016 13:07, Alexis Lee wrote:
> I propose the ant, known for discovering probabilistic graph traversal
> algorithms.
> 
> My other thought, the bee, has already been taken by refstack.
> 
> 
> lxsli
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

AFAIK #infra is going to chose the ant.

I propose the "supernova":
https://de.wikipedia.org/wiki/Supernova#/media/File:SN1994D.jpg

https://en.wikipedia.org/wiki/Nova#/media/File:GKPersei-MiniSuperNova-20150316.jpg


-- 
Regards, Markus Zoeller (markus_z)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api][os-api-ref] openstackdocstheme integration

[Hayes, Graham]

On 05/08/2016 19:15, Doug Hellmann wrote:
> Excerpts from Hayes, Graham's message of 2016-08-05 17:04:35 +:
>> Hey,
>>
>> We look like we are getting close to merging the os-api-ref integration
>> with openstackdocstheme.
>>
>> Unfortunately, there is no "phased" approach available - the version
>> released with compatibility for openstackdocstheme will not work
>> with oslo.sphinx.
>
> In what way doesn't it work? Is one of the themes missing something?
>
> Doug

Both themes are laid out differently. One uses bootstrap and the other
doesn't, one has a different view on what should be hidden, and where
TOCs belong.

The end result was that for the oslosphinx integration we included extra
CSS / JS, but that code can cause conflicts with openstackdocstheme.

As one theme already uses bootstrap, the css (and the classes applied
to the HTML elements) has to be modified, and is incompatible with the
old theme, as it was only using the sideloaded bootstrap css in a
section of the page.

The review for the integration is here:

https://review.openstack.org/#/c/322430/

- Graham


>
>> So, we need a way to use oslosphinx until it is released, and the new
>> theme after it is released.
>>
>>
>> I suggest we put a temporary section of code in the `conf.py` of each
>> project using os-api-ref - I have a WIP preview for designate up for
>> review [0]
>>
>> Can I get some feedback, if people think this is a good way forward?
>>
>> The list of repos I have using os-api-ref is (from [1]:
>>
>> openstack/networking-sfc
>> openstack/ceilometer
>> openstack/glance
>> openstack/heat
>> openstack/ironic
>> openstack/keystone
>> openstack/manila
>> openstack/designate
>> openstack/neutron-lib
>> openstack/nova
>> openstack/sahara
>> openstack/searchlight
>> openstack/senlin
>> openstack/swift
>> openstack/zaqar
>>
>> Thanks,
>>
>> Graham
>>
>> 0 - https://review.openstack.org/#/c/351800/
>> 1 -
>> http://codesearch.openstack.org/?q=os_api_ref=nope=api-ref%2Fsource%2Fconf.py=
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][neutron-lbaas][stable] Cherry pick a patch from master to stable/liberty

[Ihar Hrachyshka]

zhi  wrote:


hi,
Recently, I saw a patch which fixed the listener's admin_state_up 
status[1]. This patch is already merged in master branch. But it is not merged 
into stable/liberty.

So I upload a new patch[2] to merge that patch into stable/liberty.



[1]: https://review.openstack.org/#/c/266816/
[2]: https://review.openstack.org/#/c/352238/


No need to send an email per backport patch proposed, reviewers will get to  
each of them when they have time.


As for the backport itself, two things to consider:
- Change-Id is wrong. Read:  
http://docs.openstack.org/project-team-guide/stable-branches.html#change-ids
- It’s not clear how critical bug is. The branch is in Phase II mode now:  
http://docs.openstack.org/project-team-guide/stable-branches.html#support-phases  
which means "Only critical bugfixes and security patches are acceptable”.  
AFAIU there is no formal definition of ‘criticality’ of a bug, but in  
neutron world we usually stick to ‘High and above’ in Launchpad terms. So  
we need that missing bit of information somewhere in Launchpad bug to  
assess if it still applies there.


Ihar

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Glance][Glare] External locations design

[Mikhail Fedosin]

Thanks for you comments Stuart!

I also have some thoughts on this matter: first of all, I never liked how
locations are implemented in glance v2, and not just because of
inconsistencies in multiple locations per one image, but for the reason
that there are no differences between internal and external locations.
So, I think we're on right track with glare's implementation. But I suppose
we can improve it even more:  in my understanding, the service generally
does not work with external data. I will try to explain with an example:

Let's assume that we have a link to external ubuntu image (
http://releases.ubuntu.com/16.04.1/ubuntu-16.04.1-server-amd64.iso ). When
user adds this link to his image artifact, Glare doesn't validate it, it
just creates a blob instance in DB (optionally user may specify a checksum).
When user requests 'show' on his image, the blob url will look like this:
{"url": "http://releases.ubuntu.com/16.04.1/ubuntu-16.04.1-server-amd64.iso"},
and if user wants to download it, the image won't be proxied with glare -
it will be a direct download from ubuntu.com from the client.

In case of internal locations their urls are {"url":
"/artifacts/images//file"} and these downloads are done with
Glare as a proxy between the client and internal cloud storage, like Swift.

For compatibility, internal urls should work with external blobs as well,
but there will be a redirect to the external url:
"/artifacts/images//file" -> "
http://releases.ubuntu.com/16.04.1/ubuntu-16.04.1-server-amd64.iso;.

We can discuss it on the artifact meeting at 1730 UTC on Mondays at
#openstack-meeting-alt

Best,
Mike

On Tue, Aug 2, 2016 at 1:16 PM,  wrote:

> Hi Kairat,
>
> I think it's great to try and tease through the various issues here.
> I added some comments to the etherpad.
>
> -Stuart
>
> Hello all,
>>
>> I would like to start to describe some design decisions we made in Glare
>> code (https://review.openstack.org/#/q/topic:bp/glare-api+status:open).
>> If
>> you are not familiar with Glare I suggest you to read the following spec:
>>
>> https://github.com/openstack/glance-specs/blob/master/specs/
>> newton/approved/glance/glare-api.rst
>>
>> I hope it will help other folks to understand Glare approach and provide
>> some constructive feedback for Glare. I think that we can also use Glare
>> solution for Glance in near future to address some drawbacks we have in
>> Glance.
>>
>> Glare locations
>>
>> Glance and Glare have possibility to set some external url as
>> image(artifact) location. This feature is quite useful for users who would
>> like to refer to some external image or artifact (for example, Fedora
>> image
>> on official Fedora site) and not to store this image or artifact in the
>> cloud.
>>
>> External locations in Glance have several specialities:
>>
>>   1.
>>
>>   It is possible to setup multiple locations for an image. Glance uses
>>   special location strategy to define which location to use. This strategy
>>   defined in glance codebase and can be configured in glance conf.
>>   2.
>>
>>   Glance doesn?t differ image locations specified by url and image
>>   locations uploaded to Glance backend. Glance has some restrictions about
>>   which urls to use for locations (see Glance docs for more info).
>>
>>
>> Glare external locations designed in different way to address some
>> drawbacks we have in Glance. So the approach is the following:
>>
>>   1.
>>
>>   Glare doesn?t support multiple locations, you can specify dict of blobs
>>   in artifact type and add url for each blob in dict. User must define a
>>   name(f.e. region name or priority) for blob in dict and this name can be
>>   used to retrieve this blob from artifact. So decision about which
>> location
>>   to use will be outside of Glare.
>>   2.
>>
>>   Glare adds a special flag to database for external locations. So they
>>   will be treated differently in Glare when delete artifact. If blob
>> value is
>>   external url then we don?t need to pass this url to backend and just
>> delete
>>   the record in DB. For now, Glare allows only http(s) locations set but
>> it
>>   may be extended in future but the idea still the same: external location
>>   are just records in DB.
>>   3.
>>
>>   Glare saves blob size and checksum when specifying external url. When
>>   user specified url Glare downloads the blob by url, calculates its size
>> and
>>   checksum. Of course, it leads to some performance degradation but we can
>>   ensure that the external blob is immutable. We made this because
>> security
>>   seems more important for Glare than performance. Also there are plans to
>>   extend this approach to support subscriptions for external locations so
>> we
>>   can increase secureness of that operation.
>>
>>
>> I think that some of the features above can be implemented in Glance. For
>> example, we can treat our locations as only read-only links if external
>> flag will be implemented.  It will allow us to 

Re: [openstack-dev] [Cinder] [stable] [all] Changing stable policy for drivers

[Ihar Hrachyshka]

Jeremy Stanley  wrote:


On 2016-08-06 17:51:02 -0400 (-0400), Ben Swartzlander wrote:
[...]

when it's no longer to run dsvm jobs on them (because those jobs
WILL eventually break as infra stops maintaining support for very
old releases) then we simply remove those jobs and rely on vendor
CI + minimal upstream tests (pep8, unit tests).


This suggestion has been resisted in the past as it's not up to our
community's QA standards, and implying there is "support" when we
can no longer test that changes don't cause breakage is effectively
dishonest. In the past we've held that if a branch is no longer
testable, then there's not much reason to collaborate on code
reviewing proposed backports in the first place. If we're reducing
these branches to merely a holding place for "fixes" that "might
work" it doesn't sound particularly beneficial.


Right. At this point the branch does not really rely on openstack  
infrastructure services (particularly CI), and so there is little reason to  
keep the code on OpenStack premises.


A similar suggestion was proposed earlier for extending life for stable  
branches while deprovisioning gate jobs, allowing interested parties to  
collaborate on eg. CVE fixes in this branch. It was not supported in the  
past, because CI is an integral part of the promise of ‘safe fixes’ that we  
give as part of our stable policy.


Ihar

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova] Nova mascot

[Alexis Lee]

I propose the ant, known for discovering probabilistic graph traversal
algorithms.

My other thought, the bee, has already been taken by refstack.


lxsli

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Cinder] [stable] [all] Changing stable policy for drivers

[Ihar Hrachyshka]

Sean McGinnis  wrote:


This may mostly be a Cinder concern, but putting it out there to get
wider input.

For some time now there has been some debate about moving third party
drivers in Cinder to be out of tree. I won't go into that too much,
other than to point out one of the major drivers for this desire that
was brought up at our recent Cinder midcycle.

It turned out at least part of the desire to move drivers out of tree
came down to the difficulty in getting bug fixes out to end users that
were on older stable versions, whether because that's what their distro
was still using, or because of some other internal constraint that
prevented them from upgrading.

A lot of times what several vendors ended up doing is forking Cinder to
their own github repo and keeping that in sync with backports, plus
including driver fixes they needed to get out to their end users. This
has a few drawbacks:


If you would at least provide a public (more or less) stable driver API for  
vendors to use, like neutron does, then your vendors would not need to fork  
the whole Cinder tree. Instead, they would 1) work with community on bug  
fixes while stable/* is supported; 2) once stable/* is EOL, fork it into  
their own repo (on their own premises!) and maintain it from there.  
Consumers will then decide whether they trust the vendor shipped code as  
much as upstream maintained version of it that is now EOL.


Why don't vendors feel like maintaining their drivers out of tree? Is it  
technically possible? Is it too much of a burden?




1- this is more work for the vendor to keep this fork up to date
2- end users don't necessarily know where to go to find these without
   calling in to a support desk (that then troubleshoots a known issue
   and hopefully eventually ends up contacting the folks internally that
   actually work on Cinder that know it's been fixed and where to get
   the updates). Generally a bad taste for someone using Cinder and
   OpenStack.
3- Distros that package stable branches aren't able to pick up these
   changes, even if they are picking up stable branch updates for
   security fixes
4- We end up with a lot of patches proposed against security only stable
   branches that we need to either leave or abandon, just so a vendor
   can point end users to the patch to be able to grab the code changes

Proposed Solution
-

So part of our discussion at the midcycle was a desire to open up stable
restrictions for getting these driver bugfixes backported. At the time,
we had discussed having new branches created off of the stable branches
specifically for driver bugfixes. Something like:

stable/mitaka > stable/mitaka-drivers


How would distributions that care about quality determine which one to ship  
in their products? If the former, for as long as it’s supported by  
upstream, then how/when/whether distros are expected to transition to the  
latter branch?




After talking to the infra team, this really did sound like overkill.
The suggestion was to just change our stable policy in regards to driver
bugfix backports. No need to create and maintain more branches. No need
to set up gate jobs and things like that.



Unless you manage to get it approved for the global policy, I think you  
will effectively make your stable:follows-policy tag obsolete, and then it  
should be removed from your project. Read the requirements:


https://governance.openstack.org/reference/tags/stable_follows-policy.html#requirements

Support phases are part of the stable policy, and so if you don’t mostly  
adhere to their definitions, you should not carry the tag. Which is fine  
with me, it’s up to Cinder team to decide whether it’s worth it.



So this is a divergence from our official policy. I want to propose
we officially make a change to our stable policy to call out that
drivers bugfixes (NOT new driver features) be allowed at any time.



If that would be pushed as a global OpenStack policy, I would voice my  
concerns.


I think Neutron model is much more viable, with vendors untangled from core  
neutron release cycles, and effectively controlling their own destiny by  
relying on (more or less) stable plugin/driver API.


Then each vendor will be able to determine whether carrying new bug fixes  
is more important for them than having the stable:follows-policy tag for  
their deliverable, without compromising the promise the core project  
(Cinder) made with the tag applied.



If that's not OK with other project teams that support any kind of third
party drivers, I will just implement this policy specific to Cinder
unless there is a very strong objection, with good logic behind it, why
this should not be allowed.



Support phases are signalling consumers what to expect from new patch/minor  
releases. Without following the global policy, you leave consumers puzzled  
as to whether the next patch release from a  
widely-advertised-to-be-CVE-only branch will break 

Re: [openstack-dev] [Magnum] Adding opensuse as new driver to Magnum

[Michal Jura]

Hi,

On 08/07/2016 09:24 PM, Hongbin Lu wrote:

Added to the agenda of next team meeting: 
https://wiki.openstack.org/wiki/Meetings/Containers#Agenda_for_2016-08-09_1600_UTC
 .


Thank you, I will join tomorrow (09.08.2016) Magnum team meeting,

Have a nice day,



Best regards,
Hongbin


-Original Message-
From: Murali Allada [mailto:murali.all...@rackspace.com]
Sent: August-04-16 12:38 PM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [Magnum] Adding opensuse as new driver to
Magnum

Michal,

The right place for drivers is the /drivers folder.

Take a look at the existing drivers as an examples. You'll also need to
update this file
https://github.com/openstack/magnum/blob/master/setup.cfg#L60
and add a new entry point for the driver.

I would encourage you to hold off on this patch. We are currently
working on using stevedore to load drivers and moving all the heat
stack creation and update operations to each driver.

-Murali


From: Michal Jura 
Sent: Thursday, August 4, 2016 3:26 AM
To: openstack-dev@lists.openstack.org
Subject: [openstack-dev] [Magnum] Adding opensuse as new driver to
Magnum

Hi,

I would like to put for discussion adding new driver to Magnum. We
would like to propose opensuse with kubernetes as driver. I did some
initial work in bug

https://launchpad.net/bugs/1600197
and changes
https://review.openstack.org/339480
https://review.openstack.org/349994

I've got also some comments from you about how this should be proceed.

As maintainer for this change I can propose myself.

I have couple question about moving this driver to /contrib directory.
If I will do this how this driver should be installed from there?

Thank you for all answers and help with doing this,

Best regards,
Michal

___
___
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-
requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
___
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: OpenStack-dev-
requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [infra] Project tailored VM image for Jenkins jobs

[eran]

I have originally brought this on IRC, but though this place might be  
better adequate.
As part of the storlets functional tests Jenkins job, we do a docker  
pull from docker hub.
This operation fails many times, and requires many "recheck"s (we run  
this with every 'git review').
One solution we were thinking of is having a VM image that contains  
the Docker image we need (e.g. saved using docker save), thus  
bypassing the need to do "docker pull". Is that possible? I realise  
that so far probably devstack is the only example making a special  
tailored image.


Many thanks,
Eran


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Patches for python-fuelclient are blocked by broken master.python-fuelclient.pkgs.ubuntu.review_fuel_client

[Roman Prykhodchenko]

Vladimir,

Thanks you for the update on this. Is there any ETA available?

On Mon, Aug 8, 2016 at 12:09 PM Vladimir Kozhukalov <
vkozhuka...@mirantis.com> wrote:

> We are working on this. Will fix soon.
>
> Vladimir Kozhukalov
>
> On Mon, Aug 8, 2016 at 12:52 PM, Roman Prykhodchenko 
> wrote:
>
>> If it’s not possible to fix this job during next few hours, let’s mark in
>> as non-voting until the bug(s) are fixed.
>>
>> > 8 серп. 2016 р. о 11:48 Roman Prykhodchenko 
>> написав(ла):
>> >
>> > Folks,
>> >
>> > Since the end of last week
>> master.python-fuelclient.pkgs.ubuntu.review_fuel_client [1] blocks all
>> patches to python-fuelclient. As logs suggest this issue is caused by the
>> Xenial merge party.
>> >
>> > Please resolve the issue ASAP because some folks are blocked and cannot
>> finish their jobs in time.
>> >
>> >
>> > 1.
>> https://ci.fuel-infra.org/job/master.python-fuelclient.pkgs.ubuntu.review_fuel_client/
>> >
>> > - romcheg
>> >
>> >
>> >
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [magnum][heat] 2 million requests / sec, 100s of nodes

[Tim Bell]

On 08 Aug 2016, at 11:51, Ricardo Rocha 
> wrote:

Hi.

On Mon, Aug 8, 2016 at 1:52 AM, Clint Byrum 
> wrote:
Excerpts from Steve Baker's message of 2016-08-08 10:11:29 +1200:
On 05/08/16 21:48, Ricardo Rocha wrote:
Hi.

Quick update is 1000 nodes and 7 million reqs/sec :) - and the number
of requests should be higher but we had some internal issues. We have
a submission for barcelona to provide a lot more details.

But a couple questions came during the exercise:

1. Do we really need a volume in the VMs? On large clusters this is a
burden, and local storage only should be enough?

2. We observe a significant delay (~10min, which is half the total
time to deploy the cluster) on heat when it seems to be crunching the
kube_minions nested stacks. Once it's done, it still adds new stacks
gradually, so it doesn't look like it precomputed all the info in advance

Anyone tried to scale Heat to stacks this size? We end up with a stack
with:
* 1000 nested stacks (depth 2)
* 22000 resources
* 47008 events

And already changed most of the timeout/retrial values for rpc to get
this working.

This delay is already visible in clusters of 512 nodes, but 40% of the
time in 1000 nodes seems like something we could improve. Any hints on
Heat configuration optimizations for large stacks very welcome.

Yes, we recommend you set the following in /etc/heat/heat.conf [DEFAULT]:
max_resources_per_stack = -1

Enforcing this for large stacks has a very high overhead, we make this
change in the TripleO undercloud too.


Wouldn't this necessitate having a private Heat just for Magnum? Not
having a resource limit per stack would leave your Heat engines
vulnerable to being DoS'd by malicious users, since one can create many
many thousands of resources, and thus python objects, in just a couple
of cleverly crafted templates (which is why I added the setting).

This makes perfect sense in the undercloud of TripleO, which is a
private, single tenant OpenStack. But, for Magnum.. now you're talking
about the Heat that users have access to.

We have it already at -1 for these tests. As you say a malicious user
could DoS, right now this is manageable in our environment. But maybe
move it to a per tenant value, or some special policy? The stacks are
created under a separate domain for magnum (for trustees), we could
also use that for separation.


If there was a quota system within Heat for items like stacks and resources, 
this could be
controlled through that.

Looks like https://blueprints.launchpad.net/heat/+spec/add-quota-api-for-heat 
did not make it into upstream though.

Tim

A separate heat instance sounds like an overkill.

Cheers,
Ricardo


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [docker] [magnum] Magnum account on Docker Hub

[Spyros Trigazis]

Hello team,

I just acquired the openstackmagnum account [1] on docker Hub. It's an
organization
account so all core team members can be owners. Cores, please share with me
your
docker Hub ID or registered e-mail and I'll add you. I already added Adrian
and Egor.

In organization accounts we can have different teams with different
permissions. [2]

Cheers,
Spyros

[1] https://hub.docker.com/u/openstackmagnum/
[2] https://docs.docker.com/docker-hub/orgs/

On 5 August 2016 at 18:12, Steven Dake (stdake)  wrote:

> Tango,
>
> Sorry to hear that, but glad I could help clarify things :)
>
> Regards
> -steve
>
> From: Ton Ngo 
> Reply-To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> Date: Friday, August 5, 2016 at 7:38 AM
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> Subject: Re: [openstack-dev] [docker] [magnum] Magnum account on Docker
> Hub
>
> Thanks Steve, Spyros. I checked with Docker Hub support and the "magnum"
> account is not registered to Steve,
> so we will just use the new account "openstackmagnum".
> Ton,
>
> [image: Inactive hide details for Spyros Trigazis ---08/02/2016 09:27:38
> AM---I just filed a ticket to acquire the username openstackma]Spyros
> Trigazis ---08/02/2016 09:27:38 AM---I just filed a ticket to acquire the
> username openstackmagnum. I included Hongbin's contact informat
>
> From: Spyros Trigazis 
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-dev@lists.openstack.org>
> Date: 08/02/2016 09:27 AM
> Subject: Re: [openstack-dev] [docker] [magnum] Magnum account on Docker
> Hub
> --
>
>
>
> I just filed a ticket to acquire the username openstackmagnum.
>
> I included Hongbin's contact information explaining that he's the
> project's PTL.
>
> Thanks Steve,
> Spyros
>
>
> On 2 August 2016 at 13:29, Steven Dake (stdake) <*std...@cisco.com*
> > wrote:
>
>Ton,
>
>I may or may not have set it up early in Magnum's development.  I just
>don't remember.  My recommendation is to file a support ticket with docker
>and see if they will tell you who it belongs to (as in does it belong to
>one of the founders of Magnum) or if it belongs to some other third party.
>Their support is very fast.  They may not be able to give you the answer if
>its not an openstacker.
>
>Regards
>-steve
>
>
>*From: *Ton Ngo <*t...@us.ibm.com* >
> * Reply-To: *"OpenStack Development Mailing List (not for usage
>questions)" <*openstack-dev@lists.openstack.org*
>>
> * Date: *Monday, August 1, 2016 at 1:06 PM
> * To: *OpenStack Development Mailing List <
>*openstack-dev@lists.openstack.org* 
>>
> * Subject: *[openstack-dev] [docker] [magnum] Magnum account on Docker Hub
>Hi everyone,
>  At the last IRC meeting, the team discussed the need for hosting
>  some container images on Docker Hub
>  to facilitate development. There is currently a Magnum account
>  on Docker Hub, but this is not owned by anyone
>  on the team, so we would like to find who the owner is and
>  whether this account was set up for OpenStack Magnum.
>  Thanks in advance!
>  Ton Ngo,
>
>
>__
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe:
>*openstack-dev-requ...@lists.openstack.org?subject:unsubscribe*
>
> *http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev*
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [scheduler] Use ResourceProviderTags instead of ResourceClass?

[Chris Dent]

On Mon, 8 Aug 2016, Alex Xu wrote:


Chris, thanks for the blog to explain your idea! It helps me understand
your idea better.


Thanks for reading it. As I think I've mentioned a few times I'm not
really trying to sell the idea, just make sure it is clear enough to
be evaluated.


I agree the goal for API interface design in your blog. But one point I
guess you also agree, that is "The interface is easy to understand for API
user". So look at the example of API request flow with gabbi,  it is pretty
clear for me even I didn't spend any time to learn the gabbi. That means:
gabbi is cool and the interface is clear! But the only confuse is "total:
∞". And the related ResourceClass is "ssd", does it mean disk size is
infinite? For a user, he is learning our API, he needs to search the
document, due to he want to know "what is this special usage way means to".
If user can understand our API without any document, so that is prefect.


I think the main source of confusion is that where I find it pretty
easy to think of qualitative characteristics as being an "infinity
of a quantity" that's not an easy concept in general.

In the "ssd" example what it means is not that disk size is infinite
but taht the ssd-ness of the resource provider is infinite. So, for
example a resource provider which provides disk space that is hosted
on ssd has (at least) two resource classes:

   DISK_GB: 
   SSD: 

When some ssd-ness is consumed, all of it (infinity) is still left
over.

For me it is easier: a resource provider has just one way of
describing what it can do: classes of inventory (that provide
gigabytes of disk that are ssd). When we add tags/capabilities
we have another mode of description.

/me yields

--
Chris Dent   ┬─┬ノ( º _ ºノ) http://anticdent.org/
freenode: cdent tw: @anticdent__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [glare][kolla] timing of release of glare and request for technical interview on IRC wherever the glare team wants to have it

[Mikhail Fedosin]

Hello Steven!
Our plans for Glare in Newton are: 1. Implement beta version of
App-Catalog, based on Glare v1; 2. Develop an experimental support (POC)
for Murano, Heat and probably Tacker. It means that all big updates will be
in Ocata, and despite the fact that the code will be ready in this cycle, I
do not think we need an immediate adoption in Kolla.

And for sure, I'll be glad to answer all your questions, 1600 UTC today
works for me. So, let's meet at this time in your channel.

Best,
Mike

On Sat, Aug 6, 2016 at 2:21 PM, Steven Dake (stdake) 
wrote:

> Hey folks,
>
> I guess the split of glare and glance have been in the works for awhile.
> It is challenging for operational deployment managers (ODMs) such as Kolla
> to keep up with the internal goings-on of every big tent project (or
> projects that shave off from big-tent projects).  Just to be clear up
> front, the Kolla community doesn't care that glare split the work out.  The
> Kolla development team adapts very quickly to upstream changes.  What we do
> care about is that we present an accurate deployment for Newton that
> represents the best that OpenStack has to offer and offer a seamless
> operational experience – within Kolla's capacity constraints.
>
>  I need information on when the code base will be ready to consume (from a
> tarball on tarballs.oo).  Is this planned for milestone 3 – or post
> Newton?  I'd recommend post-Newton for the split to be consumable – it
> would ease the difficulty of adoption if the various ODMs (and Operators)
> had more then 3 weeks to work with on what is clearly a project required by
> every deployment based upon the threads I read.
>
> I have some other technical questions related to how the glance registry
> disappears (I believe this point was mentioned in another thread by Jay) as
> well as the upgrade mechanism (if any is needed) that would best be served
> by a high bandwidth conversation on IRC (and those conversations are
> recorded on eavesdrop for others to benefit).
>
> Would one of the technical cats from glare team drop by #opentack-kolla so
> we can get a quick (30 minutes) technical interview on the work to
> understand how this change impacts OpenStack in the short term (prior to
> Newton) and the long  term layout of the two projects so we can make a
> determination a to how to proceed technically?  I don't necessarily need to
> be there – any of our core reviewer team can handle the Q – but would
> like to be there if possible.
>
> If that doesn't work for the glare team, could we get 30 minutes of agenda
> time in what I'm sure is a busy glare team meeting to have the same
> technical discussion?
>
> If that doesn't work for the glare team, we can host the topic in the
> kolla team meeting (UTC1600 on Wednesdays) if a glare core reviewer or the
> glare PTL can stop by.
>
> Please let me know how you wish to proceed.
>
> TIA
> -steve
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [puppet] [tripleo] Host configuration for CPUAffinity and IRQ Pinning

[Saravanan KR]

Hello,

For using DPDK, CPUs on compute host has to be isolated between Host,
DPDK PMD and Guests. In order, to configure the host to use only
specified CPUs, the CPUAffinity [1] configuration in
/etc/systemd/system.conf needs to be used. Along with CPUAffinity, IRQ
repining[2] needs to be done, to pin the interrupt requests to the
CPUs dedicated to host processes.

We are planning to do the changes for configuring CPUAffinity and IRQ
repining via puppet. We couldn't relate this configuration to any
existing module. Could you please help us with the direction to enable
these configurations?

Regards,
Saravanan KR


Note: It is possible to use isolcpus via grub parameter, but it has
implications [3] on load balancing. So it is recommended to use
CPUAffinity to restrict CPUs for host process.

[1] 
https://www.freedesktop.org/software/systemd/man/systemd-system.conf.html#CPUAffinity=
[2] 
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_MRG/1.3/html/Realtime_Tuning_Guide/sect-Realtime_Tuning_Guide-General_System_Tuning-Interrupt_and_Process_Binding.html
[3] https://lists.freedesktop.org/archives/systemd-devel/2016-July/037187.html

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [magnum] 2 million requests / sec, 100s of nodes

[Ricardo Rocha]

On Mon, Aug 8, 2016 at 11:51 AM, Ricardo Rocha  wrote:
> Hi.
>
> On Mon, Aug 8, 2016 at 1:52 AM, Clint Byrum  wrote:
>> Excerpts from Steve Baker's message of 2016-08-08 10:11:29 +1200:
>>> On 05/08/16 21:48, Ricardo Rocha wrote:
>>> > Hi.
>>> >
>>> > Quick update is 1000 nodes and 7 million reqs/sec :) - and the number
>>> > of requests should be higher but we had some internal issues. We have
>>> > a submission for barcelona to provide a lot more details.
>>> >
>>> > But a couple questions came during the exercise:
>>> >
>>> > 1. Do we really need a volume in the VMs? On large clusters this is a
>>> > burden, and local storage only should be enough?
>>> >
>>> > 2. We observe a significant delay (~10min, which is half the total
>>> > time to deploy the cluster) on heat when it seems to be crunching the
>>> > kube_minions nested stacks. Once it's done, it still adds new stacks
>>> > gradually, so it doesn't look like it precomputed all the info in advance
>>> >
>>> > Anyone tried to scale Heat to stacks this size? We end up with a stack
>>> > with:
>>> > * 1000 nested stacks (depth 2)
>>> > * 22000 resources
>>> > * 47008 events
>>> >
>>> > And already changed most of the timeout/retrial values for rpc to get
>>> > this working.
>>> >
>>> > This delay is already visible in clusters of 512 nodes, but 40% of the
>>> > time in 1000 nodes seems like something we could improve. Any hints on
>>> > Heat configuration optimizations for large stacks very welcome.
>>> >
>>> Yes, we recommend you set the following in /etc/heat/heat.conf [DEFAULT]:
>>> max_resources_per_stack = -1
>>>
>>> Enforcing this for large stacks has a very high overhead, we make this
>>> change in the TripleO undercloud too.
>>>
>>
>> Wouldn't this necessitate having a private Heat just for Magnum? Not
>> having a resource limit per stack would leave your Heat engines
>> vulnerable to being DoS'd by malicious users, since one can create many
>> many thousands of resources, and thus python objects, in just a couple
>> of cleverly crafted templates (which is why I added the setting).
>>
>> This makes perfect sense in the undercloud of TripleO, which is a
>> private, single tenant OpenStack. But, for Magnum.. now you're talking
>> about the Heat that users have access to.
>
> We have it already at -1 for these tests. As you say a malicious user
> could DoS, right now this is manageable in our environment. But maybe
> move it to a per tenant value, or some special policy? The stacks are
> created under a separate domain for magnum (for trustees), we could
> also use that for separation.

For reference we also changed max_stacks_per_tenant, which is:
# Maximum number of stacks any one tenant may have active at one time. (integer
# value)

For the 1000 node bay test we had to increase it.

>
> A separate heat instance sounds like an overkill.
>
> Cheers,
> Ricardo
>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev