Re: [openstack-dev] [magnum] Difference between certs stored in keystone and certs stored in barbican

[Fox, Kevin M]

https://blueprints.launchpad.net/nova/+spec/instance-users

Please see the above spec. Nova, Keystone and Barbican have been working 
together on it this cycle and are hoping to implement it in Mitaka

The problem of secrets from the secret store is not isolated to just Magnum.

Thanks,
Kevin

From: John Dennis [jden...@redhat.com]
Sent: Tuesday, September 01, 2015 10:03 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [magnum] Difference between certs stored in 
keystone and certs stored in barbican

On 09/01/2015 10:57 AM, Clark, Robert Graham wrote:
>
>> The reason that is compelling is that you can have Barbican generate,
>> sign, and store a keypair without transmitting the private key over the
>> network to the client that originates the signing request. It can be
>> directly stored, and made available only to the clients that need access
>> to it.
>
> This is absolutely _not_ how PKI for TLS is supposed to work, yes Barbican
> can create keypairs etc because sometimes that¹s useful but in the
> public-private PKI model that TLS expects this is completely wrong. Magnum
> nodes should be creating their own private key and CSR and submitting them
> to some CA for signing.
>
> Now this gets messy because you probably don¹t want to push keystone
> credentials onto each node (that they would use to communicate with
> Barbican).
>
> I¹m a bit conflicted writing this next bit because I¹m not particularly
> familiar with the Kubernetes/Magnum architectures and also because I¹m one
> of the core developers for Anchor but here goesŠ.
>
> Have you considered using Anchor for this? It¹s a pretty lightweight
> ephemeral CA that is built to work well in small PKI communities (like a
> Kubernetes cluster) you can configure multiple methods for authentication
> and build pretty simple validation rules for deciding if a host should be
> given a certificate. Anchor is built to provide short-lifetime
> certificates where each node re-requests a certificate typically every
> 12-24 hours, this has some really nice properties like ³passive
> revocation² (Think revocation that actually works) and strong ways to
> enforce issuing logic on a per host basis.
>
> Anchor or not, I¹d like to talk to you more about how you¹re attempting to
> secure Magnum - I think it¹s an extremely interesting project that I¹d
> like to help out with.
>
> -Rob
> (Security Project PTL / Anchor flunkie)

Let's not reinvent the wheel. I can't comment on what Magnum is doing
but I do know the members of the Barbican project are PKI experts and
understand CSR's, key escrow, revocation, etc. Some of the design work
is being done by engineers who currently contribute to products in use
by the Dept. of Defense, an agency that takes their PKI infrastructure
very seriously. They also have been involved with Keystone. I work with
these engineers on a regular basis.

The Barbican blueprint states:

Barbican supports full lifecycle management including provisioning,
expiration, reporting, etc. A plugin system allows for multiple
certificate authority support (including public and private CAs).

Perhaps Anchor would be a great candidate for a Barbican plugin.

What I don't want to see is spinning our wheels, going backward, or
inventing one-off solutions to a very demanding and complex problem
space. There have been way too many one-off solutions in the past, we
want to consolidate the expertise in one project that is designed by
experts and fully vetted, this is the role of Barbican. Would you like
to contribute to Barbican? I'm sure your skills would be a tremendous
asset.


--
John

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][ceilometer] cputime value resets on restart/shutdown

[gord chung]

On 01/09/2015 12:10 PM, Julien Danjou wrote:

On Tue, Sep 01 2015, gord chung wrote:


but if it's not actually cumulative in Ceilometer (pre-storage), should we
really be tagging it as such?

We only have 3 meters type, and the cumulative definition I wrote
somewhere back in 2012 states that it can reset to 0. Sorry. :-)


we shouldn't redefine real words -- no one reads definitions for words 
they already know... also, i'm not sure the 'reset to 0' definition is 
easily accessible (i'm not sure where it exists) :-\





so i was thinking rather than modify the existing meter, we build a new
cpu.delta meter, which is gives the delta. it seems like giving a delta meter
would make the behaviour more consistent.

…with data loss if you restart the polling agent and it then ignores the
previous value of the meter.

Except if you connect the polling system to the previous state of the
meter, which requires to either:
1. Connect the polling system to the storage system
2. Store locally the latest value you fetched

Option 1 sounds crazy, option 2 sounds less crazy, but still hackish.

Whereas having a system that can compute the delta afterward with all
the value at its reach sounds way better – that's why I'm in favor of
doing that in the storage system (e.g. Gnocchi).

i realise now we're just rehashing the same topic from 3 years ago[1]. 
it seemed like the issue then was we couldn't scale out multiple 
notification consumers... which we can now.


i agree that (1) is bad. (2) has potential data loss on agent restart... 
that said, you already have data loss regardless as when you poll 
'cumulative' meter, the cputime between poll cycles is loss during 
restart. if you handle this at storage level, every subsequent sample 
will be be off by an arbitrary amount after each restart where the delta 
solution, you'll only be off a single sample whenever instance/agent 
restarts.


[1]http://lists.openstack.org/pipermail/openstack-dev/2012-November/003297.html

--
gord


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [magnum] Difference between certs stored in keystone and certs stored in barbican

[Adrian Otto]

John and Robert,

On Sep 1, 2015, at 10:03 AM, John Dennis 
> wrote:

On 09/01/2015 10:57 AM, Clark, Robert Graham wrote:

The reason that is compelling is that you can have Barbican generate,
sign, and store a keypair without transmitting the private key over the
network to the client that originates the signing request. It can be
directly stored, and made available only to the clients that need access
to it.

This is absolutely _not_ how PKI for TLS is supposed to work, yes Barbican
can create keypairs etc because sometimes that¹s useful but in the
public-private PKI model that TLS expects this is completely wrong. Magnum
nodes should be creating their own private key and CSR and submitting them
to some CA for signing.

Now this gets messy because you probably don¹t want to push keystone
credentials onto each node (that they would use to communicate with
Barbican).

Exactly. Using Keystone trust tokens was one option we discussed, but placing 
those on the bay nodes is problematic. They currently offer us the rough 
equivalent of a regular keystone token because not all OpenStack services check 
for the scope of the token used to auth against the service, meaning that a 
trust token is effectively a bearer token for interacting with most of 
OpenStack. We woodenly to land patches in *every* OpenStack project in order to 
work around this gap. This is simply not practical in the time we have before 
the Liberty release, and is much more work than our contributors signed up for. 
Both our Magnum team and our Barbican team met about this together at our 
recent Midcycle meetup. We did talk about how to put additional trust support 
capabilities into Barbican to allow delegation and restricted use of Barbican 
by individual service accounts.

The bottom line is that we need a functional TLS implementation in Magnum for 
Kubernetes and Docker Swarm to use now, and we can’t in good conscious claim 
that Magnum is suitable for production workloads until we address this. If we 
have to take some shortcuts to get this done, then that’s fine, as long as we 
commit to revisiting our design compromises and correcting them.

There is another ML thread that references a new Nova spec for “Instance Users” 
which is still in concept stage:

https://review.openstack.org/186617

We need something now, even if it’s not perfect. The thing we must solve for 
first is that we don’t have Kubernetes and Docker API’s that are open on public 
networks that are unprotected (no authentication), and allow anyone to just 
start stuff up on your container clusters. We are going to crawl before we 
walk/run here. We plan to use a TLS certificate to work as an authentication 
mechanism so that if you don’t have the correct certificate, you can’t 
communicate with the TLS enabled API port.

This is what we are doing as a first step:

https://review.openstack.org/#/q/status:open+project:openstack/magnum+branch:master+topic:bp/magnum-as-a-ca,n,z

I¹m a bit conflicted writing this next bit because I¹m not particularly
familiar with the Kubernetes/Magnum architectures and also because I¹m one
of the core developers for Anchor but here goesŠ.

Have you considered using Anchor for this? It¹s a pretty lightweight
ephemeral CA that is built to work well in small PKI communities (like a
Kubernetes cluster) you can configure multiple methods for authentication
and build pretty simple validation rules for deciding if a host should be
given a certificate. Anchor is built to provide short-lifetime
certificates where each node re-requests a certificate typically every
12-24 hours, this has some really nice properties like ³passive
revocation² (Think revocation that actually works) and strong ways to
enforce issuing logic on a per host basis.

Anchor or not, I¹d like to talk to you more about how you¹re attempting to
secure Magnum - I think it¹s an extremely interesting project that I¹d
like to help out with.

-Rob
(Security Project PTL / Anchor flunkie)

Let's not reinvent the wheel. I can't comment on what Magnum is doing but I do 
know the members of the Barbican project are PKI experts and understand CSR's, 
key escrow, revocation, etc. Some of the design work is being done by engineers 
who currently contribute to products in use by the Dept. of Defense, an agency 
that takes their PKI infrastructure very seriously. They also have been 
involved with Keystone. I work with these engineers on a regular basis.

The Barbican blueprint states:

Barbican supports full lifecycle management including provisioning, expiration, 
reporting, etc. A plugin system allows for multiple certificate authority 
support (including public and private CAs).

Perhaps Anchor would be a great candidate for a Barbican plugin.

That would be cool. I’m not sure that the use case for Anchor exactly fits into 
Barbican’s concept of a CA, but if there were a clean integration point there, 
I’d love to use it.

What I don't want to 

Re: [openstack-dev] [trove] [heat] Multi region support

[Zane Bitter]

On 01/09/15 11:41, Lowery, Mathew wrote:

This is a Trove question but including Heat as they seem to have solved
this problem.

Summary: Today, it seems that Trove is not capable of creating a cluster
spanning multiple regions. Is that the case and, if so, are there any
plans to work on that? Also, are we aware of any precedent solutions
(e.g. remote stacks in Heat) or even partially completed spec/code in Trove?

More details:

I found this nice diagram

 created
for Heat. As far as I understand it,


Clarifications below...


#1 is the absence of multi-region
support (i.e. what we have today). #2 seems to be a 100% client-based
solution. In other words, the Heat services never know about the other
stacks.


I guess you could say that.


In fact, there is nothing tying these stacks together at all.


I wouldn't go that far. The regional stacks still appear as resources in 
their parent stack, so they're tied together by whatever inputs and 
outputs are connected up in that stack.



#3
seems to show a "master" Heat server that understands "remote stacks"
and simply converts those "remote stacks" into calls on regional Heats.
I assume here the master stack record is stored by the master Heat.
Because the "remote stacks" are full-fledged stacks, they can be managed
by their regional Heats if availability of master or other regional
Heats is lost.


Yeah.


#4, the diagram doesn't seem to match the description
(instead of one global Heat, it seems the diagram should show two
regional Heats).


It does (they're the two orange boxes).


In this one, a single arbitrary region becomes the
owner of the stack and remote (low-level not stack) resources are
created as needed. One problem is the manageability is lost if the Heat
in the owning region is lost. Finally, #5. In #5, it's just #4 but with
one and only one Heat.

It seems like Heat solved this 
using #3 (Master Orchestrator)


No, we implemented #2.


but where there isn't necessarily a
separate master Heat. Remote stacks can be created by any regional stack.


Yeah, that was the difference between #3 and #2 :)

cheers,
Zane.


Trove questions:

 1. Having sub-clusters (aka remote clusters aka nested clusters) seems
to be useful (i.e. manageability isn't lost when a region is lost).
But then again, does it make sense to perform a cluster operation on
a sub-cluster?
 2. You could forego sub-clusters and just create full-fledged remote
standalone Trove instances.
 3. If you don't create full-fledged remote Trove instances (but instead
just call remote Nova), then you cannot do simple things like
getting logs from a node without going through the owning region's
Trove. This is an extra hop and a single point of failure.
 4. Even with sub-clusters, the only record of them being related lives
only in the "owning" region. Then again, some ID tying them all
together could be passed to the remote regions.
 5. Do we want to allow the piecing together of clusters (sort of like
Heat's "adopt")?

These are some questions floating around my head and I'm sure there are
plenty more. Any thoughts on any of this?

Thanks,
Mat


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [trove] Anyone using containers?

[Lowery, Mathew]

Just curious if anyone is using containers in their deployments. If so, in what 
capacity? What are the advantages, gotchas, and pain points?
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo][versionedobjects][ceilometer] explain the benefits of ceilometer+versionedobjects

[gord chung]

On 28/08/2015 5:18 PM, Alec Hothan (ahothan) wrote:





On 8/28/15, 11:39 AM, "gord chung"  wrote:


i should start by saying i re-read my subject line and it arguably comes
off aggressive -- i should probably have dropped 'explain' :)

On 28/08/15 01:47 PM, Alec Hothan (ahothan) wrote:

On 8/28/15, 10:07 AM, "gord chung"  wrote:


On 28/08/15 12:18 PM, Roman Dobosz wrote:

So imagine we have new versions of the schema for the events, alarms or
samples in ceilometer introduced in Mitaka release while you have all
your ceilo services on Liberty release. To upgrade ceilometer you'll
have to stop all services to avoid data corruption. With
versionedobjects you can do this one by one without disrupting
telemetry jobs.

are versions checked for every single message? has anyone considered the
overhead to validating each message? since ceilometer is queue based, we
could technically just publish to a new queue when schema changes... and
the consuming services will listen to the queue it knows of.

ie. our notification service changes schema so it will now publish to a
v2 queue, the existing collector service consumes the v1 queue until
done at which point you can upgrade it and it will listen to v2 queue.

this way there is no need to validate/convert anything and you can still
take services down one at a time. this support doesn't exist currently
(i just randomly thought of it) but assuming there's no flaw in my idea
(which there may be) isn't this more efficient?

If high performance is a concern for ceilometer (and it should) then maybe
there might be better options than JSON?
JSON is great for many applications but can be inappropriate for other
demanding applications.
There are other popular open source encoding options that yield much more
compact wire payload, more efficient encoding/decoding and handle
versioning to a reasonable extent.

i should clarify. we let oslo.messaging serialise our dictionary how it
does... i believe it's JSON. i'd be interested to switch it to something
more efficient. maybe it's time we revive the msgpacks patch[1] or are
there better alternatives? (hoping i didn't just unleash a storm of
'this is better' replies)

I'd be curious to know if there is any benchmark on the oslo serializer for 
msgpack and how it compares to JSON?
More important is to make sure we're optimizing in the right area.
Do we have a good understanding of where ceilometer needs to improve to scale 
or is it still not quite clear cut?


re: serialisation, that probably isn't the biggest concern for 
Ceilometer performance. the main items are storage -- to be addressed by 
Gnocchi/tsdb, and polling load. i just thought i'd point out an existing 
serialisation patch since we were on the topic :-)





Queue based versioning might be less runtime overhead per message but at
the expense of a potentially complex queue version management (which can
become tricky if you have more than 2 versions).
I think Neutron was considering to use versioned queues as well for its
rolling upgrade (along with versioned objects) and I already pointed out
that managing the queues could be tricky.

In general, trying to provide a versioning framework that allows to do
arbitrary changes between versions is quite difficult (and often bound to
fail).


yeah, so that's what a lot of the devs are debating about right now.
performance is our key driver so if we do something we think/know will
negatively impact performance, it better bring a whole lot more of
something else. if queue based versioning offers comparable
functionalities, i'd personally be more interested to explore that route
first. is there a thread/patch/log that we could read to see what
Neutron discovered when they looked into it?

The versioning comments are buried in this mega patch if you are brave enough 
to dig in:

https://review.openstack.org/#/c/190635

The (offline) conclusion was that this was WIP and deserved more discussion 
(need to check back with Miguel and Ihar from the Neutron team).
One option considered in that discussion was to use oslo messaging topics to 
manage flows of messages that had different versions (and still use 
versionedobjects). So if you have 3 versions in your cloud you'd end up with 3 
topics (and as many queues when it comes to Rabbit). What is complex is to 
manage the queues/topic names (how to name them), how to discover them and how 
to deal with all the corner cases (like a new node coming in with an arbitrary 
version, nodes going away at any moment, downgrade cases).


conceptually, i would think only the consumers need to know about all 
the queues and even then, it should only really need to know about the 
ones it understands. the producers (polling agents) can just fire off to 
the correct versioned queue and be done... thanks for the above link 
(it'll help with discussion/spec design).


cheers,

--
gord


__

Re: [openstack-dev] [magnum] Difference between certs stored in keystone and certs stored in barbican

[Douglas Mendizábal]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Added a few comments inline.

- - Douglas Mendizábal

On 9/1/15 12:03 PM, John Dennis wrote:
> On 09/01/2015 10:57 AM, Clark, Robert Graham wrote:
>> 
>>> The reason that is compelling is that you can have Barbican 
>>> generate, sign, and store a keypair without transmitting the 
>>> private key over the network to the client that originates the
>>>  signing request. It can be directly stored, and made available
>>>  only to the clients that need access to it.
>> 
>> This is absolutely _not_ how PKI for TLS is supposed to work,
>> yes Barbican can create keypairs etc because sometimes that¹s
>> useful but in the public-private PKI model that TLS expects this
>> is completely wrong. Magnum nodes should be creating their own 
>> private key and CSR and submitting them to some CA for signing.
>> 

Barbican provides a lot of options for provisioning certificates. We
do support provisioning certs by only passing a CSR so that clients
can keep ownership of their keys, if that's what the client prefers.

Of course, when you're provisioning keys for every node in a cluster
for many clusters then key management becomes an issue, and if these
are not throwaway keys, then storing them in Barbican makes sense.

We can also provision the keys, and create CSRs on the Barbican side,
so we make it very easy for clients who don't want to do any of this
locally.

>> Now this gets messy because you probably don¹t want to push 
>> keystone credentials onto each node (that they would use to 
>> communicate with Barbican).
>> 

Kevin Fox is working on a Nova spec to provide identity to VMs. I'm
really hoping this spec gains some traction because it's a problem
that not only Barbican, but all other user-facing projects can benefit
from.

See: https://blueprints.launchpad.net/nova/+spec/instance-users

>> I¹m a bit conflicted writing this next bit because I¹m not 
>> particularly familiar with the Kubernetes/Magnum architectures 
>> and also because I¹m one of the core developers for Anchor but 
>> here goesŠ.
>> 
>> Have you considered using Anchor for this? It¹s a pretty 
>> lightweight ephemeral CA that is built to work well in small PKI
>>  communities (like a Kubernetes cluster) you can configure 
>> multiple methods for authentication and build pretty simple 
>> validation rules for deciding if a host should be given a 
>> certificate. Anchor is built to provide short-lifetime 
>> certificates where each node re-requests a certificate typically
>>  every 12-24 hours, this has some really nice properties like 
>> ³passive revocation² (Think revocation that actually works) and 
>> strong ways to enforce issuing logic on a per host basis.
>> 

Someone from the Magnum team can correct me if I'm wrong, but I do
believe they considered Anchor for certificate provisioning.

As I understand the Magnum use case, they will be provisioning many
clusters across different tenants. While Anchor would work well for a
single cluster, they need the ability to provision new CA roots for
each and every cluster, and then provision certs off that root for
every node in the cluster. This way node certs are only valid in the
context of the cluster.

A new feature for Barbican Liberty will be the ability to add new CA
roots scoped to a tenant via API, which will address the Magnum
requirements of separating the certificate roots per cluster.

>> Anchor or not, I¹d like to talk to you more about how you¹re 
>> attempting to secure Magnum - I think it¹s an extremely 
>> interesting project that I¹d like to help out with.
>> 
>> -Rob (Security Project PTL / Anchor flunkie)
> 
> Let's not reinvent the wheel. I can't comment on what Magnum is 
> doing but I do know the members of the Barbican project are PKI 
> experts and understand CSR's, key escrow, revocation, etc. Some of
>  the design work is being done by engineers who currently
> contribute to products in use by the Dept. of Defense, an agency
> that takes their PKI infrastructure very seriously. They also have
> been involved with Keystone. I work with these engineers on a
> regular basis.
> 
> The Barbican blueprint states:
> 
> Barbican supports full lifecycle management including
> provisioning, expiration, reporting, etc. A plugin system allows
> for multiple certificate authority support (including public and
> private CAs).
> 
> Perhaps Anchor would be a great candidate for a Barbican plugin.
> 

We've talked about this before with the Security team, and I agree
that adding a CA plugin to Barbican to support Anchor would be awesome.

> What I don't want to see is spinning our wheels, going backward,
> or inventing one-off solutions to a very demanding and complex
> problem space. There have been way too many one-off solutions in
> the past, we want to consolidate the expertise in one project that
> is designed by experts and fully vetted, this is the role of
> Barbican. Would you like to contribute to Barbican? I'm sure your
> skills 

Re: [openstack-dev] [puppet][keystone] Keystone resource naming with domain support - no '::domain' if 'Default'

[Rich Megginson]

To close this thread: 
http://lists.openstack.org/pipermail/openstack-dev/2015-August/072878.html


puppet-openstack will support Keystone domain scoped resource names 
without a '::domain' in the name, only if the 'default_domain_id' 
parameter in Keystone has _not_ been set. That is, if the default domain 
is 'Default'.  This means that if the user/operator doesn't care about 
domains at all, the operator doesn't have to deal with them.  However, 
once the user/operator uses `keystone_domain`, and uses `is_default => 
true`, this means the user/operator _must_ use '::domain' with _all_ 
domain scoped Keystone resource names.


In addition:

* In the OpenStack L release:
   If 'default_domain_id' is set, puppet will issue a warning if a name 
is used without '::domain'. I think this is a good thing to do, just in 
case someone sets the default_domain_id by mistake.


* In OpenStack M release:
   Puppet will issue a warning if a name is used without '::domain'.

* From Openstack N release:
   A name must be used with '::domain'.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Heat] convergence rally test results (so far)

[Clint Byrum]

Excerpts from Anant Patil's message of 2015-08-30 23:01:29 -0700:
> Hi Angus,
> 
> Thanks for doing the tests with convergence. We are now assured that
> convergence has not impacted the performance in a negative way. Given
> that, in convergence, a stack provisioning process goes through a lot of
> RPC calls, it puts a lot of load on the message broker and the request
> looses time in network traversal etc., and in effect would hamper the
> performance. As the results show, having more than 2 engines will always
> yield better results with convergence. Since the deployments usually
> have 2 or more engines, this works in favor of convergence.
> 
> I have always held that convergence is more for scale (how much/many)
> than for performance (response time), due to it's design of distributing
> load (resource provisioning from single stack) among heat engines and
> also due to the fact that heat actually spends a lot of time waiting for
> the delegated resource request to be completed, not doing much
> computation. However, with these tests, we can eliminate any
> apprehension of performance issues which would have inadvertently
> sneaked in, with our focus more on scalability and reliability, than on
> performance.
> 
> I was thinking we should be doing some scale testing where we have many
> bigger stacks provisioned and compare the results with legacy, where we
> measure memory, CPU and network bandwidth.
> 

Convergence would be worth it if it was 2x slower in response time, and
scaled 10% worse. Because while scalability is super important, the main
point is resilience to failure of an engine. Add in engine restarts,
failures, etc, to these tests, and I think the advantages will be quite
a bit more skewed toward convergence.

Really nice work everyone!

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [magnum] Difference between certs stored in keystone and certs stored in barbican

[John Dennis]

On 09/01/2015 10:57 AM, Clark, Robert Graham wrote:



The reason that is compelling is that you can have Barbican generate,
sign, and store a keypair without transmitting the private key over the
network to the client that originates the signing request. It can be
directly stored, and made available only to the clients that need access
to it.


This is absolutely _not_ how PKI for TLS is supposed to work, yes Barbican
can create keypairs etc because sometimes that¹s useful but in the
public-private PKI model that TLS expects this is completely wrong. Magnum
nodes should be creating their own private key and CSR and submitting them
to some CA for signing.

Now this gets messy because you probably don¹t want to push keystone
credentials onto each node (that they would use to communicate with
Barbican).

I¹m a bit conflicted writing this next bit because I¹m not particularly
familiar with the Kubernetes/Magnum architectures and also because I¹m one
of the core developers for Anchor but here goesŠ.

Have you considered using Anchor for this? It¹s a pretty lightweight
ephemeral CA that is built to work well in small PKI communities (like a
Kubernetes cluster) you can configure multiple methods for authentication
and build pretty simple validation rules for deciding if a host should be
given a certificate. Anchor is built to provide short-lifetime
certificates where each node re-requests a certificate typically every
12-24 hours, this has some really nice properties like ³passive
revocation² (Think revocation that actually works) and strong ways to
enforce issuing logic on a per host basis.

Anchor or not, I¹d like to talk to you more about how you¹re attempting to
secure Magnum - I think it¹s an extremely interesting project that I¹d
like to help out with.

-Rob
(Security Project PTL / Anchor flunkie)


Let's not reinvent the wheel. I can't comment on what Magnum is doing 
but I do know the members of the Barbican project are PKI experts and 
understand CSR's, key escrow, revocation, etc. Some of the design work 
is being done by engineers who currently contribute to products in use 
by the Dept. of Defense, an agency that takes their PKI infrastructure 
very seriously. They also have been involved with Keystone. I work with 
these engineers on a regular basis.


The Barbican blueprint states:

Barbican supports full lifecycle management including provisioning, 
expiration, reporting, etc. A plugin system allows for multiple 
certificate authority support (including public and private CAs).


Perhaps Anchor would be a great candidate for a Barbican plugin.

What I don't want to see is spinning our wheels, going backward, or 
inventing one-off solutions to a very demanding and complex problem 
space. There have been way too many one-off solutions in the past, we 
want to consolidate the expertise in one project that is designed by 
experts and fully vetted, this is the role of Barbican. Would you like 
to contribute to Barbican? I'm sure your skills would be a tremendous 
asset.



--
John

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [all] Criteria for applying vulnerability:managed tag

[Jeremy Stanley]

Bringing OpenStack vulnerability management processes to the Big
Top started a couple months ago with creation of a deliverable tag
called vulnerability:managed, the definition of which can be found
at:

http://governance.openstack.org/reference/tags/vulnerability_managed.html

Its initial application is based on a crufty old wiki page which
previously listed what repos the VMT tracks for security
vulnerabilities with no discernible rationale (later extended from
repos to deliverables, as the TC decided against having per-repo
tags). As such, the requirements to qualify for this tag are
currently a rather dissatisfyingly non-transparent "ask the VMT and
we'll finalize it with the TC." It's past time we fix this.

In the spirit of proper transparency, I'm initiating a frank and
open dialogue on what our criteria for direct vulnerability
management within the VMT would require of a deliverable and its
controlling project-team. In the past a number of possible
requirements have been mentioned, so I'll enumerate the ones I can
recall here along with some pros and cons, and others can add to
this as they see fit...

1. Since the vulnerability:managed governance tag applies to
deliverables, all repos within a given deliverable must meet the
qualifying criteria. This means that if some repos in a deliverable
are in good enough shape to qualify, their vulnerability management
could be held back by other repos in the same deliverable. It might
be argued that perhaps this makes them separate deliverables (in
which case the governance projects.yaml should get an update to
reflect that), or maybe that we really have a use case for per-repo
tags and that the TC needs to consider deliverable and repo tags as
separate ideas.

2. The deliverable must have a dedicated point of contact for
security issues (which could be shared by multiple deliverables in a
given project-team if needed), so that the VMT can engage them to
triage reports of potential vulnerabilities. Deliverables with more
than a handful of core reviewers should (so as to limit the
unnecessary exposure of private reports) settle on a subset of these
to act as security core reviewers whose responsibility it is to be
able to confirm whether a bug report is accurate/applicable or at
least know other subject matter experts they can in turn subscribe
to perform those activities in a timely manner. They should also be
able to review and provide pre-approval of patches attached to
private bugs, which is why they are expected to be core reviewers
for the deliverable. These should be members of a group contact (for
example a -coresec team in launchpad) so that the VMT can
easily subscribe them to new bugs.

3. The PTL for the deliverable should agree to act as (or delegate)
a vulnerability management liaison, serving as a point of escalation
for the VMT in situations where severe or lingering vulnerability
reports are failing to gain traction toward timely and thorough
resolution.

4. The bug tracker for the repos within the deliverable should be
configured to initially only provide access for the VMT to
privately-reported vulnerabilities. It is the responsibility of the
VMT to determine whether suspected vulnerabilities are reported
against the correct deliverable and redirect them when possible,
since reporters are often unfamiliar with our project structure and
may choose incorrectly. For Launchpad, this means making sure that
https://launchpad.net//+sharing shows only the
OpenStack Vulnerability Management Team (openstack-vuln-mgmt) with
sharing for Private Security: All. It implies some loss of control
for the project team over initial triage of bugs reported privately
as suspected vulnerabilities, but in some cases helps reduce the
number of people who have knowledge of them prior to public
disclosure.

5. The deliverable's repos should undergo a third-party review/audit
looking for obvious signs of insecure design or risky implementation
which could imply a large number of future vulnerability reports. As
much as anything this is a measure to keep the VMT's workload down,
since it is by necessity a group of constrained size and some of its
processes simply can't be scaled safely. It's not been identified
who would actually perform this review, though this is one place
members of the OpenStack Security project-team might volunteer to
provide their expertise and assistance.

I still have a few open questions as well...

A. Can the VMT accept deliverables in any programming language?
OpenStack has a lot of expertise in Python, so it's possible for us
to find people able to confidently review Python source code for
unsafe practices. We have tools and workflows which make it easier
to set up development and testing environments to confirm reported
vulnerabilities and test fixes. For Python-based deliverables we
have lots of solutions in place to improve code quality, and could
for example even require that proposed changes get gated by bandit.
On the other hand, 

Re: [openstack-dev] [magnum] Difference between certs stored in keystone and certs stored in barbican

[Tim Bell]

> -Original Message-
> From: Adrian Otto [mailto:adrian.o...@rackspace.com]
> Sent: 01 September 2015 07:03
> To: OpenStack Development Mailing List (not for usage questions)
> 
> Subject: Re: [openstack-dev] [magnum] Difference between certs stored in
> keystone and certs stored in barbican
> 
> Simply put, Keystone is designed to generate  tokens that are to be used for
> authentication and RBAC. Systems like Kunernetes do not support Keystone
> auth, but do support TLS. Using TLS provides a solution that is compatible
> with using these systems outside of an OpenStack cloud.
> 
> Barbican is designed for secure storage of arbitrary secrets, and currently
> also has a CA function. The reason that is compelling is that you can have
> Barbican generate, sign, and store a keypair without transmitting the private
> key over the network to the client that originates the signing request. It can
> be directly stored, and made available only to the clients that need access to
> it.
> 

Will it also be possible to use a different CA ? In some environments, there is 
already a corporate certificate authority server. This would ensure compliance 
with site security standards.

Tim

> We are taking an iterative approach to TLS integration, so we can gradually
> take advantage of both keystone and Barbican features as they allow us to
> iterate toward a more secure integration.
> 
> Adrian
> 
> > On Aug 31, 2015, at 9:05 PM, Vikas Choudhary
>  wrote:
> >
> > Hi,
> >
> > Can anybody please point me out some etherpad discussion page/spec
> that can help me understand why we are going to introduce barbican  for
> magnum when we already had keystone for security management?
> >
> >
> >
> >
> > -Vikas Choudhary
> >
> >
> >
> 
> __
> >  OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: OpenStack-dev-
> requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [magnum] Difference between certs stored in keystone and certs stored in barbican

[Vikas Choudhary]

Is it like keystone authenticating between magnum-client and magnum conductor,

and barbican certs will be used b/w conductor and k8s/swarm?



Thanks

Vikas Choudhary

_
Simply put, Keystone is designed to generate  tokens that are to be
used for authentication and RBAC. Systems like Kunernetes do not
support Keystone auth, but do support TLS. Using TLS provides a
solution that is compatible with using these systems outside of an
OpenStack cloud.

Barbican is designed for secure storage of arbitrary secrets, and
currently also has a CA function. The reason that is compelling is
that you can have Barbican generate, sign, and store a keypair without
transmitting the private key over the network to the client that
originates the signing request. It can be directly stored, and made
available only to the clients that need access to it.

We are taking an iterative approach to TLS integration, so we can
gradually take advantage of both keystone and Barbican features as
they allow us to iterate toward a more secure integration.

Adrian

>* On Aug 31, 2015, at 9:05 PM, Vikas Choudhary  
>>> wrote:
*>>* Hi,
*>>* Can anybody please point me out some etherpad discussion
page/spec  that can help me understand why we are going to introduce
barbican  for magnum when we already had keystone for security
management?
*>* -Vikas Choudhary
*>>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [puppet] [security] applying for vulnerability:managed tag

[Thierry Carrez]

Emilien Macchi wrote:
> I would like the feedback from the community about applying (or not) to
> the vulnerability:managed tag [1].
> Being part of OpenStack ecosystem and the big tent, Puppet OpenStack
> project might want to follow some other projects in order to be
> consistent in Security management procedures.

Hi Emilien,

This tag is not managed by the TC, it's directly managed by the
Vulnerability Management Team (under the Security project team). It's
their decision to evaluate if your code is mature enough and if your
team is organized enough that they feel confident they can tackle the
additional workload.

You can find their contact details at:
https://security.openstack.org/

-- 
Thierry Carrez (ttx)



signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [magnum] Difference between certs stored in keystone and certs stored in barbican

[Vikas Choudhary]

Thanks Adrian for description.

If i could understand correctly,magnum clients will be use barbican
for authentication as coes dont support keystone for operations like
pod-creation

etc and for rest of the project clients like nova/glance
pythonclients, keystone will continue to serve.

Please correct me.



-Vikas


_
*Simply put, Keystone is designed to generate  tokens that are to be
used for authentication and RBAC. Systems like Kunernetes do not
support Keystone auth, but do support TLS. Using TLS provides a
solution that is compatible with using these systems outside of an
OpenStack cloud.

Barbican is designed for secure storage of arbitrary secrets, and
currently also has a CA function. The reason that is compelling is
that you can have Barbican generate, sign, and store a keypair without
transmitting the private key over the network to the client that
originates the signing request. It can be directly stored, and made
available only to the clients that need access to it.

We are taking an iterative approach to TLS integration, so we can
gradually take advantage of both keystone and Barbican features as
they allow us to iterate toward a more secure integration.

Adrian

> On Aug 31, 2015, at 9:05 PM, Vikas Choudhary  > wrote:
>
> Hi,
>
> Can anybody please point me out some etherpad discussion page/spec  that can 
> help me understand why we are going to introduce barbican  for magnum when we 
> already had keystone for security management?
>
>
>
>
> -Vikas Choudhary
>
>
> __*
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [murano] Proposing Nikolai Starodubtsev for core

[Dmitro Dovbii]

+1

2015-09-01 2:24 GMT+03:00 Serg Melikyan :

> +1
>
> On Mon, Aug 31, 2015 at 3:45 PM, Kirill Zaitsev 
> wrote:
>
>> I’m pleased to nominate Nikolai for Murano core.
>>
>> He’s been actively participating in development of murano during liberty
>> and is among top5 contributors during last 90 days. He’s also leading the
>> CloudFoundry integration initiative.
>>
>> Here are some useful links:
>>
>> Overall contribution: http://stackalytics.com/?user_id=starodubcevna
>> List of reviews:
>> https://review.openstack.org/#/q/reviewer:%22Nikolay+Starodubtsev%22,n,z
>> Murano contribution during latest 90 days
>> http://stackalytics.com/report/contribution/murano/90
>>
>> Please vote with +1/-1 for approval/objections
>>
>> --
>> Kirill Zaitsev
>> Murano team
>> Software Engineer
>> Mirantis, Inc
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
>
> --
> Serg Melikyan, Senior Software Engineer at Mirantis, Inc.
> http://mirantis.com | smelik...@mirantis.com
>
> +7 (495) 640-4904, 0261
> +7 (903) 156-0836
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [murano] Let's minimaze the list of pylint exceptions

[Dmitro Dovbii]

Hi folks!

We have a long list of pylint exceptions in code of Murano (please see
example
).
I would like to propose you to take a part in refactoring of code and
minimization of this list.
I've created blueprint

and etherpad
document  describing
the structure of participation. Please feel free to choose some type of
warning and several modules containing it, then make notice in document,
and finally fix issues.
Let's make murano code more clear together :)

Best regards,
Dmytro Dovbii
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [murano] Let's minimaze the list of pylint exceptions

[Nikolay Starodubtsev]

+1, good initiative



Nikolay Starodubtsev

Software Engineer

Mirantis Inc.


Skype: dark_harlequine1

2015-09-01 10:30 GMT+03:00 Dmitro Dovbii :

> Hi folks!
>
> We have a long list of pylint exceptions in code of Murano (please see
> example
> ).
> I would like to propose you to take a part in refactoring of code and
> minimization of this list.
> I've created blueprint
> 
> and etherpad document 
> describing the structure of participation. Please feel free to choose some
> type of warning and several modules containing it, then make notice in
> document, and finally fix issues.
> Let's make murano code more clear together :)
>
> Best regards,
> Dmytro Dovbii
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Netaddr 0.7.16 and gate breakage

[Sachi King]

On Tue, Sep 1, 2015 at 3:15 AM, Carl Baldwin  wrote:
> On Mon, Aug 31, 2015 at 11:02 AM, Armando M.  wrote:
>> On 31 August 2015 at 09:53, Jeremy Stanley  wrote:
>>> On 2015-08-31 10:33:07 -0600 (-0600), Carl Baldwin wrote:
>>> > I was under the impression that we had a plan to stop allowing these
>>> > external updates break our gate jobs.
>>> [...]
>>>
>>> We do, and it succeeded in protecting master branch integration test
>>> jobs from this new netaddr release:
>>>
>>> https://review.openstack.org/218737
>>>
>>> This was able to get implemented fairly early because DevStack
>>> already contained mechanisms for relying on the requirements repo to
>>> define its behavior WRT dependencies. The logistics for pinning
>>> versions in every project's unit tests, however, are more complex
>>> and not yet in place (but are in progress). Also where grenade is
>>> concerned, the breakage is on the stable/kilo side where we don't
>>> have constraints pinning implemented (since that work has been
>>> primarily in master this cycle and targeting liberty).
>
> Thanks for the update Jeremy.  It looks like there is still some work
> to do that I wasn't aware of.  Is there somewhere where we can follow
> the progress of this work?

Hi Carl, currently we're working on getting the required work into project
config, which means the change to tox.ini is currently going through some
revisions.

Project config: https://review.openstack.org/207262/
Nova tox.ini example: https://review.openstack.org/205931

I've created a change for Neutron's tox.ini.  This doesn't change anything,
so it should pass CI, but if there's a merge conflict with a external CI's
tox.ini this will flush it out.

https://review.openstack.org/219134/

>> Great...if there is any collaboration required by the individual projects,
>> please do not hesitate to reach out...we'd be happy to do our part.
>
> +1  Let us know how to pitch in.

For now there's not much to do, the project-config change is the change
that has to go in first.  Once that is in we can get the 'constraints'
factor into individual projects tox.ini and add the constraints jobs.

If neutron would like to be one of the initial projects to enable these
tests that would be quite helpful as I've yet to recruit any initial
projects.

Cheers,
Sachi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] subnetallocation is in core resource, while there is a extension for it?

[Tidwell, Ryan]

This was a compromise we made toward the end of Kilo.  The subnetpools resource 
was implemented as a core resource, but for purposes of Horizon interaction and 
a lack of another method for evolving the Neutron API we deliberately added a 
shim extension.  I believe this was done with a couple other “extensions” like 
VLAN transparent networks.  I don’t think we want to remove the shim extension.

-Ryan

From: gong_ys2004 [mailto:gong_ys2...@aliyun.com]
Sent: Monday, August 31, 2015 9:45 PM
To: openstack-dev
Subject: [openstack-dev] [neutron] subnetallocation is in core resource, while 
there is a extension for it?




Hi, neutron guys,

look at 
https://github.com/openstack/neutron/blob/master/neutron/extensions/subnetallocation.py,

which defines an extension Subnetallocation but defines no extension resource. 
Actually, it is implemented

in core resource.

So I think we should remove this extension.



I filed a bug for it:

https://bugs.launchpad.net/neutron/+bug/1490815





Regards,

yong sheng gong
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Heat] convergence rally test results (so far)

[Fox, Kevin M]

You can default it to the number of cores, but please make it configurable. 
Some ops cram lots of services onto one node, and one service doesn't get to 
monopolize all cores.

Thanks,
Kevin

From: Angus Salkeld [asalk...@mirantis.com]
Sent: Tuesday, September 01, 2015 4:53 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Heat] convergence rally test results (so far)

On Tue, Sep 1, 2015 at 10:45 PM Steven Hardy 
> wrote:
On Fri, Aug 28, 2015 at 01:35:52AM +, Angus Salkeld wrote:
>Hi
>I have been running some rally tests against convergence and our existing
>implementation to compare.
>So far I have done the following:
> 1. defined a template with a resource
>groupA 
> https://github.com/asalkeld/convergence-rally/blob/master/templates/resource_group_test_resource.yaml.template
> 2. the inner resource looks like
>this:A 
> https://github.com/asalkeld/convergence-rally/blob/master/templates/server_with_volume.yaml.templateA
>  (it
>uses TestResource to attempt to be a reasonable simulation of a
>server+volume+floatingip)
> 3. defined a rally
>job:A 
> https://github.com/asalkeld/convergence-rally/blob/master/increasing_resources.yamlA
>  that
>creates X resources then updates to X*2 then deletes.
> 4. I then ran the above with/without convergence and with 2,4,8
>heat-engines
>Here are the results compared:
>
> https://docs.google.com/spreadsheets/d/12kRtPsmZBl_y78aw684PTBg3op1ftUYsAEqXBtT800A/edit?usp=sharing
>Some notes on the results so far:
>  * A convergence with only 2 engines does suffer from RPC overload (it
>gets message timeouts on larger templates). I wonder if this is the
>problem in our convergence gate...
>  * convergence does very well with a reasonable number of engines
>running.
>  * delete is slightly slower on convergence
>Still to test:
>  * the above, but measure memory usage
>  * many small templates (run concurrently)

So, I tried running my many-small-templates here with convergence enabled:

https://bugs.launchpad.net/heat/+bug/1489548

In heat.conf I set:

max_resources_per_stack = -1
convergence_engine = true

Most other settings (particularly RPC and DB settings) are defaults.

Without convergence (but with max_resources_per_stack disabled) I see the
time to create a ResourceGroup of 400 nested stacks (each containing one
RandomString resource) is about 2.5 minutes (core i7 laptop w/SSD, 4 heat
workers e.g the default for a 4 core machine).

With convergence enabled, I see these errors from sqlalchemy:

File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 652, in
_checkout\nfairy = _ConnectionRecord.checkout(pool)\n', u'  File
"/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 444, in
checkout\nrec = pool._do_get()\n', u'  File
"/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 980, in
_do_get\n(self.size(), self.overflow(), self._timeout))\n',
u'TimeoutError: QueuePool limit of size 5 overflow 10 reached, connection
timed out, timeout 30\n'].

I assume this means we're loading the DB much more in the convergence case
and overflowing the QueuePool?

Yeah, looks like it.


This seems to happen when the RPC call from the ResourceGroup tries to
create some of the 400 nested stacks.

Interestingly after this error, the parent stack moves to CREATE_FAILED,
but the engine remains (very) busy, to the point of being partially
responsive, so it looks like maybe the cancel-on-fail isnt' working (I'm
assuming it isn't error_wait_time because the parent stack has been marked
FAILED and I'm pretty sure it's been more than 240s).

I'll dig a bit deeper when I get time, but for now you might like to try
the stress test too.  It's a bit of a synthetic test, but it turns out to
be a reasonable proxy for some performance issues we observed when creating
large-ish TripleO deployments (which also create a large number of nested
stacks concurrently).

Thanks a lot for testing Steve! I'll make 2 bugs for what you have raised
1. limit the number of resource actions in parallel (maybe base on the number 
of cores)
2. the cancel on fail error

-Angus


Steve

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api][keystone][openstackclient] Standards for object name attributes and filtering

[Dolph Mathews]

Does anyone have an example of an API outside of OpenStack that would
return 400 in this situation (arbitrary query string parameters)? Based on
my past experience, I'd expect them to be ignored, but I can't think of a
reason why a 400 would be a bad idea (but I suspect there's some prior art
/ discussion out there).

On Mon, Aug 31, 2015 at 10:53 AM, Ryan Brown  wrote:

> On 08/27/2015 11:28 PM, Chen, Wei D wrote:
> >
> > I agree that return 400 is good idea, thus client user would know what
> happened.
> >
>
> +1, I think a 400 is the sensible choice here. It'd be much more likely
> to help devs catch their errors .
>
> --
> Ryan Brown / Software Engineer, Openstack / Red Hat, Inc.
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [magnum] Difference between certs stored in keystone and certs stored in barbican

[Clark, Robert Graham]

I’ve requested several Security Project slots on the summit timetable, I’d
be happy to dedicate a fishbowl session to this on the security track.

-Rob

On 01/09/2015 14:31, "Fox, Kevin M"  wrote:

>Awesome. Thanks. :)
>
>Are there any plans for the summit yet? I think we should all get
>together and talk about it.
>
>Thanks,
>Kevin
>
>From: Clark, Robert Graham [robert.cl...@hp.com]
>Sent: Tuesday, September 01, 2015 1:35 PM
>To: OpenStack Development Mailing List (not for usage questions)
>Subject: Re: [openstack-dev] [magnum] Difference between certs stored in
>keystone and certs stored in barbican
>
>Extremely interesting.
>
>This is something that we are looking at during the Security mid-cycle
>(happening this week) see "Secure communications between control plane and
>tenant plane” under
>https://etherpad.openstack.org/p/security-liberty-midcycle
>
>This is problem for a lot of different projects, we’ve added your
>blueprint and hopefully we’ll be able to help with this moving forward.
>
>-Rob
>
>
>
>On 01/09/2015 11:11, "Fox, Kevin M"  wrote:
>
>>https://blueprints.launchpad.net/nova/+spec/instance-users
>>
>>Please see the above spec. Nova, Keystone and Barbican have been working
>>together on it this cycle and are hoping to implement it in Mitaka
>>
>>The problem of secrets from the secret store is not isolated to just
>>Magnum.
>>
>>Thanks,
>>Kevin
>>
>>From: John Dennis [jden...@redhat.com]
>>Sent: Tuesday, September 01, 2015 10:03 AM
>>To: OpenStack Development Mailing List (not for usage questions)
>>Subject: Re: [openstack-dev] [magnum] Difference between certs stored in
>>keystone and certs stored in barbican
>>
>>On 09/01/2015 10:57 AM, Clark, Robert Graham wrote:
>>>
 The reason that is compelling is that you can have Barbican generate,
 sign, and store a keypair without transmitting the private key over
the
 network to the client that originates the signing request. It can be
 directly stored, and made available only to the clients that need
access
 to it.
>>>
>>> This is absolutely _not_ how PKI for TLS is supposed to work, yes
>>>Barbican
>>> can create keypairs etc because sometimes that¹s useful but in the
>>> public-private PKI model that TLS expects this is completely wrong.
>>>Magnum
>>> nodes should be creating their own private key and CSR and submitting
>>>them
>>> to some CA for signing.
>>>
>>> Now this gets messy because you probably don¹t want to push keystone
>>> credentials onto each node (that they would use to communicate with
>>> Barbican).
>>>
>>> I¹m a bit conflicted writing this next bit because I¹m not particularly
>>> familiar with the Kubernetes/Magnum architectures and also because I¹m
>>>one
>>> of the core developers for Anchor but here goesŠ.
>>>
>>> Have you considered using Anchor for this? It¹s a pretty lightweight
>>> ephemeral CA that is built to work well in small PKI communities (like
>>>a
>>> Kubernetes cluster) you can configure multiple methods for
>>>authentication
>>> and build pretty simple validation rules for deciding if a host should
>>>be
>>> given a certificate. Anchor is built to provide short-lifetime
>>> certificates where each node re-requests a certificate typically every
>>> 12-24 hours, this has some really nice properties like ³passive
>>> revocation² (Think revocation that actually works) and strong ways to
>>> enforce issuing logic on a per host basis.
>>>
>>> Anchor or not, I¹d like to talk to you more about how you¹re attempting
>>>to
>>> secure Magnum - I think it¹s an extremely interesting project that I¹d
>>> like to help out with.
>>>
>>> -Rob
>>> (Security Project PTL / Anchor flunkie)
>>
>>Let's not reinvent the wheel. I can't comment on what Magnum is doing
>>but I do know the members of the Barbican project are PKI experts and
>>understand CSR's, key escrow, revocation, etc. Some of the design work
>>is being done by engineers who currently contribute to products in use
>>by the Dept. of Defense, an agency that takes their PKI infrastructure
>>very seriously. They also have been involved with Keystone. I work with
>>these engineers on a regular basis.
>>
>>The Barbican blueprint states:
>>
>>Barbican supports full lifecycle management including provisioning,
>>expiration, reporting, etc. A plugin system allows for multiple
>>certificate authority support (including public and private CAs).
>>
>>Perhaps Anchor would be a great candidate for a Barbican plugin.
>>
>>What I don't want to see is spinning our wheels, going backward, or
>>inventing one-off solutions to a very demanding and complex problem
>>space. There have been way too many one-off solutions in the past, we
>>want to consolidate the expertise in one project that is designed by
>>experts and fully vetted, this is the role of Barbican. Would you like
>>to contribute to Barbican? I'm sure your skills would be a 

Re: [openstack-dev] [magnum] Difference between certs stored in keystone and certs stored in barbican

[Fox, Kevin M]

Nice. Thank you.
Kevin

From: Clark, Robert Graham [robert.cl...@hp.com]
Sent: Tuesday, September 01, 2015 3:16 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [magnum] Difference between certs stored in 
keystone and certs stored in barbican

I’ve requested several Security Project slots on the summit timetable, I’d
be happy to dedicate a fishbowl session to this on the security track.

-Rob

On 01/09/2015 14:31, "Fox, Kevin M"  wrote:

>Awesome. Thanks. :)
>
>Are there any plans for the summit yet? I think we should all get
>together and talk about it.
>
>Thanks,
>Kevin
>
>From: Clark, Robert Graham [robert.cl...@hp.com]
>Sent: Tuesday, September 01, 2015 1:35 PM
>To: OpenStack Development Mailing List (not for usage questions)
>Subject: Re: [openstack-dev] [magnum] Difference between certs stored in
>keystone and certs stored in barbican
>
>Extremely interesting.
>
>This is something that we are looking at during the Security mid-cycle
>(happening this week) see "Secure communications between control plane and
>tenant plane” under
>https://etherpad.openstack.org/p/security-liberty-midcycle
>
>This is problem for a lot of different projects, we’ve added your
>blueprint and hopefully we’ll be able to help with this moving forward.
>
>-Rob
>
>
>
>On 01/09/2015 11:11, "Fox, Kevin M"  wrote:
>
>>https://blueprints.launchpad.net/nova/+spec/instance-users
>>
>>Please see the above spec. Nova, Keystone and Barbican have been working
>>together on it this cycle and are hoping to implement it in Mitaka
>>
>>The problem of secrets from the secret store is not isolated to just
>>Magnum.
>>
>>Thanks,
>>Kevin
>>
>>From: John Dennis [jden...@redhat.com]
>>Sent: Tuesday, September 01, 2015 10:03 AM
>>To: OpenStack Development Mailing List (not for usage questions)
>>Subject: Re: [openstack-dev] [magnum] Difference between certs stored in
>>keystone and certs stored in barbican
>>
>>On 09/01/2015 10:57 AM, Clark, Robert Graham wrote:
>>>
 The reason that is compelling is that you can have Barbican generate,
 sign, and store a keypair without transmitting the private key over
the
 network to the client that originates the signing request. It can be
 directly stored, and made available only to the clients that need
access
 to it.
>>>
>>> This is absolutely _not_ how PKI for TLS is supposed to work, yes
>>>Barbican
>>> can create keypairs etc because sometimes that¹s useful but in the
>>> public-private PKI model that TLS expects this is completely wrong.
>>>Magnum
>>> nodes should be creating their own private key and CSR and submitting
>>>them
>>> to some CA for signing.
>>>
>>> Now this gets messy because you probably don¹t want to push keystone
>>> credentials onto each node (that they would use to communicate with
>>> Barbican).
>>>
>>> I¹m a bit conflicted writing this next bit because I¹m not particularly
>>> familiar with the Kubernetes/Magnum architectures and also because I¹m
>>>one
>>> of the core developers for Anchor but here goesŠ.
>>>
>>> Have you considered using Anchor for this? It¹s a pretty lightweight
>>> ephemeral CA that is built to work well in small PKI communities (like
>>>a
>>> Kubernetes cluster) you can configure multiple methods for
>>>authentication
>>> and build pretty simple validation rules for deciding if a host should
>>>be
>>> given a certificate. Anchor is built to provide short-lifetime
>>> certificates where each node re-requests a certificate typically every
>>> 12-24 hours, this has some really nice properties like ³passive
>>> revocation² (Think revocation that actually works) and strong ways to
>>> enforce issuing logic on a per host basis.
>>>
>>> Anchor or not, I¹d like to talk to you more about how you¹re attempting
>>>to
>>> secure Magnum - I think it¹s an extremely interesting project that I¹d
>>> like to help out with.
>>>
>>> -Rob
>>> (Security Project PTL / Anchor flunkie)
>>
>>Let's not reinvent the wheel. I can't comment on what Magnum is doing
>>but I do know the members of the Barbican project are PKI experts and
>>understand CSR's, key escrow, revocation, etc. Some of the design work
>>is being done by engineers who currently contribute to products in use
>>by the Dept. of Defense, an agency that takes their PKI infrastructure
>>very seriously. They also have been involved with Keystone. I work with
>>these engineers on a regular basis.
>>
>>The Barbican blueprint states:
>>
>>Barbican supports full lifecycle management including provisioning,
>>expiration, reporting, etc. A plugin system allows for multiple
>>certificate authority support (including public and private CAs).
>>
>>Perhaps Anchor would be a great candidate for a Barbican plugin.
>>
>>What I don't want to see is spinning our wheels, going backward, or
>>inventing one-off 

Re: [openstack-dev] [trove] [heat] Multi region support

[Angus Salkeld]

On Wed, Sep 2, 2015 at 8:30 AM Lowery, Mathew  wrote:

> Thank you Zane for the clarifications!
>
> I misunderstood #2 and that led to the other misunderstandings.
>
> Further questions:
> * Are nested stacks aware of their nested-ness? In other words, given any
> nested stack (colocated with parent stack or not), can I trace it back to
> the parent stack? (On a possibly related note, I see that adopting a stack
>

Yes, there is a link (url) to the parent_stack in the links section of show
stack.


> is an option to reassemble a new parent stack from its regional parts in
> the event that the old parent stack is lost.)
> * Has this design met the users' needs? In other words, are there any
> plans to make major modifications to this design?
>

AFAIK we have had zero feedback from the multi region feature.
No more plans, but we would obviously love feedback and suggestions
on how to improve region support.

-Angus


>
> Thanks!
>
> On 9/1/15, 1:47 PM, "Zane Bitter"  wrote:
>
> >On 01/09/15 11:41, Lowery, Mathew wrote:
> >> This is a Trove question but including Heat as they seem to have solved
> >> this problem.
> >>
> >> Summary: Today, it seems that Trove is not capable of creating a cluster
> >> spanning multiple regions. Is that the case and, if so, are there any
> >> plans to work on that? Also, are we aware of any precedent solutions
> >> (e.g. remote stacks in Heat) or even partially completed spec/code in
> >>Trove?
> >>
> >> More details:
> >>
> >> I found this nice diagram
> >>
> >><
> https://wiki.openstack.org/wiki/Heat/Blueprints/Multi_Region_Support_for
> >>_Heat/The_Missing_Diagram> created
> >> for Heat. As far as I understand it,
> >
> >Clarifications below...
> >
> >> #1 is the absence of multi-region
> >> support (i.e. what we have today). #2 seems to be a 100% client-based
> >> solution. In other words, the Heat services never know about the other
> >> stacks.
> >
> >I guess you could say that.
> >
> >> In fact, there is nothing tying these stacks together at all.
> >
> >I wouldn't go that far. The regional stacks still appear as resources in
> >their parent stack, so they're tied together by whatever inputs and
> >outputs are connected up in that stack.
> >
> >> #3
> >> seems to show a "master" Heat server that understands "remote stacks"
> >> and simply converts those "remote stacks" into calls on regional Heats.
> >> I assume here the master stack record is stored by the master Heat.
> >> Because the "remote stacks" are full-fledged stacks, they can be managed
> >> by their regional Heats if availability of master or other regional
> >> Heats is lost.
> >
> >Yeah.
> >
> >> #4, the diagram doesn't seem to match the description
> >> (instead of one global Heat, it seems the diagram should show two
> >> regional Heats).
> >
> >It does (they're the two orange boxes).
> >
> >> In this one, a single arbitrary region becomes the
> >> owner of the stack and remote (low-level not stack) resources are
> >> created as needed. One problem is the manageability is lost if the Heat
> >> in the owning region is lost. Finally, #5. In #5, it's just #4 but with
> >> one and only one Heat.
> >>
> >> It seems like Heat solved this  >
> >> using #3 (Master Orchestrator)
> >
> >No, we implemented #2.
> >
> >> but where there isn't necessarily a
> >> separate master Heat. Remote stacks can be created by any regional
> >>stack.
> >
> >Yeah, that was the difference between #3 and #2 :)
> >
> >cheers,
> >Zane.
> >
> >> Trove questions:
> >>
> >>  1. Having sub-clusters (aka remote clusters aka nested clusters) seems
> >> to be useful (i.e. manageability isn't lost when a region is lost).
> >> But then again, does it make sense to perform a cluster operation on
> >> a sub-cluster?
> >>  2. You could forego sub-clusters and just create full-fledged remote
> >> standalone Trove instances.
> >>  3. If you don't create full-fledged remote Trove instances (but instead
> >> just call remote Nova), then you cannot do simple things like
> >> getting logs from a node without going through the owning region's
> >> Trove. This is an extra hop and a single point of failure.
> >>  4. Even with sub-clusters, the only record of them being related lives
> >> only in the "owning" region. Then again, some ID tying them all
> >> together could be passed to the remote regions.
> >>  5. Do we want to allow the piecing together of clusters (sort of like
> >> Heat's "adopt")?
> >>
> >> These are some questions floating around my head and I'm sure there are
> >> plenty more. Any thoughts on any of this?
> >>
> >> Thanks,
> >> Mat
> >>
> >>
> >>
> >>_
> >>_
> >> OpenStack Development Mailing List (not for usage questions)
> >> Unsubscribe:
> >>openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> >> 

[openstack-dev] [oslo.messaging]

[Nader Lahouti]

Hi,

I am considering to use oslo.messaging to read messages from a rabbit
queue. The messages are put into the queue by an external process.
In order to do that I need to specify routing_key in addition to other
parameters (i.e. exchange and queue,... name) for accessing the queue.  I
was looking at the oslo.messaging API and wasn't able to find anywhere to
specify the routing key.

Is it possible to set routing_key when using oslo.messaging? if so, can you
please point me to the document.


Regards,
Nader.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Heat] convergence rally test results (so far)

[Angus Salkeld]

On Tue, Sep 1, 2015 at 10:45 PM Steven Hardy  wrote:

> On Fri, Aug 28, 2015 at 01:35:52AM +, Angus Salkeld wrote:
> >Hi
> >I have been running some rally tests against convergence and our
> existing
> >implementation to compare.
> >So far I have done the following:
> > 1. defined a template with a resource
> >groupA
> https://github.com/asalkeld/convergence-rally/blob/master/templates/resource_group_test_resource.yaml.template
> > 2. the inner resource looks like
> >this:A
> https://github.com/asalkeld/convergence-rally/blob/master/templates/server_with_volume.yaml.templateA
> (it
> >uses TestResource to attempt to be a reasonable simulation of a
> >server+volume+floatingip)
> > 3. defined a rally
> >job:A
> https://github.com/asalkeld/convergence-rally/blob/master/increasing_resources.yamlA
> that
> >creates X resources then updates to X*2 then deletes.
> > 4. I then ran the above with/without convergence and with 2,4,8
> >heat-engines
> >Here are the results compared:
> >
> https://docs.google.com/spreadsheets/d/12kRtPsmZBl_y78aw684PTBg3op1ftUYsAEqXBtT800A/edit?usp=sharing
> >Some notes on the results so far:
> >  * A convergence with only 2 engines does suffer from RPC overload
> (it
> >gets message timeouts on larger templates). I wonder if this is
> the
> >problem in our convergence gate...
> >  * convergence does very well with a reasonable number of engines
> >running.
> >  * delete is slightly slower on convergence
> >Still to test:
> >  * the above, but measure memory usage
> >  * many small templates (run concurrently)
>
> So, I tried running my many-small-templates here with convergence enabled:
>
> https://bugs.launchpad.net/heat/+bug/1489548
>
> In heat.conf I set:
>
> max_resources_per_stack = -1
> convergence_engine = true
>
> Most other settings (particularly RPC and DB settings) are defaults.
>
> Without convergence (but with max_resources_per_stack disabled) I see the
> time to create a ResourceGroup of 400 nested stacks (each containing one
> RandomString resource) is about 2.5 minutes (core i7 laptop w/SSD, 4 heat
> workers e.g the default for a 4 core machine).
>
> With convergence enabled, I see these errors from sqlalchemy:
>
> File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 652, in
> _checkout\nfairy = _ConnectionRecord.checkout(pool)\n', u'  File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 444, in
> checkout\nrec = pool._do_get()\n', u'  File
> "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 980, in
> _do_get\n(self.size(), self.overflow(), self._timeout))\n',
> u'TimeoutError: QueuePool limit of size 5 overflow 10 reached, connection
> timed out, timeout 30\n'].
>
> I assume this means we're loading the DB much more in the convergence case
> and overflowing the QueuePool?
>

Yeah, looks like it.


>
> This seems to happen when the RPC call from the ResourceGroup tries to
> create some of the 400 nested stacks.
>
> Interestingly after this error, the parent stack moves to CREATE_FAILED,
> but the engine remains (very) busy, to the point of being partially
> responsive, so it looks like maybe the cancel-on-fail isnt' working (I'm
> assuming it isn't error_wait_time because the parent stack has been marked
> FAILED and I'm pretty sure it's been more than 240s).
>
> I'll dig a bit deeper when I get time, but for now you might like to try
> the stress test too.  It's a bit of a synthetic test, but it turns out to
> be a reasonable proxy for some performance issues we observed when creating
> large-ish TripleO deployments (which also create a large number of nested
> stacks concurrently).
>

Thanks a lot for testing Steve! I'll make 2 bugs for what you have raised
1. limit the number of resource actions in parallel (maybe base on the
number of cores)
2. the cancel on fail error

-Angus


>
> Steve
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] correction: Re: [puppet][keystone] Keystone resource naming with domain support - no '::domain' if 'Default'

[Rich Megginson]

Slight correction below:

On 09/01/2015 10:56 AM, Rich Megginson wrote:
To close this thread: 
http://lists.openstack.org/pipermail/openstack-dev/2015-August/072878.html


puppet-openstack will support Keystone domain scoped resource names 
without a '::domain' in the name, only if the 'default_domain_id' 
parameter in Keystone has _not_ been set.


Or if the 'default_domain_id' parameter has been set to 'default'.

That is, if the default domain is 'Default'.  This means that if the 
user/operator doesn't care about domains at all, the operator doesn't 
have to deal with them.  However, once the user/operator uses 
`keystone_domain`, and uses `is_default => true`, this means the 
user/operator _must_ use '::domain' with _all_ domain scoped Keystone 
resource names.


Note that the domain named 'Default' with the UUID 'default' is created 
automatically by Keystone, so no need for puppet to create it or ensure 
that it exists.




In addition:

* In the OpenStack L release:
   If 'default_domain_id' is set,

or if 'default_domain_id' is not 'default',
puppet will issue a warning if a name is used without '::domain'. I 
think this is a good thing to do, just in case someone sets the 
default_domain_id by mistake.


* In OpenStack M release:
   Puppet will issue a warning if a name is used without '::domain'.

* From Openstack N release:
   A name must be used with '::domain'.


__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [horizon] URL Sanity

[Richard Jones]

Interesting idea, and in general I'm for consistency. I can't speak
directly to the network/port question, though it seems to me that if ports
must be attached to networks then it makes sense for the URL to reflect
that.

On the other hand, some could argue that the django URL routing is ...
legacy ... and shouldn't me messed with :)

On the gripping hand, thinking about this could inform future angular
routing planning...

On 2 September 2015 at 00:39, Rob Cresswell (rcresswe) 
wrote:

> Hi all,
>
> I recently started looking into properly implementing breadcrumbs to make
> navigation clearer, especially around nested resources (Subnets Detail
> page, for example). The idea is to use the request.path to form a logical
> breadcrumb that isn’t dependent on browser history (
> https://review.openstack.org/#/c/129985/3/horizon/browsers/breadcrumb.py ).
> Unfortunately, this breaks down quite quickly because we use odd patterns
> like `//detail`, and `/`
> doesn’t exist.
>
> This made me realise how much of an inconsistent mess the URL patterns
> are.  I’ve started cleaning them up, so we move from these patterns:
>
> `/admin/networks//detail` - Detail page for a Network
> `/admin/networks//addsubnet` - Create page for a Subnet
>
> To patterns in line with usual CRUD usages, such as:
>
> `/admin/networks/` - Detail page for a Network
> `/admin/networks//subnets/create` - Create page for a Subnet
>
> This is mostly trivial, just removing extraneous words and adding
> consistency, with end goal being every panel following patterns like:
>
> `/` - Index page
> `//` - Detail page for a single resource
> `//create` - Create new resource
> `///update` - Update a resource
>
> This gets a little complex around nested items. Should a Port for example,
> which has a unique ID, be reachable in Horizon by just its ID? Ports must
> always be attached to a network as I understand it. There are multiple ways
> to express this:
>
> `/networks/ports/` - Current implementation
> `/networks//ports/` - My preferred structure
> `/ports/` - Another option
>
> Does anyone have any opinions on how to handle this structuring, or if
> it’s even necessary?
>
> Regards,
> Rob
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Code review process in Fuel and related issues

[Tomasz Napierala]

> On 01 Sep 2015, at 03:43, Igor Kalnitsky  wrote:
> 
> Hi folks,
> 
> So basically..
> 
> * core reviewers won't be feature leads anymore
> * core reviewers won't be assigned to features (or at least not full-time)
> * core reviewers will spend time doing review and participate design meetings
> * core reviewers will spend time triaging bugs
> 
> Is that correct?
> 

>From what I understand, it is not correct. Core reviewer will still do all 
>this activities in most cases. What we are trying to achieve, is to get core's 
>attention only to reviews, that are already reviewed by SMEs and peers. We
hope this will increase the quality of code core reviewers are getting.

Regards,
-- 
Tomasz 'Zen' Napierala
Product Engineering - Poland







__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [magnum] Difference between certs stored in keystone and certs stored in barbican

[Clark, Robert Graham]

On 01/09/2015 11:38, "Douglas Mendizábal"
 wrote:

This turned into exactly what I was trying to avoid, I probably shouldn’t
have mentioned Anchor, but as I started us down this road (where really I
was just expressing some concerns over certificate lifecycle) please see
my comments below :)

>-BEGIN PGP SIGNED MESSAGE-
>Hash: SHA512
>
>Added a few comments inline.
>
>- - Douglas Mendizábal
>
>On 9/1/15 12:03 PM, John Dennis wrote:
>> On 09/01/2015 10:57 AM, Clark, Robert Graham wrote:
>>> 
 The reason that is compelling is that you can have Barbican
 generate, sign, and store a keypair without transmitting the
 private key over the network to the client that originates the
  signing request. It can be directly stored, and made available
  only to the clients that need access to it.
>>> 
>>> This is absolutely _not_ how PKI for TLS is supposed to work,
>>> yes Barbican can create keypairs etc because sometimes that¹s
>>> useful but in the public-private PKI model that TLS expects this
>>> is completely wrong. Magnum nodes should be creating their own
>>> private key and CSR and submitting them to some CA for signing.
>>> 
>
>Barbican provides a lot of options for provisioning certificates. We
>do support provisioning certs by only passing a CSR so that clients
>can keep ownership of their keys, if that's what the client prefers.
>
>Of course, when you're provisioning keys for every node in a cluster
>for many clusters then key management becomes an issue, and if these
>are not throwaway keys, then storing them in Barbican makes sense.
>
>We can also provision the keys, and create CSRs on the Barbican side,
>so we make it very easy for clients who don't want to do any of this
>locally.

I wasn’t bashing Barbican, just the way that it looks like it’s being used
here, as I said, there’s good reasons why Barbican works the way it does,
but not all available options are appropriate for all types of deployments.



>
>>> Now this gets messy because you probably don¹t want to push
>>> keystone credentials onto each node (that they would use to
>>> communicate with Barbican).
>>> 
>
>Kevin Fox is working on a Nova spec to provide identity to VMs. I'm
>really hoping this spec gains some traction because it's a problem
>that not only Barbican, but all other user-facing projects can benefit
>from.
>
>See: https://blueprints.launchpad.net/nova/+spec/instance-users
>
>>> I¹m a bit conflicted writing this next bit because I¹m not
>>> particularly familiar with the Kubernetes/Magnum architectures
>>> and also because I¹m one of the core developers for Anchor but
>>> here goesŠ.
>>> 
>>> Have you considered using Anchor for this? It¹s a pretty
>>> lightweight ephemeral CA that is built to work well in small PKI
>>>  communities (like a Kubernetes cluster) you can configure
>>> multiple methods for authentication and build pretty simple
>>> validation rules for deciding if a host should be given a
>>> certificate. Anchor is built to provide short-lifetime
>>> certificates where each node re-requests a certificate typically
>>>  every 12-24 hours, this has some really nice properties like
>>> ³passive revocation² (Think revocation that actually works) and
>>> strong ways to enforce issuing logic on a per host basis.
>>> 
>
>Someone from the Magnum team can correct me if I'm wrong, but I do
>believe they considered Anchor for certificate provisioning.
>
>As I understand the Magnum use case, they will be provisioning many
>clusters across different tenants. While Anchor would work well for a
>single cluster, they need the ability to provision new CA roots for
>each and every cluster, and then provision certs off that root for
>every node in the cluster. This way node certs are only valid in the
>context of the cluster.
>
>A new feature for Barbican Liberty will be the ability to add new CA
>roots scoped to a tenant via API, which will address the Magnum
>requirements of separating the certificate roots per cluster.

Interestingly, we recently added support for this, you can have multiple
CA roots within Anchor. When we announced it a few summits back we
focussed on how you can use Anchor in silo’d deployments where you would
spin up an instance for each cluster but recently we added the ability to
run Anchor with multiple CA roots, each can have it’s own auth and it’s
own validation chain.


>
>>> Anchor or not, I¹d like to talk to you more about how you¹re
>>> attempting to secure Magnum - I think it¹s an extremely
>>> interesting project that I¹d like to help out with.
>>> 
>>> -Rob (Security Project PTL / Anchor flunkie)
>> 
>> Let's not reinvent the wheel. I can't comment on what Magnum is
>> doing but I do know the members of the Barbican project are PKI
>> experts and understand CSR's, key escrow, revocation, etc. Some of
>>  the design work is being done by engineers who currently
>> contribute to products in use by the Dept. of Defense, an agency
>> that 

Re: [openstack-dev] [Ironic] Command structure for OSC plugin

[Dean Troyer]

[late catch-up]

On Mon, Aug 24, 2015 at 2:56 PM, Doug Hellmann 
wrote:

> Excerpts from Brad P. Crochet's message of 2015-08-24 15:35:59 -0400:
> > On 24/08/15 18:19 +, Tim Bell wrote:
> > >
> > >From a user perspective, where bare metal and VMs are just different
> flavors (with varying capabilities), can we not use the same commands
> (server create/rebuild/...) ? Containers will create the same conceptual
> problems.
> > >
> > >OSC can provide a converged interface but if we just replace '$ ironic
> ' by '$ openstack baremetal ', this seems to be a missed
> opportunity to hide the complexity from the end user.
> > >
> > >Can we re-use the existing server structures ?
>

I've wondered about how users would see doing this, we've done it already
with the quota and limits commands (blurring the distinction between
project APIs).  At some level I am sure users really do not care about some
of our project distinctions.


> To my knowledge, overriding or enhancing existing commands like that
>
> is not possible.
>
> You would have to do it in tree, by making the existing commands
> smart enough to talk to both nova and ironic, first to find the
> server (which service knows about something with UUID XYZ?) and
> then to take the appropriate action on that server using the right
> client. So it could be done, but it might lose some of the nuance
> between the server types by munging them into the same command. I
> don't know what sorts of operations are different, but it would be
> worth doing the analysis to see.
>

I do have an experimental plugin that hooks the server create command to
add some options and change its behaviour so it is possible, but right now
I wouldn't call it supported at all.  That might be something that we could
consider doing though for things like this.

The current model for commands calling multiple project APIs is to put them
in openstackclient.common, so yes, in-tree.

Overall, though, to stay consistent with OSC you would map operations into
the current verbs as much as possible.  It is best to think in terms of how
the CLI user is thinking and what she wants to do, and not how the REST or
Python API is written.  In this case, 'baremetal' is a type of server, a
set of attributes of a server, etc.  As mentioned earlier, containers will
also have a similar paradigm to consider.

dt

-- 

Dean Troyer
dtro...@gmail.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Murano] Documentation on how to Start Contributing

[Vahid S Hashemian]

Hi Jeremy,

Thanks for sharing these tips.

What is your advice on debugging a PyPi package? I'm modifying the code 
for python-muranoclient and would like to be able to debug using eclipse 
(in which I'm coding) or any other convenient means.

Regards,
--Vahid Hashemian__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Blazar] Anyone interested?

[Ildikó Váncsa]

Hi,

I'm glad to see the interest and I also support the idea of using the IRC 
channel that is already set up for further communication. Should we aim for a 
meeting/discussion there around the end of this week or during next week?

@Nikolay, Sylvain: Thanks for support and bringing together a list of action 
items as very first steps.

@Pierre: You wrote that you are using Blazar. Are you using it as is with an 
older version of OpenStack or you have a modified version of the project/code?

Best Regards,
Ildikó

> -Original Message-
> From: Sylvain Bauza [mailto:sba...@redhat.com]
> Sent: September 01, 2015 16:18
> To: OpenStack Development Mailing List (not for usage questions)
> Subject: Re: [openstack-dev] [Blazar] Anyone interested?
> 
> 
> 
> Le 01/09/2015 15:46, Nikolay Starodubtsev a écrit :
> 
> 
>   Also, if we decided to continue development we should add blazar here 
> [1] according to the email [2]
>   So, my suggestion is to setup some timeframe on this week or next week 
> and hold some kind of meeting.
>   [1]: https://wiki.openstack.org/wiki/Stackforge_Namespace_Retirement
>   [2]: 
> http://lists.openstack.org/pipermail/openstack-dev/2015-August/073071.html
> 
> 
> 
> 
> That's just what I said I was OK to add, for sure.
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
>   Nikolay Starodubtsev
> 
> 
>   Software Engineer
> 
>   Mirantis Inc.
> 
> 
> 
> 
> 
>   Skype: dark_harlequine1
> 
> 
> 
>   2015-09-01 12:42 GMT+03:00 Nikolay Starodubtsev 
> :
> 
> 
>   Sylvain,
>   First of all we need to reanimate blazar gate-jobs, or we can't 
> merge anything. I tried to do it a year ago,
> but can't get the point of the tests,
>   so better decision can be to rewrite them from scratch.
> 
> 
> 
> 
> 
> 
> 
> 
>   Nikolay Starodubtsev
> 
> 
>   Software Engineer
> 
>   Mirantis Inc.
> 
> 
> 
> 
> 
>   Skype: dark_harlequine1
> 
> 
> 
>   2015-09-01 11:26 GMT+03:00 Sylvain Bauza :
> 
> 
> 
> 
> 
>   Le 01/09/2015 06:52, Nikolay Starodubtsev a écrit :
> 
> 
>   All,
>   I'd like to propose use of #openstack-blazar 
> for further communication and
> coordination.
> 
> 
> 
> 
>   +2 to that. That's the first step of any communication. 
> The channel logs are also recorded
> here, for async communication :
>   
> http://eavesdrop.openstack.org/irclogs/%23openstack-blazar/
> 
>   I don't see at the moment much benefits of running a 
> weekly meeting. We can chat on
> purpose if needed.
> 
>   Like I said to Ildiko, I'm fine to help some people 
> discovering Blazar but I won't have time
> lots of time for actually working on it.
> 
>   IMHO, the first things to do with Blazar is to reduce 
> the tech debt by :
>1/ finising the Climate->Blazar renaming
>2/ updating and using the latest oslo librairies 
> instead of using the old incubator
>3/ using Nova V2.1 API (which could be a bit difficult 
> because there are no more
> extensions)
> 
>   If I see some progress with Blazar, I'm OK with asking 
> -infra to move Blazar to the
> OpenStack namespace like it was asked by James Blair here because it seems 
> Blazar is not defunct :
>   
> http://lists.openstack.org/pipermail/openstack-dev/2015-August/072140.html
> 
>   -Sylvain
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
>   Nikolay Starodubtsev
> 
> 
>   Software Engineer
> 
>   Mirantis Inc.
> 
> 
> 
> 
> 
>   Skype: dark_harlequine1
> 
> 
> 
>   2015-08-31 19:44 GMT+03:00 Fuente, Pablo A 
> :
> 
> 
>   Yes, Blazar is a really interesting 
> project. I worked on it some
> time ago and I really enjoy it. Sadly my obligations at work don't let me to 
> still working on it, but I', happy that there still some interest
> in Blazar.
> 
>   Pablo.
>   On 31/08/15 09:19, Zhenyu Zheng wrote:
>   Hello,
>   It seems like an interesting project.
> 
>   On Fri, Aug 28, 2015 at 7:54 PM, Pierre 
> Riteau
> > wrote:
>   Hello,
> 
>   The NSF-funded Chameleon project
> (https://www.chameleoncloud.org) uses Blazar to provide advance reservations 
> of resources for running cloud computing
> experiments.
> 
>  

Re: [openstack-dev] [Nova] Placing VMs based on multiple criteria

[Sourabh Patwardhan]

Hi Sundar,

Have you considered writing your own custom filter for hosts as described
in [1] ?

Thanks,
Sourabh

[1] http://docs.openstack.org/developer/nova/devref/filter_scheduler.html


On Mon, Aug 31, 2015 at 9:58 AM, Sundar Nadathur 
wrote:

> Hi all,
>
> I’d appreciate if the experts can point me in one direction or
> another. If there are existing mechanisms, we don’t want to reinvent the
> wheel. If there aren’t, I’d be interested in exploring clean ways to extend
> and enhance nova scheduling.
>
>
>
> Thank you very much.
>
>
>
> Cheers,
>
> Sundar
>
>
>
> *From:* Sundar Nadathur
> *Sent:* Monday, August 24, 2015 10:48 PM
> *To:* 'openstack-dev@lists.openstack.org'
> *Subject:* [Nova] Placing VMs based on multiple criteria
>
>
>
> Hi,
>
>Please advise me whether the following scenario requires changes to
> nova scheduler or can be handled with existing scheduling mechanisms.
>
>
>
> I have a type of PCIe device (not necessarily a NIC or HBA). The device
> can be configured with a set of user-defined resources – say A, B, C. Each
> resource can be shared between a limited number of VMs -- say A can be
> shared among 4 VMs, B among 8, etc. A VM image may request the need for a
> specific list of features, say A and B. Then I want to place the VM on a
> host according to these criteria:
>
> 1.   If there are hosts with a PCIe device that already has A and B
> configured, and has a free instance each of A and B, the VM  must be placed
> on one of those hosts.
>
> 2.   Otherwise, find a host with this PCIe device that can be
> configured with one instance each of A and B.
>
>
>
> It is not clear that this can be handled through 3rd party metadata.
> Suppose we create host aggregates with properties like “resource=A” and
> “resource=B”, and also associate properties like “resource=A” with VM
> images. (A and B are UUIDs representing user-defined resources.) Perhaps Nova
> scheduler can match the properties to select host aggregates that have all
> properties that the VM requires. However:
>
> a.   This would not be dynamic (i.e. track the free instances of each
> resource), and
>
> b.  This addresses only #1 above.
>
>
>
> Is there any way I can leverage existing scheduler mechanisms to  solve
> this VM placement problem? If not, do you have thoughts/comments on what
> changes are needed?
>
>
>
> Thanks, and apologies in advance if I am not clear. Please feel free to
> ask questions.
>
>
>
> Cheers,
>
> Sundar
>
> --
>
> Confidentiality Notice.
> This message may contain information that is confidential or otherwise
> protected from disclosure. If you are not the intended recipient, you are
> hereby notified that any use, disclosure, dissemination, distribution, or
> copying of this message, or any attachments, is strictly prohibited. If you
> have received this message in error, please advise the sender by reply
> e-mail, and delete the message and any attachments. Thank you.
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [magnum] Difference between certs stored in keystone and certs stored in barbican

[Clark, Robert Graham]

Extremely interesting.

This is something that we are looking at during the Security mid-cycle
(happening this week) see "Secure communications between control plane and
tenant plane” under
https://etherpad.openstack.org/p/security-liberty-midcycle

This is problem for a lot of different projects, we’ve added your
blueprint and hopefully we’ll be able to help with this moving forward.

-Rob



On 01/09/2015 11:11, "Fox, Kevin M"  wrote:

>https://blueprints.launchpad.net/nova/+spec/instance-users
>
>Please see the above spec. Nova, Keystone and Barbican have been working
>together on it this cycle and are hoping to implement it in Mitaka
>
>The problem of secrets from the secret store is not isolated to just
>Magnum.
>
>Thanks,
>Kevin
>
>From: John Dennis [jden...@redhat.com]
>Sent: Tuesday, September 01, 2015 10:03 AM
>To: OpenStack Development Mailing List (not for usage questions)
>Subject: Re: [openstack-dev] [magnum] Difference between certs stored in
>keystone and certs stored in barbican
>
>On 09/01/2015 10:57 AM, Clark, Robert Graham wrote:
>>
>>> The reason that is compelling is that you can have Barbican generate,
>>> sign, and store a keypair without transmitting the private key over the
>>> network to the client that originates the signing request. It can be
>>> directly stored, and made available only to the clients that need
>>>access
>>> to it.
>>
>> This is absolutely _not_ how PKI for TLS is supposed to work, yes
>>Barbican
>> can create keypairs etc because sometimes that¹s useful but in the
>> public-private PKI model that TLS expects this is completely wrong.
>>Magnum
>> nodes should be creating their own private key and CSR and submitting
>>them
>> to some CA for signing.
>>
>> Now this gets messy because you probably don¹t want to push keystone
>> credentials onto each node (that they would use to communicate with
>> Barbican).
>>
>> I¹m a bit conflicted writing this next bit because I¹m not particularly
>> familiar with the Kubernetes/Magnum architectures and also because I¹m
>>one
>> of the core developers for Anchor but here goesŠ.
>>
>> Have you considered using Anchor for this? It¹s a pretty lightweight
>> ephemeral CA that is built to work well in small PKI communities (like a
>> Kubernetes cluster) you can configure multiple methods for
>>authentication
>> and build pretty simple validation rules for deciding if a host should
>>be
>> given a certificate. Anchor is built to provide short-lifetime
>> certificates where each node re-requests a certificate typically every
>> 12-24 hours, this has some really nice properties like ³passive
>> revocation² (Think revocation that actually works) and strong ways to
>> enforce issuing logic on a per host basis.
>>
>> Anchor or not, I¹d like to talk to you more about how you¹re attempting
>>to
>> secure Magnum - I think it¹s an extremely interesting project that I¹d
>> like to help out with.
>>
>> -Rob
>> (Security Project PTL / Anchor flunkie)
>
>Let's not reinvent the wheel. I can't comment on what Magnum is doing
>but I do know the members of the Barbican project are PKI experts and
>understand CSR's, key escrow, revocation, etc. Some of the design work
>is being done by engineers who currently contribute to products in use
>by the Dept. of Defense, an agency that takes their PKI infrastructure
>very seriously. They also have been involved with Keystone. I work with
>these engineers on a regular basis.
>
>The Barbican blueprint states:
>
>Barbican supports full lifecycle management including provisioning,
>expiration, reporting, etc. A plugin system allows for multiple
>certificate authority support (including public and private CAs).
>
>Perhaps Anchor would be a great candidate for a Barbican plugin.
>
>What I don't want to see is spinning our wheels, going backward, or
>inventing one-off solutions to a very demanding and complex problem
>space. There have been way too many one-off solutions in the past, we
>want to consolidate the expertise in one project that is designed by
>experts and fully vetted, this is the role of Barbican. Would you like
>to contribute to Barbican? I'm sure your skills would be a tremendous
>asset.
>
>
>--
>John
>
>__
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>__
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 

Re: [openstack-dev] [Tacker][NFV] Heads up: switch over to master

[Sridhar Ramaswamy]

The switch over to master is now complete. Please follow the updated (and a
bit easier) Installation steps here,

https://wiki.openstack.org/wiki/Tacker/Installation


On Mon, Aug 31, 2015 at 1:32 PM, Sridhar Ramaswamy 
wrote:

> Tacker dev & user teams:
>
> This shouldn't be a news to folks regularly attending tacker weekly
> meetings.
>
> For others, until now Tacker code base was using stable/kilo dependencies.
> Now we will be switching over to use master based dependencies using [1]
> and [2]. We are now close to merging these two patchsets. However this will
> cause your existing devstack setups incompatible with the latest tacker
> code base. Note, before merging these patchsets a stable/kilo branch will
> be pulled.
>
> Now you've two options to continue tacker activites,
>
> 1) Switch over to use new devstack setup post merge (preferred)
> 2) If you are in middle of something and want to continue your activity
> (say a PoC in flight) consider switching tacker & python-tackerclient repo
> to stable/kilo
>
> We understand this will cause some disruption. But this is essential to
> cut over to master and get ready for the upcoming liberty release.
>
> Thanks for your support. Questions / comments welcome!
>
> - Sridhar
>
> [1] https://review.openstack.org/#/c/211403/
> [2] https://review.openstack.org/#/c/211405/
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Blazar] Anyone interested?

[Shamail]

Hi everyone,

Is the information on the wiki still up to date?
https://wiki.openstack.org/wiki/Blazar

I have copied the Product WG mailing list as well since one of the user 
stories[1] being discussed in that forum has to do with capacity management and 
resource reservation.

[1] 
http://specs.openstack.org/openstack/openstack-user-stories/user-stories/draft/capacity_management.html

Thanks,
Shamail 



> On Sep 1, 2015, at 10:18 AM, Sylvain Bauza  wrote:
> 
> 
> 
> Le 01/09/2015 15:46, Nikolay Starodubtsev a écrit :
>> Also, if we decided to continue development we should add blazar here [1] 
>> according to the email [2]
>> So, my suggestion is to setup some timeframe on this week or next week and 
>> hold some kind of meeting.
>> [1]: https://wiki.openstack.org/wiki/Stackforge_Namespace_Retirement
>> [2]: 
>> http://lists.openstack.org/pipermail/openstack-dev/2015-August/073071.html
> 
> That's just what I said I was OK to add, for sure.
> 
>> 
>>   
>> Nikolay Starodubtsev
>> Software Engineer
>> Mirantis Inc.
>> 
>> Skype: dark_harlequine1
>> 
>> 2015-09-01 12:42 GMT+03:00 Nikolay Starodubtsev :
>>> Sylvain,
>>> First of all we need to reanimate blazar gate-jobs, or we can't merge 
>>> anything. I tried to do it a year ago, but can't get the point of the tests,
>>> so better decision can be to rewrite them from scratch.
>>> 
>>>   
>>> Nikolay Starodubtsev
>>> Software Engineer
>>> Mirantis Inc.
>>> 
>>> Skype: dark_harlequine1
>>> 
>>> 2015-09-01 11:26 GMT+03:00 Sylvain Bauza :
 
 
 Le 01/09/2015 06:52, Nikolay Starodubtsev a écrit :
> All,
> I'd like to propose use of #openstack-blazar for further communication 
> and coordination.
 
 
 +2 to that. That's the first step of any communication. The channel logs 
 are also recorded here, for async communication :
 http://eavesdrop.openstack.org/irclogs/%23openstack-blazar/
 
 I don't see at the moment much benefits of running a weekly meeting. We 
 can chat on purpose if needed.
 
 Like I said to Ildiko, I'm fine to help some people discovering Blazar but 
 I won't have time lots of time for actually working on it.
 
 IMHO, the first things to do with Blazar is to reduce the tech debt by :
  1/ finising the Climate->Blazar renaming
  2/ updating and using the latest oslo librairies instead of using the old 
 incubator
  3/ using Nova V2.1 API (which could be a bit difficult because there are 
 no more extensions)
 
 If I see some progress with Blazar, I'm OK with asking -infra to move 
 Blazar to the OpenStack namespace like it was asked by James Blair here 
 because it seems Blazar is not defunct :
 http://lists.openstack.org/pipermail/openstack-dev/2015-August/072140.html
 
 -Sylvain
 
 
 
 
>   
> Nikolay Starodubtsev
> Software Engineer
> Mirantis Inc.
> 
> Skype: dark_harlequine1
> 
> 2015-08-31 19:44 GMT+03:00 Fuente, Pablo A :
>> Yes, Blazar is a really interesting project. I worked on it some time 
>> ago and I really enjoy it. Sadly my obligations at work don't let me to 
>> still working on it, but I', happy that there still some interest in 
>> Blazar.
>> 
>> Pablo.
>> On 31/08/15 09:19, Zhenyu Zheng wrote:
>> Hello,
>> It seems like an interesting project.
>> 
>> On Fri, Aug 28, 2015 at 7:54 PM, Pierre Riteau 
>> > wrote:
>> Hello,
>> 
>> The NSF-funded Chameleon project (https://www.chameleoncloud.org) uses 
>> Blazar to provide advance reservations of resources for running cloud 
>> computing experiments.
>> 
>> We would be interested in contributing as well.
>> 
>> Pierre Riteau
>> 
>> On 28 Aug 2015, at 07:56, Ildikó Váncsa 
>> <ildiko.van...@ericsson.com>
>>  wrote:
>> 
>> > Hi All,
>> >
>> > The resource reservation topic pops up time to time on different 
>> > forums to cover use cases in terms of both IT and NFV. The Blazar 
>> > project was intended to address this need, but according to my 
>> > knowledge due to earlier integration and other difficulties the work 
>> > has been stopped.
>> >
>> > My question is that who would be interested in resurrecting the Blazar 
>> > project and/or working on a reservation system in OpenStack?
>> >
>> > Thanks and Best Regards,
>> > Ildikó
>> >
>> > __
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe: 
>> > 

Re: [openstack-dev] [Blazar] Anyone interested?

[Sylvain Bauza]

Le 01/09/2015 22:31, Ildikó Váncsa a écrit :

Hi,

I'm glad to see the interest and I also support the idea of using the IRC 
channel that is already set up for further communication. Should we aim for a 
meeting/discussion there around the end of this week or during next week?

@Nikolay, Sylvain: Thanks for support and bringing together a list of action 
items as very first steps.

@Pierre: You wrote that you are using Blazar. Are you using it as is with an 
older version of OpenStack or you have a modified version of the project/code?
I'm actually really surprised to see 
https://www.chameleoncloud.org/docs/user-guides/bare-metal-user-guide/ 
which describes quite fine how to use Blazar/Climate either using the 
CLI or by Horizon.


The latter is actually not provided within the git tree, so I guess 
Chameleon added it downstream. That's fine, maybe something we could 
upstream if Pierre and his team are okay ?


-Sylvain





Best Regards,
Ildikó


-Original Message-
From: Sylvain Bauza [mailto:sba...@redhat.com]
Sent: September 01, 2015 16:18
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [Blazar] Anyone interested?



Le 01/09/2015 15:46, Nikolay Starodubtsev a écrit :


Also, if we decided to continue development we should add blazar here 
[1] according to the email [2]
So, my suggestion is to setup some timeframe on this week or next week 
and hold some kind of meeting.
[1]: https://wiki.openstack.org/wiki/Stackforge_Namespace_Retirement
[2]: 
http://lists.openstack.org/pipermail/openstack-dev/2015-August/073071.html




That's just what I said I was OK to add, for sure.










Nikolay Starodubtsev


Software Engineer

Mirantis Inc.





Skype: dark_harlequine1



2015-09-01 12:42 GMT+03:00 Nikolay Starodubtsev 
:


Sylvain,
First of all we need to reanimate blazar gate-jobs, or we can't 
merge anything. I tried to do it a year ago,
but can't get the point of the tests,
so better decision can be to rewrite them from scratch.








Nikolay Starodubtsev


Software Engineer

Mirantis Inc.





Skype: dark_harlequine1



2015-09-01 11:26 GMT+03:00 Sylvain Bauza :





Le 01/09/2015 06:52, Nikolay Starodubtsev a écrit :


All,
I'd like to propose use of #openstack-blazar 
for further communication and
coordination.




+2 to that. That's the first step of any communication. 
The channel logs are also recorded
here, for async communication :

http://eavesdrop.openstack.org/irclogs/%23openstack-blazar/

I don't see at the moment much benefits of running a 
weekly meeting. We can chat on
purpose if needed.

Like I said to Ildiko, I'm fine to help some people 
discovering Blazar but I won't have time
lots of time for actually working on it.

IMHO, the first things to do with Blazar is to reduce 
the tech debt by :
 1/ finising the Climate->Blazar renaming
 2/ updating and using the latest oslo librairies 
instead of using the old incubator
 3/ using Nova V2.1 API (which could be a bit difficult 
because there are no more
extensions)

If I see some progress with Blazar, I'm OK with asking 
-infra to move Blazar to the
OpenStack namespace like it was asked by James Blair here because it seems 
Blazar is not defunct :

http://lists.openstack.org/pipermail/openstack-dev/2015-August/072140.html

-Sylvain











Nikolay Starodubtsev


Software Engineer

Mirantis Inc.





Skype: dark_harlequine1



2015-08-31 19:44 GMT+03:00 Fuente, Pablo A 
:


Yes, Blazar is a really interesting 
project. I worked on it some
time ago and I really enjoy it. Sadly my obligations at work don't let me to 
still working on it, but I', happy that there still some interest
in Blazar.

Pablo.
On 31/08/15 09:19, Zhenyu Zheng wrote:
Hello,
It seems like an interesting project.

On Fri, Aug 28, 2015 at 7:54 PM, Pierre 
Riteau
> wrote:
Hello,

  

Re: [openstack-dev] [trove] [heat] Multi region support

[Lowery, Mathew]

Thank you Zane for the clarifications!

I misunderstood #2 and that led to the other misunderstandings.

Further questions:
* Are nested stacks aware of their nested-ness? In other words, given any
nested stack (colocated with parent stack or not), can I trace it back to
the parent stack? (On a possibly related note, I see that adopting a stack
is an option to reassemble a new parent stack from its regional parts in
the event that the old parent stack is lost.)
* Has this design met the users' needs? In other words, are there any
plans to make major modifications to this design?

Thanks!

On 9/1/15, 1:47 PM, "Zane Bitter"  wrote:

>On 01/09/15 11:41, Lowery, Mathew wrote:
>> This is a Trove question but including Heat as they seem to have solved
>> this problem.
>>
>> Summary: Today, it seems that Trove is not capable of creating a cluster
>> spanning multiple regions. Is that the case and, if so, are there any
>> plans to work on that? Also, are we aware of any precedent solutions
>> (e.g. remote stacks in Heat) or even partially completed spec/code in
>>Trove?
>>
>> More details:
>>
>> I found this nice diagram
>> 
>>>_Heat/The_Missing_Diagram> created
>> for Heat. As far as I understand it,
>
>Clarifications below...
>
>> #1 is the absence of multi-region
>> support (i.e. what we have today). #2 seems to be a 100% client-based
>> solution. In other words, the Heat services never know about the other
>> stacks.
>
>I guess you could say that.
>
>> In fact, there is nothing tying these stacks together at all.
>
>I wouldn't go that far. The regional stacks still appear as resources in
>their parent stack, so they're tied together by whatever inputs and
>outputs are connected up in that stack.
>
>> #3
>> seems to show a "master" Heat server that understands "remote stacks"
>> and simply converts those "remote stacks" into calls on regional Heats.
>> I assume here the master stack record is stored by the master Heat.
>> Because the "remote stacks" are full-fledged stacks, they can be managed
>> by their regional Heats if availability of master or other regional
>> Heats is lost.
>
>Yeah.
>
>> #4, the diagram doesn't seem to match the description
>> (instead of one global Heat, it seems the diagram should show two
>> regional Heats).
>
>It does (they're the two orange boxes).
>
>> In this one, a single arbitrary region becomes the
>> owner of the stack and remote (low-level not stack) resources are
>> created as needed. One problem is the manageability is lost if the Heat
>> in the owning region is lost. Finally, #5. In #5, it's just #4 but with
>> one and only one Heat.
>>
>> It seems like Heat solved this 
>> using #3 (Master Orchestrator)
>
>No, we implemented #2.
>
>> but where there isn't necessarily a
>> separate master Heat. Remote stacks can be created by any regional
>>stack.
>
>Yeah, that was the difference between #3 and #2 :)
>
>cheers,
>Zane.
>
>> Trove questions:
>>
>>  1. Having sub-clusters (aka remote clusters aka nested clusters) seems
>> to be useful (i.e. manageability isn't lost when a region is lost).
>> But then again, does it make sense to perform a cluster operation on
>> a sub-cluster?
>>  2. You could forego sub-clusters and just create full-fledged remote
>> standalone Trove instances.
>>  3. If you don't create full-fledged remote Trove instances (but instead
>> just call remote Nova), then you cannot do simple things like
>> getting logs from a node without going through the owning region's
>> Trove. This is an extra hop and a single point of failure.
>>  4. Even with sub-clusters, the only record of them being related lives
>> only in the "owning" region. Then again, some ID tying them all
>> together could be passed to the remote regions.
>>  5. Do we want to allow the piecing together of clusters (sort of like
>> Heat's "adopt")?
>>
>> These are some questions floating around my head and I'm sure there are
>> plenty more. Any thoughts on any of this?
>>
>> Thanks,
>> Mat
>>
>>
>> 
>>_
>>_
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: 
>>openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
>__
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

Re: [openstack-dev] [magnum] Difference between certs stored in keystone and certs stored in barbican

[Fox, Kevin M]

Awesome. Thanks. :)

Are there any plans for the summit yet? I think we should all get together and 
talk about it.

Thanks,
Kevin

From: Clark, Robert Graham [robert.cl...@hp.com]
Sent: Tuesday, September 01, 2015 1:35 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [magnum] Difference between certs stored in 
keystone and certs stored in barbican

Extremely interesting.

This is something that we are looking at during the Security mid-cycle
(happening this week) see "Secure communications between control plane and
tenant plane” under
https://etherpad.openstack.org/p/security-liberty-midcycle

This is problem for a lot of different projects, we’ve added your
blueprint and hopefully we’ll be able to help with this moving forward.

-Rob



On 01/09/2015 11:11, "Fox, Kevin M"  wrote:

>https://blueprints.launchpad.net/nova/+spec/instance-users
>
>Please see the above spec. Nova, Keystone and Barbican have been working
>together on it this cycle and are hoping to implement it in Mitaka
>
>The problem of secrets from the secret store is not isolated to just
>Magnum.
>
>Thanks,
>Kevin
>
>From: John Dennis [jden...@redhat.com]
>Sent: Tuesday, September 01, 2015 10:03 AM
>To: OpenStack Development Mailing List (not for usage questions)
>Subject: Re: [openstack-dev] [magnum] Difference between certs stored in
>keystone and certs stored in barbican
>
>On 09/01/2015 10:57 AM, Clark, Robert Graham wrote:
>>
>>> The reason that is compelling is that you can have Barbican generate,
>>> sign, and store a keypair without transmitting the private key over the
>>> network to the client that originates the signing request. It can be
>>> directly stored, and made available only to the clients that need
>>>access
>>> to it.
>>
>> This is absolutely _not_ how PKI for TLS is supposed to work, yes
>>Barbican
>> can create keypairs etc because sometimes that¹s useful but in the
>> public-private PKI model that TLS expects this is completely wrong.
>>Magnum
>> nodes should be creating their own private key and CSR and submitting
>>them
>> to some CA for signing.
>>
>> Now this gets messy because you probably don¹t want to push keystone
>> credentials onto each node (that they would use to communicate with
>> Barbican).
>>
>> I¹m a bit conflicted writing this next bit because I¹m not particularly
>> familiar with the Kubernetes/Magnum architectures and also because I¹m
>>one
>> of the core developers for Anchor but here goesŠ.
>>
>> Have you considered using Anchor for this? It¹s a pretty lightweight
>> ephemeral CA that is built to work well in small PKI communities (like a
>> Kubernetes cluster) you can configure multiple methods for
>>authentication
>> and build pretty simple validation rules for deciding if a host should
>>be
>> given a certificate. Anchor is built to provide short-lifetime
>> certificates where each node re-requests a certificate typically every
>> 12-24 hours, this has some really nice properties like ³passive
>> revocation² (Think revocation that actually works) and strong ways to
>> enforce issuing logic on a per host basis.
>>
>> Anchor or not, I¹d like to talk to you more about how you¹re attempting
>>to
>> secure Magnum - I think it¹s an extremely interesting project that I¹d
>> like to help out with.
>>
>> -Rob
>> (Security Project PTL / Anchor flunkie)
>
>Let's not reinvent the wheel. I can't comment on what Magnum is doing
>but I do know the members of the Barbican project are PKI experts and
>understand CSR's, key escrow, revocation, etc. Some of the design work
>is being done by engineers who currently contribute to products in use
>by the Dept. of Defense, an agency that takes their PKI infrastructure
>very seriously. They also have been involved with Keystone. I work with
>these engineers on a regular basis.
>
>The Barbican blueprint states:
>
>Barbican supports full lifecycle management including provisioning,
>expiration, reporting, etc. A plugin system allows for multiple
>certificate authority support (including public and private CAs).
>
>Perhaps Anchor would be a great candidate for a Barbican plugin.
>
>What I don't want to see is spinning our wheels, going backward, or
>inventing one-off solutions to a very demanding and complex problem
>space. There have been way too many one-off solutions in the past, we
>want to consolidate the expertise in one project that is designed by
>experts and fully vetted, this is the role of Barbican. Would you like
>to contribute to Barbican? I'm sure your skills would be a tremendous
>asset.
>
>
>--
>John
>
>__
>OpenStack Development Mailing List (not for usage questions)
>Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

Re: [openstack-dev] [murano] Proposing Nikolai Starodubtsev for core

[Ekaterina Chernova]

+1

On Tue, Sep 1, 2015 at 10:03 AM, Dmitro Dovbii  wrote:

> +1
>
> 2015-09-01 2:24 GMT+03:00 Serg Melikyan :
>
>> +1
>>
>> On Mon, Aug 31, 2015 at 3:45 PM, Kirill Zaitsev 
>> wrote:
>>
>>> I’m pleased to nominate Nikolai for Murano core.
>>>
>>> He’s been actively participating in development of murano during liberty
>>> and is among top5 contributors during last 90 days. He’s also leading the
>>> CloudFoundry integration initiative.
>>>
>>> Here are some useful links:
>>>
>>> Overall contribution: http://stackalytics.com/?user_id=starodubcevna
>>> List of reviews:
>>> https://review.openstack.org/#/q/reviewer:%22Nikolay+Starodubtsev%22,n,z
>>> Murano contribution during latest 90 days
>>> http://stackalytics.com/report/contribution/murano/90
>>>
>>> Please vote with +1/-1 for approval/objections
>>>
>>> --
>>> Kirill Zaitsev
>>> Murano team
>>> Software Engineer
>>> Mirantis, Inc
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>>
>> --
>> Serg Melikyan, Senior Software Engineer at Mirantis, Inc.
>> http://mirantis.com | smelik...@mirantis.com
>>
>> +7 (495) 640-4904, 0261
>> +7 (903) 156-0836
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Re keystone to keystone federation

[sreeja kannagundla]

Hi

I am working on keystone2keystone federatoin and using kilo version for
both keystone-sp and keystone idp
After configuring keystone-sp and keystone-idp I am trying to use the
command :

openstack federation project list -os-auth-type v3unscopedsaml
--os-identity-provider k2k  --os-auth-url https://10.63.13.161:35357/v3
--os-identity-provider-url
https://10.63.13.163:35357/v3/OS-FEDERATION/saml2/idp --os-username user
--os-password password

It returns an error:

ERROR: openstack Expecting to find application/json in Content-Type header
- the server could not comply with the request since it is either malformed
or otherwise incorrect. The client is assumed to be in error. (HTTP 400)
(Request-ID: req-4839f349-e3ed-403f-b456-dfc0d1aecbe4)

This is because in keystoneclient/contrib/auth/v3/saml2.py, while sending a
request to keystone.idp for saml assertion, the content type used is
text/xml

idp_response = session.post(
self.identity_provider_url,
headers={'Content-type': 'text/xml'},
data=etree.tostring(idp_saml2_authn_request),
requests_auth=(self.username, self.password),
authenticated=False, log=False)

why is keystone.idp not accepting the content type: text/xml?
what can be the workaroung for this issue

Thanks
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] ProviderFirewallRules still available?

[Hans Lindgren]

Some drivers (libvirt, xen) still query the database and try to set up provider 
rules. However, there is no api for this feature as the mail thread you link 
points out and therefore no way to add/remove any rules. Due to this the setup 
of provider rules that is done today will effectively do nothing.

An attempt at cleaning out all code related to this can be found here: 
https://review.openstack.org/184027


Hans



On 2015-08-31 21:23, "Asselin, Ramy"  wrote:

>I saw this thread, “Undead DB objects: ProviderFirewallRule and 
>InstanceGroupPolicy?” [1]
>
> 
>Seems to be in the process of getting removed from the DB, but are provider 
>level firewalls still available? I still see a lot of code that indicates it 
>does [2]
> 
>I traced it to here, but not sure where the nova context is stored [3]
>
> 
>Ramy
>Irc:asselin
>[1] 
>http://lists.openstack.org/pipermail/openstack-dev/2014-November/050959.html 
>
>[2] 
>https://github.com/openstack/nova/search?l=python=provider=%E2%9C%93 
>
>[3] 
>https://github.com/openstack/nova/blob/b0854ba0c697243aa3d91170d1a22896aed60e02/nova/conductor/rpcapi.py#L215
> 
>
> 
> 
> 
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Number of IP addresses in a public network

[Igor Kalnitsky]

Hello,

My 5 cents on it.

I don't think it's really a High or Critical bug for 7.0. If there's
not enough IPs the CheckBeforeDeploymentTask will fail. And that's
actually Ok, it may fail by different reason without starting actual
deployment (sending message to Astute).

But I agree it's kinda strange that we don't check IPs during network
verification step. The good fix in my opinion is to move this check
into network checker (perhaps keep it here either), but that
definitely shouldn't be done in 7.0.

Thanks,
Igor


On Mon, Aug 31, 2015 at 2:54 PM, Roman Prykhodchenko  wrote:
> Hi folks!
>
> Recently a problem that network check does not tell whether there’s enough IP 
> addresses in a public network [1] was reported. That check is performed by 
> CheckBeforeDeployment task, but there is two problems that happen because 
> this verification is done that late:
>
>  - A deployment fails, if there’s not enough addresses in specified ranges
>  - If a user wants to get network configuration they will get an error
>
> The solution for this problems seems to be easy and a straightforward patch 
> [2] was proposed. However, there is a hidden problem which is that patch does 
> not address which is that installed plugins may reserve VIPs for their needs. 
> The issue is that they do it just before deployment and so it’s not possible 
> to get those reservations when a user wants to check their network set up.
>
> The important issue we have to address here is that network configuration 
> generator will fail, if specified ranges don’t fit all VIPs. There were 
> several proposals to fix that, I’d like to highlight two of them:
>
>  a) Allow VIPs to not have an IP address assigned, if network config 
> generator works for API output.
>  That will prevent GET requests from failure, but since IP addresses for 
> VIPs are required, generator will have to fail, if it generates a 
> configuration for the orchestrator.
>  b) Add a release note that users have to calculate IP addresses manually and 
> put sane ranges in order to not shoot their own legs. Then it’s also possible 
> to change network verification output to remind users to check the ranges 
> before starting a deployment.
>
> In my opinion we cannot follow (a) because it only masks a problem instead of 
> providing a fix. Also it requires to change the API which is not a good thing 
> to do after the SCF. If we choose (b), then we can work on a firm solution in 
> 8.0 and fix the problem for real.
>
>
> P. S. We can still merge [2], because it checks, if IP ranges can at least 
> fit the basic configuration. If you agree, I will update it soon.
>
> [1] https://bugs.launchpad.net/fuel/+bug/1487996
> [2] https://review.openstack.org/#/c/217267/
>
>
>
> - romcheg
>
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Blazar] Anyone interested?

[Sylvain Bauza]

Le 01/09/2015 06:52, Nikolay Starodubtsev a écrit :

All,
I'd like to propose use of #openstack-blazar for further communication 
and coordination.





+2 to that. That's the first step of any communication. The channel logs 
are also recorded here, for async communication :

http://eavesdrop.openstack.org/irclogs/%23openstack-blazar/

I don't see at the moment much benefits of running a weekly meeting. We 
can chat on purpose if needed.


Like I said to Ildiko, I'm fine to help some people discovering Blazar 
but I won't have time lots of time for actually working on it.


IMHO, the first things to do with Blazar is to reduce the tech debt by :
 1/ finising the Climate->Blazar renaming
 2/ updating and using the latest oslo librairies instead of using the 
old incubator
 3/ using Nova V2.1 API (which could be a bit difficult because there 
are no more extensions)


If I see some progress with Blazar, I'm OK with asking -infra to move 
Blazar to the OpenStack namespace like it was asked by James Blair here 
because it seems Blazar is not defunct :

http://lists.openstack.org/pipermail/openstack-dev/2015-August/072140.html

-Sylvain




*__*

Nikolay Starodubtsev

Software Engineer

Mirantis Inc.


Skype: dark_harlequine1


2015-08-31 19:44 GMT+03:00 Fuente, Pablo A >:


Yes, Blazar is a really interesting project. I worked on it some
time ago and I really enjoy it. Sadly my obligations at work don't
let me to still working on it, but I', happy that there still some
interest in Blazar.

Pablo.
On 31/08/15 09:19, Zhenyu Zheng wrote:
Hello,
It seems like an interesting project.

On Fri, Aug 28, 2015 at 7:54 PM, Pierre Riteau
>> wrote:
Hello,

The NSF-funded Chameleon project (https://www.chameleoncloud.org)
uses Blazar to provide advance reservations of resources for
running cloud computing experiments.

We would be interested in contributing as well.

Pierre Riteau

On 28 Aug 2015, at 07:56, Ildikó Váncsa
<>ildiko.van...@ericsson.com
>> wrote:

> Hi All,
>
> The resource reservation topic pops up time to time on different
forums to cover use cases in terms of both IT and NFV. The Blazar
project was intended to address this need, but according to my
knowledge due to earlier integration and other difficulties the
work has been stopped.
>
> My question is that who would be interested in resurrecting the
Blazar project and/or working on a reservation system in OpenStack?
>
> Thanks and Best Regards,
> Ildikó
>
>
__
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe


>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe


http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

?subject:unsubscribe>
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

Re: [openstack-dev] [Blazar] Anyone interested?

[Nikolay Starodubtsev]

Sylvain,
First of all we need to reanimate blazar gate-jobs, or we can't merge
anything. I tried to do it a year ago, but can't get the point of the tests,
so better decision can be to rewrite them from scratch.



Nikolay Starodubtsev

Software Engineer

Mirantis Inc.


Skype: dark_harlequine1

2015-09-01 11:26 GMT+03:00 Sylvain Bauza :

>
>
> Le 01/09/2015 06:52, Nikolay Starodubtsev a écrit :
>
> All,
> I'd like to propose use of #openstack-blazar for further communication and
> coordination.
>
>
>
> +2 to that. That's the first step of any communication. The channel logs
> are also recorded here, for async communication :
> http://eavesdrop.openstack.org/irclogs/%23openstack-blazar/
>
> I don't see at the moment much benefits of running a weekly meeting. We
> can chat on purpose if needed.
>
> Like I said to Ildiko, I'm fine to help some people discovering Blazar but
> I won't have time lots of time for actually working on it.
>
> IMHO, the first things to do with Blazar is to reduce the tech debt by :
>  1/ finising the Climate->Blazar renaming
>  2/ updating and using the latest oslo librairies instead of using the old
> incubator
>  3/ using Nova V2.1 API (which could be a bit difficult because there are
> no more extensions)
>
> If I see some progress with Blazar, I'm OK with asking -infra to move
> Blazar to the OpenStack namespace like it was asked by James Blair here
> because it seems Blazar is not defunct :
> http://lists.openstack.org/pipermail/openstack-dev/2015-August/072140.html
>
> -Sylvain
>
>
>
>
>
>
> Nikolay Starodubtsev
>
> Software Engineer
>
> Mirantis Inc.
>
>
> Skype: dark_harlequine1
>
> 2015-08-31 19:44 GMT+03:00 Fuente, Pablo A :
>
>> Yes, Blazar is a really interesting project. I worked on it some time ago
>> and I really enjoy it. Sadly my obligations at work don't let me to still
>> working on it, but I', happy that there still some interest in Blazar.
>>
>> Pablo.
>> On 31/08/15 09:19, Zhenyu Zheng wrote:
>> Hello,
>> It seems like an interesting project.
>>
>> On Fri, Aug 28, 2015 at 7:54 PM, Pierre Riteau > > wrote:
>> Hello,
>>
>> The NSF-funded Chameleon project (https://www.chameleoncloud.org) uses
>> Blazar to provide advance reservations of resources for running cloud
>> computing experiments.
>>
>> We would be interested in contributing as well.
>>
>> Pierre Riteau
>>
>> On 28 Aug 2015, at 07:56, Ildikó Váncsa <> ildiko.van...@ericsson.com>ildiko.van...@ericsson.com> ildiko.van...@ericsson.com>> wrote:
>>
>> > Hi All,
>> >
>> > The resource reservation topic pops up time to time on different forums
>> to cover use cases in terms of both IT and NFV. The Blazar project was
>> intended to address this need, but according to my knowledge due to earlier
>> integration and other difficulties the work has been stopped.
>> >
>> > My question is that who would be interested in resurrecting the Blazar
>> project and/or working on a reservation system in OpenStack?
>> >
>> > Thanks and Best Regards,
>> > Ildikó
>> >
>> >
>> __
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<
>> http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<
>> http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: 
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribehttp://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: 

Re: [openstack-dev] Intel PCI CI down today due to testing scripts bug

[Anita Kuno]

On 08/31/2015 10:29 PM, He, Yongli wrote:
> Hello OpenStackers!
> 
> The Intel PCI CI, due to test scripts bug( recently added) is failing to 
> report results to OpenStack Jenkins third-party tests. 
> 
> We had identify this bug and the scripts had rolled back to verified version. 
> Now we are discussing this with infra team to 
> bring it back ASAP.
> 
> Sorry for the inconvenience and your patience.
> 
> Regards
> Yongli He
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
This matter is being dealt with in this thread:
http://lists.openstack.org/pipermail/third-party-announce/2015-August/000263.html

Please let us conduct the conversation in one place on one mailing list,
in this case the third-party-announce mailing list where it was started,
rather than having split conversations.

If you want to convey the status of your third party system to the
development community, please update your wikipage.

Thank you,
Anita.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [third-party][CI] Third-party oses in devstack-gate

[Evgeny Antyshev]

Hello!

This letter I address to those third-party CI maintainers who needs to 
amend

the upstream devstack-gate to satisfy their environment.

Some folks that I know use inline patching at job level,
some make private forks of devstack-gate (I even saw one on github).
There have been a few improvements to devstack-gate, which made it 
easier to use it
downstream, f.e. introducing DEVSTACK_LOCAL_CONFIG 
(https://review.openstack.org/145321)


We particularly need it to recognize our rhel-based distribution as a 
Fedora OS.
We cannot define it explicitly in is_fedora() as it is not officially 
supported upstream,

but we can introduce the variable DEVSTACK_GATE_IS_FEDORA which makes
is_fedora() agnostic to distributions and to succeed if run on an 
undefined OS.


Here is the change: https://review.openstack.org/215029
I welcome everyone interested in the matter
to tell us if we do it right or not, and to review the change.

--
Best regards,
Evgeny Antyshev.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova][vmware] compute log files are flooded with unnecessary data

[Gary Kotton]

Hi,
The commit 
https://github.com/openstack/nova/commit/bcc002809894f39c84dca5c46034468ed0469c2b
 results in a ton of data in log files. An example is that a tempest runs is 
about 80G of data in the nova compute log file. This makes debugging issues 
terribly difficult.
I have proposed a revert of this in - https://review.openstack.org/219225
Thanks
Gary
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][db] reviewers: please mind the branch a script belongs to

[Miguel Angel Ajo]

Good reminder, I believe automation will help us most of the time. But we need 
to have a good eye on contract/expand branches,

Ihar Hrachyshka wrote:

Hi reviewers,

several days ago, a semantically expand-only migration script was merged into 
contract branch [1]. This is not a disaster, though it would be a tiny one if a 
contract-only migration script would be merged into expand branch.

Please make sure you know the new migration strategy described in [2].

Previously, we introduced a check that validates that we don’t mix 
down_revision heads, linking e.g. expand script to contract revision, or vice 
versa [3]. Apparently, it’s not enough.

Ann is looking into introducing another check for semantical correctness of 
scripts. I don’t believe it may work for all complex cases we may need to solve 
manually, but at least it should be able to catch add_* operations in contract 
scripts, or drop_* operations in expand branch. Since there may be exceptions 
to general automation, we may also need a mechanism to disable such a sanity 
check for specific scripts.

So all in all, I kindly ask everyone to become aware of how we now manage 
migration scripts, and what it implies in how we should review code (f.e. 
looking at paths as well as the code of alembic scripts). That is especially 
important before the test that Ann is looking to implement is not merged.

[1]: https://bugs.launchpad.net/neutron/+bug/1490767
[2]: http://docs.openstack.org/developer/neutron/devref/alembic_migrations.html
[3]: https://review.openstack.org/#/c/206746/

Thanks
Ihar
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][bugs] more specific tags?

[Markus Zoeller]

Matt Riedemann  wrote on 08/31/2015 10:16:23 
PM:

> From: Matt Riedemann 
> To: openstack-dev@lists.openstack.org
> Date: 08/31/2015 10:21 PM
> Subject: Re: [openstack-dev] [nova][bugs] more specific tags?
> 
> 
> 
> On 8/28/2015 10:37 AM, Markus Zoeller wrote:
> > This is a proposal to enhance the list of offical tags for our bugs
> > in Launchpad. During the tagging process in the last weeks it seems
> > to me that some of the tags are too coarse-grained. Would you see a
> > benefit in enhancing the official list to more fine-grained tags?
> >
> > [...]
> >
> > Would you see this as benefitial? If yes, which tags would you like to
> > have additionally? If no, what did I miss or overlook?
> >
> > Regards,
> > Markus Zoeller (markus_z)
> 
> Given my cynical nature, I have to ask, are the bug tags really helping 
> with focusing triage by sub-teams?  Or are we somehow pulling or 
> planning to pull this info into some analysis, like, 'we have a lot of 
> bugs in component x, use case y, so we need to refactor and/or beef up 
> testing there'.  Otherwise I feel like we're just adding more official 
> tags that very few people will be aware of or care about.
> 
> -- 
> 
> Thanks,
> 
> Matt Riedemann

If this would help the sub-teams to split things up cannot be answered
by me. I hope some of the sub-team leaders will give their opinion here.

>From a users point of view I don't care, as an example, if the libvirt
driver is (theoretically) without bugs. I'm interested if I can do a 
live-migration/snapshot/reboot/ and if I 
can do so, what are my restrictions. So, yes, the rationale behind the
proposal was indeed to get an impression where it makes sense to invest
time and effort, mainly from a use case perspective through the layers.
We have enough open bugs to be busy all the time and I don't want to 
make any orders of direction. You could see it as a navigation system.
It makes nice suggestions which way you could go, but you are free to
take another route because of personal experience or unforeseen events.

Regards,
Markus Zoeller (markus_z)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [murano] Proposing Nikolai Starodubtsev for core

[GERSHENZON, Michal (Michal)]

+1

From: Dmitro Dovbii [mailto:ddov...@mirantis.com]
Sent: Tuesday, September 01, 2015 10:03 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [murano] Proposing Nikolai Starodubtsev for core

+1

2015-09-01 2:24 GMT+03:00 Serg Melikyan 
>:
+1

On Mon, Aug 31, 2015 at 3:45 PM, Kirill Zaitsev 
> wrote:
I’m pleased to nominate Nikolai for Murano core.

He’s been actively participating in development of murano during liberty and is 
among top5 contributors during last 90 days. He’s also leading the CloudFoundry 
integration initiative.

Here are some useful links:

Overall contribution: http://stackalytics.com/?user_id=starodubcevna
List of reviews: 
https://review.openstack.org/#/q/reviewer:%22Nikolay+Starodubtsev%22,n,z
Murano contribution during latest 90 days 
http://stackalytics.com/report/contribution/murano/90

Please vote with +1/-1 for approval/objections

--
Kirill Zaitsev
Murano team
Software Engineer
Mirantis, Inc

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



--
Serg Melikyan, Senior Software Engineer at Mirantis, Inc.
http://mirantis.com | 
smelik...@mirantis.com

+7 (495) 640-4904, 0261
+7 (903) 156-0836

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [neutron][db] reviewers: please mind the branch a script belongs to

[Ihar Hrachyshka]

Hi reviewers,

several days ago, a semantically expand-only migration script was merged into 
contract branch [1]. This is not a disaster, though it would be a tiny one if a 
contract-only migration script would be merged into expand branch.

Please make sure you know the new migration strategy described in [2].

Previously, we introduced a check that validates that we don’t mix 
down_revision heads, linking e.g. expand script to contract revision, or vice 
versa [3]. Apparently, it’s not enough.

Ann is looking into introducing another check for semantical correctness of 
scripts. I don’t believe it may work for all complex cases we may need to solve 
manually, but at least it should be able to catch add_* operations in contract 
scripts, or drop_* operations in expand branch. Since there may be exceptions 
to general automation, we may also need a mechanism to disable such a sanity 
check for specific scripts.

So all in all, I kindly ask everyone to become aware of how we now manage 
migration scripts, and what it implies in how we should review code (f.e. 
looking at paths as well as the code of alembic scripts). That is especially 
important before the test that Ann is looking to implement is not merged.

[1]: https://bugs.launchpad.net/neutron/+bug/1490767
[2]: http://docs.openstack.org/developer/neutron/devref/alembic_migrations.html
[3]: https://review.openstack.org/#/c/206746/

Thanks
Ihar


signature.asc
Description: Message signed with OpenPGP using GPGMail
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] periodic task

[Gary Kotton]

On 8/31/15, 9:22 PM, "Matt Riedemann"  wrote:

>
>
>On 8/27/2015 1:22 AM, Gary Kotton wrote:
>> 
>> 
>> On 8/25/15, 2:43 PM, "Andrew Laski"  wrote:
>> 
>>> On 08/25/15 at 06:08pm, Gary Kotton wrote:


 On 8/25/15, 9:10 AM, "Matt Riedemann" 
wrote:

>
>
> On 8/25/2015 10:03 AM, Gary Kotton wrote:
>>
>>
>> On 8/25/15, 7:04 AM, "Matt Riedemann" 
>> wrote:
>>
>>>
>>>
>>> On 8/24/2015 9:32 PM, Gary Kotton wrote:
 In item #2 below the reboot is down via the guest and not the nova
 api¹s :)

 From: Gary Kotton >
 Reply-To: OpenStack List >
 Date: Monday, August 24, 2015 at 7:18 PM
 To: OpenStack List >
 Subject: [openstack-dev] [nova] periodic task

 Hi,
 A couple of months ago I posted a patch for bug
 https://launchpad.net/bugs/1463688. The issue is as follows: the
 periodic task detects that the instance state does not match the
 state
 on the hypervisor and it shuts down the running VM. There are a
 number
 of ways that this may happen and I will try and explain:

1. Vmware driver example: a host where the instances are
running
 goes
   down. This could be a power outage, host failure, etc. The
 first
   iteration of the perdioc task will determine that the actual
   instacne is down. This will update the state of the
instance to
   DOWN. The VC has the ability to do HA and it will start the
 instance
   up and running again. The next iteration of the periodic
task
 will
   determine that the instance is up and the compute manager
will
 stop
   the instance.
2. All drivers. The tenant decides to do a reboot of the
instance
 and
   that coincides with the periodic task state validation. At
this
   point in time the instance will not be up and the compute
node
 will
   update the state of the instance as DWON. Next iteration the
 states
   will differ and the instance will be shutdown

 Basically the issue hit us with our CI and there was no CI running
 for a
 couple of hours due to the fact that the compute node decided to
 shutdown the running instances. The hypervisor should be the
source
 of
 truth and it should not be the compute node that decides to
shutdown
 instances. I posted a patch to deal with this
 https://review.openstack.org/#/c/190047/. Which is the reason for
 this
 mail. The patch is backwards compatible so that the existing
 deployments
 and random shutdown continues as it works today and the admin now
 has
 an
 ability just to do a log if there is a inconsistency.

 We do not want to disable the periodic task as knowing the current
 state
 of the instance is very important and has a ton of value, we just
do
 not
 want the periodic to task to shut down a running instance.

 Thanks
 Gary




 
___
__
 __
 __
 _
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

>>>
>>> In #2 the guest shouldn't be rebooted by the user (tenant) outside
>>>of
>>> the nova-api.  I'm not sure if it's actually formally documented in
>>> the
>>> nova documentation, but from what I've always heard/known, nova is
>>> the
>>> control plane and you should be doing everything with your
>>>instances
>>> via
>>> the nova-api.  If the user rebooted via nova-api, the task_state
>>> would
>>> be set and the periodic task would ignore the instance.
>>
>> Matt, this is one case that I showed where the problem occurs. There
>> are
>> others and I can invest time to see them. The fact that the periodic
>> task
>> is there is important. What I don¹t understand is why having an
>>option
>> of
>> log indication for an admin is something that is not useful and
>> instead
>> we
>> are going 

Re: [openstack-dev] [api] [wsme] [ceilometer] Replacing WSME with _____ ?

[Chris Dent]

On Fri, 28 Aug 2015, Chris Dent wrote:


The problem with the spec is that it doesn't know what to replace
WSME with.


Thanks to everyone who provided some input.

The summary is: A lot of support for Flask. A fair bit of support
for the idea of using JSONSchema and publishing it.

Since this isn't a new API we may find that doing Flask to exactly
replicate the existing API is not possible, but if it is it seems
like a good candidate.

Thanks again.

--
Chris Dent tw:@anticdent freenode:cdent
https://tank.peermore.com/tanks/cdent

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ironic] 100% failure in pxe_ssh job

[Lucas Alvares Gomes]

Hi,

> The current failure rate for the ironic pxe_ssh job is 100% -
> http://graphite.openstack.org/render/?from=-200hours=500=now=800=ff=00=100=0=color(alias(movingAverage(asPercent(stats.zuul.pipeline.check.job.gate-tempest-dsvm-ironic-pxe_ssh.FAILURE,sum(stats.zuul.pipeline.check.job.gate-tempest-dsvm-ironic-pxe_ssh.{SUCCESS,FAILURE})),%2712hours%27),%20%27gate-tempest-dsvm-ironic-pxe_ssh%27),%27orange%27)
>
> The reason is something wrong with disk image builder and upstream ubuntu.
>
> Which raises a much more pressing issue, why is an ironic integration
> job building, from scratch a disk image builder image on every go (and
> connecting to the internet to do it)? Especially as this job sits on a
> bunch of other projects beyond ironic. Architecturally this is not sound
> enough to be a voting job.
>
> I'm proposing we make it non-voting immediately, and until it's redone
> so it's no long dependent on pulling images directly from upstream, we
> don't let it be voting.
>

Yeah, I had few time in the morning but I put a potential fix for that
problem here to test if would solve the problem:
https://review.openstack.org/#/c/219199/

This is one of the main jobs for Ironic and it would be great if we
could keep it voting. Could we perhaps change the base OS to something
else until Ubuntu is fixed ? ( Fedora / CentOS / Debian )

Cheers,
Lucas

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Debug tool for neutron

[bharath thiruveedula]

Hi,
We have some troubleshooting guides for openstack neutron. But many people who 
are new to neutron find it difficult to follow the guides, as they are not 
aware of what is happening behind the scenes. So is there any tool which tracks 
the packet flow from the VM to debug issues like why the VM  is not getting the 
IP Address or why internet is not reachable from the VM? If any, can you please 
suggest some of them?

RegardsBharath__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ironic] 100% failure in pxe_ssh job

[Sergey Nikitin]

Maybe problem in ubuntu hash sums. I tried to compare hash sums of images
from here http://cloud-images.ubuntu.com/trusty/current/ but image names in
file with sums SHA256SUMS differ from actual images names.

pxe_ssh job failing with

+ grep trusty-server-cloudimg-amd64-root.tar.gz
/opt/stack/new/.cache/image-create/SHA256SUMS.ubuntu.trusty.amd64
2015-09-01 09:01:01.512 | + sha256sum --check -
2015-09-01 09:01:01.514 | sha256sum: standard input: no properly formatted
SHA256 checksum lines found

I guess it couldn't grep name of this image from SHA256SUMS file. I pinged
ubuntu guys in IRC #ubuntu-delev. They said that it sounds like a bug.


2015-09-01 14:32 GMT+03:00 Lucas Alvares Gomes :

> Hi,
>
> > The current failure rate for the ironic pxe_ssh job is 100% -
> >
> http://graphite.openstack.org/render/?from=-200hours=500=now=800=ff=00=100=0=color(alias(movingAverage(asPercent(stats.zuul.pipeline.check.job.gate-tempest-dsvm-ironic-pxe_ssh.FAILURE,sum(stats.zuul.pipeline.check.job.gate-tempest-dsvm-ironic-pxe_ssh.{SUCCESS,FAILURE})),%2712hours%27),%20%27gate-tempest-dsvm-ironic-pxe_ssh%27),%27orange%27)
> >
> > The reason is something wrong with disk image builder and upstream
> ubuntu.
> >
> > Which raises a much more pressing issue, why is an ironic integration
> > job building, from scratch a disk image builder image on every go (and
> > connecting to the internet to do it)? Especially as this job sits on a
> > bunch of other projects beyond ironic. Architecturally this is not sound
> > enough to be a voting job.
> >
> > I'm proposing we make it non-voting immediately, and until it's redone
> > so it's no long dependent on pulling images directly from upstream, we
> > don't let it be voting.
> >
>
> Yeah, I had few time in the morning but I put a potential fix for that
> problem here to test if would solve the problem:
> https://review.openstack.org/#/c/219199/
>
> This is one of the main jobs for Ironic and it would be great if we
> could keep it voting. Could we perhaps change the base OS to something
> else until Ubuntu is fixed ? ( Fedora / CentOS / Debian )
>
> Cheers,
> Lucas
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [murano] Proposing Nikolai Starodubtsev for core

[Victor Ryzhenkin]

+1 from me ;)

-- 
Victor Ryzhenkin
Junior QA Engeneer
freerunner on #freenode

Включено 1 сентября 2015 г. в 12:18:19, Ekaterina Chernova 
(efedor...@mirantis.com) написал:

+1

On Tue, Sep 1, 2015 at 10:03 AM, Dmitro Dovbii  wrote:
+1

2015-09-01 2:24 GMT+03:00 Serg Melikyan :
+1

On Mon, Aug 31, 2015 at 3:45 PM, Kirill Zaitsev  wrote:
I’m pleased to nominate Nikolai for Murano core.

He’s been actively participating in development of murano during liberty and is 
among top5 contributors during last 90 days. He’s also leading the CloudFoundry 
integration initiative.

Here are some useful links:

Overall contribution: http://stackalytics.com/?user_id=starodubcevna
List of reviews: 
https://review.openstack.org/#/q/reviewer:%22Nikolay+Starodubtsev%22,n,z
Murano contribution during latest 90 days 
http://stackalytics.com/report/contribution/murano/90

Please vote with +1/-1 for approval/objections

-- 
Kirill Zaitsev
Murano team
Software Engineer
Mirantis, Inc

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




--
Serg Melikyan, Senior Software Engineer at Mirantis, Inc.
http://mirantis.com | smelik...@mirantis.com

+7 (495) 640-4904, 0261
+7 (903) 156-0836

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__  
OpenStack Development Mailing List (not for usage questions)  
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe  
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev  
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [murano] Proposing Nikolai Starodubtsev for core

[Alexander Tivelkov]

+1. Well deserved.

--
Regards,
Alexander Tivelkov

On Tue, Sep 1, 2015 at 2:47 PM, Victor Ryzhenkin 
wrote:

> +1 from me ;)
>
> --
> Victor Ryzhenkin
> Junior QA Engeneer
> freerunner on #freenode
>
> Включено 1 сентября 2015 г. в 12:18:19, Ekaterina Chernova (
> efedor...@mirantis.com) написал:
>
> +1
>
> On Tue, Sep 1, 2015 at 10:03 AM, Dmitro Dovbii 
> wrote:
>
>> +1
>>
>> 2015-09-01 2:24 GMT+03:00 Serg Melikyan :
>>
>>> +1
>>>
>>> On Mon, Aug 31, 2015 at 3:45 PM, Kirill Zaitsev 
>>> wrote:
>>>
 I’m pleased to nominate Nikolai for Murano core.

 He’s been actively participating in development of murano during
 liberty and is among top5 contributors during last 90 days. He’s also
 leading the CloudFoundry integration initiative.

 Here are some useful links:

 Overall contribution: http://stackalytics.com/?user_id=starodubcevna
 List of reviews:
 https://review.openstack.org/#/q/reviewer:%22Nikolay+Starodubtsev%22,n,z
 Murano contribution during latest 90 days
 http://stackalytics.com/report/contribution/murano/90

 Please vote with +1/-1 for approval/objections

 --
 Kirill Zaitsev
 Murano team
 Software Engineer
 Mirantis, Inc


 __
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
 openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


>>>
>>>
>>> --
>>> Serg Melikyan, Senior Software Engineer at Mirantis, Inc.
>>> http://mirantis.com | smelik...@mirantis.com
>>>
>>> +7 (495) 640-4904, 0261
>>> +7 (903) 156-0836
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [ironic] 100% failure in pxe_ssh job

[Sean Dague]

The current failure rate for the ironic pxe_ssh job is 100% -
http://graphite.openstack.org/render/?from=-200hours=500=now=800=ff=00=100=0=color(alias(movingAverage(asPercent(stats.zuul.pipeline.check.job.gate-tempest-dsvm-ironic-pxe_ssh.FAILURE,sum(stats.zuul.pipeline.check.job.gate-tempest-dsvm-ironic-pxe_ssh.{SUCCESS,FAILURE})),%2712hours%27),%20%27gate-tempest-dsvm-ironic-pxe_ssh%27),%27orange%27)

The reason is something wrong with disk image builder and upstream ubuntu.

Which raises a much more pressing issue, why is an ironic integration
job building, from scratch a disk image builder image on every go (and
connecting to the internet to do it)? Especially as this job sits on a
bunch of other projects beyond ironic. Architecturally this is not sound
enough to be a voting job.

I'm proposing we make it non-voting immediately, and until it's redone
so it's no long dependent on pulling images directly from upstream, we
don't let it be voting.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] FW: [cinder] Microsoft CI Still Disabled

[Duncan Thomas]

Mike is on vacation at the moment, somebody else will evaluate in his stead
as reply here shortly

On 1 September 2015 at 14:20, Octavian Ciuhandu <
ociuha...@cloudbasesolutions.com> wrote:

> Hi Mike,
>
> I think the first mail was sent only on openstack-dev. We have had already
> over
> a week of reliable results on the Microsoft Cinder CI on all three jobs,
> we have
> an updated set of results available at [1].
>
> Please re-evaluate the activity of our CI and re-enable the gerrit account.
>
> Thank you,
>
> Octavian.
>
> [1]: http://paste.openstack.org/show/437509/
>
>
>
>
>
> On 24/8/15 20:19, "Octavian Ciuhandu" 
> wrote:
>
> >Hi Mike,
> >
> >We have stabilised the Microsoft Cinder CI and ensured dry-runs testing
> over
> >the past days.
> >
> >The reason why we didnt report earlier on the status is that we were
> waiting
> >for the completion of a series of hardware maintenance activities
> performed in
> >the Microsoft datacenter where the CI is running, as metioned by Hashir
> in the
> >previous email.
> >
> >In the meantime we also improved the CI software infrastructure,
> including:
> >
> >1. Adding local pip and deb cache servers to reduce transient package
> download
> >issues 2. Improved resiliency in the under and overcloud deployment
> scripts [1]
> >3. Improved monitoring to reduce the respose time on transient issues,
> >especially for undercloud networking which was the source of a significant
> >amount of failures due to a switch misconfiguration in the datacenter,
> recently
> >identified and solved.
> >
> >We have a list of recent runs for all 3 drivers available at [2].
> >
> >Please let us know if there’s anything else that prevents re-enabling the
> >Microsoft Cinder CI account.
> >
> >Thank you,
> >
> >Octavian.
> >
> >[1]: https://github.com/cloudbase/cinder-ci
> >[2]: http://paste.openstack.org/show/426144/
> >
> >
> >
> >
> >
> >
> >On 24/8/15 16:48, "Hashir Abdi"  wrote:
> >
> >>Mike:
> >>
> >>Apologies for the delayed response.
> >>
> >>We have been addressing bug-fixes and infrastructure changes for our CI,
> while working through vacation schedules.
> >>
> >>We have now been able to conduct successful dry runs of our CI, and I'll
> let Octavian follow up here with those latest successful results.
> >>
> >>Sincerely
> >>
> >>Hashir Abdi
> >>
> >>Principal Software Engineering Manager
> >>Microsoft
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>-Original Message-
> >>From: Mike Perez [mailto:thin...@gmail.com]
> >>Sent: Friday, August 21, 2015 8:39 PM
> >>To: OpenStack Development Mailing List <
> openstack-dev@lists.openstack.org>
> >>Cc: Microsoft Cinder CI ; Octavian
> Ciuhandu 
> >>Subject: [cinder] Microsoft CI Still Disabled
> >>
> >>The Microsoft CI has been disabled since July 22nd [1].
> >>
> >>Last I heard from the Cinder midcycle sprint, this CI was still not
> ready and it hasn't been for 30 days now.
> >>
> >>Where are we with things, and why has communication been so poor with
> the Cloud Base solutions team?
> >>
> >>[1] -
> https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2flists.openstack.org%2fpipermail%2fthird-party-announce%2f2015-July%2f000249.html=01%7c01%7chabdi%40exchange.microsoft.com%7c33ba5a82486e4dc6d06508d2aa8a115d%7c72f988bf86f141af91ab2d7cd011db47%7c1=mPalAeUjUVBu9sVn6fDnehA4TN3uZOUB4DAKS6qcFaU%3d
> >>
> >>--
> >>Mike Perez
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
-- 
Duncan Thomas
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] FW: [cinder] Microsoft CI Still Disabled

[Duncan Thomas]

Mike is on vacation at the moment, somebody else will evaluate in his stead
Hi Mike,

I think the first mail was sent only on openstack-dev. We have had already
over
a week of reliable results on the Microsoft Cinder CI on all three jobs, we
have
an updated set of results available at [1].

Please re-evaluate the activity of our CI and re-enable the gerrit account.

Thank you,

Octavian.

[1]: http://paste.openstack.org/show/437509/





On 24/8/15 20:19, "Octavian Ciuhandu" 
wrote:

>Hi Mike,
>
>We have stabilised the Microsoft Cinder CI and ensured dry-runs testing
over
>the past days.
>
>The reason why we didnt report earlier on the status is that we were
waiting
>for the completion of a series of hardware maintenance activities
performed in
>the Microsoft datacenter where the CI is running, as metioned by Hashir in
the
>previous email.
>
>In the meantime we also improved the CI software infrastructure, including:
>
>1. Adding local pip and deb cache servers to reduce transient package
download
>issues 2. Improved resiliency in the under and overcloud deployment
scripts [1]
>3. Improved monitoring to reduce the respose time on transient issues,
>especially for undercloud networking which was the source of a significant
>amount of failures due to a switch misconfiguration in the datacenter,
recently
>identified and solved.
>
>We have a list of recent runs for all 3 drivers available at [2].
>
>Please let us know if there’s anything else that prevents re-enabling the
>Microsoft Cinder CI account.
>
>Thank you,
>
>Octavian.
>
>[1]: https://github.com/cloudbase/cinder-ci
>[2]: http://paste.openstack.org/show/426144/
>
>
>
>
>
>
>On 24/8/15 16:48, "Hashir Abdi"  wrote:
>
>>Mike:
>>
>>Apologies for the delayed response.
>>
>>We have been addressing bug-fixes and infrastructure changes for our CI,
while working through vacation schedules.
>>
>>We have now been able to conduct successful dry runs of our CI, and I'll
let Octavian follow up here with those latest successful results.
>>
>>Sincerely
>>
>>Hashir Abdi
>>
>>Principal Software Engineering Manager
>>Microsoft
>>
>>
>>
>>
>>
>>
>>
>>-Original Message-
>>From: Mike Perez [mailto:thin...@gmail.com]
>>Sent: Friday, August 21, 2015 8:39 PM
>>To: OpenStack Development Mailing List 
>>Cc: Microsoft Cinder CI ; Octavian
Ciuhandu 
>>Subject: [cinder] Microsoft CI Still Disabled
>>
>>The Microsoft CI has been disabled since July 22nd [1].
>>
>>Last I heard from the Cinder midcycle sprint, this CI was still not ready
and it hasn't been for 30 days now.
>>
>>Where are we with things, and why has communication been so poor with the
Cloud Base solutions team?
>>
>>[1] -
https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2flists.openstack.org%2fpipermail%2fthird-party-announce%2f2015-July%2f000249.html=01%7c01%7chabdi%40exchange.microsoft.com%7c33ba5a82486e4dc6d06508d2aa8a115d%7c72f988bf86f141af91ab2d7cd011db47%7c1=mPalAeUjUVBu9sVn6fDnehA4TN3uZOUB4DAKS6qcFaU%3d
>>
>>--
>>Mike Perez
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [magnum] Difference between certs stored in keystone and certs stored in barbican

[John Dennis]

On 09/01/2015 02:49 AM, Tim Bell wrote:

Will it also be possible to use a different CA ? In some
environments, there is already a corporate certificate authority
server. This would ensure compliance with site security standards.


A configurable CA was one of the original design goals when the Barbican 
work began. I have not tracked the work so I don't know if that is still 
the case, Ade Lee would know for sure.


--
John

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [cinder] L3 low pri review queue starvation

[Tom Barron]

[Yesterday while discussing the following issue on IRC, jgriffith
suggested that I post to the dev list in preparation for a discussion in
Wednesday's cinder meeting.]

Please take a look at the 10 "Low" priority reviews in the cinder
Liberty 3 etherpad that were punted to Mitaka yesterday. [1]

Six of these *never* [2] received a vote from a core reviewer. With the
exception of the first in the list, which has 35 patch sets, none of the
others received a vote before Friday, August 28.  Of these, none had
more than -1s on minor issues, and these have been remedied.

Review https://review.openstack.org/#/c/213855 "Implement
manage/unmanage snapshot in Pure drivers" is a great example:

   * approved blueprint for a valuable feature
   * pristine code
   * passes CI and Jenkins (and by the deadline)
   * never reviewed

We have 11 core reviewers, all of whom were very busy doing reviews
during L3, but evidently this set of reviews didn't really have much
chance of making it.  This looks like a classic case where the
individually rational priority decisions of each core reviewer
collectively resulted in starving the Low Priority review queue.

One way to remedy would be for the 11 core reviewers to devote a day or
two to cleaning up this backlog of 10 outstanding reviews rather than
punting all of them out to Mitaka.

Thanks for your time and consideration.

Respectfully,

-- Tom Barron

[1] https://etherpad.openstack.org/p/cinder-liberty-3-reviews
[2] At the risk of stating the obvious, in this count I ignore purely
procedural votes such as the final -2.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Heat] convergence rally test results (so far)

[Steven Hardy]

On Fri, Aug 28, 2015 at 01:35:52AM +, Angus Salkeld wrote:
>Hi
>I have been running some rally tests against convergence and our existing
>implementation to compare.
>So far I have done the following:
> 1. defined a template with a resource
>groupA 
> https://github.com/asalkeld/convergence-rally/blob/master/templates/resource_group_test_resource.yaml.template
> 2. the inner resource looks like
>this:A 
> https://github.com/asalkeld/convergence-rally/blob/master/templates/server_with_volume.yaml.templateA
>  (it
>uses TestResource to attempt to be a reasonable simulation of a
>server+volume+floatingip)
> 3. defined a rally
>job:A 
> https://github.com/asalkeld/convergence-rally/blob/master/increasing_resources.yamlA
>  that
>creates X resources then updates to X*2 then deletes.
> 4. I then ran the above with/without convergence and with 2,4,8
>heat-engines
>Here are the results compared:
>
> https://docs.google.com/spreadsheets/d/12kRtPsmZBl_y78aw684PTBg3op1ftUYsAEqXBtT800A/edit?usp=sharing
>Some notes on the results so far:
>  * A convergence with only 2 engines does suffer from RPC overload (it
>gets message timeouts on larger templates). I wonder if this is the
>problem in our convergence gate...
>  * convergence does very well with a reasonable number of engines
>running.
>  * delete is slightly slower on convergence
>Still to test:
>  * the above, but measure memory usage
>  * many small templates (run concurrently)

So, I tried running my many-small-templates here with convergence enabled:

https://bugs.launchpad.net/heat/+bug/1489548

In heat.conf I set:

max_resources_per_stack = -1
convergence_engine = true

Most other settings (particularly RPC and DB settings) are defaults.

Without convergence (but with max_resources_per_stack disabled) I see the
time to create a ResourceGroup of 400 nested stacks (each containing one
RandomString resource) is about 2.5 minutes (core i7 laptop w/SSD, 4 heat
workers e.g the default for a 4 core machine).

With convergence enabled, I see these errors from sqlalchemy:

File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 652, in
_checkout\nfairy = _ConnectionRecord.checkout(pool)\n', u'  File
"/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 444, in
checkout\nrec = pool._do_get()\n', u'  File
"/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 980, in
_do_get\n(self.size(), self.overflow(), self._timeout))\n',
u'TimeoutError: QueuePool limit of size 5 overflow 10 reached, connection
timed out, timeout 30\n'].

I assume this means we're loading the DB much more in the convergence case
and overflowing the QueuePool?

This seems to happen when the RPC call from the ResourceGroup tries to
create some of the 400 nested stacks.

Interestingly after this error, the parent stack moves to CREATE_FAILED,
but the engine remains (very) busy, to the point of being partially
responsive, so it looks like maybe the cancel-on-fail isnt' working (I'm
assuming it isn't error_wait_time because the parent stack has been marked
FAILED and I'm pretty sure it's been more than 240s).

I'll dig a bit deeper when I get time, but for now you might like to try
the stress test too.  It's a bit of a synthetic test, but it turns out to
be a reasonable proxy for some performance issues we observed when creating
large-ish TripleO deployments (which also create a large number of nested
stacks concurrently).

Steve

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [ec2api][puppet] EC2 api puppet module

[Marcos Fermin Lobo]

Hi all,

The standalone EC2 api project https://github.com/stackforge/ec2-api does not 
have puppet module yet. I want to develop this puppet module and my idea is 
start in a public Github repo and get feedback from the community. All feedback 
and collaborations will be very welcome.

I would like to start requesting your suggestions about the Github repository 
name. I would suggest: puppet-ec2-api

Sounds good for you people?, more suggestions?.

Thank you.

Regards,
Marcos.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Ironic] Command structure for OSC plugin

[Tim Bell]

That would be great to have plugins on the commands which are relevant to 
multiple projects… avoiding exposing all of the underlying projects as prefixes 
and getting more consistency would be very appreciated by the users.

Tim

From: Dean Troyer [mailto:dtro...@gmail.com]
Sent: 01 September 2015 22:47
To: OpenStack Development Mailing List (not for usage questions) 

Subject: Re: [openstack-dev] [Ironic] Command structure for OSC plugin

[late catch-up]

On Mon, Aug 24, 2015 at 2:56 PM, Doug Hellmann 
> wrote:
Excerpts from Brad P. Crochet's message of 2015-08-24 15:35:59 -0400:
> On 24/08/15 18:19 +, Tim Bell wrote:
> >
> >From a user perspective, where bare metal and VMs are just different flavors 
> >(with varying capabilities), can we not use the same commands (server 
> >create/rebuild/...) ? Containers will create the same conceptual problems.
> >
> >OSC can provide a converged interface but if we just replace '$ ironic ' 
> >by '$ openstack baremetal ', this seems to be a missed opportunity to 
> >hide the complexity from the end user.
> >
> >Can we re-use the existing server structures ?

I've wondered about how users would see doing this, we've done it already with 
the quota and limits commands (blurring the distinction between project APIs).  
At some level I am sure users really do not care about some of our project 
distinctions.


> To my knowledge, overriding or enhancing existing commands like that
> is not possible.

You would have to do it in tree, by making the existing commands
smart enough to talk to both nova and ironic, first to find the
server (which service knows about something with UUID XYZ?) and
then to take the appropriate action on that server using the right
client. So it could be done, but it might lose some of the nuance
between the server types by munging them into the same command. I
don't know what sorts of operations are different, but it would be
worth doing the analysis to see.

I do have an experimental plugin that hooks the server create command to add 
some options and change its behaviour so it is possible, but right now I 
wouldn't call it supported at all.  That might be something that we could 
consider doing though for things like this.

The current model for commands calling multiple project APIs is to put them in 
openstackclient.common, so yes, in-tree.

Overall, though, to stay consistent with OSC you would map operations into the 
current verbs as much as possible.  It is best to think in terms of how the CLI 
user is thinking and what she wants to do, and not how the REST or Python API 
is written.  In this case, 'baremetal' is a type of server, a set of attributes 
of a server, etc.  As mentioned earlier, containers will also have a similar 
paradigm to consider.

dt

--

Dean Troyer
dtro...@gmail.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Heat] convergence rally test results (so far)

[Anant Patil]

When the stack fails, it is marked as FAILED and all the sync points
that are needed to trigger the next set of resources are deleted. The
resources at same level in the graph, like here, they are suppose to
timeout or fail for an exception. Many DB hits means that the cache
data we were maintaining is not being used in the way we intended.

I don't see if really need 1; if it works with legacy w/o putting any such
constraints, it should work with convergence as well.

--
Anant

On Wed, Sep 2, 2015 at 5:23 AM, Angus Salkeld  wrote:

> On Tue, Sep 1, 2015 at 10:45 PM Steven Hardy  wrote:
>
>> On Fri, Aug 28, 2015 at 01:35:52AM +, Angus Salkeld wrote:
>> >Hi
>> >I have been running some rally tests against convergence and our
>> existing
>> >implementation to compare.
>> >So far I have done the following:
>> > 1. defined a template with a resource
>> >groupA
>> https://github.com/asalkeld/convergence-rally/blob/master/templates/resource_group_test_resource.yaml.template
>> > 2. the inner resource looks like
>> >this:A
>> https://github.com/asalkeld/convergence-rally/blob/master/templates/server_with_volume.yaml.templateA
>> (it
>> >uses TestResource to attempt to be a reasonable simulation of a
>> >server+volume+floatingip)
>> > 3. defined a rally
>> >job:A
>> https://github.com/asalkeld/convergence-rally/blob/master/increasing_resources.yamlA
>> that
>> >creates X resources then updates to X*2 then deletes.
>> > 4. I then ran the above with/without convergence and with 2,4,8
>> >heat-engines
>> >Here are the results compared:
>> >
>> https://docs.google.com/spreadsheets/d/12kRtPsmZBl_y78aw684PTBg3op1ftUYsAEqXBtT800A/edit?usp=sharing
>> >Some notes on the results so far:
>> >  * A convergence with only 2 engines does suffer from RPC overload
>> (it
>> >gets message timeouts on larger templates). I wonder if this is
>> the
>> >problem in our convergence gate...
>> >  * convergence does very well with a reasonable number of engines
>> >running.
>> >  * delete is slightly slower on convergence
>> >Still to test:
>> >  * the above, but measure memory usage
>> >  * many small templates (run concurrently)
>>
>> So, I tried running my many-small-templates here with convergence enabled:
>>
>> https://bugs.launchpad.net/heat/+bug/1489548
>>
>> In heat.conf I set:
>>
>> max_resources_per_stack = -1
>> convergence_engine = true
>>
>> Most other settings (particularly RPC and DB settings) are defaults.
>>
>> Without convergence (but with max_resources_per_stack disabled) I see the
>> time to create a ResourceGroup of 400 nested stacks (each containing one
>> RandomString resource) is about 2.5 minutes (core i7 laptop w/SSD, 4 heat
>> workers e.g the default for a 4 core machine).
>>
>> With convergence enabled, I see these errors from sqlalchemy:
>>
>> File "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 652, in
>> _checkout\nfairy = _ConnectionRecord.checkout(pool)\n', u'  File
>> "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 444, in
>> checkout\nrec = pool._do_get()\n', u'  File
>> "/usr/lib64/python2.7/site-packages/sqlalchemy/pool.py", line 980, in
>> _do_get\n(self.size(), self.overflow(), self._timeout))\n',
>> u'TimeoutError: QueuePool limit of size 5 overflow 10 reached, connection
>> timed out, timeout 30\n'].
>>
>> I assume this means we're loading the DB much more in the convergence case
>> and overflowing the QueuePool?
>>
>
> Yeah, looks like it.
>
>
>>
>> This seems to happen when the RPC call from the ResourceGroup tries to
>> create some of the 400 nested stacks.
>>
>> Interestingly after this error, the parent stack moves to CREATE_FAILED,
>> but the engine remains (very) busy, to the point of being partially
>> responsive, so it looks like maybe the cancel-on-fail isnt' working (I'm
>> assuming it isn't error_wait_time because the parent stack has been marked
>> FAILED and I'm pretty sure it's been more than 240s).
>>
>> I'll dig a bit deeper when I get time, but for now you might like to try
>> the stress test too.  It's a bit of a synthetic test, but it turns out to
>> be a reasonable proxy for some performance issues we observed when
>> creating
>> large-ish TripleO deployments (which also create a large number of nested
>> stacks concurrently).
>>
>
> Thanks a lot for testing Steve! I'll make 2 bugs for what you have raised
> 1. limit the number of resource actions in parallel (maybe base on the
> number of cores)
> 2. the cancel on fail error
>
> -Angus
>
>
>>
>> Steve
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> 

Re: [openstack-dev] [Heat] convergence rally test results (so far)

[Robert Collins]

On 2 September 2015 at 11:53, Angus Salkeld  wrote:

> 1. limit the number of resource actions in parallel (maybe base on the
> number of cores)

I'm having trouble mapping that back to 'and heat-engine is running on
3 separate servers'.

-Rob

-- 
Robert Collins 
Distinguished Technologist
HP Converged Cloud

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [mistral] Displaying wf hierarchy in CLI

[Lingxian Kong]

Hi, Renat,

Actually, I have the same idea months ago, but what I thought is to provide
task dependencies information in a workflow definition, since as a workflow
designer, I have no idea about how my workflow 'looks like', unless I
create an execution with that, especially when there are a lot of tasks
within workflow definition.

And what you want to address here is to get execution/task-execution
dependencies information in run time, which also will give users more
detailed information about what happened in the system, I'd like to see it
landed in Mistral.

To achieve that, we should record the execution/task-execution relationship
during an execution is running, because we have no such info currently.

On the CLI side, I agree we add option to 'execution-get' command, and
accordingly we could add a new column 'dependent on' or 'parent' or
something else to the result.

On Tue, Sep 1, 2015 at 1:47 AM, Joshua Harlow  wrote:

> Would u guys have any use for the following being split out into its own
> library?
>
> https://github.com/openstack/taskflow/blob/master/taskflow/types/tree.py
>
> It already has a pformat method that could be used to do your drawing of
> the 'tree'...
>
>
> http://docs.openstack.org/developer/taskflow/types.html#taskflow.types.tree.Node.pformat
>
> Might be useful for u folks? Taskflow uses it to be able to show
> information that is tree-like to the developer/user for similar purposes
> (it also supports using pydot to dump things out in dot graph format):
>
> For example http://tempsend.com/A8AA89F397/4663/car.pdf is the graph of
> an example (in particular
> https://github.com/openstack/taskflow/blob/master/taskflow/examples/build_a_car.py
> )
>
> -Josh
>
> Renat Akhmerov wrote:
>
>> Team,
>>
>> I’d like to discuss
>> https://blueprints.launchpad.net/mistral/+spec/mistral-execution-origin.
>>
>> To summarize what it’s about: imagine that we have a workflow which
>> calls other workflows and those workflows call some workflows again,
>> etc. etc. In other words, we have a tree of workflows. Right now there
>> isn’t a convenient way to track down the whole execution tree in CLI.
>> For example, see a running workflow but I have no idea whether it was
>> started by user manually or called by another (parent) workflow. In many
>> cases it’s crucial to know, otherwise it’s really painful if we need to
>> debug something or just figure out the whole picture of what’s going on.
>>
>> What this BP offers is that we have an “origin ID” that would always
>> tell the top level (the highest one) workflow execution since which it
>> all started. This is kind of simple solution though and I thought we
>> could massage this idea a little bit and could come up with something
>> more interesting. For example, could we add a new option (i.e.
>> --detailed or --recursive) for ‘mistral execution-get’ command and if
>> it’s provided then we print out information not only about this wf
>> execution itself but about it’s children as well? The only question is:
>> how do we display a tree in CLI?
>>
>> I also created an empty etherpad where we can sketch out how it could
>> look like:
>> https://etherpad.openstack.org/p/mistral-cli-workflow-execution-tree
>>
>> Any other ideas? Thoughts?
>>
>> Renat Akhmerov
>> @ Mirantis Inc.
>>
>>
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
*Regards!*
*---*
*Lingxian Kong*
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ec2api][puppet] EC2 api puppet module

[Alex Schultz]

Hey Marcos,

On Tue, Sep 1, 2015 at 7:50 AM, Marcos Fermin Lobo <
marcos.fermin.l...@cern.ch> wrote:

> Hi all,
>
> The standalone EC2 api project https://github.com/stackforge/ec2-api does
> not have puppet module yet. I want to develop this puppet module and my
> idea is start in a public Github repo and get feedback from the community.
> All feedback and collaborations will be very welcome.
>
> I would like to start requesting your suggestions about the Github
> repository name. I would suggest: puppet-ec2-api
>
>

Just a note I don't think dashes are allowed in the puppet module name or
if they are it may lead to some weirdness[0][1]. So puppet-ec2api might be
better.




> Sounds good for you people?, more suggestions?.
>
> Thank you.
>
> Regards,
> Marcos.
>
>

Thanks,
-Alex



[0] https://projects.puppetlabs.com/issues/5268
[1] https://tickets.puppetlabs.com/browse/PUP-1397
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [horizon] URL Sanity

[Rob Cresswell (rcresswe)]

Hi all,

I recently started looking into properly implementing breadcrumbs to make 
navigation clearer, especially around nested resources (Subnets Detail page, 
for example). The idea is to use the request.path to form a logical breadcrumb 
that isn’t dependent on browser history ( 
https://review.openstack.org/#/c/129985/3/horizon/browsers/breadcrumb.py ). 
Unfortunately, this breaks down quite quickly because we use odd patterns like 
`//detail`, and `/` doesn’t 
exist.

This made me realise how much of an inconsistent mess the URL patterns are.  
I’ve started cleaning them up, so we move from these patterns:

`/admin/networks//detail` - Detail page for a Network
`/admin/networks//addsubnet` - Create page for a Subnet

To patterns in line with usual CRUD usages, such as:

`/admin/networks/` - Detail page for a Network
`/admin/networks//subnets/create` - Create page for a Subnet

This is mostly trivial, just removing extraneous words and adding consistency, 
with end goal being every panel following patterns like:

`/` - Index page
`//` - Detail page for a single resource
`//create` - Create new resource
`///update` - Update a resource

This gets a little complex around nested items. Should a Port for example, 
which has a unique ID, be reachable in Horizon by just its ID? Ports must 
always be attached to a network as I understand it. There are multiple ways to 
express this:

`/networks/ports/` - Current implementation
`/networks//ports/` - My preferred structure
`/ports/` - Another option

Does anyone have any opinions on how to handle this structuring, or if it’s 
even necessary?

Regards,
Rob
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ec2api][puppet] EC2 api puppet module

[Emilien Macchi]

On 09/01/2015 08:50 AM, Marcos Fermin Lobo wrote:
> Hi all,
> 
> The standalone EC2 api project https://github.com/stackforge/ec2-api
> does not have puppet module yet. I want to develop this puppet module
> and my idea is start in a public Github repo and get feedback from the
> community. All feedback and collaborations will be very welcome.
> 
> I would like to start requesting your suggestions about the Github
> repository name. I would suggest: puppet-ec2-api
> 
> Sounds good for you people?, more suggestions?.

I would look at https://wiki.openstack.org/wiki/Puppet/New_module
For hosting place, I guess using 'openstack' namespace is fine.

Ping us on IRC #puppet-openstack if you need more informations,

Best,

> 
> Thank you.
> 
> Regards,
> Marcos.
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [magnum] Difference between certs stored in keystone and certs stored in barbican

[Clark, Robert Graham]

>The reason that is compelling is that you can have Barbican generate,
>sign, and store a keypair without transmitting the private key over the
>network to the client that originates the signing request. It can be
>directly stored, and made available only to the clients that need access
>to it.

This is absolutely _not_ how PKI for TLS is supposed to work, yes Barbican
can create keypairs etc because sometimes that¹s useful but in the
public-private PKI model that TLS expects this is completely wrong. Magnum
nodes should be creating their own private key and CSR and submitting them
to some CA for signing.

Now this gets messy because you probably don¹t want to push keystone
credentials onto each node (that they would use to communicate with
Barbican).

I¹m a bit conflicted writing this next bit because I¹m not particularly
familiar with the Kubernetes/Magnum architectures and also because I¹m one
of the core developers for Anchor but here goesŠ.

Have you considered using Anchor for this? It¹s a pretty lightweight
ephemeral CA that is built to work well in small PKI communities (like a
Kubernetes cluster) you can configure multiple methods for authentication
and build pretty simple validation rules for deciding if a host should be
given a certificate. Anchor is built to provide short-lifetime
certificates where each node re-requests a certificate typically every
12-24 hours, this has some really nice properties like ³passive
revocation² (Think revocation that actually works) and strong ways to
enforce issuing logic on a per host basis.

Anchor or not, I¹d like to talk to you more about how you¹re attempting to
secure Magnum - I think it¹s an extremely interesting project that I¹d
like to help out with.

-Rob
(Security Project PTL / Anchor flunkie)



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova][vmware][qa] vmware nsx CI appears gone

[Matt Riedemann]

I haven't seen the vmware nsx CI reporting on anything in awhile but 
don't see any outage events here:


https://wiki.openstack.org/wiki/NovaVMware/Minesweeper/Status

Is there some status?

--

Thanks,

Matt Riedemann


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ironic] 100% failure in pxe_ssh job

[Jim Rollenhagen]

On Tue, Sep 01, 2015 at 07:17:56AM -0400, Sean Dague wrote:
> The current failure rate for the ironic pxe_ssh job is 100% -
> http://graphite.openstack.org/render/?from=-200hours=500=now=800=ff=00=100=0=color(alias(movingAverage(asPercent(stats.zuul.pipeline.check.job.gate-tempest-dsvm-ironic-pxe_ssh.FAILURE,sum(stats.zuul.pipeline.check.job.gate-tempest-dsvm-ironic-pxe_ssh.{SUCCESS,FAILURE})),%2712hours%27),%20%27gate-tempest-dsvm-ironic-pxe_ssh%27),%27orange%27)
> 
> The reason is something wrong with disk image builder and upstream ubuntu.
> 
> Which raises a much more pressing issue, why is an ironic integration
> job building, from scratch a disk image builder image on every go (and
> connecting to the internet to do it)? Especially as this job sits on a
> bunch of other projects beyond ironic. Architecturally this is not sound
> enough to be a voting job.
> 
> I'm proposing we make it non-voting immediately, and until it's redone
> so it's no long dependent on pulling images directly from upstream, we
> don't let it be voting.

Quick update here after talking with Sean in IRC:

* pxe_ssh job has been made non-voting.

* We're switching to pxe_ipa job for Nova. This job uses a prebuilt
  ironic-python-agent ramdisk from tarballs.o.o, and is really what we
  want anyway since we're deprecating the bash ramdisk. This will remain
  non-voting.

* mtreinish and I are going to work on getting the full tempest job
  working for the Ironic driver, as has been the plan basically forever.
  Would love some extra hands if anyone is interested. :)

// jim

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ironic] 100% failure in pxe_ssh job

[Julia Kreger]

On Tue, Sep 1, 2015 at 9:17 AM, Jim Rollenhagen 
wrote:

>
>
> > On Sep 1, 2015, at 04:32, Lucas Alvares Gomes 
> wrote:
> >
> > Hi,
> >
> >> The current failure rate for the ironic pxe_ssh job is 100% -
> >>
> http://graphite.openstack.org/render/?from=-200hours=500=now=800=ff=00=100=0=color(alias(movingAverage(asPercent(stats.zuul.pipeline.check.job.gate-tempest-dsvm-ironic-pxe_ssh.FAILURE,sum(stats.zuul.pipeline.check.job.gate-tempest-dsvm-ironic-pxe_ssh.{SUCCESS,FAILURE})),%2712hours%27),%20%27gate-tempest-dsvm-ironic-pxe_ssh%27),%27orange%27)
> >>
> >> The reason is something wrong with disk image builder and upstream
> ubuntu.
> >>
> >> Which raises a much more pressing issue, why is an ironic integration
> >> job building, from scratch a disk image builder image on every go (and
> >> connecting to the internet to do it)? Especially as this job sits on a
> >> bunch of other projects beyond ironic. Architecturally this is not sound
> >> enough to be a voting job.
> >>
> >> I'm proposing we make it non-voting immediately, and until it's redone
> >> so it's no long dependent on pulling images directly from upstream, we
> >> don't let it be voting.
> >
> > Yeah, I had few time in the morning but I put a potential fix for that
> > problem here to test if would solve the problem:
> > https://review.openstack.org/#/c/219199/
> >
> > This is one of the main jobs for Ironic and it would be great if we
> > could keep it voting. Could we perhaps change the base OS to something
> > else until Ubuntu is fixed ? ( Fedora / CentOS / Debian )
>
> Why don't we just use the pre-built agent ramdisks we already publish?
> AFAIK we already have a job for it, just need to switch the name in
> project-config for Nova.
>
> // jim
>
>
We need to make sure the dib element is in working shape, which means means
it actually has to be utilized somewhere along the line.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] FW: [cinder] Microsoft CI Still Disabled

[Octavian Ciuhandu]

Thank you for your help on this,

Octavian.

From: Duncan Thomas
Reply-To: "OpenStack Development Mailing List (not for usage questions)"
Date: Tuesday, 1 September 2015 14:48
To: "OpenStack Development Mailing List (not for usage questions)"
Subject: Re: [openstack-dev] FW: [cinder] Microsoft CI Still Disabled

Mike is on vacation at the moment, somebody else will evaluate in his stead as 
reply here shortly

On 1 September 2015 at 14:20, Octavian Ciuhandu 
> 
wrote:
Hi Mike,

I think the first mail was sent only on openstack-dev. We have had already over
a week of reliable results on the Microsoft Cinder CI on all three jobs, we have
an updated set of results available at [1].

Please re-evaluate the activity of our CI and re-enable the gerrit account.

Thank you,

Octavian.

[1]: http://paste.openstack.org/show/437509/





On 24/8/15 20:19, "Octavian Ciuhandu" 
> 
wrote:

>Hi Mike,
>
>We have stabilised the Microsoft Cinder CI and ensured dry-runs testing over
>the past days.
>
>The reason why we didnt report earlier on the status is that we were waiting
>for the completion of a series of hardware maintenance activities performed in
>the Microsoft datacenter where the CI is running, as metioned by Hashir in the
>previous email.
>
>In the meantime we also improved the CI software infrastructure, including:
>
>1. Adding local pip and deb cache servers to reduce transient package download
>issues 2. Improved resiliency in the under and overcloud deployment scripts [1]
>3. Improved monitoring to reduce the respose time on transient issues,
>especially for undercloud networking which was the source of a significant
>amount of failures due to a switch misconfiguration in the datacenter, recently
>identified and solved.
>
>We have a list of recent runs for all 3 drivers available at [2].
>
>Please let us know if there’s anything else that prevents re-enabling the
>Microsoft Cinder CI account.
>
>Thank you,
>
>Octavian.
>
>[1]: https://github.com/cloudbase/cinder-ci
>[2]: http://paste.openstack.org/show/426144/
>
>
>
>
>
>
>On 24/8/15 16:48, "Hashir Abdi" 
>> wrote:
>
>>Mike:
>>
>>Apologies for the delayed response.
>>
>>We have been addressing bug-fixes and infrastructure changes for our CI, 
>>while working through vacation schedules.
>>
>>We have now been able to conduct successful dry runs of our CI, and I'll let 
>>Octavian follow up here with those latest successful results.
>>
>>Sincerely
>>
>>Hashir Abdi
>>
>>Principal Software Engineering Manager
>>Microsoft
>>
>>
>>
>>
>>
>>
>>
>>-Original Message-
>>From: Mike Perez [mailto:thin...@gmail.com]
>>Sent: Friday, August 21, 2015 8:39 PM
>>To: OpenStack Development Mailing List 
>>>
>>Cc: Microsoft Cinder CI 
>>>;
>> Octavian Ciuhandu 
>>>
>>Subject: [cinder] Microsoft CI Still Disabled
>>
>>The Microsoft CI has been disabled since July 22nd [1].
>>
>>Last I heard from the Cinder midcycle sprint, this CI was still not ready and 
>>it hasn't been for 30 days now.
>>
>>Where are we with things, and why has communication been so poor with the 
>>Cloud Base solutions team?
>>
>>[1] - 
>>https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2flists.openstack.org%2fpipermail%2fthird-party-announce%2f2015-July%2f000249.html=01%7c01%7chabdi%40exchange.microsoft.com%7c33ba5a82486e4dc6d06508d2aa8a115d%7c72f988bf86f141af91ab2d7cd011db47%7c1=mPalAeUjUVBu9sVn6fDnehA4TN3uZOUB4DAKS6qcFaU%3d
>>
>>--
>>Mike Perez
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



--
--
Duncan Thomas
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ironic] 100% failure in pxe_ssh job

[Jim Rollenhagen]

> On Sep 1, 2015, at 04:32, Lucas Alvares Gomes  wrote:
> 
> Hi,
> 
>> The current failure rate for the ironic pxe_ssh job is 100% -
>> http://graphite.openstack.org/render/?from=-200hours=500=now=800=ff=00=100=0=color(alias(movingAverage(asPercent(stats.zuul.pipeline.check.job.gate-tempest-dsvm-ironic-pxe_ssh.FAILURE,sum(stats.zuul.pipeline.check.job.gate-tempest-dsvm-ironic-pxe_ssh.{SUCCESS,FAILURE})),%2712hours%27),%20%27gate-tempest-dsvm-ironic-pxe_ssh%27),%27orange%27)
>> 
>> The reason is something wrong with disk image builder and upstream ubuntu.
>> 
>> Which raises a much more pressing issue, why is an ironic integration
>> job building, from scratch a disk image builder image on every go (and
>> connecting to the internet to do it)? Especially as this job sits on a
>> bunch of other projects beyond ironic. Architecturally this is not sound
>> enough to be a voting job.
>> 
>> I'm proposing we make it non-voting immediately, and until it's redone
>> so it's no long dependent on pulling images directly from upstream, we
>> don't let it be voting.
> 
> Yeah, I had few time in the morning but I put a potential fix for that
> problem here to test if would solve the problem:
> https://review.openstack.org/#/c/219199/
> 
> This is one of the main jobs for Ironic and it would be great if we
> could keep it voting. Could we perhaps change the base OS to something
> else until Ubuntu is fixed ? ( Fedora / CentOS / Debian )

Why don't we just use the pre-built agent ramdisks we already publish? AFAIK we 
already have a job for it, just need to switch the name in project-config for 
Nova. 

// jim 


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ironic] 100% failure in pxe_ssh job

[Sean Dague]

On 09/01/2015 09:21 AM, Julia Kreger wrote:
> 
> On Tue, Sep 1, 2015 at 9:17 AM, Jim Rollenhagen  > wrote:

> 
> Why don't we just use the pre-built agent ramdisks we already
> publish? AFAIK we already have a job for it, just need to switch the
> name in project-config for Nova.
> 
> // jim
> 
> 
> We need to make sure the dib element is in working shape, which means
> means it actually has to be utilized somewhere along the line.

Which should be a functional test on dib, or at some lower level, not
randomly done in the integration tests of unrelated projects. The
current ball of mud testing leads to high coupling and high failure
rates, because everything is secondarily tested. It should never be run
during nova patches, because there is 0 chance that a Nova patch can
impact that code.

We spent a lot of time trying to decouple libraries from projects and
put the tests in the right places to actually verify the behavior closer
to the code in question. dib / ironic is still doing gating like other
projects were a year ago, which was fragile and led to unrelated lock
ups like this really often. So that needs to be prioritized soon.

-Sean

-- 
Sean Dague
http://dague.net

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ironic] 100% failure in pxe_ssh job

[Jim Rollenhagen]

On Tue, Sep 01, 2015 at 09:21:39AM -0400, Julia Kreger wrote:
> On Tue, Sep 1, 2015 at 9:17 AM, Jim Rollenhagen 
> wrote:
> 
> >
> > Why don't we just use the pre-built agent ramdisks we already publish?
> > AFAIK we already have a job for it, just need to switch the name in
> > project-config for Nova.
> >
> > // jim
> >
> >
> We need to make sure the dib element is in working shape, which means means
> it actually has to be utilized somewhere along the line.

It's still running in Ironic's gate, I don't see any reason why it also
needs to be in Nova's gate.

// jim

> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Debug tool for neutron

[Jay Pipes]

Check out https://github.com/yeasy/easyOVS for OVS-based setups.

Best,
-jay

On 09/01/2015 07:50 AM, bharath thiruveedula wrote:

Hi,

We have some troubleshooting guides for openstack neutron. But many
people who are new to neutron find it difficult to follow the guides, as
they are not aware of what is happening behind the scenes. So is there
any tool which tracks the packet flow from the VM to debug issues like
why the VM  is not getting the IP Address or why internet is not
reachable from the VM? If any, can you please suggest some of them?


Regards
Bharath


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Fuel] Number of IP addresses in a public network

[Andrey Danin]

+1 to Igor.

It's definitely not a High bug. The biggest problem I see here is a
confusing error message with a wrong number of required IPs. AFAIU we
cannot fix it easily now so let's postpone it to 8.0 but change a message
itself [0] in 7.0.

[0]
https://github.com/stackforge/fuel-web/blob/master/nailgun/nailgun/task/task.py#L1160-L1163

On Tue, Sep 1, 2015 at 1:39 PM, Igor Kalnitsky 
wrote:

> Hello,
>
> My 5 cents on it.
>
> I don't think it's really a High or Critical bug for 7.0. If there's
> not enough IPs the CheckBeforeDeploymentTask will fail. And that's
> actually Ok, it may fail by different reason without starting actual
> deployment (sending message to Astute).
>
> But I agree it's kinda strange that we don't check IPs during network
> verification step. The good fix in my opinion is to move this check
> into network checker (perhaps keep it here either), but that
> definitely shouldn't be done in 7.0.
>
> Thanks,
> Igor
>
>
> On Mon, Aug 31, 2015 at 2:54 PM, Roman Prykhodchenko 
> wrote:
> > Hi folks!
> >
> > Recently a problem that network check does not tell whether there’s
> enough IP addresses in a public network [1] was reported. That check is
> performed by CheckBeforeDeployment task, but there is two problems that
> happen because this verification is done that late:
> >
> >  - A deployment fails, if there’s not enough addresses in specified
> ranges
> >  - If a user wants to get network configuration they will get an error
> >
> > The solution for this problems seems to be easy and a straightforward
> patch [2] was proposed. However, there is a hidden problem which is that
> patch does not address which is that installed plugins may reserve VIPs for
> their needs. The issue is that they do it just before deployment and so
> it’s not possible to get those reservations when a user wants to check
> their network set up.
> >
> > The important issue we have to address here is that network
> configuration generator will fail, if specified ranges don’t fit all VIPs.
> There were several proposals to fix that, I’d like to highlight two of them:
> >
> >  a) Allow VIPs to not have an IP address assigned, if network config
> generator works for API output.
> >  That will prevent GET requests from failure, but since IP addresses
> for VIPs are required, generator will have to fail, if it generates a
> configuration for the orchestrator.
> >  b) Add a release note that users have to calculate IP addresses
> manually and put sane ranges in order to not shoot their own legs. Then
> it’s also possible to change network verification output to remind users to
> check the ranges before starting a deployment.
> >
> > In my opinion we cannot follow (a) because it only masks a problem
> instead of providing a fix. Also it requires to change the API which is not
> a good thing to do after the SCF. If we choose (b), then we can work on a
> firm solution in 8.0 and fix the problem for real.
> >
> >
> > P. S. We can still merge [2], because it checks, if IP ranges can at
> least fit the basic configuration. If you agree, I will update it soon.
> >
> > [1] https://bugs.launchpad.net/fuel/+bug/1487996
> > [2] https://review.openstack.org/#/c/217267/
> >
> >
> >
> > - romcheg
> >
> >
> >
> >
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> >
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Andrey Danin
ada...@mirantis.com
skype: gcon.monolake
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Blazar] Anyone interested?

[Nikolay Starodubtsev]

Also, if we decided to continue development we should add blazar here [1]
according to the email [2]
So, my suggestion is to setup some timeframe on this week or next week and
hold some kind of meeting.
[1]: https://wiki.openstack.org/wiki/Stackforge_Namespace_Retirement
[2]:
http://lists.openstack.org/pipermail/openstack-dev/2015-August/073071.html




Nikolay Starodubtsev

Software Engineer

Mirantis Inc.


Skype: dark_harlequine1

2015-09-01 12:42 GMT+03:00 Nikolay Starodubtsev 
:

> Sylvain,
> First of all we need to reanimate blazar gate-jobs, or we can't merge
> anything. I tried to do it a year ago, but can't get the point of the tests,
> so better decision can be to rewrite them from scratch.
>
>
>
> Nikolay Starodubtsev
>
> Software Engineer
>
> Mirantis Inc.
>
>
> Skype: dark_harlequine1
>
> 2015-09-01 11:26 GMT+03:00 Sylvain Bauza :
>
>>
>>
>> Le 01/09/2015 06:52, Nikolay Starodubtsev a écrit :
>>
>> All,
>> I'd like to propose use of #openstack-blazar for further communication
>> and coordination.
>>
>>
>>
>> +2 to that. That's the first step of any communication. The channel logs
>> are also recorded here, for async communication :
>> http://eavesdrop.openstack.org/irclogs/%23openstack-blazar/
>>
>> I don't see at the moment much benefits of running a weekly meeting. We
>> can chat on purpose if needed.
>>
>> Like I said to Ildiko, I'm fine to help some people discovering Blazar
>> but I won't have time lots of time for actually working on it.
>>
>> IMHO, the first things to do with Blazar is to reduce the tech debt by :
>>  1/ finising the Climate->Blazar renaming
>>  2/ updating and using the latest oslo librairies instead of using the
>> old incubator
>>  3/ using Nova V2.1 API (which could be a bit difficult because there are
>> no more extensions)
>>
>> If I see some progress with Blazar, I'm OK with asking -infra to move
>> Blazar to the OpenStack namespace like it was asked by James Blair here
>> because it seems Blazar is not defunct :
>> http://lists.openstack.org/pipermail/openstack-dev/2015-August/072140.html
>>
>> -Sylvain
>>
>>
>>
>>
>>
>>
>> Nikolay Starodubtsev
>>
>> Software Engineer
>>
>> Mirantis Inc.
>>
>>
>> Skype: dark_harlequine1
>>
>> 2015-08-31 19:44 GMT+03:00 Fuente, Pablo A :
>>
>>> Yes, Blazar is a really interesting project. I worked on it some time
>>> ago and I really enjoy it. Sadly my obligations at work don't let me to
>>> still working on it, but I', happy that there still some interest in Blazar.
>>>
>>> Pablo.
>>> On 31/08/15 09:19, Zhenyu Zheng wrote:
>>> Hello,
>>> It seems like an interesting project.
>>>
>>> On Fri, Aug 28, 2015 at 7:54 PM, Pierre Riteau >> > wrote:
>>> Hello,
>>>
>>> The NSF-funded Chameleon project (https://www.chameleoncloud.org) uses
>>> Blazar to provide advance reservations of resources for running cloud
>>> computing experiments.
>>>
>>> We would be interested in contributing as well.
>>>
>>> Pierre Riteau
>>>
>>> On 28 Aug 2015, at 07:56, Ildikó Váncsa <>> ildiko.van...@ericsson.com>ildiko.van...@ericsson.com>> ildiko.van...@ericsson.com>> wrote:
>>>
>>> > Hi All,
>>> >
>>> > The resource reservation topic pops up time to time on different
>>> forums to cover use cases in terms of both IT and NFV. The Blazar project
>>> was intended to address this need, but according to my knowledge due to
>>> earlier integration and other difficulties the work has been stopped.
>>> >
>>> > My question is that who would be interested in resurrecting the Blazar
>>> project and/or working on a reservation system in OpenStack?
>>> >
>>> > Thanks and Best Regards,
>>> > Ildikó
>>> >
>>> >
>>> __
>>> > OpenStack Development Mailing List (not for usage questions)
>>> > Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<
>>> http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
>>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe<
>>> http://openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>>
>>>
>>>
>>> __
>>> OpenStack Development Mailing List (not for usage questions)
>>> Unsubscribe:
>>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe>
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>>
>>>
>>>
>>>
>>> __
>>> OpenStack 

Re: [openstack-dev] [Blazar] Anyone interested?

[Sylvain Bauza]

Le 01/09/2015 15:46, Nikolay Starodubtsev a écrit :
Also, if we decided to continue development we should add blazar here 
[1] according to the email [2]
So, my suggestion is to setup some timeframe on this week or next week 
and hold some kind of meeting.

[1]: https://wiki.openstack.org/wiki/Stackforge_Namespace_Retirement
[2]: 
http://lists.openstack.org/pipermail/openstack-dev/2015-August/073071.html




That's just what I said I was OK to add, for sure.



*__*

Nikolay Starodubtsev

Software Engineer

Mirantis Inc.


Skype: dark_harlequine1


2015-09-01 12:42 GMT+03:00 Nikolay Starodubtsev 
>:


Sylvain,
First of all we need to reanimate blazar gate-jobs, or we can't
merge anything. I tried to do it a year ago, but can't get the
point of the tests,
so better decision can be to rewrite them from scratch.

*__*

Nikolay Starodubtsev

Software Engineer

Mirantis Inc.


Skype: dark_harlequine1


2015-09-01 11:26 GMT+03:00 Sylvain Bauza >:



Le 01/09/2015 06:52, Nikolay Starodubtsev a écrit :

All,
I'd like to propose use of #openstack-blazar for further
communication and coordination.




+2 to that. That's the first step of any communication. The
channel logs are also recorded here, for async communication :
http://eavesdrop.openstack.org/irclogs/%23openstack-blazar/

I don't see at the moment much benefits of running a weekly
meeting. We can chat on purpose if needed.

Like I said to Ildiko, I'm fine to help some people
discovering Blazar but I won't have time lots of time for
actually working on it.

IMHO, the first things to do with Blazar is to reduce the tech
debt by :
 1/ finising the Climate->Blazar renaming
 2/ updating and using the latest oslo librairies instead of
using the old incubator
 3/ using Nova V2.1 API (which could be a bit difficult
because there are no more extensions)

If I see some progress with Blazar, I'm OK with asking -infra
to move Blazar to the OpenStack namespace like it was asked by
James Blair here because it seems Blazar is not defunct :

http://lists.openstack.org/pipermail/openstack-dev/2015-August/072140.html

-Sylvain





*__*

Nikolay Starodubtsev

Software Engineer

Mirantis Inc.


Skype: dark_harlequine1


2015-08-31 19:44 GMT+03:00 Fuente, Pablo A
>:

Yes, Blazar is a really interesting project. I worked on
it some time ago and I really enjoy it. Sadly my
obligations at work don't let me to still working on it,
but I', happy that there still some interest in Blazar.

Pablo.
On 31/08/15 09:19, Zhenyu Zheng wrote:
Hello,
It seems like an interesting project.

On Fri, Aug 28, 2015 at 7:54 PM, Pierre Riteau
>> wrote:
Hello,

The NSF-funded Chameleon project
(https://www.chameleoncloud.org) uses Blazar to provide
advance reservations of resources for running cloud
computing experiments.

We would be interested in contributing as well.

Pierre Riteau

On 28 Aug 2015, at 07:56, Ildikó Váncsa
<>ildiko.van...@ericsson.com

>> wrote:

> Hi All,
>
> The resource reservation topic pops up time to time on
different forums to cover use cases in terms of both IT
and NFV. The Blazar project was intended to address this
need, but according to my knowledge due to earlier
integration and other difficulties the work has been stopped.
>
> My question is that who would be interested in
resurrecting the Blazar project and/or working on a
reservation system in OpenStack?
>
> Thanks and Best Regards,
> Ildikó
>
>

__
> OpenStack Development Mailing List (not for usage
questions)
> Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

Re: [openstack-dev] [cinder] L3 low pri review queue starvation

[John Griffith]

On Tue, Sep 1, 2015 at 5:57 AM, Tom Barron  wrote:

> [Yesterday while discussing the following issue on IRC, jgriffith
> suggested that I post to the dev list in preparation for a discussion in
> Wednesday's cinder meeting.]
>
> Please take a look at the 10 "Low" priority reviews in the cinder
> Liberty 3 etherpad that were punted to Mitaka yesterday. [1]
>
> Six of these *never* [2] received a vote from a core reviewer. With the
> exception of the first in the list, which has 35 patch sets, none of the
> others received a vote before Friday, August 28.  Of these, none had
> more than -1s on minor issues, and these have been remedied.
>
> Review https://review.openstack.org/#/c/213855 "Implement
> manage/unmanage snapshot in Pure drivers" is a great example:
>
>* approved blueprint for a valuable feature
>* pristine code
>* passes CI and Jenkins (and by the deadline)
>* never reviewed
>
> We have 11 core reviewers, all of whom were very busy doing reviews
> during L3, but evidently this set of reviews didn't really have much
> chance of making it.  This looks like a classic case where the
> individually rational priority decisions of each core reviewer
> collectively resulted in starving the Low Priority review queue.
>
> One way to remedy would be for the 11 core reviewers to devote a day or
> two to cleaning up this backlog of 10 outstanding reviews rather than
> punting all of them out to Mitaka.
>
> Thanks for your time and consideration.
>
> Respectfully,
>
> -- Tom Barron
>
> [1] https://etherpad.openstack.org/p/cinder-liberty-3-reviews
> [2] At the risk of stating the obvious, in this count I ignore purely
> procedural votes such as the final -2.
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

​Thanks Tom, this is sadly an ongoing problem every release.  I think we
have a number of things we can talk about at the summit to try and make
some of this better.  I honestly think that if people were to actually
"use" launchpad instead of creating tracking etherpads everywhere it would
help.  What I mean is that there is a ranked targeting of items in
Launchpad and we should use it, core team members should know that as the
source of truth and things that must get reviewed.

As far as Liberty and your patches; Yesterday was the freeze point, the
entire Cinder team agreed on that (yourself included both at the mid-cycle
meet up and at the team meeting two weeks ago when Thingee reiterated the
deadlines).  If you noticed last week that your patches weren't going
anywhere YOU should've wrangled up some reviews.

Furthermore, I've explained every release for the last 3 or 4 years that
there's no silver bullet, no magic process when it come to review
throughput.  ESPECIALLY when it comes to the 3'rd milestone.  You can try
landing strips, priority listed etherpads, sponsors etc etc but the fact is
that things happen, the gate slows down (or we completely break on the
Cinder side like we did yesterday).  This doesn't mean "oh, well then you
get another day or two", it means stuff happens and it sucks but first
course of action is drop low priority items.  It just means if you really
wanted it you probably should've made it happen earlier.  Just so you know,
I run into this every release as well.  I had a number of things in
progress that I had hoped to finish last week and yesterday, BUT my
priority shifted to trying to help get the cinder patches back on track and
get the items in Launchpad updated to actually reflect something that was
somewhat possible.

The only thing that works is "submit early and review often" it's simple.

Finally, I pointed out to you yesterday that we could certainly discuss as
a team what to do with your patches.  BUT that given how terribly far
behind we were in the process that I wanted reviewers to focus on medium,
high and critical prioritized items.  That's what prioritization's are for,
it means when crunch time hits and things hit the fan it's usually the
"low" priority things that get bumped.

Thanks,
John
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Swift] Swift 2.4.0 release

[John Dickinson]

I'm pleased to announce that Swift 2.4.0 is available. As always, you can 
upgrade to
this version with no end-user downtime. This release has several very nice 
features,
so I recommend that you upgrade as soon as possible.

You can find the release at:
 https://launchpad.net/swift/liberty/2.4.0

Please note the dependency and config changes in the full release notes:
 https://github.com/openstack/swift/blob/master/CHANGELOG

This release is the result of 55 code contributors, including 29 first-time code
contributors. These are the first-time code contributors:

Timur Alperovich   Kazuhiro Miyahara
Minwoo Bae Takashi Natsume
Tim Burke  Ondrej Novy
Carlos Cavanna Falk Reimann
Emmanuel Cazenave  Brian Reitz
Clément ContiniEran Rom
Oshrit Feder   Hamdi Roumani
Charles HsuAtsushi Sakai
Joanna H. HuangAzhagu Selvan SP
Bill Huber Alexandra Settle
Jaivish KothariPradeep Kumar Singh
Zhao Lei   Victor Stinner
Ben Martin Akihito Takai
Kenichiro Matsuda  Kai Zhang
Michael Matur


I'd like to call out a few significant changes in this release. Please read
the release notes in the link above for more information about these new
features.

* Allow one or more object servers per disk deployment

  This new deployment model allows you to set the number of object servers per
  drive. The basic change is that you add every drive into the ring at a
  different port, and Swift will automatically run the configured number of
  server processes for each drive. This results in per-disk isolated IO so
  that one slow disk does not slow down every operation to that server. This
  change can dramatically lower request latency and increase requests per
  second.

* Improve performance for object writes to large containers

  When an object is written in Swift, the object servers will attempt to
  update the listing in the appropriate container server before giving a
  response to the client. If the container server is busy, object latency
  increases as the object server waits for the container server. In this
  release, the object server waits less time (and the wait time is
  configurable) for the container server response, thus lowering overall
  latency. Effectively, this means that object writes will no longer get
  slower as the number of objects in a container increases.

* Users can set per-object metadata with bulk upload

  Bulk uploads allow users to upload an archive (.tar) to Swift and store the
  individual referenced files as separate objects in the system. Swift now
  observes extended attributes set on these files and appropriately sets user
  metadata on the stored objects.

* Allow SLO PUTs to forgo per-segment integrity checks.

  Previously, each segment referenced in the manifest also needed the correct
  etag and bytes setting. These fields now allow the "null" value to skip
  those particular checks on the given segment.

There are many other changes that have gone into this release. I encourage you 
to review
the release notes and upgrade to this release.

Thank you to everyone who contributed to this release. If you would like to get 
involved
in Swift, please join us in #openstack-swift on freenode IRC.


--John





signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [trove] [heat] Multi region support

[Lowery, Mathew]

This is a Trove question but including Heat as they seem to have solved this 
problem.

Summary: Today, it seems that Trove is not capable of creating a cluster 
spanning multiple regions. Is that the case and, if so, are there any plans to 
work on that? Also, are we aware of any precedent solutions (e.g. remote stacks 
in Heat) or even partially completed spec/code in Trove?

More details:

I found this nice 
diagram
 created for Heat. As far as I understand it, #1 is the absence of multi-region 
support (i.e. what we have today). #2 seems to be a 100% client-based solution. 
In other words, the Heat services never know about the other stacks. In fact, 
there is nothing tying these stacks together at all. #3 seems to show a 
"master" Heat server that understands "remote stacks" and simply converts those 
"remote stacks" into calls on regional Heats. I assume here the master stack 
record is stored by the master Heat. Because the "remote stacks" are 
full-fledged stacks, they can be managed by their regional Heats if 
availability of master or other regional Heats is lost. #4, the diagram doesn't 
seem to match the description (instead of one global Heat, it seems the diagram 
should show two regional Heats). In this one, a single arbitrary region becomes 
the owner of the stack and remote (low-level not stack) resources are created 
as needed. One problem is the manageability is lost if the Heat in the owning 
region is lost. Finally, #5. In #5, it's just #4 but with one and only one Heat.

It seems like Heat solved this using 
#3 (Master Orchestrator) but where there isn't necessarily a separate master 
Heat. Remote stacks can be created by any regional stack.

Trove questions:

  1.  Having sub-clusters (aka remote clusters aka nested clusters) seems to be 
useful (i.e. manageability isn't lost when a region is lost). But then again, 
does it make sense to perform a cluster operation on a sub-cluster?
  2.  You could forego sub-clusters and just create full-fledged remote 
standalone Trove instances.
  3.  If you don't create full-fledged remote Trove instances (but instead just 
call remote Nova), then you cannot do simple things like getting logs from a 
node without going through the owning region's Trove. This is an extra hop and 
a single point of failure.
  4.  Even with sub-clusters, the only record of them being related lives only 
in the "owning" region. Then again, some ID tying them all together could be 
passed to the remote regions.
  5.  Do we want to allow the piecing together of clusters (sort of like Heat's 
"adopt")?

These are some questions floating around my head and I'm sure there are plenty 
more. Any thoughts on any of this?

Thanks,
Mat
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][ceilometer] cputime value resets on restart/shutdown

[gord chung]

On 31/08/2015 3:36 PM, Julien Danjou wrote:

On Mon, Aug 31 2015, gord chung wrote:


i'm not sure Gnocchi is where we should be fixing this as it really only
(potentially) fixes it for Gnocchi and not for any of the other ways Ceilometer
data can be consumed.

The ideal way is to send the data as it is collected and do the
treatment in the backend, otherwise you end up implementing the world in
Ceilometer.


but if it's not actually cumulative in Ceilometer (pre-storage), should 
we really be tagging it as such?



a proposed solution in bug and in a previous thread suggests that a 'delta'
meter be built from cputime value libvirt provides which would better handle
the reset scenario. that said, is there another option to truly have a
cumulative meter?

Yes, the hackish way might be to convert those cumulative to delta with
a transfomer in the pipeline and be done with it. Though you'll probably
have data points missing on restart, which is not ultimate solution. As
soon as you start losing information in the Ceilometer pipeline by
transforming I'll stand and say "bad idea". :-)


so i was thinking rather than modify the existing meter, we build a new 
cpu.delta meter, which is gives the delta. it seems like giving a delta 
meter would make the behaviour more consistent.




So yes, you have to have a smart storage solution in the end.

ideally, i was thinking Nova would capture this data and there would be 
absolutely no data loss as Nova knows exactly when instances are 
restarted/stopped... that said, it's arguably outside the scope of Nova.


cheers,

--
gord


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [puppet] weekly meeting #49

[Emilien Macchi]

On 08/31/2015 08:40 AM, Emilien Macchi wrote:
> Hello,
> 
> Here's an initial agenda for our weekly meeting, tomorrow at 1500 UTC
> in #openstack-meeting-4:
> 
> https://etherpad.openstack.org/p/puppet-openstack-weekly-meeting-20150901
> 
> Please add additional items you'd like to discuss.
> If our schedule allows it, we'll make bug triage during the meeting.

We did our meeting:
http://eavesdrop.openstack.org/meetings/puppet_openstack/2015/puppet_openstack.2015-09-01-15.00.html

Reminder: tomorrow is our 3-days mid-cycle!
Don't miss to look
https://etherpad.openstack.org/p/puppet-liberty-mid-cycle if you want to
participate.

Happy hacking,
-- 
Emilien Macchi



signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][ceilometer] cputime value resets on restart/shutdown

[Julien Danjou]

On Tue, Sep 01 2015, gord chung wrote:

> but if it's not actually cumulative in Ceilometer (pre-storage), should we
> really be tagging it as such?

We only have 3 meters type, and the cumulative definition I wrote
somewhere back in 2012 states that it can reset to 0. Sorry. :-)

> so i was thinking rather than modify the existing meter, we build a new
> cpu.delta meter, which is gives the delta. it seems like giving a delta meter
> would make the behaviour more consistent.

…with data loss if you restart the polling agent and it then ignores the
previous value of the meter.

Except if you connect the polling system to the previous state of the
meter, which requires to either:
1. Connect the polling system to the storage system
2. Store locally the latest value you fetched

Option 1 sounds crazy, option 2 sounds less crazy, but still hackish.

Whereas having a system that can compute the delta afterward with all
the value at its reach sounds way better – that's why I'm in favor of
doing that in the storage system (e.g. Gnocchi).

-- 
Julien Danjou
;; Free Software hacker
;; http://julien.danjou.info


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] FW: [cinder] Microsoft CI Still Disabled

[Octavian Ciuhandu]

Hi Mike,

I think the first mail was sent only on openstack-dev. We have had already over 
a week of reliable results on the Microsoft Cinder CI on all three jobs, we 
have 
an updated set of results available at [1].

Please re-evaluate the activity of our CI and re-enable the gerrit account.

Thank you,

Octavian.

[1]: http://paste.openstack.org/show/437509/





On 24/8/15 20:19, "Octavian Ciuhandu"  wrote:

>Hi Mike,
>
>We have stabilised the Microsoft Cinder CI and ensured dry-runs testing over
>the past days.
>
>The reason why we didnt report earlier on the status is that we were waiting 
>for the completion of a series of hardware maintenance activities performed in 
>the Microsoft datacenter where the CI is running, as metioned by Hashir in the
>previous email.
>
>In the meantime we also improved the CI software infrastructure, including:
>
>1. Adding local pip and deb cache servers to reduce transient package download
>issues 2. Improved resiliency in the under and overcloud deployment scripts [1]
>3. Improved monitoring to reduce the respose time on transient issues,
>especially for undercloud networking which was the source of a significant
>amount of failures due to a switch misconfiguration in the datacenter, recently
>identified and solved.
>
>We have a list of recent runs for all 3 drivers available at [2].
>
>Please let us know if there’s anything else that prevents re-enabling the 
>Microsoft Cinder CI account.
>
>Thank you,
>
>Octavian.
>
>[1]: https://github.com/cloudbase/cinder-ci
>[2]: http://paste.openstack.org/show/426144/
>
>
>
>
>
>
>On 24/8/15 16:48, "Hashir Abdi"  wrote:
>
>>Mike:
>>
>>Apologies for the delayed response.
>>
>>We have been addressing bug-fixes and infrastructure changes for our CI, 
>>while working through vacation schedules.
>>
>>We have now been able to conduct successful dry runs of our CI, and I'll let 
>>Octavian follow up here with those latest successful results.
>>
>>Sincerely
>>
>>Hashir Abdi
>>
>>Principal Software Engineering Manager
>>Microsoft
>>
>>
>>
>>
>>
>>
>>
>>-Original Message-
>>From: Mike Perez [mailto:thin...@gmail.com]
>>Sent: Friday, August 21, 2015 8:39 PM
>>To: OpenStack Development Mailing List 
>>Cc: Microsoft Cinder CI ; Octavian 
>>Ciuhandu 
>>Subject: [cinder] Microsoft CI Still Disabled
>>
>>The Microsoft CI has been disabled since July 22nd [1].
>>
>>Last I heard from the Cinder midcycle sprint, this CI was still not ready and 
>>it hasn't been for 30 days now.
>>
>>Where are we with things, and why has communication been so poor with the 
>>Cloud Base solutions team?
>>
>>[1] - 
>>https://na01.safelinks.protection.outlook.com/?url=http%3a%2f%2flists.openstack.org%2fpipermail%2fthird-party-announce%2f2015-July%2f000249.html=01%7c01%7chabdi%40exchange.microsoft.com%7c33ba5a82486e4dc6d06508d2aa8a115d%7c72f988bf86f141af91ab2d7cd011db47%7c1=mPalAeUjUVBu9sVn6fDnehA4TN3uZOUB4DAKS6qcFaU%3d
>>
>>--
>>Mike Perez
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev