[openstack-dev] [magnum] Stable/liberty functional tests

[Bunting, Niall]

Hi,


There was work on a patch [1] to fix the liberty stable branch. However, we are 
running into problems with the functional tests, because the functional tests 
were set up differently in liberty eg. The current [2] magnum.yaml compared to 
the old one [3].

This leads us with two options to try and get stable/liberty passing the gate 
again.
1. To create a 'new' test that will have the same test setup as used in liberty.
2. To back port all the necessary changes for the tests to run successfully.

I think option one would be the better route, as that well be less likely to 
break any stable rules. It would also have the affect of future proofing the 
tests against any changes in the future.

I'm wondering what the rest of you think would be the best way to approach this 
problem?


Niall


1. https://review.openstack.org/#/c/303420/

2. 
https://github.com/openstack-infra/project-config/blob/master/jenkins/jobs/magnum.yaml

3. 
https://github.com/openstack-infra/project-config/blob/3d6764223981170081f151364f0d398db59513ce/jenkins/jobs/magnum.yaml

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [glance] Remove `run_tests.sh` and `tools/*`

[Bunting, Niall]

> Hey Folks,
> 
> I'm looking at doing some cleanups in our repo and I would like to start by
> deprecating the `run_tests` script and the contents in the `tools/` dir.
> 
> As far as I can tell, no one is using this code - we're not even using it in 
> the
> gate - as it was broken until recently, I believe. The recommended way to run
> tests is using `tox` and I believe having this script in the code base 
> misleads
> new contributors and other users.
> 
> So, before we do this. I wanted to get feedback from a broader audience and 
> give
> a heads up to folks that might be using this code.
> 
> Any objections? Something I'm missing?
> 
> Flavio

This is not strictly related however, it might be worth having some 
documentation or some links to info. So the new  contributors have some 
information about how to run the various tox tests and utilities such as the 
config regen.

Niall

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova][glance] Nova using service tokens with protected properties in Glance

[Bunting, Niall]

Hi,

There is the following spec for review:
https://review.openstack.org/#/c/249950/ for allowing the modification
of protected properties with service tokens in Glance.

John & Stuart suggest using service tokens for the following Nova spec:
https://review.openstack.org/#/c/206431/ (Inheritable admin image
properties).

It would be great to see some Nova input on the Glance spec taking into
consideration the proposed Nova spec.

Thanks,
Niall Bunting

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [glance] Auth_version from 'old style' URLs in the database

[Bunting, Niall]

Hi,

Currently glance will use an auth_url if in the database. Eg. 10.0.0.8:5000/v2.0

However glance currently takes the auth_version from the config
files. Therefore this can lead to a mismatch of keystone version to be used
between the url and the config files. This is problematic due to a different
resource id being required in different version of keystone (in keystone v2
it was /v2.0/tokens in keystone v3 it is /v3/auth/tokens).

Using a v2 url and config file with keystone v3:
10.0.0.8:5000/v2.0/auth/tokens -- Fails to authenticate the user,
and user can't download image.

See https://bugs.launchpad.net/glance-store/+bug/1507610 for a bug report
on this.

This means that the fix proposed by
https://review.openstack.org/#/c/238074/ parses the URL for an auth_version
and then if found will use the parsed value as the auth_version rather than
the one from the config files. Taking the url as the true source.
Therefore the image will still work as the auth_version used by glance is the
one defined in the URL meaning the correct resource id appended.

Whilst discussing it with Kairat it was proposed that we ignore the
keystone version in the URL and if it does not support the auth_version
in the configs, then the image would fail to be downloaded. This is due to a
preference to have a centralised auth_version value.

I am wondering what people would prefer to do, to support the 'old style' urls
and therefore parse the version from the url. Or to make the auth_version
common and potentially break the 'old style' database entries.

Thanks,
Niall Bunting


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Glance] Process to clean up the review queue from non-active patches

[Bunting, Niall]

> From: Julien Danjou [jul...@danjou.info]
> Sent: 07 October 2015 10:12
> 
> On Wed, Oct 07 2015, Flavio Percoco wrote:
> 
> > I'm not trying to solve the lack of reviews in Liberty by removing
> > patches. What I'd like to do, though, is help to keep around patches
> > that really matter.
> 
> I think that's where you are making a mistake. They are contributors,
> like me or Victor, are knocking on the Glance doors for months now,
> sending patches that resolve technical debt rather than adding new
> debt^Wfeatures. Currently, these patches are not seen as important and
> are often dismissed. So I'm pretty sure they are going to expire with
> this new system.
> 
> Imagine that if you were merging patches from me, Victor, and people
> like us, we would continue to send many of them, and mid-term, you'd get
> some new blood on your core team.
> 
> What is proposed here is really focusing on making life easier for the
> current core team which is in large majority inactive.
> 
> Don't read me wrong. I know you and Nikhil are both well-intentioned by
> proposing that. I just think it's going to be worse, because it won't
> improve much and you're going to push new contributors away.

If your patches are sitting there waiting for review once they get off
the top couple of pages they are likely to become buried, as they are
waiting for review. This is unlikely to benefit anyone.

How would an active user keep their patches it the active/up for review
state? By just posting bump comments?

Any type of change Juilen is talking about could keep on being dismissed,
and this could end up in some sort of game to keep your patch above
the non-review line for it just to be ignored. This would definitely be
a bad thing, therefore I think any patches that are picked up as old,
should be reviewed before being marked as WIP. As this would mean if
the patch is moved out of WIP it would be less likely to get stuck in
some sort of loop as it has a review.

We also have to be careful about alienating contributors, we should make
sure they know why there work got marked WIP with a link to a wiki page
explaining the process. However if this forces a review, they may also
be happy that they eventually got a review on their patch.

My thoughts,
Niall

Edit: As this did not send to the list originally.
Flavio points out that they aim to review patches that are unmarked wip,
I think that system could work as long as it avoids the problem of patches
potentially becoming stuck in a circle.

And It should be kept in mind that even old reviews could still be relevant,
and the best course of action may not to just mark them wip without thought.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [glance][api][tc] Response when a illegal body is sent

[Bunting, Niall]

> Excerpts from Ian Cordasco's message of 2015-07-24 11:22:33 -0700:
> >
> > On 7/24/15, 13:16, "Clint Byrum"  wrote:
> >
> > >Excerpts from Ian Cordasco's message of 2015-07-24 08:58:06 -0700:
> > >>
> > >> On 7/23/15, 19:38, "michael mccune"  wrote:
> > >>
> > >> >On 07/23/2015 12:43 PM, Ryan Brown wrote:
> > >> >> On 07/23/2015 12:13 PM, Jay Pipes wrote:
> > >> >>> On 07/23/2015 10:53 AM, Bunting, Niall wrote:
> > >> >>>> Hi,
> > >> >>>>
> > >> >>>> Currently when a body is passed to an API operation that explicitly
> > >> >>>> does not allow bodies Glance throws a 500.
> > >> >>>>
> > >> >>>> Such as in this bug report:
> > >> >>>> https://bugs.launchpad.net/glance/+bug/1475647 This is an example
> > >>of
> > >> >>>> a GET however this also applies to other requests.
> > >> >>>>
> > >> >>>> What should Glance do rather than throwing a 500, should it return
> > >>a
> > >> >>>> 400 as the user provided an illegal body
> > >> >>>
> > >> >>> Yep, this.
> > >> >>
> > >> >> +1, this should be a 400. It would also be acceptable (though less
> > >> >> preferable) to ignore any body on GET requests and execute the
> > >>request
> > >> >> as normal.
> > >> >>
> > >> >>> Best,
> > >> >>> -jay
> > >> >
> > >> >i'm also +1 on the 400 band wagon
> > >>
> > >> 400 feels right for when Glance is operating without anything in front
> > >>of
> > >> it. However, let me present a hypothetical situation:
> > >>
> > >> Company X is operating Glance behind a load-balancing proxy. Most users
> > >> talk to Glance behind the LB. If someone writes a quick script to send a
> > >> GET and (for whatever reason) includes a body, they'll get a 200 with
> > >>the
> > >> data that would otherwise have been sent if they didn't include a body.
> > >> This is because most such proxies will strip the body on a GET (even
> > >> though RFC 7231 allows for bodies on a GET and explicitly refuses to
> > >> define semantic meaning for them). If later that script is updated to
> > >>work
> > >> behind the load balancer it will be broken, because Glance is choosing
> > >>to
> > >> error instead of ignoring it.
> > >>
> > >> Note: I'm not arguing that the user is correct in sending a body when
> > >> there shouldn't be one sent, just that we're going to confuse a lot of
> > >> people with this.
> > >>
> > >> I'm also fine with either a 400 or a 200.
> > >>
> > >
> > >Nice succinct description of an interesting corner case.
> > >
> > >This is indeed one of those scenarios that should be defended against
> > >at the edges, but it's worth considering what will make things simplest
> > >for users.
> > >
> > >If we believe in Postel's robustness principle[1], then Glance would
> > >probably just drop the body as something we liberally accept because
> > >it doesn't harm anything to do so. If we don't believe thats a good
> > >principle, then 400 or maybe 413 would be the right codes I think.
> > >
> > >So the real question is, do we follow Postel's principle or not? That
> > >might even be something to add to OpenStack's design principles... which
> > >I seem to remember at one time we had written down somewhere.
> > >
> > >[1] https://en.wikipedia.org/wiki/Robustness_principle
> >
> > Just to throw a monkey-wrench in,
> > https://tools.ietf.org/html/draft-thomson-postel-was-wrong-00
> 
> To be clear, I agree with Thomson, and think that's the way to go.
> 
> However, I believe we haven't stated either in our principles (and if
> somebody has a link to those principles, or a clear assertion that we
> do not have them and why we don't have them, that would be helpful).
> 
> Adding tc to bump the people most likely to respond to that.

It may not always be possible to check whether a body exists, as the has body 
can sometimes end up being ignored depending in on the HTTP method being used 
when using chunked encoding. Unless anyone knows how to always check for a 
body, as webobs implementation is to use the HTTP method to make an informed 
guess it appears.

If we try and return a 400. This could lead to different results such as a body 
with a non chunked encoding returning a 400, and a body with a chunked encoding 
not returning a 400. Therefore would it be better to ignore the body in all 
cases, as that would mean the results will always be the same with different 
encodings.

Niall
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [glance][api] Response when a illegal body is sent

[Bunting, Niall]

Hi,

Currently when a body is passed to an API operation that explicitly does not 
allow bodies Glance throws a 500.

Such as in this bug report: https://bugs.launchpad.net/glance/+bug/1475647 This 
is an example of a GET however this also applies to other requests.

What should Glance do rather than throwing a 500, should it return a 400 as the 
user provided an illegal body or should Glance ignore the body and continue?

Regards,
Niall

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev