Re: [openstack-dev] Proposal for a process to keep up with Python releases

[William M Edmonds]

Zane Bitter  wrote on 10/22/2018 03:12:46 PM:
> On 22/10/18 10:33 AM, Thomas Goirand wrote:
> > On 10/19/18 5:17 PM, Zane Bitter wrote:



> >> Integration Tests
> >> -
> >>
> >> Integration tests do test, amongst other things, integration with
> >> non-openstack-supplied things in the distro, so it's important that we
> >> test on the actual distros we have identified as popular.[2] It's also
> >> important that every project be testing on the same distro at the end
of
> >> a release, so we can be sure they all work together for users.
> >
> > I find very disturbing to see the project only leaning toward these
only
> > 2 distributions. Why not SuSE & Debian?
>
> The bottom line is it's because targeting those two catches 88% of our
> users. (For once I did not make this statistic up.)
>
> Also note that in practice I believe almost everything is actually
> tested on Ubuntu LTS, and only TripleO is testing on CentOS. It's
> difficult to imagine how to slot another distro into the mix without
> doubling up on jobs.

I think you meant 78%, assuming you were looking at the latest User Survey
results [1], page 55. Still a hefty number.

It is important to note that the User Survey lumps all versions of a given
OS together, whereas the TC reference [2] only considers the latest
LTS/stable version. If the User Survey split out latests LTS/stable
versions vs. others (e.g. Ubuntu 16.04 LTS), I expect we'd see Ubuntu 18.04
LTS + Centos 7 adding up to much less than 78%.

[1] https://www.openstack.org/assets/survey/April2017SurveyReport.pdf
[2]
https://governance.openstack.org/tc/reference/project-testing-interface.html#linux-distributions
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][limits] Does ANYONE at all use the quota class functionality in Nova?

[William M Edmonds]

melanie witt  wrote on 10/25/2018 02:14:40 AM:
> On Thu, 25 Oct 2018 14:12:51 +0900, ボーアディネシュ[bhor Dinesh] wrote:
> > We were having a similar use case like *Preemptible Instances* called
as
> > *Rich-VM’s* which
> >
> > are high in resources and are deployed each per hypervisor. We have a
> > custom code in
> >
> > production which tracks the quota for such instances separately and for

> > the same reason
> >
> > we have *rich_instances* custom quota class same as *instances* quota
class.
>
> Please see the last reply I recently sent on this thread. I have been
> thinking the same as you about how we could use quota classes to
> implement the quota piece of preemptible instances. I think we can
> achieve the same thing using unified limits, specifically registered
> limits [1], which span across all projects. So, I think we are covered
> moving forward with migrating to unified limits and deprecation of quota
> classes. Let me know if you spot any issues with this idea.

And we could finally close https://bugs.launchpad.net/nova/+bug/1602396
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

[William M Edmonds]

Corey Bryant  wrote on 10/15/2018 05:34:24 PM:
...
> From an ubuntu perspective, ubuntu is going to support stein on 18.
> 04 LTS (3.6) and 19.04 (3.7) only.
...

So folks with Ubuntu 16.04 LTS compute nodes will have to upgrade them all
to 18.04 before upgrading to Stein? Of course this would be a distro
statement, and would not preclude someone from building their own
environment from source/pypi on Ubuntu 16.04. And 16.04 is still pretty
heavily used, right?

Ubuntu 18.04 LTS is not supported on PowerVM compute nodes, so the PowerVM
CI will not be able to switch to running under py3 if code that doesn't
work in py35 is introduced. At least until RHEL 8 comes out, at which point
we could switch to using that in our CI. But please don't allow such
changes before the RHEL 8 release.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [zVM] [python3] tox/zuul issues for zVM OpenStack

[William M Edmonds]

The current tox.ini for ceilometer-zvm includes this line [1] similar to
what ceilometer-powervm was doing up until recently:

   -egit+https://github.com/openstack/ceilometer@master#egg=ceilometer

We found that this no longer works since ceilometer was added to
upper-constraints [2]. We first got things working again by [3] and are now
improving on that by [4]. I expect you will need to make similar changes.

I was going to propose this to ceilometer-zvm for you, but then noticed
that you don't even have a zuul.yaml file. You're missing changes like [5]
adding lower-constraints checking and [6] for the python3-first effort. So
that is probably a bigger issue to address first (and for
networking-powervm and nova-zvm-virt-driver as well as ceilometer-zvm).
When you get to the python3 stuff, I suggest you work with dhellmann on
that. He's got scripts to generate at least some of those commits.

[1] http://git.openstack.org/cgit/openstack/ceilometer-zvm/tree/tox.ini#n19
[2] https://review.openstack.org/#/c/601498
[3] https://review.openstack.org/#/c/609058/
[4] https://review.openstack.org/#/c/609823/
[5] https://review.openstack.org/#/c/555358/
[6] https://review.openstack.org/#/c/594984/

W. Matthew Edmonds
Sr. Software Engineer, IBM Power Systems
Email: edmon...@us.ibm.com
Phone: (919) 543-7538 / Tie-Line: 441-7538
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [goal][python3] week 7 update

[William M Edmonds]

Doug Hellmann  wrote on 09/26/2018 06:29:11 PM:

> * We do not want to set the override once in testenv, because that
>   breaks the more specific versions used in default environments like
>   py35 and py36 (at least under older versions of tox).


I assume that something like
https://git.openstack.org/cgit/openstack/nova-powervm/commit/?id=fa64a93c965e6a6692711962ad6584534da81695
 should be a perfectly acceptable alternative in at least some cases.
Agreed?
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tempest][CI][nova compute] Skipping non-compute-driver tests

[William M Edmonds]

Ghanshyam Mann  wrote on 09/07/2018 02:18:13 AM:

snip..

> neutron-tempest-plugin or other service test you can always avoid to
> run with regex. And i do not think compute negative or DB test will
> take much time to run. But still if you want to avoid to run then, I
> think it is easy to maintain a whitelist regex file on CI side which
> can run only compute driver tests(61 in this case).
>
> Tagging compute driver on tempest side is little hard to maintain i
> feel and it can goes out of date very easily. If you have any other
> idea on that, we can surly talk in PTG on this.

The concern that I have with whitelisting in a given CI is that it has to
be done independently in every compute driver CI. So while I agree that it
won't be easy to maintain tagging for compute driver on the tempest side,
at least that's one place / easier than doing it in every driver CI. When
anyone figures out that an change is needed, all of the CIs would benefit
together if there is a shared solution.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [goal][python3] week 2 update

[William M Edmonds]

Doug Hellmann  wrote on 08/20/2018 11:27:09 AM:
> If your team is ready to have your zuul settings migrated, please
> let us know by following up to this email. We will start with the
> volunteers, and then work our way through the other teams.

I think PowerVMStackers is ready (so nova-powervm, networking-powervm,
ceilometer-powervm).
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] keypair quota usage info for user

[William M Edmonds]

Ghanshyam Mann  wrote on 07/25/2018 05:44:46 AM:
... snip ...
> 1. is it ok to show the keypair used info via API ? any original
> rational not to do so or it was just like that from starting.

keypairs aren't tied to a tenant/project, so how could nova track/report a
quota for them on a given tenant/project? Which is how the API is
constructed... note the "tenant_id" in
GET /os-quota-sets/{tenant_id}/detail

> 2. Because this change will show the keypair used quota information
> in API's existing filed 'in_use', it is API behaviour change (not
> interface signature change in backward incompatible way) which can
> cause interop issue. Should we bump microversion for this change?

If we find a meaningful way to return in_use data for keypairs, then yes, I
would expect a microversion bump so that callers can distinguish between a)
talking to an older installation where in_use is always 0 vs. b) talking to
a newer installation where in_use is 0 because there are really none in
use. Or if we remove keypairs from the response, which at a glance seems to
make more sense, that should also have a microversion bump so that someone
who expects the old response format will still get it.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][requirements] a plan to stop syncing requirements into projects

[William M Edmonds]

Doug Hellmann  wrote on 05/14/2018 08:52:08 AM:
>
... snip ...
>
> We still have about 50 open patches related to adding the
> lower-constraints test job. I'll keep those open until the third
> milestone of the Rocky development cycle, and then abandon the rest to
> clear my gerrit view so it is usable again.
>
> If you want to add lower-constraints tests to your project and have
> an open patch in the list [1], please take it over and fix the
> settings then approve the patch (the fix usually involves making
> the values in lower-constraints.txt match the values in the various
> requirements.txt files).
>
> If you don't want the job, please leave a comment on the patch to
> tell me and I will abandon it.
>
> Doug
>
> [1] https://review.openstack.org/#/q/topic:requirements-stop-syncing
+status:open

I believe we're stuck for nova-powervm [1] and ceilometer-powervm [2]
until/unless nova and ceilometer, respectively, post releases to pypi. Is
anyone working on that?

Even then, I don't love what we've had to do to get this working for
networking-powervm [3][4], which is what we'd do for nova-powervm and
ceilometer-powervm as well once they're on pypi. When you consider master,
it's a really nasty hack (including a non-master version in
requirements.txt because obviously master can't be on pypi). It's better
than not testing, but if someone has a better idea...

And I'd appreciate -infra reviews on [4] since I have no idea how to ensure
that's doing what it's intended to do.

[1] https://review.openstack.org/#/c/555964/
[2] https://review.openstack.org/#/c/555358/
[3] https://review.openstack.org/#/c/555936/
[4] https://review.openstack.org/#/c/569104/
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [requirements][horizon][neutron] plugins depending on services

[William M Edmonds]

Monty Taylor  wrote on 04/25/2018 09:40:47 AM:
...
> Introduce a whitelist of git repo urls, starting with:
>
>* https://git.openstack.org/openstack/neutron
>* https://git.openstack.org/openstack/horizon
>
We would also need to include at least nova (e.g. [1]) and ceilometer (e.g.
[2]).

[1] https://github.com/openstack/nova-powervm
[2] https://github.com/openstack/ceilometer-powervm
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [docs] When should we say 'run as root' in the docs?

[William M Edmonds]

Matt Riedemann  wrote on 04/19/2018 06:11:58 PM:
> How loose are we with saying things like, "you should run this as root"
> in the docs?
>
> I was triaging this nova bug [1] which is saying that the docs should
> tell you to run nova-status (which implies also nova-manage) as root,
> but isn't running admin-level CLIs implied that you need root, or
> something with access to those commands (sudo)?
>
> I'm not sure how prescriptive we should be with stuff like this in the
> docs because if we did start saying this, I feel like we'd have to say
> it everywhere.
>
> [1] https://bugs.launchpad.net/nova/+bug/1764530

Maybe instead of treating this as a docs bug, we should fix the command to
return a nicer error when run as non-root. Presumably the caller has root
access, but forgot they were logged in as something else or forgot sudo.
Dumping that stack trace on them is more likely to confuse than anything.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] VMware NSX CI - no longer running?

[William M Edmonds]

melanie witt  wrote on 03/29/2018 06:03:26 AM:

> I would like to see the VMware CI running again and it need only run on
> changes under the nova/virt/vmwareapi/ tree, to save on your resources.
> And on our side, I'd like us to add VMware subteam members to VMware
> driver patch reviews (I believe most of the active team members are
> listed on the priorities etherpad [0]) and to be sure we consult VMware
> CI votes when we review.

running only on virt/vmwareapi changes would not catch problems caused by
changes elsewhere, such as compute/manager.py or virt/driver.py
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [requirements] we are now unfrozen and branched

[William M Edmonds]

I'm not seeing a stable/queens branch for openstack/requirements yet. Is
that not what you meant? When is that projected?

Matthew Thode  wrote on 02/12/2018 11:25:44 AM:
> This means we are back to business as usual.
>
> cycle trailing projects have been warned not to merge requirements
> updates until they branch or get an ack from a requirements core.
>
> --
> Matthew Thode (prometheanfire)
>
__
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [osc][python-openstackclient] Consistency of option name

[William M Edmonds]

Graham Hayes <g...@ham.ie> wrote on 02/12/2018 11:17:45 AM:
> On 12/02/18 16:04, William M Edmonds wrote:
> > keystone may have taken "domain", but it didn't take "dns-domain"
>
> No, but the advice at the time was to move to zone, and match DNS
> RFCs, and not namespace objects with the service type.
>

I wasn't trying to criticize or question history but rather to look
forward. IF we change the name, "dns-domain" could be an option. That is
all.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [osc][python-openstackclient] Consistency of option name

[William M Edmonds]

keystone may have taken "domain", but it didn't take "dns-domain"


Dean Troyer  wrote on 02/12/2018 10:24:05 AM:
>
> On Mon, Feb 12, 2018 at 9:13 AM, Graham Hayes  wrote:
> > OSC only predates Designate by 5 months ...
>
> My bad, I didn't check dates.
>
> > "Zone" was what we were recommend to use by the OSC devs at the time we
> > wrote our OSC plugin, and at the time we were also *not* supposed to
> > name space commands inside service parent (e.g. openstack zone create
vs
> > openstack dns zone create).
>
> Namespacing commands and naming options are totally separate things.
> It is likely I suggested --zone at the time, and in the context of DNS
> commands it is very clear.  Also, in the context of Compute commands,
> --zone meaning availability zone is also clear.
>
> > For command flags --dns-zone seems like a good idea - but having a
plain
> > --zone is confusing when we have a top level "zone" object in the CLI,
> > when the type of object that "--zone" refers to is different to
> > "openstack zone "
>
> Again, if there is confusion, things should be more specifically named
> to remove the confusion.  Maybe allowing "zone" to be assumed to be a
> DNS zone was a mistake, I've made plenty of those in OSC already, so
> there is precedent, but it seemed reasonable at the time and we (OSC
> team) do not control what external plugins do.
>
> For example, I really resist using abbreviations in OSC, but in some
> places to not do so is to buck trends that any semi-experienced user
> in the field would expect.  The last discussion of this was last week
> regarding "MTU" in Network commands.
>
> These are not hard rules, but strong guidelines that can and should be
> interpreted in the context that they will be applied.  And in the end,
> the result should be one that is understandable, clear and even
> expected by the users.
>
> dt
>
> --
>
> Dean Troyer
> dtro...@gmail.com
>
>
__
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> https://urldefense.proofpoint.com/v2/url?
>
u=http-3A__lists.openstack.org_cgi-2Dbin_mailman_listinfo_openstack-2Ddev=DwIGaQ=jf_iaSHvJObTbx-

>
siA1ZOg=uPMq7DJxi29v-9CkM5RT0pxLlwteWvldJgmFhLURdvg=Fr9TF_mDZVJgACWKoyXcnphs-6rMDWufyRhpQEtUask=m5wXNx8okCgs7CbNoMhHEQev0xJCFIq61pcmnWBugSs=

>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone] Keystone Team Update - Week of 15 January 2018

[William M Edmonds]

welcome, Gage! Congrats!

Boris, Steve, Brant, Brad... you are and will be missed.


W. Matthew Edmonds
Sr. Software Engineer, IBM Power Systems
Email: edmon...@us.ibm.com
Phone: (919) 543-7538 / Tie-Line: 441-7538



From:   Colleen Murphy 
To: "OpenStack Development Mailing List (not for usage questions)"

Date:   01/19/2018 12:55 PM
Subject:[openstack-dev] [keystone] Keystone Team Update - Week of 15
January 2018



# Keystone Team Update - Week of 15 January 2018

## News

### Core team changes

We added a new core reviewer! Thanks Gage Hugo for all your hard work
and for stepping up to take on more responsibility!

We also lost some core members: Boris Bobrov, Steve Martinelli, Brant
Knudson and Brad Topol have stepped down from core membership after
having made enormous contributions over the years. We're grateful to
them for everything they've done to make keystone better and welcome
them back any time.

### Proposed community goals for Rocky

There are five community goals[1][2][3][4][5] proposed for Rocky that
are under discussion. In the meeting this week we had some confusion
and conerns over whether the proposed goal about pagination links[3]
would apply to us. We don't paginate anything in keystone, so the goal
wouldn't apply to us. The one that would potentially apply to keystone
is about mutable configuration[5]. If you have thoughts on any of
these potential community goals, including whether the team has the
capacity to take on this work, make your voice heard on the reviews.

### PTG Planning

We still need to put some thought into our agenda for the PTG. Add
your ideas to the etherpad[6] and also add your name if you're going
to be attending so that we can organize a team dinner.

I noticed that no one requested a BM/VM room for the cross-project
days of the PTG[7]. If we want to organize discussions with those
teams we might want to start thinking about that now, but we will be
able to book rooms spontaneously if we want to.

[1]
https://urldefense.proofpoint.com/v2/url?u=https-3A__review.openstack.org_513875=DwIGaQ=jf_iaSHvJObTbx-siA1ZOg=uPMq7DJxi29v-9CkM5RT0pxLlwteWvldJgmFhLURdvg=WZyK2gn2PyQ8sv1VSFd0wmEw1KZMyiZucOwFiQVLhy4=1DNtvlP9hEpIKo3pCN68D0H4y4F46EqgJGWo4yGs_x8=

[2]
https://urldefense.proofpoint.com/v2/url?u=https-3A__review.openstack.org_532361=DwIGaQ=jf_iaSHvJObTbx-siA1ZOg=uPMq7DJxi29v-9CkM5RT0pxLlwteWvldJgmFhLURdvg=WZyK2gn2PyQ8sv1VSFd0wmEw1KZMyiZucOwFiQVLhy4=JHB0pEqz9O1eVNcmGvRbRrbhuqEt8ZYG8DwwU59szN0=

[3]
https://urldefense.proofpoint.com/v2/url?u=https-3A__review.openstack.org_532627=DwIGaQ=jf_iaSHvJObTbx-siA1ZOg=uPMq7DJxi29v-9CkM5RT0pxLlwteWvldJgmFhLURdvg=WZyK2gn2PyQ8sv1VSFd0wmEw1KZMyiZucOwFiQVLhy4=J8PVKWbG2l-pwPXNV9WGgjmRpDYPIxkE7r-NsNCr6DA=

[4]
https://urldefense.proofpoint.com/v2/url?u=https-3A__review.openstack.org_533544=DwIGaQ=jf_iaSHvJObTbx-siA1ZOg=uPMq7DJxi29v-9CkM5RT0pxLlwteWvldJgmFhLURdvg=WZyK2gn2PyQ8sv1VSFd0wmEw1KZMyiZucOwFiQVLhy4=G76CnD1Jab84Ng4YY3YDClK5_tuxwL5SIAmq7ZOpess=

[5]
https://urldefense.proofpoint.com/v2/url?u=https-3A__review.openstack.org_534605=DwIGaQ=jf_iaSHvJObTbx-siA1ZOg=uPMq7DJxi29v-9CkM5RT0pxLlwteWvldJgmFhLURdvg=WZyK2gn2PyQ8sv1VSFd0wmEw1KZMyiZucOwFiQVLhy4=2sGC1vqKoMD5TPi6CepPxg_Px2gF0udR01_JRoze6IU=

[6]
https://urldefense.proofpoint.com/v2/url?u=https-3A__etherpad.openstack.org_p_keystone-2Drocky-2Dptg=DwIGaQ=jf_iaSHvJObTbx-siA1ZOg=uPMq7DJxi29v-9CkM5RT0pxLlwteWvldJgmFhLURdvg=WZyK2gn2PyQ8sv1VSFd0wmEw1KZMyiZucOwFiQVLhy4=AK0hgcNrTZn050SWudyNJMXhB3nq5Hvm1YK8EP_TIGU=

[7]
https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.openstack.org_pipermail_openstack-2Ddev_2018-2DJanuary_126335.html=DwIGaQ=jf_iaSHvJObTbx-siA1ZOg=uPMq7DJxi29v-9CkM5RT0pxLlwteWvldJgmFhLURdvg=WZyK2gn2PyQ8sv1VSFd0wmEw1KZMyiZucOwFiQVLhy4=ojAJr5Ffr8mZcDP98KavDm9eEI4WEKflLHDa6CTETbI=


## Recently Merged Changes

Search query:
https://urldefense.proofpoint.com/v2/url?u=https-3A__goo.gl_hdD9Kw=DwIGaQ=jf_iaSHvJObTbx-siA1ZOg=uPMq7DJxi29v-9CkM5RT0pxLlwteWvldJgmFhLURdvg=WZyK2gn2PyQ8sv1VSFd0wmEw1KZMyiZucOwFiQVLhy4=46ojFnkizMRWuTYLT_r1aH2xUaWypLtFZWh08wDI2uM=


We merged 38 changes this week. Lots of these were major stepping
stones for our new features.

## Changes that need Attention

Search query:
https://urldefense.proofpoint.com/v2/url?u=https-3A__goo.gl_h9knRA=DwIGaQ=jf_iaSHvJObTbx-siA1ZOg=uPMq7DJxi29v-9CkM5RT0pxLlwteWvldJgmFhLURdvg=WZyK2gn2PyQ8sv1VSFd0wmEw1KZMyiZucOwFiQVLhy4=xeWa9B-DRwjQ781pFfjgfna8ogWhj3TYKU8NtF61HZA=


There are 55 changes that are passing CI, not in merge conflict, have
no negative reviews and aren't proposed by bots. Please prioritize
reviews for python-keystoneclient and our major feature initiatives
(see below).

## Milestone Outlook

Re: [openstack-dev] [nova] Working toward Queens feature freeze and RC1

[William M Edmonds]

> From: William M Edmonds/Raleigh/IBM
> To: "OpenStack Development Mailing List \(not for usage questions\)"
> <openstack-dev@lists.openstack.org>
> Date: 01/08/2018 03:11 PM
> Subject: Re: [openstack-dev] [nova] Working toward Queens feature
> freeze and RC1
>
> > From: Matt Riedemann <mriede...@gmail.com>
> > To: "OpenStack Development Mailing List (not for usage questions)"
> > <openstack-dev@lists.openstack.org>
> > Date: 01/03/2018 07:03 PM
> > Subject: [openstack-dev] [nova] Working toward Queens feature freeze
and RC1
> >
> ... snip ...
> > The rest of the blueprints are tracked here:
> >
> > https://etherpad.openstack.org/p/nova-queens-blueprint-status
>
> I updated that etherpad with the latest status for the powervm
> blueprint. Should have 2 of the 3 remaining patches ready for review
> in the next day or two, and the last later in the week.

All of the powervm patches are ready for core reviews, and the etherpad has
been updated accordingly. Thanks in advance!
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Working toward Queens feature freeze and RC1

[William M Edmonds]

> From: Matt Riedemann 
> To: "OpenStack Development Mailing List (not for usage questions)"
> 
> Date: 01/03/2018 07:03 PM
> Subject: [openstack-dev] [nova] Working toward Queens feature freeze and
RC1
>
... snip ...
> The rest of the blueprints are tracked here:
>
> https://urldefense.proofpoint.com/v2/url?
>
u=https-3A__etherpad.openstack.org_p_nova-2Dqueens-2Dblueprint-2Dstatus=DwIGaQ=jf_iaSHvJObTbx-

>
siA1ZOg=uPMq7DJxi29v-9CkM5RT0pxLlwteWvldJgmFhLURdvg=HVyvQHTZ4ft1C3JEJ9ij0uXwEy5_y3egSY7kNu_BvcU=mmvsEIKWRecnDlvYgLPwBAfPlVQQV5HEtHYMdDuaRME=


I updated that etherpad with the latest status for the powervm blueprint.
Should have 2 of the 3 remaining patches ready for review in the next day
or two, and the last later in the week.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [policy] AWS IAM session

[William M Edmonds]

+1

Lance Bragstad  wrote on 10/03/2017 04:08:31 PM:
> Hey all,
>
> It was mentioned in today's keystone meeting [0] that it would be useful
> to go through AWS IAM (or even GKE) as a group. With all the recent
> policy discussions and work, it seems useful to get our eyes on another
> system. The idea would be to spend time using a video conference/screen
> share to go through and play with policy together. The end result should
> keep us focused on the implementations we're working on today, but also
> provide clarity for the long-term vision of OpenStack's RBAC system.
>
> Are you interested in attending? If so, please respond to the thread.
> Once we have some interest, we can gauge when to hold the meeting, which
> tools we can use, and setting up a test IAM account.
>
> Thanks,
>
> Lance
>
> [0]
> http://eavesdrop.openstack.org/meetings/keystone/2017/keystone.
> 2017-10-03-18.00.log.html#l-119
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone][zuul] A Sad Farewell

[William M Edmonds]

Jamie Lennox  wrote on 10/02/2017 10:13:49 PM:
>
> Hi All,

> I'm really sad to announce that I'll be leaving the OpenStack
> community (at least for a while), I've accepted a new position
> unrelated to OpenStack that'll begin in a few weeks, and am going to
> be mostly on holiday until then.
>
> I want to thank everyone I've had the pleasure of working with over
> the last few years - but particularly the Keystone community. I feel
> we as a team and I personally grew a lot over that time, we made
> some amazing achievements, and I couldn't be prouder to have worked
> with all of you.
>
> Obviously I'll be around at least during the night for some of the
> Sydney summit and will catch up with some of you there, and
> hopefully see some of you at linux.conf.au. To everyone else, thank
> you and I hope we'll meet again.
>
> Jamie Lennox, Stacker.
>

Boo! You will be greatly missed.

But I hope you enjoy your new position. Whatever it is, they are getting a
good one.

Will look for you in Sydney!

-matthew
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone] Does the policy.json for trustsworks?

[William M Edmonds]

Adrian Turjak  wrote on 09/18/2017 01:39:20 AM:
>
> Bug submitted:
>
> https://urldefense.proofpoint.com/v2/url?
>
u=https-3A__bugs.launchpad.net_keystone_-2Bbug_1717847=DwIGaQ=jf_iaSHvJObTbx-

>
siA1ZOg=uPMq7DJxi29v-9CkM5RT0pxLlwteWvldJgmFhLURdvg=pc-9BTikvQSYJU9gcS334Ut4ER1gN6c2hXl3vGzdTPY=9S9InbF78aSW8ums9lJm8snzR6XbHYUibLuPMFLmnFU=

>
> Note that this is an odd one, since the current state (while unhelpful)
> is safe, fixing it has a chance of exposing an API to users that
> shouldn't be able to use it if operators don't update their policy file
> to match the new default we'd add.
>
>

I think we're actually mostly ok here. The one rule that looks off is the
one that I think you may have thought was correct... create_trust. I
updated the bug with reasoning. Please take a look and comment if I've
missed something or you've got further questions. Specific examples that
you've tried and got unexpected results would provide useful talking
points. Thanks!
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] summit free passes

[William M Edmonds]

It has been stated that attending the PTG will get you a free code for the
summit. Is that still true even though the PTG falls after the early
registration deadline in this case, or will it simply be a discount of the
early-registration cost?

Similarly, would someone with a free registration code from the Atlanta PTG
have to use it by the early-registration deadline in order for their
registration to be completely free and not simply discounted? That has been
the case in the past, but the emails that went out with those codes seemed
to imply that you could use them for free registration until October 27th.


W. Matthew Edmonds
Sr. Software Engineer, IBM Power Systems
Email: edmon...@us.ibm.com
Phone: (919) 543-7538 / Tie-Line: 441-7538
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [release] [telemetry] Ceilometer stable/pike branch outlook

[William M Edmonds]

Thierry Carrez <thie...@openstack.org> wrote on 08/17/2017 03:35:20 AM:
> From: Thierry Carrez <thie...@openstack.org>
> To: openstack-dev@lists.openstack.org
> Date: 08/17/2017 03:37 AM
> Subject: Re: [openstack-dev] [release] [telemetry] Ceilometer
> stable/pike branch outlook
>
> Tony Breeds wrote:
> > On Wed, Aug 16, 2017 at 02:37:50PM -0400, William M Edmonds wrote:
> >>
> >> Julien Danjou <jul...@danjou.info> wrote on 08/16/2017 02:13:10 PM:
> >>> AFAIU it's impossible to cut a branch for our projects and release a
rc1
> >>> because of the release model we use. The release team does not allow
us
> >>> to do that. We need to release directly a stable version and cut a
> >>> branch.
> >>>
> >>> I guess we'll do that in a couple of week, at release time.
> >>
> >> That doesn't fit my understanding of cycle-with-intermediary, which is
the
> >> the ceilometer release model per [0]. As I read the release model
> >> definitions [1], cycle-with-intermediary means that you can have
> >> intermediate releases *as well*, but you still have to have a
cycle-ending
> >> release in line with the projects using the cycle-with-milestones
model.
> >>
> >> Can someone on the release team clarify this for us?
> >
> > That's correct the bit you're missing is cycle-with-intermediary
doesn't
> > have pre-releases (b{1,2,3},rc{1,2}) so when the ceilometer team feels
> > they have the code in shape for a release they'll tag that release and
> > cut a stable/pike branch at the tag point.
>
> Exactly. As explained a couple weeks ago in the release countdown email
[1]:
>

that email explicitly says: "Deliverables following the
cycle-with-intermediary model should also create their Pike release branch
next week. That means potentially making a last Pike release, and in all
cases posting the stable/pike branch creation request"

And that wasn't done for ceilometer. So it needs to be done, right?

> > Deliverables following the cycle-with-intermediary model should also
> > create their Pike release branch next week. That means potentially
> > making a last Pike release, and in all cases posting the stable/pike
> > branch creation request:
> >
> > ...
> > branches:
> >   - location: YOUR.PIKE.VERSION
> > name: stable/pike
>
> [1]
>
http://lists.openstack.org/pipermail/openstack-dev/2017-August/120574.html
>
> --
> Thierry Carrez (ttx)
>
> [attachment "signature.asc" deleted by William M Edmonds/Raleigh/
> IBM]
>
__
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [release] [telemetry] Ceilometer stable/pike branch outlook

[William M Edmonds]

Julien Danjou  wrote on 08/16/2017 02:13:10 PM:
> From: Julien Danjou 
> To: "Eric S Berglund" 
> Cc: openstack-dev@lists.openstack.org
> Date: 08/16/2017 02:14 PM
> Subject: Re: [openstack-dev] [release] [telemetry] Ceilometer
> stable/pike branch outlook
>
> On Wed, Aug 16 2017, Eric S Berglund wrote:
>
> Hi Eric,
>
> > Is there an outlook for cutting a pike branch for ceilometer?
> > We currently can't run our 3rd party CI against pike without a pike
> > release branch and are deciding whether it's worth the time to
> > implement a workaround.
>
> AFAIU it's impossible to cut a branch for our projects and release a rc1
> because of the release model we use. The release team does not allow us
> to do that. We need to release directly a stable version and cut a
> branch.
>
> I guess we'll do that in a couple of week, at release time.

That doesn't fit my understanding of cycle-with-intermediary, which is the
the ceilometer release model per [0]. As I read the release model
definitions [1], cycle-with-intermediary means that you can have
intermediate releases *as well*, but you still have to have a cycle-ending
release in line with the projects using the cycle-with-milestones model.

Can someone on the release team clarify this for us?

[0]
https://github.com/openstack/releases/blob/bf890914c1ec5bcd41d70140e80ef8d39df64c86/deliverables/pike/ceilometer.yaml#L3
[1] https://releases.openstack.org/reference/release_models.html
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [keystone][api] Backwards incompatible changes based on config

[William M Edmonds]

Lance Bragstad  wrote on 08/04/2017 02:37:40 PM:
> Properly fixing this would result in a 403 -> 204 status code, which
> requires an API version bump according to the interoperability
> guidelines [5] (note that keystone has not implemented microversions at
> this point). At the same time - not fixing the issues results in a 403
> anytime a project is deleted while in this configuration.
>

The guidelines you linked actually say that this is allowed without a
version bump:

"There are two types of change which do not require a version change:... or
responding with success (when the request was properly formed, but the
server had broken handling)."
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [telemetry] [ceilometer] [panko] ceilometer API deprecation

[William M Edmonds]

On 01/10/2017 09:26 AM, gordon chung wrote:
> On 10/01/17 07:27 AM, Julien Danjou wrote:
> > On Mon, Jan 09 2017, William M Edmonds wrote:
> >
> >> I started the conversation on IRC [5], but wanted to send this to the
> >> mailing list and see if others have thoughts/concerns here and figure
out
> >> what we should do about this going forward.
> >
> > Nothing? The code has not been removed, it has been moved to a new
> > project. Ocata will be the second release for Panko, so if user did not
> > switch already during Newton, they'll have to do it for Ocata. That's a
> > lot of overlap. Two cycles to switch to a "new" service should be
enough.
>
> well it's not actually two. it'd just be the one cycle in Newton since
> it's gone in Ocata. :P
>
> that said, for me, the move to remove it is to avoid any needless
> additional work of maintaining two active codebases. we're a small team
> so it's work we don't have time for.
>
> as i mentioned in chat, i'm ok with reverting patch and leaving it for
> Ocata but if the transition is clean (similiar to how aodh was split)
> i'd rather not waste resources on maintaining residual 'dead' code.

After discussing with my team, I think we will need to propose a revert.
The deprecation process was not followed correctly here. We will start
working on moving to panko, but we are not sure we can contain that for
Ocata. Please follow the deprecation process correctly in future and we can
avoid this hassle for everyone.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [telemetry] [ceilometer] [panko] ceilometer API deprecation

[William M Edmonds]

It's just come to my attention that the Newton release notes indicated
deprecation of the events support in favor of Panko [1] and that it has
already been removed in master [2]. But I don't believe there were any
deprecation warnings for this in the Newton code, which is the typical way
many folks learn about deprecations. The only deprecation warning I've
found was merged after Newton released [3]. And while the official
OpenStack deprecation policy allows you to remove things in the next
release after deprecation, it also says "Note that this delay is a required
minimum. For significant features, it is recommended that the deprecated
feature appears at least in the next two stable release branches." [4]. So
a) we should have had a deprecation warning in the code to go along with
the releasenote in Newton and b) even then we probably shouldn't be
removing the event API support until Pike.

I started the conversation on IRC [5], but wanted to send this to the
mailing list and see if others have thoughts/concerns here and figure out
what we should do about this going forward.

[1]
http://docs.openstack.org/releasenotes/ceilometer/newton.html#deprecation-notes
[2]
https://github.com/openstack/ceilometer/commit/8d23f431ab0bd638edbf2197e56bea68d7b06a21
[3]
https://github.com/openstack/ceilometer/commit/6616a714009a80c7484fa2292c2331868617cb9c
[4]
https://governance.openstack.org/tc/reference/tags/assert_follows-standard-deprecation.html
[5]
http://eavesdrop.openstack.org/irclogs/%23openstack-telemetry/%23openstack-telemetry.2017-01-09.log.html#t2017-01-09T17:24:36


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Let's kill quota classes (again)

[William M Edmonds]

On 12/16/2016 05:03 PM, Jay Pipes wrote:
> On 12/16/2016 04:36 PM, Matt Riedemann wrote:
> > On 12/16/2016 2:20 PM, Jay Pipes wrote:
> >>
> >> For problems with placing data like this as configuration options, see
> >> the hassle we went through in making the allocation_ratio options into
> >> fields stored in the DB...
> >>
> >> Better long-term to have all this kind of configuration live in a data
> >> store (not a config file) and be exposed via an HTTP API.
> >>
> >
> > So, we could do that, we already have the quota_classes table and the
> > os-quota-class-sets REST API, and as mentioned the only usable thing
> > that goes in there is overriding global default quota.
> >
> > Would you suggest we drop the global quota limit configuration options
> > and simply populate the quota_classes table with a 'default' quota
class
> > and the limits from the config in a DB migration, then drop the config
> > options?
>
> Yeah, I think that's the best long-term strategerization.

Why would someone need to change the defaults via REST API calls? I agree
that we should plan for that now if we think that will eventually be
needed, but I'm not seeing why it would be needed. And if we're not sure
this is needed now, we could still always do this later... at which point
we could do it a lot better than the current implementation being able to
start from scratch. Perhaps as part of the implementation of a new API that
allows you to change *any* mutable config option?
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cross-project] Admin

[William M Edmonds]

Adam Young <ayo...@redhat.com> wrote on 10/22/2015 10:31:12 AM:
> On 10/22/2015 05:16 AM, William M Edmonds wrote:
> Adam Young <ayo...@redhat.com> wrote on 10/19/2015 09:53:14 AM:
> > While I tend to play up  bug 968696 for dramatic effect, the reality is

> > we have a logical contradiction on what we mean by 'admin' when talking

> > about RBAC.
> >
> > In early iterations of OpenStack, roles were global.  This is reflected

> > in many of the Policy checks that only look for the global role.
> > However, prior to the Keystone-Light rewrite, role assignments became
> > scoped to tenants.  This shows up in the Keystone git history.  As this

> > pattern got established, some people wrote policy checks that assert:
> >
> >       role==admin and tenant_id=resource.tenant_id
> >
> > This contradicts the global-ness of the admin roles.  If I assign
> > ('joeuser', 'admin','mytenant') I've just granted them the ability to
> > perform all of the admin operations.
> >
> > Thus, today we have a situation where, unless the user rewrites the
> > default policy, they have to only assign the role  admins to users that

> > are trusted to be admins on the whole deployment.
> >
>
> This all appears to be based on a misassumptions that a) checking
> the project id should be done in policy.json files and b) if it's
> not being checked in the policy file then it's not being checked.
> Neither of those is the case. Many APIs check project id in the
> code, which is where it should be checked. Tokens are scoped to
> projects, thus any use of those tokens should necessarily be scoped
> to the project... otherwise you're not obeying the token scoping.
> The few places that are not already enforcing that in their code
> need to be fixed to start enforcing that. It doesn't make sense to
> do that in policy files, since this is a hard and fast rule, not
> something someone needs to be able to change in policy, or should be
> able to change. Nor would it be practical to put this in policy
> files when you realize that this logic applies to all roles, not just
admin.
>
> I agree that project_id check is better performed in code.  That is
> not the issue here.
>
> Checking Project ID needs to be done, policy file or code does not
> matter.  The problem is more fundamental.
>
> 0. All access is done with Keystone tokens.
> 1. Admin is a role assigned on a project. Always.
> 2. Some APIs have no project with which to check the Scope.

if the URI of the API doesn't include the project, then the code should
limit the request to the scope of the token. No need to check that the
scope of the token matches the scope of the API because scope wasn't
indicated on the API, i.e. scope is implicit based on the token.

> 3. We do not, today, have a means to communicate the scope for an
> admin project.

I don't know what you mean here. "Scope for an admin project"? The scope of
a project is the things that are in that project. And admin is a role that
can exist on any project per #1, i.e., there is no "admin project". We may
need to create the concept of an admin project for keystone in order to
address 968696, but that would be a keystone concept and irrelevant to
nova, cinder, etc.

-matthew
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cross-project] Admin

[William M Edmonds]

Adam Young  wrote on 10/19/2015 09:53:14 AM:
> While I tend to play up  bug 968696 for dramatic effect, the reality is
> we have a logical contradiction on what we mean by 'admin' when talking
> about RBAC.
>
> In early iterations of OpenStack, roles were global.  This is reflected
> in many of the Policy checks that only look for the global role.
> However, prior to the Keystone-Light rewrite, role assignments became
> scoped to tenants.  This shows up in the Keystone git history.  As this
> pattern got established, some people wrote policy checks that assert:
>
>   role==admin and tenant_id=resource.tenant_id
>
> This contradicts the global-ness of the admin roles.  If I assign
> ('joeuser', 'admin','mytenant') I've just granted them the ability to
> perform all of the admin operations.
>
> Thus, today we have a situation where, unless the user rewrites the
> default policy, they have to only assign the role  admins to users that
> are trusted to be admins on the whole deployment.
>

This all appears to be based on a misassumptions that a) checking the
project id should be done in policy.json files and b) if it's not being
checked in the policy file then it's not being checked. Neither of those is
the case. Many APIs check project id in the code, which is where it should
be checked. Tokens are scoped to projects, thus any use of those tokens
should necessarily be scoped to the project... otherwise you're not obeying
the token scoping. The few places that are not already enforcing that in
their code need to be fixed to start enforcing that. It doesn't make sense
to do that in policy files, since this is a hard and fast rule, not
something someone needs to be able to change in policy, or should be able
to change. Nor would it be practical to put this in policy files when you
realize that this logic applies to all roles, not just admin.

-Matthew
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Requests + urllib3 + distro packages

[William M Edmonds]

Cory Benfield  writes:
> > The problem that occurs is the result of a few interacting things:
> >  - requests has very very specific versions of urllib3 it works with.
> > So specific they aren't always released yet.
>
> This should no longer be true. Our downstream redistributors pointedout
to us
> that this  was making their lives harder than they needed to be, so it's
now
> our policy to only  update to actual release versions of urllib3.

That's great... except that I'm confused as to why requests would continue
to repackage urllib3 if that's the case. Why not just prereq the version of
urllib3 that it needs? I thought the one and only answer to that question
had been so that requests could package non-standard versions.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Requests + urllib3 + distro packages

[William M Edmonds]

Robert Collins  writes:
>  - Linux vendors often unbundle urllib3 from requests and then apply
> what patches were needed to their urllib3; while not updating their
> requests package dependencies to reflect this.

I opened a bug on Fedora for them to update their requests package
dependencies. See https://bugzilla.redhat.com/show_bug.cgi?id=1253823. Of
course that may continue to be an issue on older versions and other
distros.

>  - if for any reason we have a distro-altered requests + a
> pip-installed urllib3, requests will [usually] break... see the 'not
> always released yet' key thing above.
>
> Now, there are lots of places this last thing can happen; they all
> depend on us having a dependency on requests that is compatible with
> the version installed by the distro, but a urllib3 dependency that
> triggers an upgrade of just urllib3. When constraints are in use, the
> requests version has to match the distro requests version exactly, but
> that will happen from time to time.

When you're using a distro, you're always going to have to worry about
someone pip installing something that conflicts with the rpm, no? That
could be for any reason, could be completely unrelated to OpenStack
dependencies. Unless the distros have a way to put in protection against
this, preventing pip install of something that is already installed by RPM?

>  - make sure none of our testing environments include distro
> requests packages.

It's not like requests is an unusual package for someone to have installed
from their distro in a base OS image. So when they take that base OS and go
to setup OpenStack, they'll be hitting this case, whether we tested it or
not. So while not testing this case seems nice from a development
perspective, it doesn't seem to fit real-world usage. I don't think it
would make operators very happy.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [glance] Removing python-swiftclient from requirements.txt

[William M Edmonds]

 From: Doug Hellmann d...@doughellmann.com
 To: openstack-dev openstack-dev@lists.openstack.org
 Date: 07/28/2015 07:59 PM
 Subject: Re: [openstack-dev] [glance] Removing python-swiftclient
 from requirements.txt


snip

 I replied on both patches, but I'll repeat it here for a broader
 audience:

 Please set up an extras entry for each backend instead of just
 removing the dependencies.  That will signal to users that you know
 what dependencies there are for a backend, but that they are optional,
 and still allow someone to do the equivalent of pip install
 glance[vmware] or pip install glance[swift] to get those
 dependencies.  Nova and oslo.versionedobjects have examples in their
 setup.cfg if you need a template.

 I didn't mention in the reviews, but this will also make integration
 tests in our gate easier, since you can put .[vmware] or .[swift] in
 the tox.ini to pull in those dependencies.

 Doug


snip

I did this... but now I'm thinking I really shouldn't have. That makes
perfect sense for glance_store, but shouldn't that only be in glance_store,
and not also in glance?

Matthew__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [glance] Removing python-swiftclient from requirements.txt

[William M Edmonds]

 From: Flavio Percoco fla...@redhat.com
 To: OpenStack Development Mailing List (not for usage questions)
 openstack-dev@lists.openstack.org
 Date: 07/28/2015 07:36 AM
 Subject: Re: [openstack-dev] [glance] Removing python-swiftclient
 from requirements.txt

 On 28/07/15 09:15 +, Kuvaja, Erno wrote:
 I do agree, we don’t depend or are cleaning the other clients out
 of the glance
 dependencies as well and I think swift should not be there either.
 
 
 
 The default store is filesystem store and if something is depending on
the
 actual store clients it should be either glance_store or deployer (well
for
 example our gate) glance itself should not have hard dependencies for
‘em.

 Agreed!

 William, would it be possible for you to spend some more time and
 create a single patch that removes all non-required dependencies?

 Cheers,
 Flavio


This all started when I opened a bug [1] saying we needed to pull out
oslo.vmware. Louis quickly threw up a patch [2] to remove that, but it was
pointed out that swiftclient falls into the same category. So I created a
separate patch to remove swiftclient [3]. Looking over what else is in
requirements.txt and running a few searches, it looks like we may also be
able to remove greenlet, kombu, and posix-ipc. Does anyone know if any of
those (greenlet?) are needed for some reason other than a direct import? In
which case I can add a comment to clarify that while I'm removing the
others.

I'd originally included the removal of oslo.vmware in [3], but I pulled
that out thinking we could go ahead and merge [2] while we were having this
discussion. But that didn't seem to fly, so I guess we want to make all
these changes together under [3] and abandon [2]? Or should we go ahead and
merge [2] while we're figuring out whether to remove greenlet, kombu, and
posix-ipc as well under [3]?

Just to be clear, it sounds to me like Flavio and Erno agree we should pull
swiftclient out of requirements.txt immediately. I'd like to see a reply
from Ian and Louis to round things out, make sure we're all on the same
page and we won't be fighting over this in the review...

[1] https://launchpad.net/bugs/1475737
[2] https://review.openstack.org/#/c/203200/
[3] https://review.openstack.org/#/c/203242/

 
 
 
 -  Erno
 
 
 
 From: William M Edmonds [mailto:edmon...@us.ibm.com]
 Sent: Monday, July 27, 2015 10:42 PM
 To: openstack-dev@lists.openstack.org
 Subject: [openstack-dev] [glance] Removing python-swiftclient from
 requirements.txt
 
 
 
 python-swiftclient is only needed by operators that are using the swift
 backend, so it really doesn't belong in requirements.txt. Listing it in
 requirements forces all operators to install it, even if they're not
going to
 use the swift backend. When I proposed a change [1] to move this from
 requirements to test-requirements (would still be needed there
 because of tests
 using the swift backend), others raised concerns about the impact this
could
 have on operators who use the swift backend today and would be upgrading
to
 Liberty. I believe everyone agreed this should not be in
 requirements, but the
 fact is that it has been, so operators may have (incorrectly) been
 depending on
 that during upgrades. If we remove it in Liberty, and there are changes
in
 Liberty that require a newer version of swiftclient, how would
 those operators
 know that they need to upgrade swiftclient?
 
 The optional dependencies spec [2] could definitely help here. I
 don't think we
 should have to wait for that, though. This is an issue we obviously
already
 have today for other backends, which indicates folks can deal with it
without
 an optional dependencies implementation.
 
 This would be a new concern for operators using the default swift
backend,
 though. So how do we get the message out to those operators? And dowe
need to
 put out a message about this change in Liberty and then wait until
Mitaka to
 actually remove this, or can we go ahead and remove in Liberty?
 
 [1] https://review.openstack.org/#/c/203242
 [2] http://specs.openstack.org/openstack/oslo-specs/specs/liberty/
 optional-deps.html
 
 -Matthew
 


__
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


 --
 @flaper87
 Flavio Percoco
 [attachment attdy18a.dat deleted by William M Edmonds/Raleigh/IBM]

__
 OpenStack Development Mailing List (not for usage questions)
 Unsubscribe:
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org

[openstack-dev] [glance] Removing python-swiftclient from requirements.txt

[William M Edmonds]

python-swiftclient is only needed by operators that are using the swift
backend, so it really doesn't belong in requirements.txt. Listing it in
requirements forces all operators to install it, even if they're not going
to use the swift backend. When I proposed a change [1] to move this from
requirements to test-requirements (would still be needed there because of
tests using the swift backend), others raised concerns about the impact
this could have on operators who use the swift backend today and would be
upgrading to Liberty. I believe everyone agreed this should not be in
requirements, but the fact is that it has been, so operators may have
(incorrectly) been depending on that during upgrades. If we remove it in
Liberty, and there are changes in Liberty that require a newer version of
swiftclient, how would those operators know that they need to upgrade
swiftclient?

The optional dependencies spec [2] could definitely help here. I don't
think we should have to wait for that, though. This is an issue we
obviously already have today for other backends, which indicates folks can
deal with it without an optional dependencies implementation.

This would be a new concern for operators using the default swift backend,
though. So how do we get the message out to those operators? And do we need
to put out a message about this change in Liberty and then wait until
Mitaka to actually remove this, or can we go ahead and remove in Liberty?

[1] https://review.openstack.org/#/c/203242
[2]
http://specs.openstack.org/openstack/oslo-specs/specs/liberty/optional-deps.html

-Matthew__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [ceilometer] upgrades from juno to kilo

[William M Edmonds]

I tracked down the cause of the check-grenade-dsvm failure on
https://review.openstack.org/#/c/167370 . As I understand it, grenade is
taking the previous stable release, deploying it, then upgrading to the
current master (plus the proposed changeset) without changing any of the
config from the stable deployment. Thus the policy.json file used in that
test is the file from stable-juno. Then if we look at oslo_policy/policy.py
we see that if the rule being looked for is missing then the default rule
will be used, but then if that default rule is also missing a KeyError is
thrown. Since the default rule was missing with ceilometer's policy.json
file in Juno, that's what would happen here. I assume that KeyError then
gets turned into the 403 Forbidden that is causing check-grenade-dsvm
failure.

I suspect the author of the already-merged
https://review.openstack.org/#/c/115717 did what they did in
ceilometer/api/rbac.py rather than what is proposed in
https://review.openstack.org/#/c/167370 just to get the grenade tests to
pass. I think they got lucky (unlucky for us), too, because I think they
actually did break what the grenade tests are meant to catch. The patch set
which was merged under https://review.openstack.org/#/c/115717 changed the
rule that is checked in get_limited_to() from context_is_admin to
segregation. But the segregation rule didn't exist in the Juno version
of ceilometer's policy.json, so if a method that calls get_limited_to() was
tested after an upgrade, I believe it would fail with a 403 Forbidden
tracing back to a KeyError looking for the segregation rule... very
similar to what we're seeing in https://review.openstack.org/#/c/167370

Am I on the right track here? How should we handle this? Is there a way to
maintain backward compatibility while fixing what is currently broken (as a
result of https://review.openstack.org/#/c/115717 ) and allowing for a fix
for https://bugs.launchpad.net/ceilometer/+bug/1435855 (the goal of
https://review.openstack.org/#/c/167370)? Or will we need to document in
the release notes that the manual step of modifying ceilometer's
policy.json is required when upgrading from Juno, and then correspondingly
modify grenade's upgrade_ceilometer file?


W. Matthew Edmonds
IBM Systems  Technology Group
Email: edmon...@us.ibm.com
Phone: (919) 543-7538 / Tie-Line: 441-7538__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev