Re: [openstack-dev] [cyborg] [nova] Poll: Name for VARs

[Nadathur, Sundar]

Thanks for all who participated in the discussion and/or voted. The most 
votes, such as there were, went for the name 'Accelerator Requests' 
abbrev. ARQs. The specs will be updated over the next couple of days.


Have a good weekend.

Best Regards,
Sundar

On 10/22/2018 11:37 AM, Nadathur, Sundar wrote:

Hi,
The name VAR (Virtual Accelerator Request) is introduced in 
https://review.openstack.org/#/c/603955/. It came up during the Stein 
PTG and is being used by default, but some folks have said they find 
the name VAR to be confusing. I would like to resolve this to 
completion, so that whatever name we choose is not subject to 
recurrent debates in the future.


Here is a poll for Cyborg and Nova developers to indicate their 
preferences for existing or proposed options:
https://docs.google.com/spreadsheets/d/179Q8J9qIJNOiVm86K7bWPxo7otTsU18XVCI32V77JaU/edit?usp=sharing 



1. Please add your name, if not already listed, and please feel free 
to propose additional options as you see fit.

2. The voting is by rank -- 1 indicates most preferred.
3. If you strongly oppose a term, you may say 'No' and justify with a 
comment.

   (Comments are added by pressing Ctrl-Alt-M on a cell.)

I'll keep this open for a minimum of two days and possibly for a week 
depending on feedback.


Regards,
Sundar


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [goals][upgrade-checkers] Week R-24 Update

[Matt Riedemann]

There isn't much news this week except some of the base framework 
changes being proposed to projects are getting merged which is nice to see.


https://storyboard.openstack.org/#!/story/2003657

https://review.openstack.org/#/q/topic:upgrade-checkers+status:merged

And there are a lot of patches that are ready for review:

https://review.openstack.org/#/q/topic:upgrade-checkers+status:open

--

Thanks,

Matt

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [puppet][tripleo] puppet 5.5.7 breaks a bunch of stuff

[Alex Schultz]

Just a heads up. I've been battling with some unit test issues with
the latest version of puppet 5.5.  I've proposed some fixes[0][1], but
it appears that there is a larger issue with legacy functions which
affects the stable branches.  I've reported the issues[2][3] upstream
to Puppetlabs, but it'll likely be some time before we have any
resolution. In the mean time I would recommend pinning to 5.5.6 if
possible.

Thanks,
-Alex

[0] https://bugs.launchpad.net/puppet-nova/+bug/1799757
[1] https://bugs.launchpad.net/tripleo/+bug/1799786
[2] https://tickets.puppetlabs.com/browse/PUP-9270
[3] https://tickets.puppetlabs.com/browse/PUP-9271

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] Update to flake8 and failures despite # flake8: noqa

[Sean McGinnis]

On Fri, Oct 26, 2018 at 12:53:17PM -0400, David Moreau Simard wrote:
> Hi openstack-dev,
> 
> I stumbled on odd and sudden pep8 failures with ARA recently and
> brought it up in #openstack-infra [1].
> 
> It was my understanding that appending "  # flake8: noqa" to a line of
> code would have flake8 ignore this line if it happened to violate any
> linting rules.
> It turns out that, at least according to the flake8 release notes [2],
> "flake8: noqa" is actually meant to ignore the linting on an entire
> file.
> 
> The correct way to ignore a specific line appears to be to append "  #
> noqa" to the line... without "flake8: ".
> Looking at codesearch [3], there is a lot of projects using the
> "flake8: noqa" approach with the intent of ignoring a specific line.
> 
> It would be important to fix that in order to make sure we're only
> ignoring the specific lines we're interested in ignoring and prevent
> upcoming failures in the jobs.
> 
> [1]: 
> http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2018-10-26.log.html#t2018-10-26T16:18:38
> [2]: http://flake8.pycqa.org/en/latest/release-notes/3.6.0.html
> [3]: http://codesearch.openstack.org/?q=flake8%3A%20noqa=nope==
> 
> David Moreau Simard
> dmsimard = [irc, github, twitter]
> 

Thanks for raising this. We have a few of these in python-cinderclient, and
after correcting the usage, it was indeed suppressing some valid errors by
skipping the entire file.

For those looking at fixing this in your repos - this may help:

for file in $(grep -r -l "flake8.*noqa" cinderclient/*); do
sed -i 's/flake8.*noqa/noqa/g' $file
done

Of course check the git diff and run any linting jobs before accepting the
changes from that.

Sean


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposal for a process to keep up with Python releases

[Zane Bitter]

On 26/10/18 5:09 AM, Thomas Goirand wrote:

On 10/22/18 9:12 PM, Zane Bitter wrote:

On 22/10/18 10:33 AM, Thomas Goirand wrote:

This can only happen if we have supporting distribution packages for it.
IMO, this is a call for using Debian Testing or even Sid in the gate.


It depends on which versions we choose to support, but if necessary yes.


If what we want is to have early detection of problems with latest
versions of Python, then there's not so many alternatives.


I think a lot depends on the relative timing of the Python release, the 
various distro release cycles, and the OpenStack release cycle. We 
established that for 3.7 that's the only way we could have done it in 
Rocky; for 3.8, who knows.



I don't really understand why you're writing that it "depends on which
version we choose to support".


The current version of the resolution[1] says that we'll choose the 
latest released version "we can feasibly use for testing", while making 
clear that availability in an Ubuntu LTS release is *not* a requirement 
for feasibility. But it doesn't require the TC to choose the latest 
version available from python.org if we're not able to build an image 
that we can successfully use for testing in time before the beginning of 
the release cycle.


[1] https://review.openstack.org/613145


That's the kind of answer which I found
very frustrating when I submit a bug, and I'm being replied "we don't
support this version". My reasoning is, the earlier we detect and fix
problems, the better, and that's orthogonal to to what version of Python
we want to support. Delaying bugfix and latest Python version compat
leads to nowhere, and best is to test with it if possible (even in a
non-voting mode).


I agree that bugs with future versions of Python are always worth fixing 
ASAP, whether or not we are able to test them in the gate.


cheers,
Zane.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [all] Update to flake8 and failures despite # flake8: noqa

[David Moreau Simard]

Hi openstack-dev,

I stumbled on odd and sudden pep8 failures with ARA recently and
brought it up in #openstack-infra [1].

It was my understanding that appending "  # flake8: noqa" to a line of
code would have flake8 ignore this line if it happened to violate any
linting rules.
It turns out that, at least according to the flake8 release notes [2],
"flake8: noqa" is actually meant to ignore the linting on an entire
file.

The correct way to ignore a specific line appears to be to append "  #
noqa" to the line... without "flake8: ".
Looking at codesearch [3], there is a lot of projects using the
"flake8: noqa" approach with the intent of ignoring a specific line.

It would be important to fix that in order to make sure we're only
ignoring the specific lines we're interested in ignoring and prevent
upcoming failures in the jobs.

[1]: 
http://eavesdrop.openstack.org/irclogs/%23openstack-infra/%23openstack-infra.2018-10-26.log.html#t2018-10-26T16:18:38
[2]: http://flake8.pycqa.org/en/latest/release-notes/3.6.0.html
[3]: http://codesearch.openstack.org/?q=flake8%3A%20noqa=nope==

David Moreau Simard
dmsimard = [irc, github, twitter]

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [tripleo] retirement of instack-undercloud

[Alex Schultz]

We have officially moved off of the instack-undercloud deployment
process in Rocky and have officially removed it's support from
python-tripleoclient in Stein.  In order to prevent confusion I have
proposed a patch to start the retirement of instack-undercloud[0].  We
will continue to support the stable branches for their life but we
don't want any further patches to instack-undercloud going forward.
Please let me know if there are any issues.

Thanks,
-Alex

[0] https://review.openstack.org/#/c/613621/

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposal for a process to keep up with Python releases

[Ben Nemec]

On 10/25/18 3:43 PM, Zane Bitter wrote:

On 25/10/18 1:38 PM, William M Edmonds wrote:

Zane Bitter  wrote on 10/22/2018 03:12:46 PM:
 > On 22/10/18 10:33 AM, Thomas Goirand wrote:
 > > On 10/19/18 5:17 PM, Zane Bitter wrote:



 > >> Integration Tests
 > >> -
 > >>
 > >> Integration tests do test, amongst other things, integration with
 > >> non-openstack-supplied things in the distro, so it's important 
that we
 > >> test on the actual distros we have identified as popular.[2] 
It's also
 > >> important that every project be testing on the same distro at 
the end of

 > >> a release, so we can be sure they all work together for users.
 > >
 > > I find very disturbing to see the project only leaning toward 
these only

 > > 2 distributions. Why not SuSE & Debian?
 >
 > The bottom line is it's because targeting those two catches 88% of our
 > users. (For once I did not make this statistic up.)
 >
 > Also note that in practice I believe almost everything is actually
 > tested on Ubuntu LTS, and only TripleO is testing on CentOS. It's
 > difficult to imagine how to slot another distro into the mix without
 > doubling up on jobs.

I think you meant 78%, assuming you were looking at the latest User 
Survey results [1], page 55. Still a hefty number.


I never know how to read those weird 3-way bar charts they have in the 
user survey, but that actually adds up to 91% by the looks of it (I 
believe you forgot to count RHEL). The numbers were actually slightly 
lower in the full-year data for 2017 that I used (from 
https://www.openstack.org/analytics - I can't give you a direct link 
because Javascript ).


It is important to note that the User Survey lumps all versions of a 
given OS together, whereas the TC reference [2] only considers the 
latest LTS/stable version. If the User Survey split out latests 
LTS/stable versions vs. others (e.g. Ubuntu 16.04 LTS), I expect we'd 
see Ubuntu 18.04 LTS + Centos 7 adding up to much less than 78%.


This is true, although we don't know by how much. (FWIW I can almost 
guarantee that virtually all of the CentOS/RHEL users are on 7, but I'm 
sure the same is not the case for Ubuntu 16.04.)


In this context I don't think the version matters though. The original 
question was why we are focusing our test efforts on Ubuntu and CentOS, 
and the answer is that ~90% of our users are on those platforms. The 
specific version they're on right now doesn't really matter - even if 
they're on an older one, chances are eventually they'll move to a newer 
release of that same OS.





[1] https://www.openstack.org/assets/survey/April2017SurveyReport.pdf
[2] 
https://governance.openstack.org/tc/reference/project-testing-interface.html#linux-distributions 




__ 


OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][limits] Does ANYONE at all use the quota class functionality in Nova?

[Jay Pipes]

On 10/25/2018 02:44 PM, melanie witt wrote:

On Thu, 25 Oct 2018 14:00:08 -0400, Jay Pipes wrote:

On 10/25/2018 01:38 PM, Chris Friesen wrote:

On 10/24/2018 9:10 AM, Jay Pipes wrote:

Nova's API has the ability to create "quota classes", which are
basically limits for a set of resource types. There is something
called the "default quota class" which corresponds to the limits in
the CONF.quota section. Quota classes are basically templates of
limits to be applied if the calling project doesn't have any stored
project-specific limits.

Has anyone ever created a quota class that is different from "default"?


The Compute API specifically says:

"Only ‘default’ quota class is valid and used to set the default quotas,
all other quota class would not be used anywhere."

What this API does provide is the ability to set new default quotas for
*all* projects at once rather than individually specifying new defaults
for each project.


It's a "defaults template", yes.

The alternative is, you know, to just set the default values in
CONF.quota, which is what I said above. Or, if you want project X to
have different quota limits from those CONF-driven defaults, then set
the quotas for the project to some different values via the
os-quota-sets API (or better yet, just use Keystone's /limits API when
we write the "limits driver" into Nova). The issue is that the
os-quota-classes API is currently blocking *me* writing that "limits
driver" in Nova because I don't want to port nova-specific functionality
(like quota classes) to a limits driver when the Keystone /limits
endpoint doesn't have that functionality and nobody I know of has ever
used it.


When you say it's blocking you from writing the "limits driver" in nova, 
are you saying you're picking up John's unified limits spec [1]? It's 
been in -W mode and hasn't been updated in 4 weeks. In the spec, 
migration from quota classes => registered limits and deprecation of the 
existing quota API and quota classes is described.


Cheers,
-melanie

[1] https://review.openstack.org/602201


Actually, I wasn't familiar with John's spec. I'll review it today.

I was referring to my own attempts to clean up the quota system and 
remove all the limits-related methods from the QuotaDriver class...


Best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [placement] update 18-43

[Chris Dent]

HTML: https://anticdent.org/placement-update-18-43.html

A placement update for you.

# Most Important

Same as last week: The major factors that need attention are
managing database migrations and associated tooling and getting the
ball rolling on properly producing documentation. More on both of
these things in the extraction section below.

Matt has sent out [an
email](http://lists.openstack.org/pipermail/openstack-dev/2018-October/136075.html)
seeking volunteers from OpenStack Ansible or TripleO to get
placement upgrade tooling in one of those projects.

# Bugs

I guess it is because of various people doing upgrades, and some of
the downstream projects starting to take more advantage of
placement, but there's been a raft of interesting bugs recently.
Many related to some of the more esoteric aspects of the
ProviderTree handling in the resource tracker, the SQL in the
placement service, or management of global state in WSGI servers.
Initially this is a bit frustrating, but it's also a good thing:
Finding and fixing bugs is the beating heart of an open source
project. So thanks to everyone finding and fixing them.

* Placement related [bugs not yet in progress](https://goo.gl/TgiPXb): 15.
  -1.
* [In progress placement bugs](https://goo.gl/vzGGDQ) 11.

# Specs

The spec review sprint happened and managed to get some specs
merged, so this list should have shrunk some.

* 
  Account for host agg allocation ratio in placement
  (Still in rocky/)

* 
  Add subtree filter for GET /resource_providers

* 
  Resource provider - request group mapping in allocation candidate

* 
  VMware: place instances on resource pool
  (still in rocky/)

* 
  Standardize CPU resource tracking

* 
  Allow overcommit of dedicated CPU
  (Has an alternative which changes allocations to a float)

* 
  Modelling passthrough devices for report to placement

* 
  Spec: allocation candidates in tree

* 
  [WIP] generic device discovery policy

* 
  Nova Cyborg interaction specification.

* 
  supporting virtual NVDIMM devices

* 
  Spec: Support filtering by forbidden aggregate

* 
  Proposes NUMA topology with RPs

* 
  Count quota based on resource class

* 
  WIP: High Precision Event Timer (HPET) on x86 guests

* 
  Add support for emulated virtual TPM

* 
  Adds spec for instance live resize

* 
  Provider config YAML file

# Main Themes

## Making Nested Useful

Work on getting nova's use of nested resource providers happy and
fixing bugs discovered in placement in the process. This is creeping
ahead. We recently confirmed that end-to-end success with nested
providers is priority one for resource provider related work.

* 

There's a topic for reshaper that still has some open patches:

* 

## Extraction

There continue to be three main tasks in regard to placement
extraction:

1. upgrade and integration testing
2. database schema migration and management
3. documentation publishing

There's been some good progress here. The [grenade job
works](https://review.openstack.org/#/c/604454/) and is ready to
merge independent of other things. The related [devstack
change](https://review.openstack.org/#/c/600162/) is still waiting
on the database management that's part of (2). As mentioned above,
volunteers from OSA or TripleO are being recruited.

That db management is making some good headway with a [working
alembic setup](https://review.openstack.org/#/c/611441/) but the
tooling to use it needs to be formalized. The [command line
hack](https://review.openstack.org/#/c/600161/) has been updated to
use the alembic setup.

We have work in progress to tune up the documentation but we are not
yet publishing documentation (3). The plan here is to incrementally
improve things as we have attention and discover things. One goal
with this is to keep the process moving and use followups to avoid
nitpicking each other too much.

# Other

Various placement changes out in the world.

* 
  Generate sample policy in placement directory
  (This is a bit stuck on not being sure what the right thing to do
  is.)

* 

Re: [openstack-dev] [Release-job-failures] Release of openstack/python-apmecclient failed

[Jeremy Stanley]

On 2018-10-26 06:46:23 -0500 (-0500), Sean McGinnis wrote:
[...]
> Release failed for this due to openstackci not being properly configured
> for the pypi package upload.

If whoever "pineunity" is can add the "openstackci" account as
another maintainer for https://pypi.org/project/python-apmecclient/
then I'm happy to reenqueue that tag into Zuul.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [all][release] Document deliverables without release management

[Thierry Carrez]

Hi,

The Release Management team is handling release management for 
deliverables from all official OpenStack teams. However, there are a 
number of exceptions:


- deliverables that are not using tags or branches for their 'release', 
or that are directly published (docs, specs, cookiecutters...)


- deployment deliverables that are released on a specific marketplace 
(Chef supermarket, Charm store...) and that are not relying on the 
OpenStack release management team help


To facilitate tracking the intention of the teams for their 
deliverables, I proposed the following change:


https://review.openstack.org/#/c/613268/

It introduces a "release-management:" key to the deliverable entries in 
the governance projects.yaml file that lists official repos. By default 
(if the key is not present), the deliverable would be handled by the 
OpenStack Release Management team using the openstack/release repository.


This change applies the key to already-known exceptions: please review 
those and suggest any other exception that I missed!


Cheers,

--
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Release-job-failures] Release of openstack/python-apmecclient failed

[Sean McGinnis]

On Fri, Oct 26, 2018 at 4:42 AM  wrote:

> Build failed.
>
> - release-openstack-python3
> http://logs.openstack.org/d3/d39466cf752f2a20a3047b9ca537b2b6adccb154/release/release-openstack-python3/345a591/
> : POST_FAILURE in 3m 57s
> - announce-release announce-release : SKIPPED
> - propose-update-constraints propose-update-constraints : SKIPPED
>
>
Release failed for this due to openstackci not being properly configured
for the pypi package upload.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Proposal for a process to keep up with Python releases

[Thomas Goirand]

On 10/22/18 9:12 PM, Zane Bitter wrote:
> On 22/10/18 10:33 AM, Thomas Goirand wrote:
>> This can only happen if we have supporting distribution packages for it.
>> IMO, this is a call for using Debian Testing or even Sid in the gate.
> 
> It depends on which versions we choose to support, but if necessary yes.

If what we want is to have early detection of problems with latest
versions of Python, then there's not so many alternatives.

I don't really understand why you're writing that it "depends on which
version we choose to support". That's the kind of answer which I found
very frustrating when I submit a bug, and I'm being replied "we don't
support this version". My reasoning is, the earlier we detect and fix
problems, the better, and that's orthogonal to to what version of Python
we want to support. Delaying bugfix and latest Python version compat
leads to nowhere, and best is to test with it if possible (even in a
non-voting mode).

Cheers,

Thomas Goirand (zigo)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev