date:20181010

On 2018-10-10 16:35:38 -0700 (-0700), Goutham Pacha Ravi wrote:
[...]
> Thanks Corey for starting this effort. I proposed changes to
> manila repos to use your template [1] [2], but the interpreter's
> not being installed, do you need to make any bindep changes to
> enable the "universe" ppa and install python3.7 and python3.7-dev?
[...]

I think we need to just make sure that the standard Python jobs
install the intended version of the interpreter. Using bindep for
that particular purpose is mildly silly. The bindep.txt file is,
first and foremost, a local developer convenience to let people know
what unexpected packages they might need to install on their systems
to run certain kinds of local tests. I really doubt any reasonable
developer will be surprised that they need to install python3.7
before being able to successfully run `tox -e py37` nor is the error
message confusing if they forget to do so.

A couple projects have added python-version-specific bindep profiles
which do nothing but install the corresponding interpreter, but
adding things to bindep.txt purely to satisfy the CI system is
backwards. Our CI jobs should do what we expect them to do by
default. If the job says it's going to run unit tests under Python
3.7 then the job should make sure a suitable interpreter is
installed to do so.
-- 
Jeremy Stanley

signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

2018-10-10 Thread Goutham Pacha Ravi

On Wed, Oct 10, 2018 at 2:10 PM Jeremy Stanley  wrote:
>
> On 2018-10-10 16:00:40 -0500 (-0500), Sean McGinnis wrote:
> [...]
> > I would rather see us testing 3.5 and 3.7 versus 3.5, 3.6, and
> > 3.7.
> [...]
>
> I might have only pointed this out on IRC so far, but the
> expectation is that testing 3.5 and 3.6 at the same time was merely
> transitional since official OpenStack projects should be moving
> their testing from Ubuntu Xenial (which provides 3.5) to Ubuntu
> Bionic (which provides 3.6 and, now, 3.7 as well) during the Stein
> cycle and so will drop 3.5 testing on master in the process.

++ on switching python3.5 jobs to testing with python3.7 on Bionic.
python3.5 wasn't supported on all distros [1][2][3][4][5]. Xenial had it,
so it was nice to test with it when developing Queens and Rocky.


Thanks Corey for starting this effort. I proposed changes to
manila repos to use your template [1] [2], but the interpreter's not
being installed,
do you need to make any bindep changes to enable the "universe" ppa and install
python3.7 and python3.7-dev?


[1] OpenSuse https://software.opensuse.org/package/python3
[2] Ubuntu https://packages.ubuntu.com/search?keywords=python3
[3] Fedora https://apps.fedoraproject.org/packages/python3
[4] Arch https://www.archlinux.org/packages/extra/x86_64/python/
[5] Gentoo https://wiki.gentoo.org/wiki/Project:Python/Implementations
[6] manila https://review.openstack.org/#/c/609558
[7] python-manilaclient https://review.openstack.org/609557

--
Goutham

> --
> Jeremy Stanley
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

On 2018-10-10 16:00:40 -0500 (-0500), Sean McGinnis wrote:
[...]
> I would rather see us testing 3.5 and 3.7 versus 3.5, 3.6, and
> 3.7.
[...]

I might have only pointed this out on IRC so far, but the
expectation is that testing 3.5 and 3.6 at the same time was merely
transitional since official OpenStack projects should be moving
their testing from Ubuntu Xenial (which provides 3.5) to Ubuntu
Bionic (which provides 3.6 and, now, 3.7 as well) during the Stein
cycle and so will drop 3.5 testing on master in the process.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

2018-10-10 Thread Sean McGinnis

> >
> >
> > What I mean is that we run too into a situation where we have a large
> > backlog of CI jobs since we have to many changes and jobs in flight.
> >
> > So, I'm asking whether there is a good way to not duplicating all jobs
> > to run on all three interpreters. Do we really need testing of all three
> > versions? Or is testing with a subset a manageable risk?
> >
> 
> Fair enough. I'm probably not the right person to answer so perhaps someone
> else can chime in. One thing worth pointing out is that it seems the jump
> from 3.5 to 3.6 wasn't nearly as painful as the jump from 3.6 to 3.7, at
> least in my experience.
> 
> Corey
> 

I share Andreas's concerns. I would rather see us testing 3.5 and 3.7 versus
3.5, 3.6, and 3.7. I would expect anything that passes on 3.7 to be fairly safe
when it comes to 3.6 runtimes.

Maybe a periodic job that exercies 3.6?

Sean

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [sahara] PTL out for a week

2018-10-10 Thread Telles Nobrega

Hi all,

I'm taking PTO from tomorrow until Monday Oct 22nd. I won't cancel the
meeting yet but let me know if you want me to.

See you all in a couple weeks.

Thanks,
-- 

TELLES NOBREGA

SOFTWARE ENGINEER

Red Hat Brasil  

Av. Brg. Faria Lima, 3900 - 8º andar - Itaim Bibi, São Paulo

tenob...@redhat.com

TRIED. TESTED. TRUSTED. 
 Red Hat é reconhecida entre as melhores empresas para trabalhar no Brasil
pelo Great Place to Work.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo][taskflow] Thoughts on moving taskflow out of openstack/oslo

2018-10-10 Thread Ben Nemec




On 10/10/18 1:35 PM, Greg Hill wrote:


I'm not sure how using pull requests instead of Gerrit changesets would
help "core reviewers being pulled on to other projects"?


The 2 +2 requirement works for larger projects with a lot of 
contributors. When you have only 3 regular contributors and 1 of them 
gets pulled on to a project and can no longer actively contribute, you 
have 2 developers who can +2 each other but nothing can get merged 
without that 3rd dev finding time to add another +2. This is what 
happened with Taskflow a few years back. Eventually the other 2 gave up 
and moved on also.


As the others have mentioned, this doesn't need to continue to be a 
blocker. If the alternative is nobody working on the project at all, a 
single approver policy is far better. In practice it's probably not much 
different from having a general oslo core rubber stamp +2 a patch that 
was already reviewed by a taskflow expert.




Is this just about preferring not having a non-human gatekeeper like
Gerrit+Zuul and being able to just have a couple people merge whatever
they want to the master HEAD without needing to talk about +2/+W rights?


We plan to still have a CI gatekeeper, probably Travis CI, to make sure 
PRs past muster before being merged, so it's not like we're wanting to 
circumvent good contribution practices by committing whatever to HEAD. 
But the +2/+W rights thing was a huge PITA to deal with with so few 
contributors, for sure.


I guess this would be the one concern I'd have about moving it out. We 
still have a fair number of OpenStack projects depending on taskflow[1] 
to one degree or another, and having taskflow fully integrated into the 
OpenStack CI system is nice for catching problems with proposed changes 
early. I think there was some work recently to get OpenStack CI voting 
on Github, but it seems inefficient to do work to move it out of 
OpenStack and then do more work to partially bring it back.


I suppose the other option is to just stop CI'ing on OpenStack and rely 
on the upper-constraints gating we do for our other dependencies. That 
would be unfortunate, but again if the alternative is no development at 
all then it might be a necessary compromise.


1: 
http://codesearch.openstack.org/?q=taskflow=nope=requirements.txt=




If it's just about preferring the pull request workflow versus the
Gerrit rebase workflow, just say so. Same for just preferring the
Github
UI versus Gerrit's UI (which I agree is awful).


I mean, yes, I personally prefer the Github UI and workflow, but that 
was not a primary consideration. I got used to using gerrit well enough. 
It was mostly the  There's also a sense that if a project is in the 
Openstack umbrella, it's not useful outside Openstack, and Taskflow is 
designed to be a general purpose library. The hope is that just making 
it a regular open source project might attract more users and 
contributors. This may or may not bear out, but as it is, there's no 
real benefit to staying an openstack project on this front since nobody 
is actively working on it within the community.


Greg

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo][taskflow] Thoughts on moving taskflow out of openstack/oslo

2018-10-10 Thread Eric Fried



On 10/10/2018 12:41 PM, Greg Hill wrote:
> I've been out of the openstack loop for a few years, so I hope this
> reaches the right folks.
> 
> Josh Harlow (original author of taskflow and related libraries) and I
> have been discussing the option of moving taskflow out of the openstack
> umbrella recently. This move would likely also include the futurist and
> automaton libraries that are primarily used by taskflow. The idea would
> be to just host them on github and use the regular Github features for
> Issues, PRs, wiki, etc, in the hopes that this would spur more
> development. Taskflow hasn't had any substantial contributions in
> several years and it doesn't really seem that the current openstack devs
> have a vested interest in moving it forward. I would like to move it
> forward, but I don't have an interest in being bound by the openstack
> workflow (this is why the project stagnated as core reviewers were
> pulled on to other projects and couldn't keep up with the review
> backlog, so contributions ground to a halt).
> 
> I guess I'm putting it forward to the larger community. Does anyone have
> any objections to us doing this? Are there any non-obvious
> technicalities that might make such a transition difficult? Who would
> need to be made aware so they could adjust their own workflows?

The PowerVM nova virt driver uses taskflow (and we love it, btw). So we
do need to be kept apprised of any movement in this area, and will need
to be able to continue tracking it as a requirement.

If it does move, I assume the maintainers will still be available and
accessible. Josh has been helpful a number of times in the past.

Other than that, I have no opinion on whether such a move is good or
bad, right or wrong, or what it should look like.

-efried

> 
> Or would it be preferable to just fork and rename the project so
> openstack can continue to use the current taskflow version without worry
> of us breaking features?
> 
> Greg
> 
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] add service discovery, proxysql, vault, fabio and FQDN endpoints

2018-10-10 Thread Piotr Misiak


On 10.10.2018 09:06, Florian Engelmann wrote:
Now I get you. I would say all configuration templates need to be 
changed to allow, eg.


$ grep http /etc/kolla/cinder-volume/cinder.conf
glance_api_servers = http://10.10.10.5:9292
auth_url = http://internal.somedomain.tld:35357
www_authenticate_uri = http://internal.somedomain.tld:5000
auth_url = http://internal.somedomain.tld:35357
auth_endpoint = http://internal.somedomain.tld:5000

to look like:

glance_api_servers = http://glance.service.somedomain.consul:9292
auth_url = http://keystone.service.somedomain.consul:35357
www_authenticate_uri = http://keystone.service.somedomain.consul:5000
auth_url = http://keystone.service.somedomain.consul:35357
auth_endpoint = http://keystone.service.somedomain.consul:5000



The idea with Consul looks interesting.

But I don't get your issue with VIP address and spine-leaf network.

What we have:
- controller1 behind leaf1 A/B pair with MLAG
- controller2 behind leaf2 A/B pair with MLAG
- controller3 behind leaf3 A/B pair with MLAG

The VIP address is active on one controller server.
When the server fail then the VIP will move to another controller server.
Where do you see a SPOF in this configuration?

Thanks


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo][taskflow] Thoughts on moving taskflow out of openstack/oslo

On 2018-10-10 13:35:00 -0500 (-0500), Greg Hill wrote:
[...]
> We plan to still have a CI gatekeeper, probably Travis CI, to make sure PRs
> past muster before being merged, so it's not like we're wanting to
> circumvent good contribution practices by committing whatever to HEAD.

Travis CI has gained the ability to prevent you from merging changes
which fail testing? Or do you mean something else when you refer to
it as a "gatekeeper" here?

> But the +2/+W rights thing was a huge PITA to deal with with so
> few contributors, for sure.
[...]

Note that this is not a technical limitation at all, merely social
convention. There are plenty of projects hosted on our
infrastructure, some even official OpenStack projects, where
contributor count is low enough that authors who are also core
reviewers just review and approve their own changes for expediency.

> There's also a sense that if a project is in the Openstack
> umbrella, it's not useful outside Openstack, and Taskflow is
> designed to be a general purpose library.
[...]

Be aware that the "OpenStack Infrastructure" is in the process of
rebranding itself as "OpenDev" and we're working to eliminate
mention of OpenStack on things that don't need it. This includes
moving services to a new domain name, switching to other repository
namespaces, putting mirroring to services like GitHub and Bitbucket
under the direct control of teams who are interested in handling
that with their own unique organizations in those platforms, and so
on. It's progressing, though perhaps too slowly to solve your
immediate concerns.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo][taskflow] Thoughts on moving taskflow out of openstack/oslo

2018-10-10 Thread Clark Boylan

On Wed, Oct 10, 2018, at 11:35 AM, Greg Hill wrote:
> > I'm not sure how using pull requests instead of Gerrit changesets would
> > help "core reviewers being pulled on to other projects"?
> >
> 
> The 2 +2 requirement works for larger projects with a lot of contributors.
> When you have only 3 regular contributors and 1 of them gets pulled on to a
> project and can no longer actively contribute, you have 2 developers who
> can +2 each other but nothing can get merged without that 3rd dev finding
> time to add another +2. This is what happened with Taskflow a few years
> back. Eventually the other 2 gave up and moved on also.
> 

To be clear this isn't enforced by anything but your reviewer practices. What 
is enforced is that you have +2 verified, a +2 code review, and +1 Workflow 
(this is a Gerrit submit requirements function that is also configurable per 
project). OpenStack requiring multiple +2 code reviews is enforced by humans 
and maybe the discussion could be "should taskflow and related tools allow 
single code review approval (and possibly self approval by any remaining 
cores)?"

It might be a worthwhile discussion to reevaluate whether or not the humans 
should continue to enforce this rule on all code bases independent of what 
happens with taskflow.

> 
> > Is this just about preferring not having a non-human gatekeeper like
> > Gerrit+Zuul and being able to just have a couple people merge whatever
> > they want to the master HEAD without needing to talk about +2/+W rights?
> >
> 
> We plan to still have a CI gatekeeper, probably Travis CI, to make sure PRs
> past muster before being merged, so it's not like we're wanting to
> circumvent good contribution practices by committing whatever to HEAD. But
> the +2/+W rights thing was a huge PITA to deal with with so few
> contributors, for sure.
> 
> If it's just about preferring the pull request workflow versus the
> > Gerrit rebase workflow, just say so. Same for just preferring the Github
> > UI versus Gerrit's UI (which I agree is awful).
> >
> 
> I mean, yes, I personally prefer the Github UI and workflow, but that was
> not a primary consideration. I got used to using gerrit well enough. It was
> mostly the  There's also a sense that if a project is in the Openstack
> umbrella, it's not useful outside Openstack, and Taskflow is designed to be
> a general purpose library. The hope is that just making it a regular open
> source project might attract more users and contributors. This may or may
> not bear out, but as it is, there's no real benefit to staying an openstack
> project on this front since nobody is actively working on it within the
> community.
> 
> Greg


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo][taskflow] Thoughts on moving taskflow out of openstack/oslo

2018-10-10 Thread Jim Rollenhagen

On Wed, Oct 10, 2018 at 2:35 PM Greg Hill  wrote:

>
> I'm not sure how using pull requests instead of Gerrit changesets would
>> help "core reviewers being pulled on to other projects"?
>>
>
> The 2 +2 requirement works for larger projects with a lot of contributors.
> When you have only 3 regular contributors and 1 of them gets pulled on to a
> project and can no longer actively contribute, you have 2 developers who
> can +2 each other but nothing can get merged without that 3rd dev finding
> time to add another +2. This is what happened with Taskflow a few years
> back. Eventually the other 2 gave up and moved on also.
>

As a note, the 2+2 requirement is only a convention, not a rule. Swift has
moved to 1+2 already, and other projects have considered it.

// jim
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo][taskflow] Thoughts on moving taskflow out of openstack/oslo

2018-10-10 Thread Greg Hill

> I'm not sure how using pull requests instead of Gerrit changesets would
> help "core reviewers being pulled on to other projects"?
>

The 2 +2 requirement works for larger projects with a lot of contributors.
When you have only 3 regular contributors and 1 of them gets pulled on to a
project and can no longer actively contribute, you have 2 developers who
can +2 each other but nothing can get merged without that 3rd dev finding
time to add another +2. This is what happened with Taskflow a few years
back. Eventually the other 2 gave up and moved on also.


> Is this just about preferring not having a non-human gatekeeper like
> Gerrit+Zuul and being able to just have a couple people merge whatever
> they want to the master HEAD without needing to talk about +2/+W rights?
>

We plan to still have a CI gatekeeper, probably Travis CI, to make sure PRs
past muster before being merged, so it's not like we're wanting to
circumvent good contribution practices by committing whatever to HEAD. But
the +2/+W rights thing was a huge PITA to deal with with so few
contributors, for sure.

If it's just about preferring the pull request workflow versus the
> Gerrit rebase workflow, just say so. Same for just preferring the Github
> UI versus Gerrit's UI (which I agree is awful).
>

I mean, yes, I personally prefer the Github UI and workflow, but that was
not a primary consideration. I got used to using gerrit well enough. It was
mostly the  There's also a sense that if a project is in the Openstack
umbrella, it's not useful outside Openstack, and Taskflow is designed to be
a general purpose library. The hope is that just making it a regular open
source project might attract more users and contributors. This may or may
not bear out, but as it is, there's no real benefit to staying an openstack
project on this front since nobody is actively working on it within the
community.

Greg
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo][taskflow] Thoughts on moving taskflow out of openstack/oslo


On 10/10/2018 01:41 PM, Greg Hill wrote:
I've been out of the openstack loop for a few years, so I hope this 
reaches the right folks.


Josh Harlow (original author of taskflow and related libraries) and I 
have been discussing the option of moving taskflow out of the openstack 
umbrella recently. This move would likely also include the futurist and 
automaton libraries that are primarily used by taskflow. The idea would 
be to just host them on github and use the regular Github features for 
Issues, PRs, wiki, etc, in the hopes that this would spur more 
development. Taskflow hasn't had any substantial contributions in 
several years and it doesn't really seem that the current openstack devs 
have a vested interest in moving it forward. I would like to move it 
forward, but I don't have an interest in being bound by the openstack 
workflow (this is why the project stagnated as core reviewers were 
pulled on to other projects and couldn't keep up with the review 
backlog, so contributions ground to a halt).


I'm not sure how using pull requests instead of Gerrit changesets would 
help "core reviewers being pulled on to other projects"?


Is this just about preferring not having a non-human gatekeeper like 
Gerrit+Zuul and being able to just have a couple people merge whatever 
they want to the master HEAD without needing to talk about +2/+W rights?


If it's just about preferring the pull request workflow versus the 
Gerrit rebase workflow, just say so. Same for just preferring the Github 
UI versus Gerrit's UI (which I agree is awful).


Anyway, it's cool with me to "free" taskflow from the onerous yoke of 
OpenStack development if that's what the contributors to it want.


Best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [oslo][taskflow] Thoughts on moving taskflow out of openstack/oslo

2018-10-10 Thread Greg Hill

I've been out of the openstack loop for a few years, so I hope this reaches
the right folks.

Josh Harlow (original author of taskflow and related libraries) and I have
been discussing the option of moving taskflow out of the openstack umbrella
recently. This move would likely also include the futurist and automaton
libraries that are primarily used by taskflow. The idea would be to just
host them on github and use the regular Github features for Issues, PRs,
wiki, etc, in the hopes that this would spur more development. Taskflow
hasn't had any substantial contributions in several years and it doesn't
really seem that the current openstack devs have a vested interest in
moving it forward. I would like to move it forward, but I don't have an
interest in being bound by the openstack workflow (this is why the project
stagnated as core reviewers were pulled on to other projects and couldn't
keep up with the review backlog, so contributions ground to a halt).

I guess I'm putting it forward to the larger community. Does anyone have
any objections to us doing this? Are there any non-obvious technicalities
that might make such a transition difficult? Who would need to be made
aware so they could adjust their own workflows?

Or would it be preferable to just fork and rename the project so openstack
can continue to use the current taskflow version without worry of us
breaking features?

Greg
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla][tc] add service discovery, proxysql, vault, fabio and FQDN endpoints


+tc topic

On 10/10/2018 11:49 AM, Fox, Kevin M wrote:

Sorry. Couldn't quite think of the name. I was meaning, openstack project tags.


I think having a tag that indicates the project is no longer using 
SELECT FOR UPDATE (and thus is safe to use multi-writer Galera) is an 
excellent idea, Kevin. ++


-jay



From: Jay Pipes [jaypi...@gmail.com]
Sent: Tuesday, October 09, 2018 12:22 PM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [kolla] add service discovery, proxysql, vault, 
fabio and FQDN endpoints

On 10/09/2018 03:10 PM, Fox, Kevin M wrote:

Oh, this does raise an interesting question... Should such information be reported by the 
projects up to users through labels? Something like, "percona_multimaster=safe" 
Its really difficult for folks to know which projects can and can not be used that way 
currently.


Are you referring to k8s labels/selectors? or are you referring to
project tags (you know, part of that whole Big Tent thing...)?

-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Openstack-operators] [nova] Supporting force live-migrate and force evacuate with nested allocations

2018-10-10 Thread Dan Smith

>> I disagree on this. I'd rather just do a simple check for >1
>> provider in the allocations on the source and if True, fail hard.
>>
>> The reverse (going from a non-nested source to a nested destination)
>> will hard fail anyway on the destination because the POST
>> /allocations won't work due to capacity exceeded (or failure to have
>> any inventory at all for certain resource classes on the
>> destination's root compute node).
>
> I agree with Jay here. If we know the source has allocations on >1
> provider, just fail fast, why even walk the tree and try to claim
> those against the destination - the nested providers aren't going to
> be the same UUIDs on the destination, *and* trying to squash all of
> the source nested allocations into the single destination root
> provider and hope it works is super hacky and I don't think we should
> attempt that. Just fail if being forced and nested allocations exist
> on the source.

Same, yeah.

--Dan

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla] add service discovery, proxysql, vault, fabio and FQDN endpoints

2018-10-10 Thread Fox, Kevin M

Sorry. Couldn't quite think of the name. I was meaning, openstack project tags.

Thanks,
Kevin

From: Jay Pipes [jaypi...@gmail.com]
Sent: Tuesday, October 09, 2018 12:22 PM
To: openstack-dev@lists.openstack.org
Subject: Re: [openstack-dev] [kolla] add service discovery, proxysql, vault, 
fabio and FQDN endpoints

On 10/09/2018 03:10 PM, Fox, Kevin M wrote:
> Oh, this does raise an interesting question... Should such information be 
> reported by the projects up to users through labels? Something like, 
> "percona_multimaster=safe" Its really difficult for folks to know which 
> projects can and can not be used that way currently.

Are you referring to k8s labels/selectors? or are you referring to
project tags (you know, part of that whole Big Tent thing...)?

-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [puppet][tripleo][all] Zuul job backlog

2018-10-10 Thread Bogdan Dobrelya


Wesley Hayutin  writes:

[snip]


The TripleO project has created a single node container based composable
OpenStack deployment [2]. It is the projects intention to replace most of
the TripleO upstream jobs with the Standalone deployment.  We would like to
reduce our multi-node usage to a total of two or three multinode jobs to
handle a basic overcloud deployment, updates and upgrades[a]. Currently in
master we are relying on multiple multi-node scenario jobs to test many of
the OpenStack services in a single job. Our intention is to move these
multinode scenario jobs to single node job(s) that tests a smaller subset
of services. The goal of this would be target the specific areas of the
TripleO code base that affect these services and only run those there. This
would replace the existing 2-3 hour two node job(s) with single node job(s)
for specific services that completes in about half the time.  This
unfortunately will reduce the overall coverage upstream but still allows us
a basic smoke test of the supported OpenStack services and their deployment
upstream.

Ideally projects other than TripleO would make use of the Standalone
deployment to test their particular service with containers, upgrades or
for various other reasons.  Additional projects using this deployment would
help ensure bugs are found quickly and resolved providing additional
resilience to the upstream gate jobs. The TripleO team will begin review to
scope out and create estimates for the above work starting on October 18
2018.  One should expect to see updates on our progress posted to the
list.  Below are some details on the proposed changes.

Thank you all for your time and patience!

Performance improvements:
  * Standalone jobs use half the nodes of multinode jobs
  * The standalone job has an average run time of 60-80 minutes, about half
the run time of our multinode jobs

Base TripleO Job Definitions (Stein onwards):
Multi-node jobs
  * containers-multinode
  * containers-multinode-updates
  * containers-multinode-upgrades
Single node jobs
  * undercloud
  * undercloud-upgrade
  * standalone

Jobs to be removed (Stein onwards):
Multi-node jobs[b]
  * scenario001-multinode
  * scenario002-multinode
  * scenario003-multinode
  * scenario004-multinode
  * scenario006-mulitinode
  * scenario007-multinode
  * scenario008-multinode
  * scenario009-multinode
  * scenario010-multinode
  * scenario011-multinode

Jobs that may need to be created to cover additional services[4] (Stein
onwards):
Single node jobs[c]
  * standalone-barbican
  * standalone-ceph[d]
  * standalone-designate
  * standalone-manila
  * standalone-octavia
  * standalone-openshift
  * standalone-sahara
  * standalone-telemetry

[1] https://gist.github.com/notmyname/8bf3dbcb7195250eb76f2a1a8996fb00
[2]
https://docs.openstack.org/tripleo-docs/latest/install/containers_deployment/standalone.html
[3]
http://lists.openstack.org/pipermail/openstack-dev/2018-September/134867.html
[4]
https://github.com/openstack/tripleo-heat-templates/blob/master/README.rst#service-testing-matrix


I wanted to follow-up that original thread [0] wrt running a default 
standalone tripleo deployment integration job for openstack-puppet 
modules to see if it breaks tripleo. There is a topic [1] to review please.


The issue (IMO) is that the default standalone setup deploys a fixed set 
of openstack services, some are disabled [2] and some go by default [3], 
which may be either an excessive or lacking coverage (like Ironic) for 
some of the puppet openstack modules.


My take is it only makes sense to deploy that standalone setup for the 
puppet-openstack-integration perhaps (and tripleo itself obviously, as 
that involves a majority of openstack-puppet modules), but not for each 
particular puppet-foo module.


Why wasting CI resources for that default job clonned for the modules 
and see, for example, puppet-keystone (and all other modules) standalone 
jobs failing because of an unrelated puppet-nova's libvirt issue [4]? 
That's pointless and inefficient. And to cover Ironic deployments, we'd 
have to keep the undercloud job as a separate.


Although that probably is acceptable as a first iteration... But ideally 
I'd like to see that standalone job composable and adapted to only test 
a deployment for the wanted components for puppet-foo modules under 
check/gate. And it also makes sense to disable tempest for the 
standalone job(s) perhaps as it is already covered by neighbour jobs.


[0] https://goo.gl/UFNtcC
[1] https://goo.gl/dPkgCH
[2] https://goo.gl/eZ1wuC
[3] https://goo.gl/H8ZnAJ
[4] https://review.openstack.org/609289

--
Best regards,
Bogdan Dobrelya,
Irc #bogdando

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] I have some problems with Prometheus alarms in vitrage.

2018-10-10 Thread Ifat Afek

Hi Won,

On Wed, Oct 10, 2018 at 11:58 AM Won  wrote:

>
> my prometheus version : 2.3.2 and alertmanager version : 0.15.2 and I
> attached files.(vitrage collector,graph logs and apache log and
> prometheus.yml alertmanager.yml alarm rule file etc..)
> I think the problem that resolved alarm does not disappear is the time
> stamp problem of the alarm.
>
> -gray alarm info
> severity:PAGE
> vitrage id: c6a94386-3879-499e-9da0-2a5b9d3294b8  ,
> e2c5eae9-dba9-4f64-960b-b964f1c01dfe , 3d3c903e-fe09-4a6f-941f-1a2adb09feca
> , 8c6e7906-9e66-404f-967f-40037a6afc83 ,
> e291662b-115d-42b5-8863-da8243dd06b4 , 8abd2a2f-c830-453c-a9d0-55db2bf72d46
> --
>
> The alarms marked with the blue circle are already resolved. However, it
> does not disappear from the entity graph and alarm list.
> There were seven more gray alarms at the top screenshot in active alarms
> like entity graph. It disappeared by deleting gray alarms from the
> vitrage-alarms table in the DB or changing the end timestamp value to an
> earlier time than the current time.
>

I checked the files that you sent, and it appears that the connection
between Prometheus and Vitrage works well. I see in vitrage-graph log that
Prometheus notified both on alert firing and on alert resolved statuses.
I still don't understand why the alarms were not removed from Vitrage,
though. Can you please send me the output of 'vitrage topology show' CLI
command?
Also, did you happen to restart vitrage-graph or vitrage-collector during
your tests?

> At the log, it seems that the first problem is that the timestamp value
> from the vitrage comes to 2001-01-01, even though the starting value in the
> Prometheus alarm information has the correct value.
> When the alarm is solved, the end time stamp value is not updated so alarm
> does not disappear from the alarm list.
>

Can you please show me where you saw the 2001 timestamp? I didn't find it
in the log.

> The second problem is that even if the time stamp problem is solved, the
> entity graph problem will not be solved. Gray alarm information is not in
> the vitage-collector log but in the vitrage graph and apache log.
> I want to know how to forcefully delete entity from a vitage graph.
>

You shouldn't do it :-) there is no API for deleting entities, and messing
with the database may cause unexpected results.
The only thing that you can safely do is to stop all Vitrage services,
execute 'vitrage-purge-data' command, and start the services again. This
will cause rebuilding of the entity graph.

> Regarding the multi nodes, I mean, 1 controll node(pc1) & 1 compute
> node(pc2). So one openstack.
>
> The test VM in the picture is an instance on compute node that has already
> been deleted. I waited for hours and checked nova.conf but it was not
> removed.
> This was not the occur in the queens version; in the rocky version,
> multinode environment, there seems to be a bug in VM creation on multi node.
> The same situation occurred in multi-node environments that were
> configured with different PCs.
>

Let me make sure I understand the problem.
When you create a new vm in Nova, does it immediately appear in the entity
graph?
When you delete a vm, it remains? does it remain in a multi-node
environment and deleted in a single node environment?

Br,
Ifat
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

On Wed, Oct 10, 2018 at 10:18 AM Jeremy Stanley  wrote:

> On 2018-10-10 16:09:21 +0200 (+0200), Andreas Jaeger wrote:
> [...]
> > So, I'm asking whether there is a good way to not duplicating all
> > jobs to run on all three interpreters. Do we really need testing
> > of all three versions? Or is testing with a subset a manageable
> > risk?
>
> OpenStack projects are hopefully switching to testing on Bionic
> instead of Xenial during the Stein cycle, so will stop testing with
> Python 3.5 on master when that happens (since Bionic provides
> 3.6/3.7 and no 3.5).
>

That would be ideal, in which case dropping py35 and adding py37 for master
in Stein shouldn't require any more resources.

Corey

-- 
> Jeremy Stanley
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

On 2018-10-10 16:09:21 +0200 (+0200), Andreas Jaeger wrote:
[...]
> So, I'm asking whether there is a good way to not duplicating all
> jobs to run on all three interpreters. Do we really need testing
> of all three versions? Or is testing with a subset a manageable
> risk?

OpenStack projects are hopefully switching to testing on Bionic
instead of Xenial during the Stein cycle, so will stop testing with
Python 3.5 on master when that happens (since Bionic provides
3.6/3.7 and no 3.5).
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

On Wed, Oct 10, 2018 at 10:09 AM Andreas Jaeger  wrote:

> On 10/10/2018 15.42, Corey Bryant wrote:
> >
> >
> > On Wed, Oct 10, 2018 at 9:26 AM Andreas Jaeger  > > wrote:
> >
> > On 10/10/2018 14.45, Corey Bryant wrote:
> >  > [...]
> >  > == Enabling py37 unit tests ==
> >  >
> >  > Ubuntu Bionic (18.04 LTS) has the 3.7.0 interpreter and I have
> > reviews
> >  > up to define the py37 zuul job and templates here:
> >  > https://review.openstack.org/#/c/609066
> >  >
> >  > I'd like to start submitting reviews to projects to enable
> >  > openstack-python37-jobs (or variant) for projects that already
> have
> >  > openstack-python36-jobs in their .zuul.yaml, zuul.yaml,
> >  > .zuul.d/project.yaml.
> >
> > We have projects testing python 3.5 and 3.6 already. Adding 3.7 to
> > it is
> > a lot of wasted VMs. Can we limit testing and not test all three,
> > please?
> >
> >
> > Well, I wouldn't call any of them wasted if they're testing against a
> > supported Python version.
>
>
> What I mean is that we run too into a situation where we have a large
> backlog of CI jobs since we have to many changes and jobs in flight.
>
> So, I'm asking whether there is a good way to not duplicating all jobs
> to run on all three interpreters. Do we really need testing of all three
> versions? Or is testing with a subset a manageable risk?
>

Fair enough. I'm probably not the right person to answer so perhaps someone
else can chime in. One thing worth pointing out is that it seems the jump
from 3.5 to 3.6 wasn't nearly as painful as the jump from 3.6 to 3.7, at
least in my experience.

Corey


> Andreas
> --
>   Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
>SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
> GF: Felix Imendörffer, Jane Smithard, Graham Norton,
> HRB 21284 (AG Nürnberg)
>  GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Supporting force live-migrate and force evacuate with nested allocations

2018-10-10 Thread Matt Riedemann


On 10/10/2018 7:46 AM, Jay Pipes wrote:

2) in the old microversions change the blind allocation copy to gather
every resource from a nested source RPs too and try to allocate that
from the destination root RP. In nested allocation cases putting this
allocation to placement will fail and nova will fail the migration /
evacuation. However it will succeed if the server does not need nested
allocation neither on the source nor on the destination host (a.k.a the
legacy case). Or if the server has nested allocation on the source host
but does not need nested allocation on the destination host (for
example the dest host does not have nested RP tree yet).


I disagree on this. I'd rather just do a simple check for >1 provider in 
the allocations on the source and if True, fail hard.


The reverse (going from a non-nested source to a nested destination) 
will hard fail anyway on the destination because the POST /allocations 
won't work due to capacity exceeded (or failure to have any inventory at 
all for certain resource classes on the destination's root compute node).


I agree with Jay here. If we know the source has allocations on >1 
provider, just fail fast, why even walk the tree and try to claim those 
against the destination - the nested providers aren't going to be the 
same UUIDs on the destination, *and* trying to squash all of the source 
nested allocations into the single destination root provider and hope it 
works is super hacky and I don't think we should attempt that. Just fail 
if being forced and nested allocations exist on the source.


--

Thanks,

Matt

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

2018-10-10 Thread Andreas Jaeger

On 10/10/2018 15.42, Corey Bryant wrote:

On Wed, Oct 10, 2018 at 9:26 AM Andreas Jaeger > wrote:

On 10/10/2018 14.45, Corey Bryant wrote:
 > [...]
 > == Enabling py37 unit tests ==
 >
 > Ubuntu Bionic (18.04 LTS) has the 3.7.0 interpreter and I have
reviews
 > up to define the py37 zuul job and templates here:
 > https://review.openstack.org/#/c/609066
 >
 > I'd like to start submitting reviews to projects to enable
 > openstack-python37-jobs (or variant) for projects that already have
 > openstack-python36-jobs in their .zuul.yaml, zuul.yaml,
 > .zuul.d/project.yaml.

We have projects testing python 3.5 and 3.6 already. Adding 3.7 to
it is
a lot of wasted VMs. Can we limit testing and not test all three,
please?

Well, I wouldn't call any of them wasted if they're testing against a 
supported Python version.

What I mean is that we run too into a situation where we have a large 
backlog of CI jobs since we have to many changes and jobs in flight.

So, I'm asking whether there is a good way to not duplicating all jobs 
to run on all three interpreters. Do we really need testing of all three 
versions? Or is testing with a subset a manageable risk?

Andreas
--
 Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
  SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
   GF: Felix Imendörffer, Jane Smithard, Graham Norton,
   HRB 21284 (AG Nürnberg)
GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Supporting force live-migrate and force evacuate with nested allocations

2018-10-10 Thread Matt Riedemann


On 10/9/2018 10:08 AM, Balázs Gibizer wrote:

Question for you as well: if we remove (or change) the force flag in a
new microversion then how should the old microversions behave when
nested allocations would be required?


Fail fast if we can detect we have nested. We don't support forcing 
those types of servers.


--

Thanks,

Matt

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests


On 10/10/2018 09:42 AM, Corey Bryant wrote:
On Wed, Oct 10, 2018 at 9:26 AM Andreas Jaeger > wrote:


On 10/10/2018 14.45, Corey Bryant wrote:
 > [...]
 > == Enabling py37 unit tests ==
 >
 > Ubuntu Bionic (18.04 LTS) has the 3.7.0 interpreter and I have
reviews
 > up to define the py37 zuul job and templates here:
 > https://review.openstack.org/#/c/609066
 >
 > I'd like to start submitting reviews to projects to enable
 > openstack-python37-jobs (or variant) for projects that already have
 > openstack-python36-jobs in their .zuul.yaml, zuul.yaml,
 > .zuul.d/project.yaml.

We have projects testing python 3.5 and 3.6 already. Adding 3.7 to
it is
a lot of wasted VMs. Can we limit testing and not test all three,
please?

Well, I wouldn't call any of them wasted if they're testing against a 
supported Python version.


++

-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

On 2018-10-10 09:38:14 -0400 (-0400), Corey Bryant wrote:
[...]
> Another option could be to use a non-LTS image to use a supported
> release.

Let's avoid creating additional images unless there is a strong
reason (every additional image means more load on our image
builders, more space consumed in our providers, et cetera). Bionic
seems like it will serve fine for this purpose now that it's got
more than a pre-release of 3.7.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

On Wed, Oct 10, 2018 at 9:26 AM Andreas Jaeger  wrote:

> On 10/10/2018 14.45, Corey Bryant wrote:
> > [...]
> > == Enabling py37 unit tests ==
> >
> > Ubuntu Bionic (18.04 LTS) has the 3.7.0 interpreter and I have reviews
> > up to define the py37 zuul job and templates here:
> > https://review.openstack.org/#/c/609066
> >
> > I'd like to start submitting reviews to projects to enable
> > openstack-python37-jobs (or variant) for projects that already have
> > openstack-python36-jobs in their .zuul.yaml, zuul.yaml,
> > .zuul.d/project.yaml.
>
> We have projects testing python 3.5 and 3.6 already. Adding 3.7 to it is
> a lot of wasted VMs. Can we limit testing and not test all three, please?
>
>
Well, I wouldn't call any of them wasted if they're testing against a
supported Python version.

Corey

Andreas
> --
>   Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
>SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
> GF: Felix Imendörffer, Jane Smithard, Graham Norton,
> HRB 21284 (AG Nürnberg)
>  GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

On Wed, Oct 10, 2018 at 9:27 AM Jeremy Stanley  wrote:

> On 2018-10-10 08:45:39 -0400 (-0400), Corey Bryant wrote:
> [...]
> > Ubuntu Bionic (18.04 LTS) has the 3.7.0 interpreter
> [...]
>
> Thanks for the heads up! Last time I looked it was still a pre-3.7.0
> beta package, but looks like that has finally been updated to a
> proper release of the interpreter for Bionic in the last few weeks?
>

Yes, it was recently updated. It was originally a sync from Debian and we
got what we got in Bionic but the foundations folks were kind enough to
update it for us. This is a universe package for Bionic so it's not
officially supported but it should be good enough for unit testing. Another
option could be to use a non-LTS image to use a supported release.

Corey

-- 
> Jeremy Stanley
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-devbut t
> 
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

2018-10-10 Thread Andreas Jaeger


On 10/10/2018 14.45, Corey Bryant wrote:

[...]
== Enabling py37 unit tests ==

Ubuntu Bionic (18.04 LTS) has the 3.7.0 interpreter and I have reviews 
up to define the py37 zuul job and templates here: 
https://review.openstack.org/#/c/609066


I'd like to start submitting reviews to projects to enable 
openstack-python37-jobs (or variant) for projects that already have 
openstack-python36-jobs in their .zuul.yaml, zuul.yaml, 
.zuul.d/project.yaml.


We have projects testing python 3.5 and 3.6 already. Adding 3.7 to it is 
a lot of wasted VMs. Can we limit testing and not test all three, please?


Andreas
--
 Andreas Jaeger aj@{suse.com,opensuse.org} Twitter: jaegerandi
  SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
   GF: Felix Imendörffer, Jane Smithard, Graham Norton,
   HRB 21284 (AG Nürnberg)
GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [python3] Enabling py37 unit tests

On 2018-10-10 08:45:39 -0400 (-0400), Corey Bryant wrote:
[...]
> Ubuntu Bionic (18.04 LTS) has the 3.7.0 interpreter
[...]

Thanks for the heads up! Last time I looked it was still a pre-3.7.0
beta package, but looks like that has finally been updated to a
proper release of the interpreter for Bionic in the last few weeks?
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [api] Open API 3.0 for OpenStack API

On 2018-10-10 13:24:28 +1100 (+1100), Gilles Dubreuil wrote:
> On 09/10/18 23:58, Jeremy Stanley wrote:
> > On 2018-10-09 08:52:52 -0400 (-0400), Jim Rollenhagen wrote:
> > [...]
> > > It seems to me that a major goal of openstacksdk is to hide
> > > differences between clouds from the user. If the user is meant
> > > to use a GraphQL library themselves, we lose this and the user
> > > needs to figure it out themselves. Did I understand that
> > > correctly?
> > This is especially useful where the SDK implements business
> > logic for common operations like "if the user requested A and
> > the cloud supports features B+C+D then use those to fulfil the
> > request, otherwise fall back to using features E+F".
> 
> The features offered to the user don't have to change, it's just a
> different architecture.
> 
> The user doesn't have to deal with a GraphQL library, only the
> client applications (consuming OpenStack APIs). And there are also
> UI tools such as GraphiQL which allow to interact directly with
> GraphQL servers.

My point was simply that SDKs provide more than a simple translation
of network API calls and feature discovery. There can also be rather
a lot of "business logic" orchestrating multiple primitive API calls
to reach some more complex outcome. The services don't want to embed
this orchestrated business logic themselves, and it makes little
sense to replicate the same algorithms in every single application
which wants to make use of such composite functionality. There are
common actions an application might wish to take which involve
speaking to multiple APIs for different services to make specific
calls in a particular order, perhaps feeding the results of one into
the next.

Can you explain how GraphQL eliminates the above reasons for an SDK?
-- 
Jeremy Stanley

signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Supporting force live-migrate and force evacuate with nested allocations

2018-10-10 Thread Balázs Gibizer



On Wed, Oct 10, 2018 at 2:46 PM, Jay Pipes  wrote:
> On 10/10/2018 06:32 AM, Balázs Gibizer wrote:
>> Hi,
>> 
>> Thanks for all the feedback. I feel the following consensus is 
>> forming:
>> 
>> 1) remove the force flag in a new microversion. I've proposed a spec
>> about that API change [1]
> 
> +1
> 
>> 2) in the old microversions change the blind allocation copy to 
>> gather
>> every resource from a nested source RPs too and try to allocate that
>> from the destination root RP. In nested allocation cases putting this
>> allocation to placement will fail and nova will fail the migration /
>> evacuation. However it will succeed if the server does not need 
>> nested
>> allocation neither on the source nor on the destination host (a.k.a 
>> the
>> legacy case). Or if the server has nested allocation on the source 
>> host
>> but does not need nested allocation on the destination host (for
>> example the dest host does not have nested RP tree yet).
> 
> I disagree on this. I'd rather just do a simple check for >1 provider 
> in the allocations on the source and if True, fail hard.
> 
> The reverse (going from a non-nested source to a nested destination) 
> will hard fail anyway on the destination because the POST 
> /allocations won't work due to capacity exceeded (or failure to have 
> any inventory at all for certain resource classes on the 
> destination's root compute node).

If we hard fail on >1 provider in an allocation on the source then we 
lose the (not really common) case when the source allocation is nested 
but the destination node does not have a nested RP tree yet and it 
would support the summarized allocation on the root RP.
But sure simply failing would be a simpler solution.

gibi

> 
> -jay
> 
>> I will start implementing #2) as part of the
>> use-nested-allocation-candidate bp soon and will continue with #1)
>> later in the cycle.
>> 
>> Nothing is set in stone yet so feedback is still very appreciated.
>> 
>> Cheers,
>> gibi
>> 
>> [1] https://review.openstack.org/#/c/609330/
>> 
>> On Tue, Oct 9, 2018 at 11:40 AM, Balázs Gibizer
>>  wrote:
>>> Hi,
>>> 
>>> Setup
>>> -
>>> 
>>> nested allocation: an allocation that contains resources from one or
>>> more nested RPs. (if you have better term for this then please
>>> suggest).
>>> 
>>> If an instance has nested allocation it means that the compute, it
>>> allocates from, has a nested RP tree. BUT if a compute has a nested
>>> RP tree it does not automatically means that the instance, 
>>> allocating
>>> from that compute, has a nested allocation (e.g. bandwidth inventory
>>> will be on a nested RPs but not every instance will require 
>>> bandwidth)
>>> 
>>> Afaiu, as soon as we have NUMA modelling in place the most trivial
>>> servers will have nested allocations as CPU and MEMORY inverntory
>>> will be moved to the nested NUMA RPs. But NUMA is still in the 
>>> future.
>>> 
>>> Sidenote: there is an edge case reported by bauzas when an instance
>>> allocates _only_ from nested RPs. This was discussed on last Friday
>>> and it resulted in a new patch[0] but I would like to keep that
>>> discussion separate from this if possible.
>>> 
>>> Sidenote: the current problem somewhat related to not just nested 
>>> PRs
>>> but to sharing RPs as well. However I'm not aiming to implement
>>> sharing support in Nova right now so I also try to keep the sharing
>>> disscussion separated if possible.
>>> 
>>> There was already some discussion on the Monday's scheduler meeting
>>> but I could not attend.
>>> http://eavesdrop.openstack.org/meetings/nova_scheduler/2018/nova_scheduler.2018-10-08-14.00.log.html#l-20
>>> 
>>> 
>>> The meat
>>> 
>>> 
>>> Both live-migrate[1] and evacuate[2] has an optional force flag on
>>> the nova REST API. The documentation says: "Force  by 
>>> not
>>> verifying the provided destination host by the scheduler."
>>> 
>>> Nova implements this statement by not calling the scheduler if
>>> force=True BUT still try to manage allocations in placement.
>>> 
>>> To have allocation on the destination host Nova blindly copies the
>>> instance allocation from the source host to the destination host
>>> during these operations. Nova can do that as 1) the whole allocation
>>> is against a single RP (the compute RP) and 2) Nova knows both the
>>> source compute RP and the destination compute RP.
>>> 
>>> However as soon as we bring nested allocations into the picture that
>>> blind copy will not be feasible. Possible cases
>>> 0) The instance has non-nested allocation on the source and would
>>> need non nested allocation on the destination. This works with 
>>> blindy
>>> copy today.
>>> 1) The instance has a nested allocation on the source and would need
>>> a nested allocation on the destination as well.
>>> 2) The instance has a non-nested allocation on the source and would
>>> need a nested allocation on the destination.
>>> 3) The instance has a nested allocation on the source and would need
>>> a non nested

Re: [openstack-dev] [api] Open API 3.0 for OpenStack API

2018-10-10 Thread Jim Rollenhagen

On Tue, Oct 9, 2018 at 10:25 PM Gilles Dubreuil  wrote:

>
>
> On 09/10/18 23:58, Jeremy Stanley wrote:
> > On 2018-10-09 08:52:52 -0400 (-0400), Jim Rollenhagen wrote:
> > [...]
> >> It seems to me that a major goal of openstacksdk is to hide differences
> >> between clouds from the user. If the user is meant to use a GraphQL
> library
> >> themselves, we lose this and the user needs to figure it out themselves.
> >> Did I understand that correctly?
> > This is especially useful where the SDK implements business logic
> > for common operations like "if the user requested A and the cloud
> > supports features B+C+D then use those to fulfil the request,
> > otherwise fall back to using features E+F".
> >
>
> The features offered to the user don't have to change, it's just a
> different architecture.
>
> The user doesn't have to deal with a GraphQL library, only the client
> applications (consuming OpenStack APIs).
> And there are also UI tools such as GraphiQL which allow to interact
> directly with GraphQL servers.
>

Right, this comes back to what I said earlier:

> That said, it seems like using this in a client like OpenStackSDK would
get messy quickly. Instead of asking for which versions are supported,
you'd have to fetch the schema, map it to actual features somehow, and
adjust queries based on this info.
>
> I guess there might be a middleground where we could fetch the REST API
version, and know from that what GraphQL queries can be made.

This isn't unsolvable, but it does sound like quite a bit of work. This
isn't to say "let's not do graphql at all", but it's important to
understand the work involved.

FWIW, I originally mentioned the SDK (as opposed to the clients speaking
graphql directly), as the client applications are currently transitioning
to use openstacksdk instead of their own API calls.

// jim

>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Supporting force live-migrate and force evacuate with nested allocations

On 10/10/2018 06:32 AM, Balázs Gibizer wrote:

Hi,

Thanks for all the feedback. I feel the following consensus is forming:

1) remove the force flag in a new microversion. I've proposed a spec
about that API change [1]

2) in the old microversions change the blind allocation copy to gather
every resource from a nested source RPs too and try to allocate that
from the destination root RP. In nested allocation cases putting this
allocation to placement will fail and nova will fail the migration /
evacuation. However it will succeed if the server does not need nested
allocation neither on the source nor on the destination host (a.k.a the
legacy case). Or if the server has nested allocation on the source host
but does not need nested allocation on the destination host (for
example the dest host does not have nested RP tree yet).

I disagree on this. I'd rather just do a simple check for >1 provider in
the allocations on the source and if True, fail hard.

The reverse (going from a non-nested source to a nested destination)
will hard fail anyway on the destination because the POST /allocations
won't work due to capacity exceeded (or failure to have any inventory at
all for certain resource classes on the destination's root compute node).

-jay

I will start implementing #2) as part of the
use-nested-allocation-candidate bp soon and will continue with #1)
later in the cycle.

Nothing is set in stone yet so feedback is still very appreciated.

Cheers,
gibi

[1] https://review.openstack.org/#/c/609330/

On Tue, Oct 9, 2018 at 11:40 AM, Balázs Gibizer
wrote:

Hi,

Setup
-

nested allocation: an allocation that contains resources from one or
more nested RPs. (if you have better term for this then please
suggest).

If an instance has nested allocation it means that the compute, it
allocates from, has a nested RP tree. BUT if a compute has a nested
RP tree it does not automatically means that the instance, allocating
from that compute, has a nested allocation (e.g. bandwidth inventory
will be on a nested RPs but not every instance will require bandwidth)

Afaiu, as soon as we have NUMA modelling in place the most trivial
servers will have nested allocations as CPU and MEMORY inverntory
will be moved to the nested NUMA RPs. But NUMA is still in the future.

Sidenote: there is an edge case reported by bauzas when an instance
allocates _only_ from nested RPs. This was discussed on last Friday
and it resulted in a new patch[0] but I would like to keep that
discussion separate from this if possible.

Sidenote: the current problem somewhat related to not just nested PRs
but to sharing RPs as well. However I'm not aiming to implement
sharing support in Nova right now so I also try to keep the sharing
disscussion separated if possible.

There was already some discussion on the Monday's scheduler meeting
but I could not attend.
http://eavesdrop.openstack.org/meetings/nova_scheduler/2018/nova_scheduler.2018-10-08-14.00.log.html#l-20

The meat

Both live-migrate[1] and evacuate[2] has an optional force flag on
the nova REST API. The documentation says: "Force by not
verifying the provided destination host by the scheduler."

Nova implements this statement by not calling the scheduler if
force=True BUT still try to manage allocations in placement.

To have allocation on the destination host Nova blindly copies the
instance allocation from the source host to the destination host
during these operations. Nova can do that as 1) the whole allocation
is against a single RP (the compute RP) and 2) Nova knows both the
source compute RP and the destination compute RP.

However as soon as we bring nested allocations into the picture that
blind copy will not be feasible. Possible cases
0) The instance has non-nested allocation on the source and would
need non nested allocation on the destination. This works with blindy
copy today.
1) The instance has a nested allocation on the source and would need
a nested allocation on the destination as well.
2) The instance has a non-nested allocation on the source and would
need a nested allocation on the destination.
3) The instance has a nested allocation on the source and would need
a non nested allocation on the destination.

Nova cannot generate nested allocations easily without reimplementing
some of the placement allocation candidate (a_c) code. However I
don't like the idea of duplicating some of the a_c code in Nova.

Nova cannot detect what kind of allocation (nested or non-nested) an
instance would need on the destination without calling placement a_c.
So knowing when to call placement is a chicken and egg problem.

Possible solutions:
A) fail fast

0) Nova can detect that the source allocatioin is non-nested and try
the blindy copy and it will succeed.
1) Nova can detect that the source allocaton is nested and fail the
operation
2) Nova only sees a non nested source allocation. Even if the dest RP
tree is nested it does not mean that the

[openstack-dev] [python3] Enabling py37 unit tests

Hi All,

I'd like to enable py37 unit tests in the gate.

== Background ==

I work on OpenStack packaging for Ubuntu. During the Rocky release (Ubuntu
Cosmic) I tried to fix py37 bugs upstream as I came across them. There
ended up being a lot of py37 issues and after a while, due to time
constraints, I resorted to just opening bugs and disabling py37 unit tests
that were failing in our package builds. Luckily enough, even though Cosmic
ships with python3.6 and python3.7, python3.6 ended up being chosen as the
default for Cosmic.

== Defaulting to python3.7 ==

The next release of Ubuntu opens in just a few weeks. It will default to
python3.7 and will not include python3.6. My hope is that if I can help
enable py37 unit tests upstream now, we can get a wider view at fixing
issues soon.

== Enabling py37 unit tests ==

Ubuntu Bionic (18.04 LTS) has the 3.7.0 interpreter and I have reviews up
to define the py37 zuul job and templates here:
https://review.openstack.org/#/c/609066

I'd like to start submitting reviews to projects to enable
openstack-python37-jobs (or variant) for projects that already have
openstack-python36-jobs in their .zuul.yaml, zuul.yaml,
.zuul.d/project.yaml.

== Coinciding work ==

There is python3-first work going on now and I completely understand that
this is going to cause more work for some projects. It seems that now is as
good of a time as ever to catch up and test with a recent python3 version.
I'm sure python3.8 and beyond will be here before we know it.

Any thoughts or concerns?

Thanks,
Corey
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] Supporting force live-migrate and force evacuate with nested allocations