date:20180214

On 02/09/2018 08:49 AM, Jiri Tomasek wrote:

*Step 2. network-environment -> NIC configs*

Second step of network configuration is NIC config. For this
network-environment.yaml is used which references NIC config templates
which define network_config in their resources section. User is
currently required to configure these templates manually. We would like
to provide interactive view which would allow user to setup these
templates using TripleO UI. A good example is a standalone tool created
by Ben Nemec [3].

There is currently work aimed for Pike to introduce jinja templating for
network environments and templates [4] (single-nic-with-vlans,
bond-with-vlans) to support composable networks and roles (integrate
data from roles_data.yaml and network_data.yaml) It would be great if we
could move this one step further by using these samples as a starting
point and let user specify full NIC configuration.

Available information at this point:
- list of roles and networks as well as which networks need to be
configured at which role's NIC Config template
- os-net-config schema which defines NIC configuration elements and
relationships [5]

- jinja templated sample NIC templates

Requirements:
- provide feedback to the user about networks assigned to role and have
not been configured in NIC config yet

I don't have much to add on this point, but I will note that because my
UI is standalone and pre-dates composable networks it takes the opposite
approach. As a user adds a network to a role, it exposes the
configuration for that network. Since you have the networks ahead of
time, you can obviously expose all of those settings up front and ensure
the correct networks are configured for each nic-config.

I say this mostly for everyone's awareness so design elements of my tool
don't get copied where they don't make sense.

- let user construct network_config section of NIC config templates for
each role (brigdes/bonds/vlans/interfaces...)
- provide means to assign network to vlans/interfaces and automatically
construct network_config section parameter references

So obviously your UI code is going to differ, but I will point out that
the code in my tool for generating the actual os-net-config data is
semi-standalone:
https://github.com/cybertron/tripleo-scripts/blob/master/net_processing.py

It's also about 600 lines of code and doesn't even handle custom roles
or networks yet. I'm not clear whether it ever will at this point given
the change in my focus.

Unfortunately the input JSON schema isn't formally documented, although
the unit tests do include a number of examples.
https://github.com/cybertron/tripleo-scripts/blob/master/test-data/all-the-things/nic-input.json
covers quite a few different cases.

- populate parameter definitions in NIC config templates based on
role/networks assignment
- populate parameter definitions in NIC config templates based on
specific elements which use them e.g. BondInterfaceOvsOptions in case
when ovs_bond is used

I guess there's two ways to handle this - you could use the new jinja
templating to generate parameters, or you could handle it in the
generation code.

I'm not sure if there's a chicken-and-egg problem with the UI generating
jinja templates, but that's probably the simplest option if it works.
The approach I took with my tool was to just throw all the parameters
into all the files and if they're unused then oh well. With jinja
templating you could do the same thing - just copy a single boilerplate
parameter header that includes the jinja from the example nic-configs
and let the templating handle all the logic for you.

It would be cleaner to generate static templates that don't need to be
templated, but it would require re-implementing all of the custom
network logic for the UI. I'm not sure being cleaner is sufficient
justification for doing that.

- store NIC config templates in deployment plan and reference them from
network-environment.yaml

Problems to solve:
As a biggest problem to solve I see defining logic which would
automatically handle assigning parameters to elements in network_config
based on Network which user assigns to the element. For example: Using
GUI, user is creating network_config for compute role based on
network/config/multiple-nics/compute.yaml, user adds an interface and
assigns the interface to Tenant network. Resulting template should then
automatically populate addresses/ip_netmask: get_param: TenantIpSubnet.
Question is whether all this logic should live in GUI or should GUI pass
simplified format to Mistral workflow which will convert it to proper
network_config format and populates the template with it.

I guess the fact that I separated the UI and config generation code in
my tool is my answer to this question. I don't remember all of my
reasons for that design, but I think the main thing was to keep the
input and generation cleanly separated. Otherwise

Re: [openstack-dev] [tripleo][python3] python3 readiness?




On 02/14/2018 03:26 PM, Haïkel wrote:

I agree it won't be simple, we will have to provide those
repositories, determine how
we will gate updates, fix puppet modules, POI, etc.. and that's only a
beginning.

That's why we won't be providing raw Fedora but rather a curated
version and at some
point, we'll likely freeze it. That's kinda similar to how EL8 is
made, but it won't be EL8. :o)


Yeesh, I hadn't looked at it that way, but we would basically be doing 
the EL8 process in parallel with the actual EL8.  I assume the reason we 
can't use the actual EL8 pre-release whenever it becomes a thing is that 
there isn't a corresponding CentOS pre-release that would be usable 
upstream?




Let's say that the time is ticking, if we want to ship a productized
OpenStack distro on
Python3, and possibly on EL8 (Hint: I don't know when it will be
released, and moreover,
I'm not the one who gets to decide when RHOSP will support EL8), we're
about to reach
the point of no-return.


[snarky comment redacted in the interest of keeping my job :-)]

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo][python3] python3 readiness?

2018-02-14 Thread Haïkel

2018-02-14 22:53 GMT+01:00 Tom Barron :
> On 13/02/18 16:53 -0600, Ben Nemec wrote:
>>
>>
>>
>> On 02/13/2018 01:57 PM, Tom Barron wrote:
>>>
>>> Since python 2.7 will not be maintained past 2020 [1] it is a reasonable
>>> conjecture that downstream distributions
>>> will drop support for python 2 between now and then, perhaps as early as
>>> next year.
>>
>>
>> I'm not sure I agree.  I suspect python 2 support will not go quietly into
>> that good night.  Personally I anticipate a lot of kicking and screaming
>> right up to the end, especially from change averse enterprise users.
>>
>> But that's neither here nor there.  I think we're all in agreement that
>> python 3 support is needed. :-)
>
>
> Yeah, but you raise a good issue.  How likely is it that EL8 will choose --
> perhaps under duress -- to support both python 2 and python 3 in the next
> big downstream release.  If this is done long enough that we can support
> TripleO deployments on CentOS 8 using python2 while at the same time testing
> TripleO deployments on CentOS using python3 then TripleO support for Fedora
> wouldn't be necessary.
>
> Perhaps this question is settled, perhaps it is open.  Let's try to nail
> down which for the record.
>

All I can say is that question is definitely settled. As far as
OpenStack is concerned,
there will be no Python2 on EL8.

>
>>
>>> In Pike, OpenStack projects, including TripleO, added python 3 unit
>>> tests.  That effort was a good start, but likely we can agree that it is
>>> *only* a start to gaining confidence that real life TripleO deployments will
>>> "just work" running python 3.  As agreed in the TripleO community meeting,
>>> this email is intended to kick off a discussion in advance of PTG on what
>>> else needs to be done.
>>>
>>> In this regard it is worth observing that TripleO currently only supports
>>> CentOS deployments and CentOS won't have python 3 support until RHEL does,
>>> which may be too late to test deploying with python3 before support for
>>> python2 is dropped.  Fedora does have support for python 3 and for this
>>> reason RDO has decided [2] to begin work to run with *stabilized* Fedora
>>> repositories in the Rocky cycle, aiming to be ready on time to migrate to
>>> Python 3 and support its use in downstream and upstream CI pipelines.
>>
>>
>> So that means we'll never have Python 3 on CentOS 7 and we need to start
>> supporting Fedora again in order to do functional testing on py3? That's
>> potentially messy.  My recollection of running TripleO CI on Fedora is that
>> it was, to put it nicely, a maintenance headache.  Even with the
>> "stabilized" repos from RDO, TripleO has a knack for hitting edge case bugs
>> in a fast-moving distro like Fedora.  I guess it's not entirely clear to me
>> what the exact plan is since there's some discussion of frozen snapshots and
>> such, which might address the fast-moving part.
>>
>> It also means more CI jobs, unless we're okay with dropping CentOS support
>> for some scenarios and switching them to Fedora.  Given the amount of
>> changes between CentOS 7 and current Fedora that's a pretty big gap in our
>> testing.
>>
>> I guess if RDO has chosen this path then we don't have much choice.  As
>> far as next steps, the first thing that would need to be done is to get
>> TripleO running on Fedora again.  I suggest starting with
>> https://github.com/openstack/instack-undercloud/blob/3e702f3bdfea21c69dc8184e690f26e142a13bff/instack_undercloud/undercloud.py#L1377
>> :-)
>>
>> -Ben
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo][python3] python3 readiness?

2018-02-14 Thread Tom Barron

On 13/02/18 16:53 -0600, Ben Nemec wrote:

On 02/13/2018 01:57 PM, Tom Barron wrote:
Since python 2.7 will not be maintained past 2020 [1] it is a
reasonable conjecture that downstream distributions
will drop support for python 2 between now and then, perhaps as
early as next year.

I'm not sure I agree. I suspect python 2 support will not go quietly
into that good night. Personally I anticipate a lot of kicking and
screaming right up to the end, especially from change averse
enterprise users.

But that's neither here nor there. I think we're all in agreement
that python 3 support is needed. :-)

Yeah, but you raise a good issue. How likely is it that EL8 will
choose -- perhaps under duress -- to support both python 2 and python
3 in the next big downstream release. If this is done long enough
that we can support TripleO deployments on CentOS 8 using python2
while at the same time testing TripleO deployments on CentOS using
python3 then TripleO support for Fedora wouldn't be necessary.

Perhaps this question is settled, perhaps it is open. Let's try to
nail down which for the record.

In Pike, OpenStack projects, including TripleO, added python 3 unit
tests. That effort was a good start, but likely we can agree that
it is *only* a start to gaining confidence that real life TripleO
deployments will "just work" running python 3. As agreed in the
TripleO community meeting, this email is intended to kick off a
discussion in advance of PTG on what else needs to be done.

In this regard it is worth observing that TripleO currently only
supports CentOS deployments and CentOS won't have python 3 support
until RHEL does, which may be too late to test deploying with
python3 before support for python2 is dropped. Fedora does have
support for python 3 and for this reason RDO has decided [2] to
begin work to run with *stabilized* Fedora repositories in the Rocky
cycle, aiming to be ready on time to migrate to Python 3 and support
its use in downstream and upstream CI pipelines.

So that means we'll never have Python 3 on CentOS 7 and we need to
start supporting Fedora again in order to do functional testing on
py3? That's potentially messy. My recollection of running TripleO CI
on Fedora is that it was, to put it nicely, a maintenance headache.
Even with the "stabilized" repos from RDO, TripleO has a knack for
hitting edge case bugs in a fast-moving distro like Fedora. I guess
it's not entirely clear to me what the exact plan is since there's
some discussion of frozen snapshots and such, which might address the
fast-moving part.

It also means more CI jobs, unless we're okay with dropping CentOS
support for some scenarios and switching them to Fedora. Given the
amount of changes between CentOS 7 and current Fedora that's a pretty
big gap in our testing.

I guess if RDO has chosen this path then we don't have much choice.
As far as next steps, the first thing that would need to be done is to
get TripleO running on Fedora again. I suggest starting with https://github.com/openstack/instack-undercloud/blob/3e702f3bdfea21c69dc8184e690f26e142a13bff/instack_undercloud/undercloud.py#L1377
:-)

-Ben

signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo][python3] python3 readiness?

2018-02-14 Thread Haïkel

2018-02-14 17:05 GMT+01:00 Ben Nemec :
>
>
> On 02/13/2018 05:30 PM, David Moreau Simard wrote:
>>
>> On Tue, Feb 13, 2018 at 5:53 PM, Ben Nemec  wrote:
>>>
>>>
>>> I guess if RDO has chosen this path then we don't have much choice.
>>
>>
>> This makes it sound like we had a choice to begin with.
>> We've already had a lot of discussions around the topic but we're
>> ultimately stuck between a rock and a hard place.
>>
>> We're in this together and it's important that everyone understands
>> what's going on.
>>
>> It's not a secret to anyone that Fedora is more or less the upstream to
>> RHEL.
>> There's no py3 available in RHEL 7.
>> The alternative to making things work in Fedora is to use Software
>> Collections [1].
>>
>> If you're not familiar with Software Collections for python, it's more
>> or less the installation of RPM packages in a virtualenv.
>> Installing the "rh-python35" SCL would:
>> - Set up a chroot in /opt/rh/rh-python35/root
>> - Set up a py35 interpreter at /opt/rh/rh-python35/root/usr/bin/python3
>>
>> And then, when you would install packages *against* that SCL, they
>> would end up being installed
>> in /opt/rh/rh-python35/root/usr/lib/python3.5/site-packages/.
>>
>> That means that you need *all* of your python packages to be built
>> against the software collections and installed in the right path.
>>
>> Python script with a #!/usr/bin/python shebang ? Probably not going to
>> work.
>> Need python-requests ? Nope, sclo-python35-python-requests.
>> Need one of the 1000+ python packages maintained by RDO ?
>> Those need to be re-built and maintained against the SCL too.
>>
>> If you want to see what it looks like in practice, here's a Zuul spec
>> file [2] or the official docs for SCL [3].
>
>
> Ick, I didn't realize SCLs were that bad.
>

And that's only the tip of the iceberg :)

> /me dons his fireproof suit
>
> I know this is a dirty word around these parts, but I note that EPEL appears
> to have python 3 packages...
>

All I can say is that option was put on the table.

> Ultimately, though, I'm not in a position to be making any definitive
> statements about how to handle this.  RDO has more consumers than just
> TripleO.  The purpose of my email was mostly to provide some historical
> perspective from back when we were doing TripleO CI on Fedora, why we're not
> doing that anymore, and in fact went so far as to explicitly disable Fedora
> in the undercloud installer.  If Fedora is still our best option then so be
> it, but I don't want anyone to think it's going to be as simple as
> s/CentOS/Fedora/ (I assume no one does, but you know what they say about
> ass-u-me :-).
>

I agree it won't be simple, we will have to provide those
repositories, determine how
we will gate updates, fix puppet modules, POI, etc.. and that's only a
beginning.

That's why we won't be providing raw Fedora but rather a curated
version and at some
point, we'll likely freeze it. That's kinda similar to how EL8 is
made, but it won't be EL8. :o)

Let's say that the time is ticking, if we want to ship a productized
OpenStack distro on
Python3, and possibly on EL8 (Hint: I don't know when it will be
released, and moreover,
I'm not the one who gets to decide when RHOSP will support EL8), we're
about to reach
the point of no-return.

H.

>
>>
>> Making stuff work on Fedora is not going to be easy for anyone but it
>> sure beats messing with 1500+ packages that we'd need to untangle
>> later.
>> Most of the hard work for Fedora is already done as far as packaging
>> is concerned, we never really stopped building packages for Fedora
>> [4].
>>
>> It means we should be prepared once RHEL 8 comes out.
>>
>> [1]: https://www.softwarecollections.org/en/
>> [2]:
>> https://softwarefactory-project.io/r/gitweb?p=scl/zuul-distgit.git;a=blob;f=zuul.spec;h=6bba6a79c1f8ff844a9ea3715ab2cef1b12d323f;hb=refs/heads/master
>> [3]:
>> https://www.softwarecollections.org/en/docs/guide/#chap-Packaging_Software_Collections
>> [4]: https://trunk.rdoproject.org/fedora-rawhide/report.html
>>
>> David Moreau Simard
>> Senior Software Engineer | OpenStack RDO
>>
>> dmsimard = [irc, github, twitter]
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe

Re: [openstack-dev] [nova] heads up to users of Aggregate[Core|Ram|Disk]Filter: behavior change in >= Ocata

2018-02-14 Thread Matt Riedemann


On 2/5/2018 9:00 PM, Matt Riedemann wrote:
Given the size and detail of this thread, I've tried to summarize the 
problems and possible solutions/workarounds in this etherpad:


https://etherpad.openstack.org/p/nova-aggregate-filter-allocation-ratio-snafu 



For those working on this, please check that what I have written down is 
correct and then we can try to make some kind of plan for resolving this.


Jay has a spec up for review now:

https://review.openstack.org/#/c/544683/

It would be great to get operator feedback on that.

--

Thanks,

Matt

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [FFE][requirements][release][oslo] osprofiler bug fix needed

On 18-02-12 02:14:51, Nguyễn Trọng Vĩnh (Tovin Seven) wrote:
> Hello,
> 
> Currently, Oslo release for Queens is out.
> However, OSProfiler faces an issue that make some Nova CLI command not
> working.
> Detail for this issue: https://launchpad.net/bugs/1743586
> 
> Patch that fix this bug: https://review.openstack.org/#/c/535219/
> Back port for this: https://review.openstack.org/#/c/537735/
> Release new version for OSProfiler with this bug fix in Queens:
> https://review.openstack.org/#/c/541645/
> 
> Therefore, I send this email to get a FFE for it.
> 

Requirements is fine with this, sorry for the delay.

-- 
Matthew Thode (prometheanfire)


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [requirements][trove][tatu][barbican][compass][daisycloud][freezer][fuel][nova][openstack-ansible][pyghmi][solum] Migration from pycrypto

On 18-02-14 13:55:53, Sean McGinnis wrote:
> On Wed, Feb 14, 2018 at 10:09:47AM -0600, Matthew Thode wrote:
> > Development has stalled, (since 2014).  It's been forked but now would
> > be a good time to move to a more actively maintained crypto library like
> > cryptography.
> > 
> > Requirements wishes to drop pycrypto.  Let me know if there's anything
> > we can do to facilitate this.
> > 
> > -- 
> > Matthew Thode (prometheanfire)
> 
> We did have a discussion on the ML, and I think a little at one of the PTGs,
> about the path forward for this. IIRC, there was one other potential supported
> package that was considered for an option, but we settled on cryptography as
> the recommended path forward to get off of pycrypto. I think it had to do with
> ease of being able to just drop in the new package with minimal affected code.
> 

Yep, I remember it, I'm not mentioning it because I'd like to focus on
moving to cryptography rather than move to the fork.

-- 
Matthew Thode (prometheanfire)


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [requirements][trove][tatu][barbican][compass][daisycloud][freezer][fuel][nova][openstack-ansible][pyghmi][solum] Migration from pycrypto

2018-02-14 Thread Sean McGinnis

On Wed, Feb 14, 2018 at 10:09:47AM -0600, Matthew Thode wrote:
> Development has stalled, (since 2014).  It's been forked but now would
> be a good time to move to a more actively maintained crypto library like
> cryptography.
> 
> Requirements wishes to drop pycrypto.  Let me know if there's anything
> we can do to facilitate this.
> 
> -- 
> Matthew Thode (prometheanfire)

We did have a discussion on the ML, and I think a little at one of the PTGs,
about the path forward for this. IIRC, there was one other potential supported
package that was considered for an option, but we settled on cryptography as
the recommended path forward to get off of pycrypto. I think it had to do with
ease of being able to just drop in the new package with minimal affected code.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [charms]

2018-02-14 Thread Billy Olsen

Seems very reasonable. +1

On 02/14/2018 05:35 AM, Alex Kavanagh wrote:
> Yes, that seems like a reasonable approach. +1
>
> On Wed, Feb 14, 2018 at 11:29 AM, Liam Young  > wrote:
>
> Hi,
>
> I would like to propose that we do not support the notifications
> method for automatically creating DNS records in Queens+. This method
> for achieving Neutron integration has been superseded both upstream
> and in the charms. By removing support for it in Queens we prevent the
> charm from attempting to make designate v1 api calls for Queens+ which
> is a positive thing given it will have been removed (
> 
> https://docs.openstack.org/releasenotes/designate/queens.html#critical-issues
> 
> 
> ).
>
> Thanks
> Liam
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> 
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
>
>
>
>
> -- 
> Alex Kavanagh - Software Engineer
> Cloud Dev Ops - Solutions & Product Engineering - Canonical Ltd
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo][python3] python3 readiness?

2018-02-14 Thread David Moreau Simard

On Wed, Feb 14, 2018 at 11:19 AM, Ben Nemec  wrote:
>
> I have to admit I don't entirely understand this constraint.  CentOS 7 is in
> support until 2024.  I would think RHEL 7's timeline is similar or even
> longer.  If Python 2 is going out of support in 2020, does that mean there
> will be no supported Python on CentOS for the last four years of its
> lifecycle?

The OpenStack community is definitely not expected to support py2 beyond
2020. If RHEL and CentOS wants to support py2 beyond that date, the burden
is on them.

The RHEL 8 release date is unknown. We can only speculate that it should
be "sometime soon" based on previous release dates [1].

I don't know if it's going to be an official goal to drop py27 support in
OpenStack for Rocky but we can't wait at the last minute -- py3 support has
been a goal for a long time [2].

It doesn't mean that RDO has to support a python2 version of OpenStack
on EL7 after upstream has dropped support for it.
It's similar to how EL6 support was eventually dropped after moving on from
py26 (or was it py25?) and we started shipping on EL7.

> In fact, the more I think about this the more I feel like there's a
> fundamental problem with the way we're handling this transition.  We're not
> the only ones who are going to feel the pain of having disjoint Python
> releases from 7 to 8.  Anyone running a Python application now gets to not
> only do a major OS upgrade, but also a major Python upgrade.  Sure, it's
> worse for us because we need to support EL 8 at release, but _everyone_ is
> going to feel some variation on this pain as they move forward.

Python 3 has been out since 2008 [3], yup, 10 years ago.. and here we are.
I remember when most of this board was red [4].

> I realize this is a discussion that's probably above my pay grade, but I
> feel I would be remiss if I didn't point out that our Python support
> strategy seems very flawed.

It's no use questioning the decisions that lead RHEL7 to ship without py3
in 2014, we can only look forward at this point :)

[1]: https://en.wikipedia.org/wiki/Red_Hat_Enterprise_Linux#Version_history
[2]: https://governance.openstack.org/tc/goals/pike/python35.html
[3]: https://en.wikipedia.org/wiki/History_of_Python#Version_3
[4]: https://python3wos.appspot.com/

David Moreau Simard
Senior Software Engineer | OpenStack RDO

dmsimard = [irc, github, twitter]

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [cinder] No Weekly Meeting 2/21/18 or 2/28/18

2018-02-14 Thread Jay S Bryant


Team,

We will not be having a weekly meeting on 2/21/2018 or 2/28/2018 due to 
people traveling to the PTG and the PTG itself.


Regular meeting will return on 3/7/2018 .

See you all at the PTG!

Jay


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][neutron][infra] zuul job definitions overrides and the irrelevant-file attribute

On Wed, Feb 14, 2018 at 4:05 PM James E. Blair  wrote:

> Andrea Frittoli  writes:
>
> >> That has no irrelevant-files matches, and so matches everything.  If you
> >> drop the use of that template, it will work as expected.  Or, if you can
> >> say with some certainty that nova's irrelevant-files set is not
> >> over-broad, you could move the irrelevant-files from nova's invocation
> >> into the template, or even the job, and drop nova's individual
> >> invocation.
> >>
> > I don't think projects in the integrated gate should remove themselves
> > from the
> > template, it really helps keeping consistency.
> >
> > The pattern I've seen is that most projects repeat the same list of
> > irrelevant files
> > over and over again in all of their integration tests, It would be handy
> in
> > future to
> > be able to set irrelevant-files on a template when it's consumed.
> > So we could have shared irrelevant files defined in the template, and
> > custom ones
> > added by each project when consuming the template. I don't this is is
> > possible today.
> > Does it sound like a reasonable feature request?
>
> A template may specify many jobs, so if we added something like that
> feature, what would the project-pipeline template application's
> irrelevant files apply to?  All of the jobs in the template?  We could
> do that.


That's what I was thinking about,


> But it only takes one exception for this approach to fall
> short, and while a lot of irrelevant-files stanzas for a project are
> similar, I don't think having exceptions will be unusual.  The only way
> to handle exceptions like that is to specify them with jobs, and we're
> back in the same situation.
>
> Also, combining irrelevant-files is very difficult to think about.
> Because it's two inverse matches, combining them ends up being the
> intersection, not the union.  So if we implemented this, I don't think
> we should have any irrelevant-files in the template, only on template
> application.
>
> I still tend to think that irrelevant-files are almost invariably
> project-specific, so we should avoid using them in templates and job
> definitions (unless absolutely certain they are universally applicable),
> and we should only define them in one place -- in the project-pipeline
> definition for individual jobs.
>

I agree with your concerns, but the problem is that the current
implementation
renders job templates rather useless. Each project has to re-add each job
in a
template in its pipeline content definition to be able to apply irrelevant
files, and
that will turn stale if a template is modified.

With the migration to zuulv3 native jobs there is a lot of job renaming and
adding/
removing jobs going on, so for instance if a job is removed what used to be
a setting
irrelevant files may become running an extra job.

I don't really have a solution for this, but perhaps someone has an idea?

Andrea Frittoli (andreaf)


> -Jim
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [oslo] Meetings for the next two weeks


Hi,

I will be on PTO for the next Oslo meeting, and the week after that is 
the PTG so there will be no meeting.  Doug Hellmann has offered to run 
the meeting next week if necessary, but as of this writing there is 
nothing new on the agenda and unless that changes I'm inclined to skip 
it.  Many of us will be meeting face-to-face the following week anyway.


Speaking of, feel free to add topics to 
https://etherpad.openstack.org/p/oslo-ptg-rocky if there's anything 
you'd like to discuss.


As always, you don't have to wait for a scheduled meeting to start any 
discussions either.  #openstack-oslo is open 24/7, although responses 
may be delayed after hours. :-)


-Ben

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [networking-ovn] Non voting jobs for networking-ovn on neutron.

2018-02-14 Thread Ihar Hrachyshka

On Mon, Dec 4, 2017 at 5:20 AM, Miguel Angel Ajo Pelayo
 wrote:
> We were thinking about the option of having a couple of non-voting jobs
> on
> the neutron check for networking-ovn. It'd be great for us, in terms of
> traceability,
> we re-use a lot of the neutron unit test base clases/etc, and sometimes
> we get hit by surprises.

I don't think unit test base classes are meant to break, and
regardless, a better path would be to move those pieces that you reuse
into neutron-lib and consume from there. I know some drivers reuse a
lot more than just the base test class, f.e. taking ml2 unit test
classes verbatim; I don't think this is the use case that we should
ultimately support.

>
> Sometimes some other changes hit us on the neutron scenario tests.
>
> So it'd be great to have them if you believe it's a reasonable thing.

Yamamoto's concern is valid in that if we don't set clear standards of
what could be included, we would open a can of worms with every
stadium driver asking for a slot in check queue. That being said, I
don't necessarily agree it's unacceptable; what I am saying is that
drivers would need to fulfill specific requirements (for example, all
tempest tests for major api extensions executed for reference
implementation would need to pass; there may be more requirements than
just that). I think having some non-voting *tempest* jobs for each
major driver (ovn, odl, midonet?) could be useful. We have a job
specific to ironic in our check queue. I would say accommodating for
better integration with our own stadium participants can be even more
helpful.

Ihar

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [zun][kuryr][kuryr-libnetwork][neutron] Gate breakage due to removal of tag extension

2018-02-14 Thread Hongbin Lu

 Hi all,

Zun's gate is currently broken due to the removal of tag extension [1] at
neutron side. The reason is that Zun has a dependency on Kuryr-libnetwork
and Kuryr-libnetwork relies on the tag extension that was removed.

A quick fixup is to revert the tag extension removal patch [2]. This will
unblock the gate immediately. Potential alternative fixes are welcome as
long as it can quickly unblock the gate. Your help is greatly appreciated.

[1] https://review.openstack.org/#/c/534964/
[2] https://review.openstack.org/#/c/544179/

Best regards,
Hongbin
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo][python3] python3 readiness?




On 02/13/2018 10:24 PM, Haïkel wrote:

RDO has *yet* to choose a plan, and people were invited to work on the
"stabilized" repository draft [0]. If anyone has a better plan that fits all the
constraints, please share it asap.
Whatever the plan, we're launching it with the Rocky cycle.

Among the constraints (but not limited to):
* EL8 is not available
* No Python3 on EL7 *and* no allocated resources to maintain it (that includes
rebuilding/maintaining *all* python modules + libraries)


I have to admit I don't entirely understand this constraint.  CentOS 7 
is in support until 2024.  I would think RHEL 7's timeline is similar or 
even longer.  If Python 2 is going out of support in 2020, does that 
mean there will be no supported Python on CentOS for the last four years 
of its lifecycle?


In fact, the more I think about this the more I feel like there's a 
fundamental problem with the way we're handling this transition.  We're 
not the only ones who are going to feel the pain of having disjoint 
Python releases from 7 to 8.  Anyone running a Python application now 
gets to not only do a major OS upgrade, but also a major Python upgrade. 
 Sure, it's worse for us because we need to support EL 8 at release, 
but _everyone_ is going to feel some variation on this pain as they move 
forward.


I realize this is a discussion that's probably above my pay grade, but I 
feel I would be remiss if I didn't point out that our Python support 
strategy seems very flawed.



* Bridge the gap between EL7 and EL8, Fedora 27/28 are the closest thing we
have to EL8 [1][2]
* SCL have a cost (and I cannot yet expose why but not jumping onto the SCL
bandwagon has proven to be the right bet)
* Have something stable enough so that upstream gate can use it.
That's why plan stress that updates will be gated (definition of how
is still open)
* Manage to align planets so that we can ship version X of OpenStack [3] on EL8
without additional delay

Well, I cannot say that I can't relate to what you're saying, though. [4]


Indeed.  This sounds like a pub track discussion if I ever heard one. :-)



Regards,
H.

[0] 
https://etherpad.openstack.org/p/stabilized-fedora-repositories-for-openstack
[1] Do not assume anything on EL8 (name included) it's more
complicated than that.
[2] Take a breath, but we might have to ship RDO as modules, not just RPMs or
Containers. I already have headaches about it.
[3] Do not ask which one, I do not know :)
[4] Good thing that next PTG will be in Dublin, I'll need a lot of
irish whiskey :)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron] [OVN] L3 traffic

2018-02-14 Thread Assaf Muller

On Tue, Feb 13, 2018 at 11:24 PM, Numan Siddique  wrote:
>
>
> On Wed, Feb 14, 2018 at 4:19 AM, Assaf Muller  wrote:
>>
>> I'm not aware of plans for OVN to supported distributed SNAT, therefor
>> a networking node will still be required for the foreseeable future.
>>
>> On Mon, Jan 15, 2018 at 2:18 AM, wenran xiao  wrote:
>> > Hey all,
>> > I have found Network OVN will support to distributed floating ip
>> >
>> > (https://docs.openstack.org/releasenotes/networking-ovn/unreleased.html),
>> > how about the snat in the future? Still need network node or not?
>> > Any suggestions are welcomed.
>
>
> OVN can select any node (or nodes if HA is enabled) to schedule a router as
> long as the node has ovn-controller service running in it and
> ovn-bridge-mappings configured properly.
> So, If you have external connectivity in your compute nodes and you are fine
> with any of these compute nodes doing the centralized snat, you don't need
> to have a network node.

To be clear that is at parity with ML2/OVS, you can install L3 agents
on any node with external connectivity, regardless if it's also a
compute node. Some deployment tools support this, like TripleO.

>
> Thanks
> Numan
>
>> >
>> >
>> > Best regards
>> > Ran
>> >
>> >
>> > __
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe:
>> > openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>> >
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [requirements][trove][tatu][barbican][compass][daisycloud][freezer][fuel][nova][openstack-ansible][pyghmi][solum] Migration from pycrypto

Development has stalled, (since 2014).  It's been forked but now would
be a good time to move to a more actively maintained crypto library like
cryptography.

Requirements wishes to drop pycrypto.  Let me know if there's anything
we can do to facilitate this.

-- 
Matthew Thode (prometheanfire)


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][neutron][infra] zuul job definitions overrides and the irrelevant-file attribute

2018-02-14 Thread James E. Blair

Andrea Frittoli  writes:

>> That has no irrelevant-files matches, and so matches everything.  If you
>> drop the use of that template, it will work as expected.  Or, if you can
>> say with some certainty that nova's irrelevant-files set is not
>> over-broad, you could move the irrelevant-files from nova's invocation
>> into the template, or even the job, and drop nova's individual
>> invocation.
>>
> I don't think projects in the integrated gate should remove themselves
> from the
> template, it really helps keeping consistency.
>
> The pattern I've seen is that most projects repeat the same list of
> irrelevant files
> over and over again in all of their integration tests, It would be handy in
> future to
> be able to set irrelevant-files on a template when it's consumed.
> So we could have shared irrelevant files defined in the template, and
> custom ones
> added by each project when consuming the template. I don't this is is
> possible today.
> Does it sound like a reasonable feature request?

A template may specify many jobs, so if we added something like that
feature, what would the project-pipeline template application's
irrelevant files apply to?  All of the jobs in the template?  We could
do that.  But it only takes one exception for this approach to fall
short, and while a lot of irrelevant-files stanzas for a project are
similar, I don't think having exceptions will be unusual.  The only way
to handle exceptions like that is to specify them with jobs, and we're
back in the same situation.

Also, combining irrelevant-files is very difficult to think about.
Because it's two inverse matches, combining them ends up being the
intersection, not the union.  So if we implemented this, I don't think
we should have any irrelevant-files in the template, only on template
application.

I still tend to think that irrelevant-files are almost invariably
project-specific, so we should avoid using them in templates and job
definitions (unless absolutely certain they are universally applicable),
and we should only define them in one place -- in the project-pipeline
definition for individual jobs.

-Jim

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo][python3] python3 readiness?

On 02/13/2018 05:30 PM, David Moreau Simard wrote:

On Tue, Feb 13, 2018 at 5:53 PM, Ben Nemec wrote:

I guess if RDO has chosen this path then we don't have much choice.

This makes it sound like we had a choice to begin with.
We've already had a lot of discussions around the topic but we're
ultimately stuck between a rock and a hard place.

We're in this together and it's important that everyone understands
what's going on.

It's not a secret to anyone that Fedora is more or less the upstream to RHEL.
There's no py3 available in RHEL 7.
The alternative to making things work in Fedora is to use Software
Collections [1].

If you're not familiar with Software Collections for python, it's more
or less the installation of RPM packages in a virtualenv.
Installing the "rh-python35" SCL would:
- Set up a chroot in /opt/rh/rh-python35/root
- Set up a py35 interpreter at /opt/rh/rh-python35/root/usr/bin/python3

And then, when you would install packages *against* that SCL, they
would end up being installed
in /opt/rh/rh-python35/root/usr/lib/python3.5/site-packages/.

That means that you need *all* of your python packages to be built
against the software collections and installed in the right path.

Python script with a #!/usr/bin/python shebang ? Probably not going to work.
Need python-requests ? Nope, sclo-python35-python-requests.
Need one of the 1000+ python packages maintained by RDO ?
Those need to be re-built and maintained against the SCL too.

If you want to see what it looks like in practice, here's a Zuul spec
file [2] or the official docs for SCL [3].

Ick, I didn't realize SCLs were that bad.

/me dons his fireproof suit

I know this is a dirty word around these parts, but I note that EPEL
appears to have python 3 packages...

Ultimately, though, I'm not in a position to be making any definitive
statements about how to handle this. RDO has more consumers than just
TripleO. The purpose of my email was mostly to provide some historical
perspective from back when we were doing TripleO CI on Fedora, why we're
not doing that anymore, and in fact went so far as to explicitly disable
Fedora in the undercloud installer. If Fedora is still our best option
then so be it, but I don't want anyone to think it's going to be as
simple as s/CentOS/Fedora/ (I assume no one does, but you know what they
say about ass-u-me :-).

Making stuff work on Fedora is not going to be easy for anyone but it
sure beats messing with 1500+ packages that we'd need to untangle
later.
Most of the hard work for Fedora is already done as far as packaging
is concerned, we never really stopped building packages for Fedora
[4].

It means we should be prepared once RHEL 8 comes out.

[1]: https://www.softwarecollections.org/en/
[2]:
https://softwarefactory-project.io/r/gitweb?p=scl/zuul-distgit.git;a=blob;f=zuul.spec;h=6bba6a79c1f8ff844a9ea3715ab2cef1b12d323f;hb=refs/heads/master
[3]:
https://www.softwarecollections.org/en/docs/guide/#chap-Packaging_Software_Collections
[4]: https://trunk.rdoproject.org/fedora-rawhide/report.html

David Moreau Simard
Senior Software Engineer | OpenStack RDO

dmsimard = [irc, github, twitter]

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ironic][triploe] support for firmware update

2018-02-14 Thread Jim Rollenhagen

On Mon, Feb 12, 2018 at 6:52 PM, Stig Telfer 
wrote:

> Hi Moshe -
>
> It seems a bit risky to automatically apply firmware updates.  For
> example, given a node will probably be rebooted for firmware updates to
> take effect, if other vendors also did this then perhaps the node could
> reboot unexpectedly in the middle of your update.  In theory.
>

This depends on how one implements automatic firmware updates. We did
something like this when I was at Rackspace via a number of hardware
managers. Essentially, we created a hardware manager class for each type of
hardware that we wanted to be able to update. We shipped the firmware in
the agent ramdisk, and hardcoded the firmware version in code (as we would
need to ship a new ramdisk to ship a new firmware anyway). Each hardware
manager had a clean step that would check if the firmware needed an update,
and do the update if required, rebooting afterwards.

As clean steps run serially, there isn't much risk of them stepping on each
other.

The approach we’ve taken on handling firmware updates[1] has been to create
> a hardware manager for verifying firmware values during node cleaning and
> raising an exception if they do not match.  The consequence is, nodes will
> drop into maintenance mode for manual inspection / intervention.  We’ve
> then booted the node into a custom image to perform the update.
>
> Hope this helps,
> Stig
>
> [1] https://github.com/stackhpc/stackhpc-ipa-hardware-managers
>
> > On 8 Feb 2018, at 07:43, Moshe Levi  wrote:
> >
> > Hi all,
> >
> > I saw that ironic-python-agent support custom hardware manager.
> > I would like to support firmware updates (In my case Mellanox nic) and I
> was wandering how custom hardware manager can be used in such case?
>

There are a few examples of hardware managers out there that might be
helpful[0][1]. These add clean steps that update firmware when the node
goes through cleaning (which is after enrollment, and after an instance is
deleted).

> How it is integrated with ironic-python agent and also is there an
> integration to tripleO as well.
>

I've never done much with tripleo, so I'm not sure if they have a built-in
way to include a hardware manager or if you'd need to build your own
ramdisk and tell tripleo to use that.

As far as integrating it with ironic-python-agent, just make your hardware
manager something that can be installed by pip, and add entrypoints similar
to the example[2]. Then, just install it alongside the agent when building
the image, and it will be included.

// jim

[0]
https://github.com/openstack/proliantutils/blob/master/proliantutils/ipa_hw_manager/hardware_manager.py
[1]
https://github.com/openstack/ipa-example-hardware-managers/blob/master/example_hardware_managers/example_device.py
[2]
https://github.com/openstack/ipa-example-hardware-managers/blob/master/setup.cfg#L19
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [mistral][release][ffe] Requesting FFE for supporting source execution id in the Mistral client

On 18-02-14 17:03:10, Renat Akhmerov wrote:
> Hi,
> 
> We were asked to do a FFE request to be able to release a new version of 
> Mistral client out of stable/queens branch.
> 
> The backport patch: https://review.openstack.org/#/c/543393/
> The release patch: https://review.openstack.org/#/c/543402
> 
> The reason to do that after the feature freeze is that we didn’t backport 
> (and release) this patch by mistake (simply missed it) whereas the 
> corresponding functionality was already included on the server side and went 
> to Queens-3 and subsequent releases.
> 
> From my side I can assure that the change is backwards compatible and very 
> much wanted in stable/queens by many users.
> 
> Hence we’re kindly asking to approve the release patch.
> 

FFE approved from the requirements side.

-- 
Matthew Thode (prometheanfire)


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [security] Security PTG Planning, x-project request for topics.

2018-02-14 Thread Pino de Candia

Hi Luke,

Omer (in CC) has confirmed that he can stand in for me if needed, but my
preference would be that you conference me in. If you won't know until the
very day whether conference equipment is available, that's fine, we can
figure it out last minute.

A projector will be useful either way.

thanks!
Pino




On Mon, Feb 12, 2018 at 2:45 AM, Luke Hinds  wrote:

>
>
> On Sun, Feb 11, 2018 at 4:01 PM, Pino de Candia <
> giuseppe.decan...@gmail.com> wrote:
>
>> I uploaded the demo video (https://youtu.be/y6ICCPO08d8) and linked it
>> from the slides.
>>
>
> Thanks Pino , i added these to the agenda:
>
> https://etherpad.openstack.org/p/security-ptg-rocky
>
> Please let me know before the PTG, if it will be your colleague or if we
> need to find a projector to conference you in.
>
>
>> On Fri, Feb 9, 2018 at 5:51 PM, Pino de Candia <
>> giuseppe.decan...@gmail.com> wrote:
>>
>>> Hi Folks,
>>>
>>> here are the slides for the Tatu presentation: https://docs.goo
>>> gle.com/presentation/d/1HI5RR3SNUu1If-A5Zi4EMvjl-3TKsBW20xEUyYHapfM
>>>
>>> I meant to record the demo video as well but I haven't gotten around to
>>> editing all the bits. Please stay tuned.
>>>
>>> thanks,
>>> Pino
>>>
>>>
>>> On Tue, Feb 6, 2018 at 10:52 AM, Giuseppe de Candia <
>>> giuseppe.decan...@gmail.com> wrote:
>>>
 Hi Luke,

 Fantastic! An hour would be great if the schedule allows - there are
 lots of different aspects we can dive into and potential future directions
 the project can take.

 thanks!
 Pino



 On Tue, Feb 6, 2018 at 10:36 AM, Luke Hinds  wrote:

>
>
> On Tue, Feb 6, 2018 at 4:21 PM, Giuseppe de Candia <
> giuseppe.decan...@gmail.com> wrote:
>
>> Hi Folks,
>>
>> I know the request is very late, but I wasn't aware of this SIG until
>> recently. Would it be possible to present a new project to the Security 
>> SIG
>> at the PTG? I need about 30 minutes. I'm hoping to drum up interest in 
>> the
>> project, sign on users and contributors and get feedback.
>>
>> For the past few months I have been working on a new project - Tatu
>> [1]- to automate the management of SSH certificates (for both users and
>> hosts) in OpenStack. Tatu allows users to generate SSH certificates with
>> principals based on their Project role assignments, and VMs automatically
>> set up their SSH host certificate (and related config) via Nova vendor
>> data. The project also manages bastions and DNS entries so that users 
>> don't
>> have to assign Floating IPs for SSH nor remember IP addresses.
>>
>> I have a working demo (including Horizon panels [2] and OpenStack CLI
>> [3]), but am still working on the devstack script and patches [4] to get
>> Tatu's repositories into OpenStack's GitHub and Gerrit. I'll try to post 
>> a
>> demo video in the next few days.
>>
>> best regards,
>> Pino
>>
>>
>> References:
>>
>>1. https://github.com/pinodeca/tatu (Please note this is still
>>very much a work in progress, lots of TODOs in the code, very little
>>testing and documentation doesn't reflect the latest design).
>>2. https://github.com/pinodeca/tatu-dashboard
>>3. https://github.com/pinodeca/python-tatuclient
>>4. https://review.openstack.org/#/q/tatu
>>
>>
>>
>>
> Hi Giuseppe, of course you can! I will add you to the agenda. We could
> get your an hour if it allows more time for presenting and post 
> discussion?
>
> We will be meeting in an allocated room on Monday (details to follow).
>
> https://etherpad.openstack.org/p/security-ptg-rocky
>
> Luke
>
>
>
>
>>
>>
>> On Wed, Jan 31, 2018 at 12:03 PM, Luke Hinds 
>> wrote:
>>
>>>
>>> On Mon, Jan 29, 2018 at 2:29 PM, Adam Young 
>>> wrote:
>>>
 Bug 968696 and System Roles.   Needs to be addressed across the
 Service catalog.

>>>
>>> Thanks Adam, will add it to the list. I see it's been open since
>>> 2012!
>>>
>>>

 On Mon, Jan 29, 2018 at 7:38 AM, Luke Hinds 
 wrote:

> Just a reminder as we have not had many uptakes yet..
>
> Are there any projects (new and old) that would like to make use
> of the security SIG for either gaining another perspective on security
> challenges / blueprints etc or for help gaining some cross project
> collaboration?
>
> On Thu, Jan 11, 2018 at 3:33 PM, Luke Hinds 
> wrote:
>
>> Hello All,
>>
>> I am seeking topics for the PTG from all projects, as this will
>> be where we try out are new form of being a SIG.
>>
>> For

Re: [openstack-dev] [nova][neutron][infra] zuul job definitions overrides and the irrelevant-file attribute

On Fri, Jan 26, 2018 at 5:57 PM James E. Blair  wrote:

> Balázs Gibizer  writes:
>
> > Hi,
> >
> > I'm getting more and more confused how the zuul job hierarchy works or
> > is supposed to work.
>
> Hi!
>
> First, you (or others) may or may not have seen this already -- some of
> it didn't exist when we first rolled out v3, and some of it has changed
> -- but here are the relevant bits of the documentation that should help
> explain what's going on.  It helps to understand freezing:
>
>   https://docs.openstack.org/infra/zuul/user/config.html#job
>
> and matching:
>
>   https://docs.openstack.org/infra/zuul/user/config.html#matchers
>
> > First there was a bug in nova that some functional tests are not
> > triggered although the job (re-)definition in the nova part of the
> > project-config should not prevent it to run [1].
> >
> > There we figured out that irrelevant-files parameter of the jobs are
> > not something that can be overriden during re-definition or through
> > parent-child relationship. The base job openstack-tox-functional has
> > an irrelevant-files attribute that lists '^doc/.*$' as a path to be
> > ignored [2]. In the other hand the nova part of the project-config
> > tries to make this ignore less broad by adding only '^doc/source/.*$'
> > . This does not work as we expected and the job did not run on changes
> > that only affected ./doc/notification_samples path. We are fixing it
> > by defining our own functional job in nova tree [4].
> >
> > [1] https://bugs.launchpad.net/nova/+bug/1742962
> > [2]
> >
> https://github.com/openstack-infra/openstack-zuul-jobs/blob/1823e3ea20e6dfaf37786a6ff79c56cb786bf12c/zuul.d/jobs.yaml#L380
> > [3]
> >
> https://github.com/openstack-infra/project-config/blob/1145ab1293f5fa4d34c026856403c22b091e673c/zuul.d/projects.yaml#L10509
> > [4] https://review.openstack.org/#/c/533210/
>
> This is correct.  The issue here is that the irrelevant-files definition
> on openstack-tox-functional is too broad.  We need to be *extremely*
> careful applying matchers to jobs like that.  Generally I think that
> irrelevant-files should be reserved for the project-pipeline invocations
> only.  That's how they were effectively used in Zuul v2, after all.
>
> Essentially, when someone puts an irrelevant-files section on a job like
> that, they are saying "this job will never apply to these files, ever."
> That's clearly not correct in this case.
>
> So our solutions are to acknowledge that it's over-broad, and reduce or
> eliminate the list in [2] and expand it elsewhere (as in [3]).  Or we
> can say "we were generally correct, but nova is extra special so it
> needs its own job".  If that's the choice, then I think [4] is a fine
> solution.
>
> > Then I started looking into other jobs to see if we made similar
> > mistakes. I found two other examples in the nova related jobs where
> > redefining the irrelevant-files of a job caused problems. In these
> > examples nova tried to ignore more paths during the override than what
> > was originally ignored in the job definition but that did not work
> > [5][6].
> >
> > [5] https://bugs.launchpad.net/nova/+bug/1745405 (temptest-full)
>
> As noted in that bug, the tempest-full job is invoked on nova via this
> stanza:
>
>
> https://github.com/openstack-infra/project-config/blob/5ddbd62a46e17dd2fdee07bec32aa65e3b637ff3/zuul.d/projects.yaml#L10674-L10688
>
> As expected, that did not match.  There is a second invocation of
> tempest-full on nova here:
>
>
> http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/zuul.d/zuul-legacy-project-templates.yaml#n126


I guess the line number changed since so this has moved to L101 [1] now :).
tempest-full is part of the integrated-gate, so all projects in it run it
through there.

[1]
http://git.openstack.org/cgit/openstack-infra/openstack-zuul-jobs/tree/zuul.d/zuul-legacy-project-templates.yaml#n101


>
>
> That has no irrelevant-files matches, and so matches everything.  If you
> drop the use of that template, it will work as expected.  Or, if you can
> say with some certainty that nova's irrelevant-files set is not
> over-broad, you could move the irrelevant-files from nova's invocation
> into the template, or even the job, and drop nova's individual
> invocation.
>
> I don't think projects in the integrated gate should remove themselves
from the
template, it really helps keeping consistency.

The pattern I've seen is that most projects repeat the same list of
irrelevant files
over and over again in all of their integration tests, It would be handy in
future to
be able to set irrelevant-files on a template when it's consumed.
So we could have shared irrelevant files defined in the template, and
custom ones
added by each project when consuming the template. I don't this is is
possible today.
Does it sound like a reasonable feature request?

Andrea Frittoli (andreaf)


> > [6] https://bugs.launchpad.net/nova/+bug/1745431

[openstack-dev] [os-upstream-institute][ptg] Working sessions at the PTG

2018-02-14 Thread Ildiko Vancsa

Hi,

As discussed earlier we would like to leverage the opportunity to face to face 
working sessions at the PTG with the training team and anyone who’s interested 
to join/help out.

We have one dedicated time slot to meet and summarize our activities during the 
week and list of action points which is __Thursday 4pm-6pm__ local time. I will 
announce the room for the discussion in a follow up email.

To extend the time to work together in smaller groups we came up with the idea 
of a fixed work space, rather than a list of fixed time slots. The idea is to 
have a table dedicated to our team in the venue where anyone who’s not occupied 
can go and meet up with others to work on tasks we need to finish before the 
next training in Vancouver. Based on an earlier check the expectation is to 
have people available from Tuesday afternoon.

You can use the IRC channel and the PTG bot to inform others about your 
availability to further encourage team work.

The topics[1] for these working sessions are mainly focused on restructuring 
the training flow and improve the exercises.

For discussions on the Contributor Guide content please check the Docs 
sessions[2], we will cover all related topics there including moving the 
lectures over from the training guides repository. With that said we will also 
need to look into the translation aspects of the planned changes to keep 
supporting the broad community who relies on our materials.

Last but not least please also try to attend the First Contact SIG 
discussions[3] as they are relevant to the training activities as well and can 
affect how we structure and execute our courses in the future.

I will update you with details as we are getting closer to the PTG.

Please let me know if you have any questions or comments.

Thanks and Best Regards,
Ildikó

[1] https://etherpad.openstack.org/p/OUI-Rocky-PTG
[2] https://etherpad.openstack.org/p/docs-i18n-ptg-rocky
[3] https://etherpad.openstack.org/p/FC_SIG_Rocky_PTG




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova][neutron][infra] zuul job definitions overrides and the irrelevant-file attribute

On Fri, Jan 26, 2018 at 2:05 PM Balázs Gibizer 
wrote:

> Hi,
>
> I'm getting more and more confused how the zuul job hierarchy works or is
> supposed to work.
>
> First there was a bug in nova that some functional tests are not triggered
> although the job (re-)definition in the nova part of the project-config
> should not prevent it to run [1].
>
> There we figured out that irrelevant-files parameter of the jobs are not
> something that can be overriden during re-definition or through
> parent-child relationship. The base job openstack-tox-functional has an
> irrelevant-files attribute that lists '^doc/.*$' as a path to be ignored
> [2]. In the other hand the nova part of the project-config tries to make
> this ignore less broad by adding only '^doc/source/.*$' . This does not
> work as we expected and the job did not run on changes that only affected
> ./doc/notification_samples path. We are fixing it by defining our own
> functional job in nova tree [4].
>
> [1] https://bugs.launchpad.net/nova/+bug/1742962
> [2]
> https://github.com/openstack-infra/openstack-zuul-jobs/blob/1823e3ea20e6dfaf37786a6ff79c56cb786bf12c/zuul.d/jobs.yaml#L380
> [3]
> https://github.com/openstack-infra/project-config/blob/1145ab1293f5fa4d34c026856403c22b091e673c/zuul.d/projects.yaml#L10509
> [4] https://review.openstack.org/#/c/533210/
>
> Then I started looking into other jobs to see if we made similar mistakes.
> I found two other examples in the nova related jobs where redefining the
> irrelevant-files of a job caused problems. In these examples nova tried to
> ignore more paths during the override than what was originally ignored in
> the job definition but that did not work [5][6].
>
> [5] https://bugs.launchpad.net/nova/+bug/1745405 (temptest-full)
> [6] https://bugs.launchpad.net/nova/+bug/1745431 (neutron-grenade)
>
> So far the problem seemed to be consistent (i.e. override does not work).
> But then I looked into neutron-grenade-multinode. That job is defined in
> neutron tree (like neutron-grenade)
>

That is wrong and it should not have happened.
Grenade jobs that are shared by all the repos in the integrated gate should
live in repos owned by QA/infra - it was never the plan for them to end up
in the neutron repo.
We're working on making grenade and multinode zuulv3 native jobs. Grenade
jobs will leave in the grenade repo, once they're ready we will remove the
legacy one from neutron side and add the new ones defined in grenade.
Changes will be done to the job template accordingly which means that for
teams that are consuming those jobs via the template only there'll be no
action.

Andrea Frittoli (andreaf)


> but nova also refers to it in nova section of the project-config with
> different irrelevant-files than their original definition. So I assumed
> that this will lead to similar problem than in case of neutron-grenade, but
> it doesn't.
>
> The neutron-grenade-multinode original definition [1] does not try to
> ignore the 'nova/tests' path but the nova side of the definition in the
> project config does try to ignore that path [8]. Interestingly a patch in
> nova that only changes under the path: nova/tests/ does not trigger the job
> [9]. So in this case overriding the irrelevant-files of a job works. (It
> seems that overriding neutron-tempest-linuxbridge irrelevant-files works
> too).
>
> [7]
> https://github.com/openstack/neutron/blob/7e3d6a18fb928bcd303a44c1736d0d6ca9c7f0ab/.zuul.yaml#L140-L159
> [8]
> https://github.com/openstack-infra/project-config/blob/5ddbd62a46e17dd2fdee07bec32aa65e3b637ff3/zuul.d/projects.yaml#L10516-L10530
> [9] https://review.openstack.org/#/c/537936/
>
> I don't see what is the difference between neutron-grenade and
> neutron-grenade-multinode jobs definitions from this perspective but it
> seems that the irrelevent-files attribute behaves  inconsistently in these
> two jobs. Could you please help me undestand how irrelevant-files in
> overriden jobs supposed to work?
>
> cheers,
> gibi
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [designate] V1 API is now fully removed

2018-02-14 Thread Graham Hayes

I saw [1] and realised that we should be more explicit about the
upcoming release.

As highlighted in [2], this email is a reminder that the long awaited
removal of the DNS V1 API is now complete with [3].

This means from Queens onwards it will not be possible to re-enable
the V1 API (we have had it off by default for a long period of time).

Horizon, Heat and the OpenStack CLI all have v2 usable resources, and
have been deprecating the v1 resources for some time.

Any deployment tooling, custom dashboards, and internal tools should
all ensure they do not require the v1 API, and do not try to enable it.

- Graham

1 -
http://lists.openstack.org/pipermail/openstack-dev/2018-February/127366.html
2 -
https://docs.openstack.org/releasenotes/designate/queens.html#critical-issues
3 -
https://github.com/openstack/designate/commit/c318106c01b2b3976049f2c3ba0c8502a874242b



signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][infra] PTG Infra Helproom Info and Signup