Re: [openstack-dev] [nova][glance] Deprecation of nova.image.download.modules extension point

[Chris Friesen]

On 05/31/2018 04:14 PM, Moore, Curt wrote:


The challenge is that transferring the Glance image transfer is _glacially slow_
when using the Glance HTTP API (~30 min for a 50GB Windows image (It’s Windows,
it’s huge with all of the necessary tools installed)).  If libvirt can instead
perform an RBD export on the image using the image download functionality, it is
able to download the same image in ~30 sec.


This seems oddly slow.  I just downloaded a 1.6 GB image from glance in slightly 
under 10 seconds.  That would map to about 5 minutes for a 50GB image.




We could look at attaching an additional ephemeral disk to the instance and have
cloudbase-init use it as the pagefile but it appears that if libvirt is using
rbd for its images_type, _all_ disks must then come from Ceph, there is no way
at present to allow the VM image to run from Ceph and have an ephemeral disk
mapped in from node-local storage.  Even still, this would have the effect of
"wasting" Ceph IOPS for the VM disk itself which could be better used for other
purposes.

Based on what I have explained about our use case, is there a better/different
way to accomplish the same goal without using the deprecated image download
functionality?  If not, can we work to "un-deprecate" the download extension
point? Should I work to get the code for this RBD download into the upstream
repository?


Have you considered using compute nodes configured for local storage but then 
use boot-from-volume with cinder and glance both using ceph?  I *think* there's 
an optimization there such that the volume creation is fast.


Assuming the volume creation is indeed fast, in this scenario you could then 
have a local ephemeral/swap disk for your pagefile.  You'd still have your VM 
root disks on ceph though.


Chris

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack] Struggling to get the s3 api interface to work with swift.

[Kota TSUYUZAKI]

Hi Shyam,

You should specify the path starts from bucket. It will be like

http://20.20.20.229:8080/testBucket

assuming you didn't configure virtual-hosted style.
Even if your *Swift* endpoint is 
http://20.20.20.229:8080/v1/AUTH_dc86f7d8787b46158268bd77098b6578,
swift3 doesn't require the account in the path so far as well as actual amazon 
s3.

Best,
Kota

(2018/05/31 21:33), Shyam Prasad N wrote:
> Hi Yuxin,
> 
> Thank you for sharing your configs.
> So I've managed to get past the Signature not matching error.
> 
> Now the error is different. InvalidbucketName:
> 
> eightkpc@objectstore1:~/s3curl$ ./s3curl.pl --debug --id=testerks
> --createBucket --
> http://20.20.20.229:8080/v1/AUTH_dc86f7d8787b46158268bd77098b6578/testBucket
> s3curl: Found the url: host=20.20.20.229; port=8080;
> uri=/v1/AUTH_dc86f7d8787b46158268bd77098b6578/testBucket; query=;
> s3curl: cname endpoint signing case
> s3curl: StringToSign='PUT\n\n\nThu, 31 May 2018 12:02:57 +\n/
> 20.20.20.229/v1/AUTH_dc86f7d8787b46158268bd77098b6578/testBucket'
> s3curl: exec curl -v -H 'Date: Thu, 31 May 2018 12:02:57 +' -H
> 'Authorization: AWS
> 76498e1413284b9d961d452db608dff4:jj/kaAEuX/vK+WUTvZyDQUUEGV0=' -L -H
> 'content-type: ' --data-binary  -X PUT
> http://20.20.20.229:8080/v1/AUTH_dc86f7d8787b46158268bd77098b6578/testBucket
> *   Trying 20.20.20.229...
> * TCP_NODELAY set
> * Connected to 20.20.20.229 (20.20.20.229) port 8080 (#0)
>> PUT /v1/AUTH_dc86f7d8787b46158268bd77098b6578/testBucket HTTP/1.1
>> Host: 20.20.20.229:8080
>> User-Agent: curl/7.58.0
>> Accept: */*
>> Date: Thu, 31 May 2018 12:02:57 +
>> Authorization: AWS
> 76498e1413284b9d961d452db608dff4:jj/kaAEuX/vK+WUTvZyDQUUEGV0=
>> Content-Length: 0
>>
> < HTTP/1.1 400 Bad Request
> < x-amz-id-2: tx18266052d5044eb2a3bc7-005b0fe471
> < x-amz-request-id: tx18266052d5044eb2a3bc7-005b0fe471
> < Content-Type: application/xml
> < X-Trans-Id: tx18266052d5044eb2a3bc7-005b0fe471
> < X-Openstack-Request-Id: tx18266052d5044eb2a3bc7-005b0fe471
> < Date: Thu, 31 May 2018 12:02:57 GMT
> < Transfer-Encoding: chunked
> * HTTP error before end of send, stop sending
> <
> 
> * Closing connection 0
> InvalidBucketNameThe specified bucket is not
> valid.tx18266052d5044eb2a3bc7-005b0fe471v1eightkpc@objectstore1:~/s3curl$
> 
> 
> My specified endpoint is
> http://20.20.20.229:8080/v1/AUTH_dc86f7d8787b46158268bd77098b6578
> What am I doing wrong?
> 
> Regards,
> Shyam
> 
> On Wed, May 30, 2018 at 7:32 PM, Yuxin Wang 
> wrote:
> 
>> Hi Shyam,
>>
>> No problem. The output of the commands is attached.
>>
>> And my test cluster is on Swift v2.15.1 with Swift3 v1.12
>>
>> Also, here is the common process when I'm creating an S3 credential and
>> using in s3curl. Hope it helps.
>>
>> 1. Create a user and a project, and assign a proper role.
>>
>> openstack project create testproject
>> openstack user create testuser --password 123
>> openstack role add --project testproject --user testuser _member_
>>
>> 2. Check accessibility to swift
>>
>> create a test-openrc file with above info
>> source test-openrc
>> swift list
>>
>> 3.Create a credential
>>
>> openstack credential create --type ec2 --project testproject testuser
>>  '{"access": "testaccess", "secret": "testsecret"}'
>>
>> 4. Use it in s3curl
>>
>> add the endpoint url to `my @endpoints` in s3curl.pl
>> add the credential to .s3curl config file
>>
>> do `s3curl.pl -i cred_name --debug -- http://endpoint -X GET`
>>
>>
>>
>>
>> 在 2018年5月25日，18:17，Shyam Prasad N  写道：
>>
>> Hi Yuxin,
>>
>> If you don't mind, can you share the output of the following commands in
>> your running swift3 setup?
>>
>> openstack credential list
>> openstack ec2 credentials list
>> cat /etc/swift/proxy-server.conf
>>
>> Also, what are the access keys and secret keys that you use?
>> I want to make sure that I'm not missing anything in configuration.
>>
>> Regards,
>> Shyam
>>
>> On Fri, May 25, 2018 at 3:05 PM, Shyam Prasad N 
>> wrote:
>>
>>> Tried that. Unfortunately same error.
>>> Is there anything I can do to troubleshoot this?
>>>
>>> On Fri, May 25, 2018 at 2:56 PM, Yuxin Wang 
>>> wrote:
>>>
 They can be any strings.

 Replace them with whatever you want.

 - Yuxin

 在 2018年5月25日，14:57，Shyam Prasad N  写道：

 Thanks. I'll try this.
 But what values do I use in place of ak and sk? I want to use some
 command to get those values, right?

 On Fri, May 25, 2018 at 9:52 AM, Yuxin Wang 
  wrote:

> I created ec2 credentials using command `openstack credential create`.
>
> i.e.
>
> openstack credential create --type ec2 --project proj user '{"access":
> "ak", "secret": "sk”}'
>
>
> It seems the two credentials are not the same thing.
>
> Ref:
>
> https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.1.1/
> com.ibm.spectrum.scale.v4r11.adv.doc/bl1adv_ConfigureOpensta
> ckEC2credentials.htm
> 

Re: [openstack-dev] [tripleo] Containerized Undercloud by default

[Emilien Macchi]

I forgot to mention Steve's effort to update the containers when deploying
the undercloud, this is a critical piece if we want to continue to test
changes in projects like tripleo-common that are embedded in Mistral
containers for example.
The patch that will enable it is https://review.openstack.org/#/c/571631/ and
thanks to this work we'll make unify the container registry for both the
undercloud and overcloud, using the same [updated] containers.
It is important to have this feature enabled in our CI to maintain the
parity with how we tested TripleO when undercloud wasn't containeirized, so
this is something we want to achieve before switching all the TripleO CI
jobs.

On Thu, May 31, 2018 at 8:58 PM, Emilien Macchi  wrote:

> Hi,
>
> During Rocky cycle we would like to switch tripleoclient to deploy
> containeirzed undercloud by default but before to get there, we want to
> switch all CI jobs to it, like it was done when enabling config-download by
> default.
> Right now we have 3 jobs which test the containerized undercloud:
>
> - tripleo-ci-centos-7-undercloud-containers: deploy a containerized
> undercloud and run Tempest
> - tripleo-ci-centos-7-containerized-undercloud-upgrades: deploy a
> non-containerized undercloud on Queens and upgrade to a containerized
> undercloud on Rocky
> - gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master: deploy a
> containerized undercloud and an overcloud with HA architecture and IPv4
> network (with introspection, SSL, etc).
>
> What's next (target is Rocky M3):
> - tripleo-ci-centos-7-containers-multinode - currently blocked by
> https://bugs.launchpad.net/tripleo/+bug/1774297
> - all multinode scenarios - current blocked by 1774297 as well but also
> https://review.openstack.org/#/c/571566/
> - gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master -
> currently blocked by https://bugs.launchpad.net/tripleo/+bug/1774557
> (with a potential fix: https://review.openstack.org/571620)
> - all other jobs that run on master, except 
> tripleo-ci-centos-7-undercloud-oooq
> that we'll probably keep during Rocky and remove in Stein if we
> successfully switch the default.
>
> Once we've reached that point, we'll change tripleoclient's default, and
> hopefully all of this before m3 :-)
>
> Any question or feedback is highly welcome,
> --
> Emilien Macchi
>



-- 
Emilien Macchi
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [tripleo] Containerized Undercloud by default

[Emilien Macchi]

Hi,

During Rocky cycle we would like to switch tripleoclient to deploy
containeirzed undercloud by default but before to get there, we want to
switch all CI jobs to it, like it was done when enabling config-download by
default.
Right now we have 3 jobs which test the containerized undercloud:

- tripleo-ci-centos-7-undercloud-containers: deploy a containerized
undercloud and run Tempest
- tripleo-ci-centos-7-containerized-undercloud-upgrades: deploy a
non-containerized undercloud on Queens and upgrade to a containerized
undercloud on Rocky
- gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master: deploy a
containerized undercloud and an overcloud with HA architecture and IPv4
network (with introspection, SSL, etc).

What's next (target is Rocky M3):
- tripleo-ci-centos-7-containers-multinode - currently blocked by
https://bugs.launchpad.net/tripleo/+bug/1774297
- all multinode scenarios - current blocked by 1774297 as well but also
https://review.openstack.org/#/c/571566/
- gate-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset035-master - currently
blocked by https://bugs.launchpad.net/tripleo/+bug/1774557 (with a
potential fix: https://review.openstack.org/571620)
- all other jobs that run on master,
except tripleo-ci-centos-7-undercloud-oooq that we'll probably keep during
Rocky and remove in Stein if we successfully switch the default.

Once we've reached that point, we'll change tripleoclient's default, and
hopefully all of this before m3 :-)

Any question or feedback is highly welcome,
-- 
Emilien Macchi
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla][vote] Nominating Steve Noyes for kolla-cli core reviewer

[Surya Singh]

+1

Thanks for work, got the good feedback and interest of kolla-cli in
*kolla-rocky-ops-and-user-feedback* session in Vancouver.


On Thu, May 31, 2018 at 10:32 PM, Borne Mace  wrote:

> Greetings all,
>
> I would like to propose the addition of Steve Noyes to the kolla-cli core
> reviewer team.  Consider this nomination as my personal +1.
>
> Steve has a long history with the kolla-cli and should be considered its
> co-creator as probably half or more of the existing code was due to his
> efforts.  He has now been working diligently since it was pushed upstream
> to improve the stability and testability of the cli and has the second most
> commits on the project.
>
> The kolla core team consists of 19 people, and the kolla-cli team of 2,
> for a total of 21.  Steve therefore requires a minimum of 11 votes (so just
> 10 more after my +1), with no veto -2 votes within a 7 day voting window to
> end on June 6th.  Voting will be closed immediately on a veto or in the
> case of a unanimous vote.
>
> As I'm not sure how active all of the 19 kolla cores are, your attention
> and timely vote is much appreciated.
>
> Thanks!
>
> -- Borne
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla][vote] Nominating Steve Noyes for kolla-cli core reviewer

[Jeffrey Zhang]

+1
On Fri, Jun 1, 2018 at 1:41 AM Mark Giles  wrote:
>
> +1
>
> On May 31, 2018 at 1:06:43 PM, Borne Mace (borne.m...@oracle.com) wrote:
>
> Greetings all,
>
> I would like to propose the addition of Steve Noyes to the kolla-cli
> core reviewer team. Consider this nomination as my personal +1.
>
> Steve has a long history with the kolla-cli and should be considered its
> co-creator as probably half or more of the existing code was due to his
> efforts. He has now been working diligently since it was pushed
> upstream to improve the stability and testability of the cli and has the
> second most commits on the project.
>
> The kolla core team consists of 19 people, and the kolla-cli team of 2,
> for a total of 21. Steve therefore requires a minimum of 11 votes (so
> just 10 more after my +1), with no veto -2 votes within a 7 day voting
> window to end on June 6th. Voting will be closed immediately on a veto
> or in the case of a unanimous vote.
>
> As I'm not sure how active all of the 19 kolla cores are, your attention
> and timely vote is much appreciated.
>
> Thanks!
>
> -- Borne
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Regards,
Jeffrey Zhang
Blog: http://xcodest.me

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Cyborg] [Nova] Cyborg traits

[Alex Xu]

I can help on it.

2018-05-31 21:49 GMT+08:00 Eric Fried :

> Yup.  I'm sure reviewers will bikeshed the names, but the review is the
> appropriate place for that to happen.
>
> A couple of test changes will also be required.  You can have a look at
> [1] as an example to follow.
>
> -efried
>
> [1] https://review.openstack.org/#/c/511180/
>
> On 05/31/2018 01:02 AM, Nadathur, Sundar wrote:
> > On 5/30/2018 1:18 PM, Eric Fried wrote:
> >> This all sounds fully reasonable to me.  One thing, though...
> >>
> * There is a resource class per device category e.g.
>   CUSTOM_ACCELERATOR_GPU, CUSTOM_ACCELERATOR_FPGA.
> >> Let's propose standard resource classes for these ASAP.
> >>
> >> https://github.com/openstack/nova/blob/d741f624c81baf89fc8b6b94a2bc20
> eb5355a818/nova/rc_fields.py
> >>
> >>
> >> -efried
> > Makes sense, Eric. The obvious names would be ACCELERATOR_GPU and
> > ACCELERATOR_FPGA. Do we just submit a patch to rc_fields.py?
> >
> > Thanks,
> > Sundar
> >
> > 
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:
> unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova][glance] Deprecation of nova.image.download.modules extension point

[Moore, Curt]

Hello.



We recently upgraded from Liberty to Pike and looking ahead to the code in 
Queens, noticed the image download deprecation notice with instructions to post 
here if this interface was in use.  As such, I'd like to explain our use case 
and see if there is a better way of accomplishing our goal or lobby for the 
"un-deprecation" of this extension point.



As with many installations, we are using Ceph for both our Glance image store 
and VM instance disks.  In a normal workflow when both Glance and libvirt are 
configured to use Ceph, libvirt reacts to the direct_url field on the Glance 
image and performs an in-place clone of the RAW disk image from the images pool 
into the vms pool all within Ceph.  The snapshot creation process is very fast 
and is thinly provisioned as it's a COW snapshot.



This underlying workflow itself works great, the issue is with performance of 
the VM's disk within Ceph, especially as the number of nodes within the cluster 
grows.  We have found, especially with Windows VMs (largely as a result of I/O 
for the Windows pagefile), that the performance of the Ceph cluster as a whole 
takes a very large hit in keeping up with all of this I/O thrashing, especially 
when Windows is booting.  This is not the case with Linux VMs as they do not 
use swap as frequently as do Windows nodes with their pagefiles.  Windows can 
be run without a pagefile but that leads to other odditites within Windows.



I should also mention that in our case, the nodes themselves are ephemeral and 
we do not care about live migration, etc., we just want raw performance.



As an aside on our Ceph setup without getting into too many details, we have 
very fast SSD based Ceph nodes for this pool (separate crush root, SSDs for 
both OSD and journals, 2 replicas), interconnected on the same switch 
backplane, each with bonded 10GB uplinks to the switch.  Our Nova nodes are 
within the same datacenter (also have bonded 10GB uplinks to their switches) 
but are distributed across different switches.  We could move the Nova nodes to 
the same switch as the Ceph nodes but that is a larger logistical challenge to 
rearrange many servers to make space.



Back to our use case, in order to isolate this heavy I/O, a subset of our 
compute nodes have a local SSD and are set to use qcow2 images instead of rbd 
so that libvirt will pull the image down from Glance into the node's local 
image cache and run the VM from the local SSD.  This allows Windows VMs to boot 
and perform their initial cloudbase-init setup/reboot within ~20 sec vs 4-5 
min, regardless of overall Ceph cluster load.  Additionally, this prevents us 
from "wasting" IOPS and instead keep them local to the Nova node, reclaiming 
the network bandwidth and Ceph IOPS for use by Cinder volumes.  This is 
essentially the use case outlined here in the "Do designate some non-Ceph 
compute hosts with low-latency local storage" section:



https://ceph.com/planet/the-dos-and-donts-for-ceph-for-openstack/



The challenge is that transferring the Glance image transfer is _glacially 
slow_ when using the Glance HTTP API (~30 min for a 50GB Windows image (It's 
Windows, it's huge with all of the necessary tools installed)).  If libvirt can 
instead perform an RBD export on the image using the image download 
functionality, it is able to download the same image in ~30 sec.  We have code 
that is performing the direct download from Glance over RBD and it works great 
in our use case which is very similar to the code in this older patch:



https://review.openstack.org/#/c/44321/



We could look at attaching an additional ephemeral disk to the instance and 
have cloudbase-init use it as the pagefile but it appears that if libvirt is 
using rbd for its images_type, _all_ disks must then come from Ceph, there is 
no way at present to allow the VM image to run from Ceph and have an ephemeral 
disk mapped in from node-local storage.  Even still, this would have the effect 
of "wasting" Ceph IOPS for the VM disk itself which could be better used for 
other purposes.



Based on what I have explained about our use case, is there a better/different 
way to accomplish the same goal without using the deprecated image download 
functionality?  If not, can we work to "un-deprecate" the download extension 
point?  Should I work to get the code for this RBD download into the upstream 
repository?



Thanks,

-Curt



CONFIDENTIALITY NOTICE: This email and any attachments are for the sole use of 
the intended recipient(s) and contain information that may be Garmin 
confidential and/or Garmin legally privileged. If you have received this email 
in error, please notify the sender by reply email and delete the message. Any 
disclosure, copying, distribution or use of this communication (including 
attachments) by someone other than the intended recipient is prohibited. Thank 
you.

[openstack-dev] [keystone] failing documentation jobs

[Lance Bragstad]

Hi all,

If you've been trying to write documentation patches, you may have
noticed them tripping over unrelated errors when building the docs. We
have a bug opened detailing why this happened [0] and a fix working its
way through the gate [1]. The docs job should be back up and running soon.

[0] https://bugs.launchpad.net/keystone/+bug/1774508
[1] https://review.openstack.org/#/c/571369/



signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack-operators] attaching network cards to VMs taking a very long time

[Matt Riedemann]

On 5/30/2018 9:30 AM, Matt Riedemann wrote:


I can start pushing some docs patches and report back here for review help.


Here are the docs patches in both nova and neutron:

https://review.openstack.org/#/q/topic:bug/1774217+(status:open+OR+status:merged)

--

Thanks,

Matt

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [openstack-dev] Winterscale: a proposal regarding the project infrastructure

[James E. Blair]

Joshua Hesketh  writes:

> So the "winterscale infrastructure council"'s purview is quite limited in
> scope to just govern the services provided?
>
> If so, would you foresee a need to maintain some kind of "Infrastructure
> council" as it exists at the moment to be the technical design body?

For the foreseeable future, I think the "winterscale infrastructure
team" can probably handle that.  If it starts to sprawl again, we can
make a new body.

> Specifically, wouldn't we still want somewhere for the "winterscale
> infrastructure team" to be represented and would that expand to any
> infrastructure-related core teams?

Can you elaborate on this?  I'm not following.

-Jim

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Forum Recap - Stein Release Goals

[Sean McGinnis]

Here's my attempt to recap the goal selection discussion we had last week at
the Forum. Feel free to correct any misstatements and continue the discussion.

For reference, here's the etherpad from the discussion:

https://etherpad.openstack.org/p/YVR-S-release-goals

Overall Goal Discussion
===

We started off by discussing the reason for having the release cycle goals and
what we should actually be trying to achieve with them. There were some who
expressed concerns about the current Rocky goal selections not being the right
things we should be focusing on. The hope with the first part was to come to
some sort of agreement, or at least common understanding, that would inform our
selections for Stein.

Some thought the goals should be entirely operator facing. So things that have
an obvious and direct improvement for operators. At least so far, our goal
selection has mostly been to try to get one goal that is a visible thing like
that while the other is more of a technical debt cleanup. The idea being that
the tech debt ones will keep us in a good and healthy position to be able to
continue to address operator and user needs more easily.

There was also a desire to make the goals more "fully baked" before making them
a goal. This would mean having the necessary changes well documented with
example patches for teams to refer to to help guide them in figuring out what
needs to be done in their own repos.

There was also the desire to make these goals something that can generate some
excitement and be things that can be more of a marketing message. Things like
"OpenStack services now support live configuration changes" vs. "OpenStack got
rid of a testing library that no one has heard of".

And I almost missed it, but there was a great suggestion to have a #goals
channel for folks to go to for help and to discuss goal implementation. I
really like this idea and will bring it up in the next TC office hour to see if
we can get something official set up.

Stein Goals
===

We ended up with only 10-15 minutes to actually discuss some ideas for goal
selection for Stein. This was expected and planned. It will take some further
discussion and thought before I think we are ready to actually pick some goals.

Some of the more popular ones brought up in the session were:

- Cold upgrade support
- Python 3 first
- Addition of "upgrade check" CLIs

We were also able to already identify some possible goals for the T cycle:

- Move all CLIs to python-openstackclient
- Adopt a larger set of default roles

We've been collecting a "goal backlog" with these and other ideas here:

https://etherpad.openstack.org/p/community-goals

---
Sean (smcginnis)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][all] A culture change (nitpicking)

[Jeremy Stanley]

On 2018-05-31 16:49:13 -0400 (-0400), John Dennis wrote:
> On 05/30/2018 08:23 PM, Jeremy Stanley wrote:
> > I think this is orthogonal to the thread. The idea is that we should
> > avoid nettling contributors over minor imperfections in their
> > submissions (grammatical, spelling or typographical errors in code
> > comments and documentation, mild inefficiencies in implementations,
> > et cetera). Clearly we shouldn't merge broken features, changes
> > which fail tests/linters, and so on. For me the rule of thumb is,
> > "will the software be better or worse if this is merged?" It's not
> > about perfection or imperfection, it's about incremental
> > improvement. If a proposed change is an improvement, that's enough.
> > If it's not perfect... well, that's just opportunity for more
> > improvement later.
> 
> I appreciate the sentiment concerning accepting any improvement yet on the
> other hand waiting for improvements to the patch to occur later is folly, it
> won't happen.
> 
> Those of us familiar with working with large bodies of code from multiple
> authors spanning an extended time period will tell you it's very confusing
> when it's obvious most of the code follows certain conventions but there are
> odd exceptions (often without comments). This inevitably leads to investing
> a lot of time trying to understand why the exception exists because "clearly
> it's there for a reason and I'm just missing the rationale" At that point
> the reason for the inconsistency is lost.
> 
> At the end of the day it is more important to keep the code base clean and
> consistent for those that follow than it is to coddle in the near term.

Sure, I suppose it comes down to your definition of "improvement." I
don't consider a change proposing incomplete or unmaintainable code
to be an improvement. On the other hand I think it's fine to approve
changes which are "good enough" even if there's room for
improvement, so long as they're "good enough" that you're fine with
them possibly never being improved on due to shifts in priorities.
I'm certainly not suggesting that it's a good idea to merge
technical debt with the expectation that someone will find time to
solve it later (any more than it's okay to merge obvious bugs in
hopes someone will come along and fix them for you).
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][all] A culture change (nitpicking)

[John Dennis]

On 05/30/2018 08:23 PM, Jeremy Stanley wrote:

I think this is orthogonal to the thread. The idea is that we should
avoid nettling contributors over minor imperfections in their
submissions (grammatical, spelling or typographical errors in code
comments and documentation, mild inefficiencies in implementations,
et cetera). Clearly we shouldn't merge broken features, changes
which fail tests/linters, and so on. For me the rule of thumb is,
"will the software be better or worse if this is merged?" It's not
about perfection or imperfection, it's about incremental
improvement. If a proposed change is an improvement, that's enough.
If it's not perfect... well, that's just opportunity for more
improvement later.


I appreciate the sentiment concerning accepting any improvement yet on 
the other hand waiting for improvements to the patch to occur later is 
folly, it won't happen.


Those of us familiar with working with large bodies of code from 
multiple authors spanning an extended time period will tell you it's 
very confusing when it's obvious most of the code follows certain 
conventions but there are odd exceptions (often without comments). This 
inevitably leads to investing a lot of time trying to understand why the 
exception exists because "clearly it's there for a reason and I'm just 
missing the rationale" At that point the reason for the inconsistency is 
lost.


At the end of the day it is more important to keep the code base clean 
and consistent for those that follow than it is to coddle in the near term.


--
John Dennis

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack-operators] [openstack-dev] [nova] proposal to postpone nova-network core functionality removal to Stein

[Matt Riedemann]

+openstack-operators

On 5/31/2018 3:04 PM, Matt Riedemann wrote:

On 5/31/2018 1:35 PM, melanie witt wrote:


This cycle at the PTG, we had decided to start making some progress 
toward removing nova-network [1] (thanks to those who have helped!) 
and so far, we've landed some patches to extract common network 
utilities from nova-network core functionality into separate utility 
modules. And we've started proposing removal of nova-network REST APIs 
[2].


At the cells v2 sync with operators forum session at the summit [3], 
we learned that CERN is in the middle of migrating from nova-network 
to neutron and that holding off on removal of nova-network core 
functionality until Stein would help them out a lot to have a safety 
net as they continue progressing through the migration.


If we recall correctly, they did say that removal of the nova-network 
REST APIs would not impact their migration and Surya Seetharaman is 
double-checking about that and will get back to us. If so, we were 
thinking we can go ahead and work on nova-network REST API removals 
this cycle to make some progress while holding off on removing the 
core functionality of nova-network until Stein.


I wanted to send this to the ML to let everyone know what we were 
thinking about this and to receive any additional feedback folks might 
have about this plan.


Thanks,
-melanie

[1] https://etherpad.openstack.org/p/nova-ptg-rocky L301
[2] https://review.openstack.org/567682
[3] 
https://etherpad.openstack.org/p/YVR18-cellsv2-migration-sync-with-operators 
L30


As a reminder, this is the etherpad I started to document the nova-net 
specific compute REST APIs which are candidates for removal:


https://etherpad.openstack.org/p/nova-network-removal-rocky




--

Thanks,

Matt

___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [openstack-dev] [nova] proposal to postpone nova-network core functionality removal to Stein

[Matt Riedemann]

+openstack-operators

On 5/31/2018 3:04 PM, Matt Riedemann wrote:

On 5/31/2018 1:35 PM, melanie witt wrote:


This cycle at the PTG, we had decided to start making some progress 
toward removing nova-network [1] (thanks to those who have helped!) 
and so far, we've landed some patches to extract common network 
utilities from nova-network core functionality into separate utility 
modules. And we've started proposing removal of nova-network REST APIs 
[2].


At the cells v2 sync with operators forum session at the summit [3], 
we learned that CERN is in the middle of migrating from nova-network 
to neutron and that holding off on removal of nova-network core 
functionality until Stein would help them out a lot to have a safety 
net as they continue progressing through the migration.


If we recall correctly, they did say that removal of the nova-network 
REST APIs would not impact their migration and Surya Seetharaman is 
double-checking about that and will get back to us. If so, we were 
thinking we can go ahead and work on nova-network REST API removals 
this cycle to make some progress while holding off on removing the 
core functionality of nova-network until Stein.


I wanted to send this to the ML to let everyone know what we were 
thinking about this and to receive any additional feedback folks might 
have about this plan.


Thanks,
-melanie

[1] https://etherpad.openstack.org/p/nova-ptg-rocky L301
[2] https://review.openstack.org/567682
[3] 
https://etherpad.openstack.org/p/YVR18-cellsv2-migration-sync-with-operators 
L30


As a reminder, this is the etherpad I started to document the nova-net 
specific compute REST APIs which are candidates for removal:


https://etherpad.openstack.org/p/nova-network-removal-rocky




--

Thanks,

Matt

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] proposal to postpone nova-network core functionality removal to Stein

[Matt Riedemann]

On 5/31/2018 1:35 PM, melanie witt wrote:


This cycle at the PTG, we had decided to start making some progress 
toward removing nova-network [1] (thanks to those who have helped!) and 
so far, we've landed some patches to extract common network utilities 
from nova-network core functionality into separate utility modules. And 
we've started proposing removal of nova-network REST APIs [2].


At the cells v2 sync with operators forum session at the summit [3], we 
learned that CERN is in the middle of migrating from nova-network to 
neutron and that holding off on removal of nova-network core 
functionality until Stein would help them out a lot to have a safety net 
as they continue progressing through the migration.


If we recall correctly, they did say that removal of the nova-network 
REST APIs would not impact their migration and Surya Seetharaman is 
double-checking about that and will get back to us. If so, we were 
thinking we can go ahead and work on nova-network REST API removals this 
cycle to make some progress while holding off on removing the core 
functionality of nova-network until Stein.


I wanted to send this to the ML to let everyone know what we were 
thinking about this and to receive any additional feedback folks might 
have about this plan.


Thanks,
-melanie

[1] https://etherpad.openstack.org/p/nova-ptg-rocky L301
[2] https://review.openstack.org/567682
[3] 
https://etherpad.openstack.org/p/YVR18-cellsv2-migration-sync-with-operators 
L30


As a reminder, this is the etherpad I started to document the nova-net 
specific compute REST APIs which are candidates for removal:


https://etherpad.openstack.org/p/nova-network-removal-rocky

--

Thanks,

Matt

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

[Sylvain Bauza]

On Thu, May 31, 2018 at 8:26 PM, Eric Fried  wrote:

> > 1. Make everything perform the pivot on compute node start (which can be
> >re-used by a CLI tool for the offline case)
> > 2. Make everything default to non-nested inventory at first, and provide
> >a way to migrate a compute node and its instances one at a time (in
> >place) to roll through.
>
> I agree that it sure would be nice to do ^ rather than requiring the
> "slide puzzle" thing.
>
> But how would this be accomplished, in light of the current "separation
> of responsibilities" drawn at the virt driver interface, whereby the
> virt driver isn't supposed to talk to placement directly, or know
> anything about allocations?  Here's a first pass:
>
>

What we usually do is to implement either at the compute service level or
at the virt driver level some init_host() method that will reconcile what
you want.
For example, we could just imagine a non-virt specific method (and I like
that because it's non-virt specific) - ie. called by compute's init_host()
that would lookup the compute root RP inventories, see whether one ore more
inventories tied to specific resource classes have to be moved from the
root RP and be attached to a child RP.
The only subtility that would require a virt-specific update would be the
name of the child RP (as both Xen and libvirt plan to use the child RP name
as the vGPU type identifier) but that's an implementation detail that a
possible virt driver update by the resource tracker would reconcile that.


The virt driver, via the return value from update_provider_tree, tells
> the resource tracker that "inventory of resource class A on provider B
> have moved to provider C" for all applicable AxBxC.  E.g.
>
> [ { 'from_resource_provider': ,
> 'moved_resources': [VGPU: 4],
> 'to_resource_provider': 
>   },
>   { 'from_resource_provider': ,
> 'moved_resources': [VGPU: 4],
> 'to_resource_provider': 
>   },
>   { 'from_resource_provider': ,
> 'moved_resources': [
> SRIOV_NET_VF: 2,
> NET_BANDWIDTH_EGRESS_KILOBITS_PER_SECOND: 1000,
> NET_BANDWIDTH_INGRESS_KILOBITS_PER_SECOND: 1000,
> ],
> 'to_resource_provider': 
>   }
> ]
>
> As today, the resource tracker takes the updated provider tree and
> invokes [1] the report client method update_from_provider_tree [2] to
> flush the changes to placement.  But now update_from_provider_tree also
> accepts the return value from update_provider_tree and, for each "move":
>
> - Creates provider C (as described in the provider_tree) if it doesn't
> already exist.
> - Creates/updates provider C's inventory as described in the
> provider_tree (without yet updating provider B's inventory).  This ought
> to create the inventory of resource class A on provider C.
> - Discovers allocations of rc A on rp B and POSTs to move them to rp C*.
> - Updates provider B's inventory.
>
> (*There's a hole here: if we're splitting a glommed-together inventory
> across multiple new child providers, as the VGPUs in the example, we
> don't know which allocations to put where.  The virt driver should know
> which instances own which specific inventory units, and would be able to
> report that info within the data structure.  That's getting kinda close
> to the virt driver mucking with allocations, but maybe it fits well
> enough into this model to be acceptable?)
>
> Note that the return value from update_provider_tree is optional, and
> only used when the virt driver is indicating a "move" of this ilk.  If
> it's None/[] then the RT/update_from_provider_tree flow is the same as
> it is today.
>
> If we can do it this way, we don't need a migration tool.  In fact, we
> don't even need to restrict provider tree "reshaping" to release
> boundaries.  As long as the virt driver understands its own data model
> migrations and reports them properly via update_provider_tree, it can
> shuffle its tree around whenever it wants.
>
> Thoughts?
>
> -efried
>
> [1]
> https://github.com/openstack/nova/blob/8753c9a38667f984d385b4783c3c2f
> c34d7e8e1b/nova/compute/resource_tracker.py#L890
> [2]
> https://github.com/openstack/nova/blob/8753c9a38667f984d385b4783c3c2f
> c34d7e8e1b/nova/scheduler/client/report.py#L1341
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

[Sylvain Bauza]

On Thu, May 31, 2018 at 7:44 PM, Chris Dent  wrote:

> On Thu, 31 May 2018, Dan Smith wrote:
>
> I kinda think we need to either:
>>
>> 1. Make everything perform the pivot on compute node start (which can be
>>   re-used by a CLI tool for the offline case)
>>
>
> This sounds effectively like: validate my inventory and allocations
> at compute node start, correcting them as required (including the
> kind of migration stuff related to nested). Is that right?
>
> That's something I'd like to be the norm. It takes us back to a sort
> of self-healing compute node.
>
> Or am I missing something (forgive me, I've been on holiday).
>


I think I understand the same as you. And I think it's actually the best
approach. Wow, Dan, you saved my life again. Should I call you Mitch
Buchannon ?



>
> I just think that forcing people to take down their data plane to work
>> around our own data model is kinda evil and something we should be
>> avoiding at this level of project maturity. What we're really saying is
>> "we know how to translate A into B, but we require you to move many GBs
>> of data over the network and take some downtime because it's easier for
>> *us* than making it seamless."
>>
>
> If we can do it, I agree that being not evil is good.
>
> --
> Chris Dent   ٩◔̯◔۶   https://anticdent.org/
> freenode: cdent tw: @anticdent
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

[Jay Pipes]

On 05/31/2018 01:09 PM, Dan Smith wrote:

My feeling is that we should not attempt to "migrate" any allocations
or inventories between root or child providers within a compute node,
period.


While I agree this is the simplest approach, it does put a lot of
responsibility on the operators to do work to sidestep this issue, which
might not even apply to them (and knowing if it does might be
difficult).


Perhaps, yes. Though the process I described is certainly not foreign to 
operators. It is a safe and well-practiced approach.



The virt drivers should simply error out of update_provider_tree() if
there are ANY existing VMs on the host AND the virt driver wishes to
begin tracking resources with nested providers.

The upgrade operation should look like this:

1) Upgrade placement
2) Upgrade nova-scheduler
3) start loop on compute nodes. for each compute node:
  3a) disable nova-compute service on node (to take it out of scheduling)
  3b) evacuate all existing VMs off of node


You mean s/evacuate/cold migrate/ of course... :)


I meant evacuate as in `nova host-evacuate-live` with a fall back to 
`nova host-servers-migrate` if live migration isn't possible).



  3c) upgrade compute node (on restart, the compute node will see no
  VMs running on the node and will construct the provider tree inside
  update_provider_tree() with an appropriate set of child providers
  and inventories on those child providers)
  3d) enable nova-compute service on node

Which is virtually identical to the "normal" upgrade process whenever
there are significant changes to the compute node -- such as upgrading
libvirt or the kernel.


Not necessarily. It's totally legit (and I expect quite common) to just
reboot the host to take kernel changes, bringing back all the instances
that were there when it resumes.


So, you're saying the normal process is to try upgrading the Linux 
kernel and associated low-level libs, wait the requisite amount of time 
that takes (can be a long time) and just hope that everything comes back 
OK? That doesn't sound like any upgrade I've ever seen. All upgrade 
procedures I have seen attempt to get the workloads off of the compute 
host before trying anything major (and upgrading a linux kernel or 
low-level lib like libvirt is a major thing IMHO).


> The "normal" case of moving things
around slide-puzzle-style applies to live migration (which isn't an 
option here).
Sorry, I was saying that for all the lovely resources that have been 
bolted on to Nova in the past 5 years (CPU pinning, NUMA topologies, PCI 
passthrough, SR-IOV PF/VFs, vGPUs, etc), that if the workload uses 
*those* resources, then live migration won't work and that the admin 
would need to fall back to nova host-servers-migrate. I wasn't saying 
that live migration for all workloads/instances would not be a possibility.



I think people that can take downtime for the instances would rather
not have to move things around for no reason if the instance has to
get shut off anyway.


Maybe. Not sure. But my line of thinking is stick to a single, already 
known procedure since that is safe and well-practiced.


Code that we don't have to write means code that doesn't have new bugs 
that we'll have to track down and fix.


I'm also thinking that we'd be tracking down and fixing those bugs while 
trying to put out a fire that was caused by trying to auto-heal 
everything at once on nova-compute startup and resulting in broken state 
and an inability of the nova-compute service to start again, essentially 
trapping instances on the failed host. ;)



Nested resource tracking is another such significant change and should
be dealt with in a similar way, IMHO.


This basically says that for anyone to move to rocky, they will have to
cold migrate every single instance in order to do that upgrade right?


No, sorry if I wasn't clear. They can live-migrate the instances off of 
the to-be-upgraded compute host. They would only need to cold-migrate 
instances that use the aforementioned non-movable resources.



I kinda think we need to either:

1. Make everything perform the pivot on compute node start (which can be
re-used by a CLI tool for the offline case)

>

2. Make everything default to non-nested inventory at first, and provide
a way to migrate a compute node and its instances one at a time (in
place) to roll through.


I would vote for Option #2 if it comes down to it.

If we are going to go through the hassle of writing a bunch of 
transformation code in order to keep operator action as low as possible, 
I would prefer to consolidate all of this code into the nova-manage (or 
nova-status) tool and put some sort of attribute/marker on each compute 
node record to indicate whether a "heal" operation has occurred for that 
compute node.


Kinda like what Matt's been playing with for the heal_allocations stuff.

At least in that case, we'd have all the transform/heal code in a single 
place and we wouldn't need to 

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

[Sylvain Bauza]

On Thu, May 31, 2018 at 7:09 PM, Dan Smith  wrote:

> > My feeling is that we should not attempt to "migrate" any allocations
> > or inventories between root or child providers within a compute node,
> > period.
>
> While I agree this is the simplest approach, it does put a lot of
> responsibility on the operators to do work to sidestep this issue, which
> might not even apply to them (and knowing if it does might be
> difficult).
>
>
Shit, I missed the point why we were discussing about migrations. When you
upgrade, you wanna move your workloads for upgrading your kernel and the
likes. Gotcha.
But, I assume that's not something mandatory for a single upgrade (say
Queens>Rocky). In that case, you just wanna upgrade your compute without
moving your instances. Or you notified your users about a maintenance and
you know you have a minimal maintenance period for breaking them.
In both cases, adding more steps for upgrading seems a tricky and dangerous
path for those operators who are afraid of making a mistake.


> > The virt drivers should simply error out of update_provider_tree() if
> > there are ANY existing VMs on the host AND the virt driver wishes to
> > begin tracking resources with nested providers.
> >
> > The upgrade operation should look like this:
> >
> > 1) Upgrade placement
> > 2) Upgrade nova-scheduler
> > 3) start loop on compute nodes. for each compute node:
> >  3a) disable nova-compute service on node (to take it out of scheduling)
> >  3b) evacuate all existing VMs off of node
>
> You mean s/evacuate/cold migrate/ of course... :)
>
> >  3c) upgrade compute node (on restart, the compute node will see no
> >  VMs running on the node and will construct the provider tree inside
> >  update_provider_tree() with an appropriate set of child providers
> >  and inventories on those child providers)
> >  3d) enable nova-compute service on node
> >
> > Which is virtually identical to the "normal" upgrade process whenever
> > there are significant changes to the compute node -- such as upgrading
> > libvirt or the kernel.
>
> Not necessarily. It's totally legit (and I expect quite common) to just
> reboot the host to take kernel changes, bringing back all the instances
> that were there when it resumes. The "normal" case of moving things
> around slide-puzzle-style applies to live migration (which isn't an
> option here). I think people that can take downtime for the instances
> would rather not have to move things around for no reason if the
> instance has to get shut off anyway.
>
>
Yeah exactly that. Accepting a downtime is fair, to the price to not have a
long list of operations to do during that downtime period.



> > Nested resource tracking is another such significant change and should
> > be dealt with in a similar way, IMHO.
>
> This basically says that for anyone to move to rocky, they will have to
> cold migrate every single instance in order to do that upgrade right? I
> mean, anyone with two socket machines or SRIOV NICs would end up with at
> least one level of nesting, correct? Forcing everyone to move everything
> to do an upgrade seems like a non-starter to me.
>
>
For the moment, we aren't providing NUMA topologies with nested RPs but
once we do that, yeah, that would imply the above, which sounds harsh to
hear from an operator perspective.



> We also need to consider the case where people would be FFU'ing past
> rocky (i.e. never running rocky computes). We've previously said that
> we'd provide a way to push any needed transitions with everything
> offline to facilitate that case, so I think we need to implement that
> method anyway.
>
> I kinda think we need to either:
>
> 1. Make everything perform the pivot on compute node start (which can be
>re-used by a CLI tool for the offline case)
>

That's another alternative I haven't explored yet. Thanks for the idea. We
already reconcile the world when we restart the compute service by checking
whether mediated devices exist, so that could be a good option actually.



> 2. Make everything default to non-nested inventory at first, and provide
>a way to migrate a compute node and its instances one at a time (in
>place) to roll through.
>
>
 We could say that Rocky isn't supporting multiple vGPU types until you
make the necessary DB migration that will create child RPs and the likes.
That's yet another approach.

We can also document "or do the cold-migration slide puzzle thing" as an
> alternative for people that feel that's more reasonable.
>
> I just think that forcing people to take down their data plane to work
> around our own data model is kinda evil and something we should be
> avoiding at this level of project maturity. What we're really saying is
> "we know how to translate A into B, but we require you to move many GBs
> of data over the network and take some downtime because it's easier for
> *us* than making it seamless."
>
> --Dan
>
> 

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

[Sylvain Bauza]

On Thu, May 31, 2018 at 5:00 PM, Jay Pipes  wrote:

> On 05/31/2018 05:10 AM, Sylvain Bauza wrote:
>
>> After considering the whole approach, discussing with a couple of folks
>> over IRC, here is what I feel the best approach for a seamless upgrade :
>>   - VGPU inventory will be kept on root RP (for the first type) in Queens
>> so that a compute service upgrade won't impact the DB
>>   - during Queens, operators can run a DB online migration script (like
>> the ones we currently have in https://github.com/openstack/n
>> ova/blob/c2f42b0/nova/cmd/manage.py#L375) that will create a new
>> resource provider for the first type and move the inventory and allocations
>> to it.
>>   - it's the responsibility of the virt driver code to check whether a
>> child RP with its name being the first type name already exists to know
>> whether to update the inventory against the root RP or the child RP.
>>
>> Does it work for folks ?
>>
>
> No, sorry, that doesn't work for me. It seems overly complex and fragile,
> especially considering that VGPUs are not moveable anyway (no support for
> live migrating them). Same goes for CPU pinning, NUMA topologies, PCI
> passthrough devices, SR-IOV PF/VFs and all the other "must have" features
> that have been added to the virt driver over the last 5 years.
>
> My feeling is that we should not attempt to "migrate" any allocations or
> inventories between root or child providers within a compute node, period.
>
>
I don't understand why you're talking of *moving* an instance. My concern
was about upgrading a compute node to Rocky where some instances were
already there, and using vGPUs.


> The virt drivers should simply error out of update_provider_tree() if
> there are ANY existing VMs on the host AND the virt driver wishes to begin
> tracking resources with nested providers.
>
> The upgrade operation should look like this:
>
> 1) Upgrade placement
> 2) Upgrade nova-scheduler
> 3) start loop on compute nodes. for each compute node:
>  3a) disable nova-compute service on node (to take it out of scheduling)
>  3b) evacuate all existing VMs off of node
>  3c) upgrade compute node (on restart, the compute node will see no
>  VMs running on the node and will construct the provider tree inside
>  update_provider_tree() with an appropriate set of child providers
>  and inventories on those child providers)
>  3d) enable nova-compute service on node
>
> Which is virtually identical to the "normal" upgrade process whenever
> there are significant changes to the compute node -- such as upgrading
> libvirt or the kernel. Nested resource tracking is another such significant
> change and should be dealt with in a similar way, IMHO.
>
>
Upgrading to Rocky for vGPUs doesn't need to also upgrade libvirt or the
kernel. So why operators should need to "evacuate" (I understood that as
"migrate")  instances if they don't need to upgrade their host OS ?

Best,
> -jay
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

[Sylvain Bauza]

On Thu, May 31, 2018 at 4:34 PM, Jay Pipes  wrote:

> On 05/29/2018 09:12 AM, Sylvain Bauza wrote:
>
>> We could keep the old inventory in the root RP for the previous vGPU type
>> already supported in Queens and just add other inventories for other vGPU
>> types now supported. That looks possibly the simpliest option as the virt
>> driver knows that.
>>
>
> What do you mean by "vGPU type"? Are you referring to the multiple GPU
> types stuff where specific virt drivers know how to handle different vGPU
> vendor types? Or are you referring to a "non-nested VGPU inventory on the
> compute node provider" versus a "VGPU inventory on multiple child
> providers, each representing a different physical GPU (or physical GPU
> group in the case of Xen)"?
>
>
I speak about a "vGPU type" because it's how we agreed to have multiple
child RPs.
See
https://specs.openstack.org/openstack/nova-specs/specs/queens/implemented/add-support-for-vgpu.html#proposed-change

For Xen, a vGPU type is a Xen GPU group. For libvirt, it's just a mdev type.
Each pGPU can support multiple types. For the moment, we only support one
type, but my spec ( https://review.openstack.org/#/c/557065/ ) explains
more about that.


-jay
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

[Eric Fried]

Chris-

>> virt driver isn't supposed to talk to placement directly, or know
>> anything about allocations?
> 
> For sake of discussion, how much (if any) easier would it be if we
> got rid of this restriction?

At this point, having implemented the update_[from_]provider_tree flow
as we have, it would probably make things harder.  We still have to do
the same steps, but any bits we wanted to let the virt driver handle
would need some kind of weird callback dance.

But even if we scrapped update_[from_]provider_tree and redesigned from
first principles, virt drivers would have a lot of duplication of the
logic that currently resides in update_from_provider_tree.

So even though the restriction seems to make things awkward, having been
embroiled in this code as I have, I'm actually seeing how it keeps
things as clean and easy to reason about as can be expected for
something that's inherently as complicated as this.

>> the resource tracker that "inventory of resource class A on provider B
>> have moved to provider C" for all applicable AxBxC.  E.g.
> 
> traits too?

The traits are part of the updated provider tree itself.  The existing
logic in update_from_provider_tree handles shuffling those around.  I
don't think the RT needs to be told about any specific trait movement in
order to reason about moving allocations.  Do you see something I'm
missing there?

> The fact that we are using what amounts to a DSL to pass
> some additional instruction back from the virt driver feels squiffy

Yeah, I don't disagree.  The provider_tree object, and updating it via
update_provider_tree, is kind of a DSL already.  The list-of-dicts
format is just a strawman; we could make it an object or whatever (not
that that would make it less DSL-ish).

Perhaps an OVO :P

-efried
.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

[Sylvain Bauza]

On Thu, May 31, 2018 at 3:54 PM, Eric Fried  wrote:

> This seems reasonable, but...
>
> On 05/31/2018 04:34 AM, Balázs Gibizer wrote:
> >
> >
> > On Thu, May 31, 2018 at 11:10 AM, Sylvain Bauza 
> wrote:
> >>>
> >>
> >> After considering the whole approach, discussing with a couple of
> >> folks over IRC, here is what I feel the best approach for a seamless
> >> upgrade :
> >>  - VGPU inventory will be kept on root RP (for the first type) in
> >> Queens so that a compute service upgrade won't impact the DB
> >>  - during Queens, operators can run a DB online migration script (like
> -^^
> Did you mean Rocky?
>


Oops, yeah of course. Queens > Rocky.

>
> >> the ones we currently have in
> >> https://github.com/openstack/nova/blob/c2f42b0/nova/cmd/manage.py#L375)
> that
> >> will create a new resource provider for the first type and move the
> >> inventory and allocations to it.
> >>  - it's the responsibility of the virt driver code to check whether a
> >> child RP with its name being the first type name already exists to
> >> know whether to update the inventory against the root RP or the child
> RP.
> >>
> >> Does it work for folks ?
> >
> > +1 works for me
> > gibi
> >
> >> PS : we already have the plumbing in place in nova-manage and we're
> >> still managing full Nova resources. I know we plan to move Placement
> >> out of the nova tree, but for the Rocky timeframe, I feel we can
> >> consider nova-manage as the best and quickiest approach for the data
> >> upgrade.
> >>
> >> -Sylvain
> >>
> >>
> >
> >
> > 
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:
> unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [api] notes from api-sig forum meeting on graphql experiment

[Michael McCune]

hi everybody,

i have added my notes to the etherpad[1] from summit.

briefly, we had a great meeting about creating a graphql experiment in
openstack and i tried to capture some of the questions and comments
that flew back and forth.

there seems to be a good consensus that a proof of concept will be
created on the neutron server, most likely in an experimental feature
branch. Gilles Dubreuil has volunteered to lead this effort (thank you
Gilles!), hopefully with some help from a few neutron savy folks ;)

it is still very early in this experiment, but i think there was a
good feeling that if this works it could create some great
opportunities for improvement across the openstack ecosystem. i really
hope it does!

peace o/


[1]: https://etherpad.openstack.org/p/YVR18-API-SIG-forum

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

[Chris Dent]

On Thu, 31 May 2018, Eric Fried wrote:


But how would this be accomplished, in light of the current "separation
of responsibilities" drawn at the virt driver interface, whereby the
virt driver isn't supposed to talk to placement directly, or know
anything about allocations?  Here's a first pass:


For sake of discussion, how much (if any) easier would it be if we
got rid of this restriction?


the resource tracker that "inventory of resource class A on provider B
have moved to provider C" for all applicable AxBxC.  E.g.


traits too?


[ { 'from_resource_provider': ,
   'moved_resources': [VGPU: 4],
   'to_resource_provider': 

[snip]


If we can do it this way, we don't need a migration tool.  In fact, we
don't even need to restrict provider tree "reshaping" to release
boundaries.  As long as the virt driver understands its own data model
migrations and reports them properly via update_provider_tree, it can
shuffle its tree around whenever it wants.


Assuming the restriction is kept, your model seems at least worth
exploring. The fact that we are using what amounts to a DSL to pass
some additional instruction back from the virt driver feels squiffy
for some reason (probably because I'm not wed to the restriction),
but it is well-contained.

--
Chris Dent   ٩◔̯◔۶   https://anticdent.org/
freenode: cdent tw: @anticdent__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[Openstack-operators] [nova] proposal to postpone nova-network core functionality removal to Stein

[melanie witt]

Hello Operators and Devs,

This cycle at the PTG, we had decided to start making some progress 
toward removing nova-network [1] (thanks to those who have helped!) and 
so far, we've landed some patches to extract common network utilities 
from nova-network core functionality into separate utility modules. And 
we've started proposing removal of nova-network REST APIs [2].


At the cells v2 sync with operators forum session at the summit [3], we 
learned that CERN is in the middle of migrating from nova-network to 
neutron and that holding off on removal of nova-network core 
functionality until Stein would help them out a lot to have a safety net 
as they continue progressing through the migration.


If we recall correctly, they did say that removal of the nova-network 
REST APIs would not impact their migration and Surya Seetharaman is 
double-checking about that and will get back to us. If so, we were 
thinking we can go ahead and work on nova-network REST API removals this 
cycle to make some progress while holding off on removing the core 
functionality of nova-network until Stein.


I wanted to send this to the ML to let everyone know what we were 
thinking about this and to receive any additional feedback folks might 
have about this plan.


Thanks,
-melanie

[1] https://etherpad.openstack.org/p/nova-ptg-rocky L301
[2] https://review.openstack.org/567682
[3] 
https://etherpad.openstack.org/p/YVR18-cellsv2-migration-sync-with-operators 
L30


___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

[openstack-dev] [nova] proposal to postpone nova-network core functionality removal to Stein

[melanie witt]

Hello Operators and Devs,

This cycle at the PTG, we had decided to start making some progress 
toward removing nova-network [1] (thanks to those who have helped!) and 
so far, we've landed some patches to extract common network utilities 
from nova-network core functionality into separate utility modules. And 
we've started proposing removal of nova-network REST APIs [2].


At the cells v2 sync with operators forum session at the summit [3], we 
learned that CERN is in the middle of migrating from nova-network to 
neutron and that holding off on removal of nova-network core 
functionality until Stein would help them out a lot to have a safety net 
as they continue progressing through the migration.


If we recall correctly, they did say that removal of the nova-network 
REST APIs would not impact their migration and Surya Seetharaman is 
double-checking about that and will get back to us. If so, we were 
thinking we can go ahead and work on nova-network REST API removals this 
cycle to make some progress while holding off on removing the core 
functionality of nova-network until Stein.


I wanted to send this to the ML to let everyone know what we were 
thinking about this and to receive any additional feedback folks might 
have about this plan.


Thanks,
-melanie

[1] https://etherpad.openstack.org/p/nova-ptg-rocky L301
[2] https://review.openstack.org/567682
[3] 
https://etherpad.openstack.org/p/YVR18-cellsv2-migration-sync-with-operators 
L30


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][all] A culture change (nitpicking)

[Julia Kreger]

Back to the topic of nitpicking!

I virtually sat down with Doug today and we hammered out the positive
aspects that we feel like are the things that we as a community want
to see as part of reviews coming out of this effort. The principles
change[1] in governance has been updated as a result.

I think we are at a point where we have to state high level
principles, and then also update guidelines or other context providing
documentation to re-enforce some of items covered in this
discussion... not just to educate new contributors, but to serve as a
checkpoint for existing reviewers when making the decision as to how
to vote change set. The question then becomes where would such
guidelines or documentation best fit? Should we explicitly detail the
cause/effect that occurs? Should we convey contributor perceptions, or
maybe even just link to this thread as there has been a massive amount
of feedback raising valid cases, points, and frustrations.

Personally, I'd lean towards a blended approach, but the question of
where is one I'm unsure of. Thoughts?

-Julia

[1]: https://review.openstack.org/#/c/570940/

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

[Eric Fried]

Rats, typo correction below.

On 05/31/2018 01:26 PM, Eric Fried wrote:
>> 1. Make everything perform the pivot on compute node start (which can be
>>re-used by a CLI tool for the offline case)
>> 2. Make everything default to non-nested inventory at first, and provide
>>a way to migrate a compute node and its instances one at a time (in
>>place) to roll through.
> 
> I agree that it sure would be nice to do ^ rather than requiring the
> "slide puzzle" thing.
> 
> But how would this be accomplished, in light of the current "separation
> of responsibilities" drawn at the virt driver interface, whereby the
> virt driver isn't supposed to talk to placement directly, or know
> anything about allocations?  Here's a first pass:
> 
> The virt driver, via the return value from update_provider_tree, tells
> the resource tracker that "inventory of resource class A on provider B
> have moved to provider C" for all applicable AxBxC.  E.g.
> 
> [ { 'from_resource_provider': ,
> 'moved_resources': [VGPU: 4],
> 'to_resource_provider': 
>   },
>   { 'from_resource_provider': ,
> 'moved_resources': [VGPU: 4],
> 'to_resource_provider': 
>   },
>   { 'from_resource_provider': ,
> 'moved_resources': [
> SRIOV_NET_VF: 2,
> NET_BANDWIDTH_EGRESS_KILOBITS_PER_SECOND: 1000,
> NET_BANDWIDTH_INGRESS_KILOBITS_PER_SECOND: 1000,
> ],
> 'to_resource_provider': 
---
s/gpu_rp2_uuid/sriovnic_rp_uuid/ or similar.

>   }
> ]
> 
> As today, the resource tracker takes the updated provider tree and
> invokes [1] the report client method update_from_provider_tree [2] to
> flush the changes to placement.  But now update_from_provider_tree also
> accepts the return value from update_provider_tree and, for each "move":
> 
> - Creates provider C (as described in the provider_tree) if it doesn't
> already exist.
> - Creates/updates provider C's inventory as described in the
> provider_tree (without yet updating provider B's inventory).  This ought
> to create the inventory of resource class A on provider C.
> - Discovers allocations of rc A on rp B and POSTs to move them to rp C*.
> - Updates provider B's inventory.
> 
> (*There's a hole here: if we're splitting a glommed-together inventory
> across multiple new child providers, as the VGPUs in the example, we
> don't know which allocations to put where.  The virt driver should know
> which instances own which specific inventory units, and would be able to
> report that info within the data structure.  That's getting kinda close
> to the virt driver mucking with allocations, but maybe it fits well
> enough into this model to be acceptable?)
> 
> Note that the return value from update_provider_tree is optional, and
> only used when the virt driver is indicating a "move" of this ilk.  If
> it's None/[] then the RT/update_from_provider_tree flow is the same as
> it is today.
> 
> If we can do it this way, we don't need a migration tool.  In fact, we
> don't even need to restrict provider tree "reshaping" to release
> boundaries.  As long as the virt driver understands its own data model
> migrations and reports them properly via update_provider_tree, it can
> shuffle its tree around whenever it wants.
> 
> Thoughts?
> 
> -efried
> 
> [1]
> https://github.com/openstack/nova/blob/8753c9a38667f984d385b4783c3c2fc34d7e8e1b/nova/compute/resource_tracker.py#L890
> [2]
> https://github.com/openstack/nova/blob/8753c9a38667f984d385b4783c3c2fc34d7e8e1b/nova/scheduler/client/report.py#L1341
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

[Eric Fried]

> 1. Make everything perform the pivot on compute node start (which can be
>re-used by a CLI tool for the offline case)
> 2. Make everything default to non-nested inventory at first, and provide
>a way to migrate a compute node and its instances one at a time (in
>place) to roll through.

I agree that it sure would be nice to do ^ rather than requiring the
"slide puzzle" thing.

But how would this be accomplished, in light of the current "separation
of responsibilities" drawn at the virt driver interface, whereby the
virt driver isn't supposed to talk to placement directly, or know
anything about allocations?  Here's a first pass:

The virt driver, via the return value from update_provider_tree, tells
the resource tracker that "inventory of resource class A on provider B
have moved to provider C" for all applicable AxBxC.  E.g.

[ { 'from_resource_provider': ,
'moved_resources': [VGPU: 4],
'to_resource_provider': 
  },
  { 'from_resource_provider': ,
'moved_resources': [VGPU: 4],
'to_resource_provider': 
  },
  { 'from_resource_provider': ,
'moved_resources': [
SRIOV_NET_VF: 2,
NET_BANDWIDTH_EGRESS_KILOBITS_PER_SECOND: 1000,
NET_BANDWIDTH_INGRESS_KILOBITS_PER_SECOND: 1000,
],
'to_resource_provider': 
  }
]

As today, the resource tracker takes the updated provider tree and
invokes [1] the report client method update_from_provider_tree [2] to
flush the changes to placement.  But now update_from_provider_tree also
accepts the return value from update_provider_tree and, for each "move":

- Creates provider C (as described in the provider_tree) if it doesn't
already exist.
- Creates/updates provider C's inventory as described in the
provider_tree (without yet updating provider B's inventory).  This ought
to create the inventory of resource class A on provider C.
- Discovers allocations of rc A on rp B and POSTs to move them to rp C*.
- Updates provider B's inventory.

(*There's a hole here: if we're splitting a glommed-together inventory
across multiple new child providers, as the VGPUs in the example, we
don't know which allocations to put where.  The virt driver should know
which instances own which specific inventory units, and would be able to
report that info within the data structure.  That's getting kinda close
to the virt driver mucking with allocations, but maybe it fits well
enough into this model to be acceptable?)

Note that the return value from update_provider_tree is optional, and
only used when the virt driver is indicating a "move" of this ilk.  If
it's None/[] then the RT/update_from_provider_tree flow is the same as
it is today.

If we can do it this way, we don't need a migration tool.  In fact, we
don't even need to restrict provider tree "reshaping" to release
boundaries.  As long as the virt driver understands its own data model
migrations and reports them properly via update_provider_tree, it can
shuffle its tree around whenever it wants.

Thoughts?

-efried

[1]
https://github.com/openstack/nova/blob/8753c9a38667f984d385b4783c3c2fc34d7e8e1b/nova/compute/resource_tracker.py#L890
[2]
https://github.com/openstack/nova/blob/8753c9a38667f984d385b4783c3c2fc34d7e8e1b/nova/scheduler/client/report.py#L1341

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[OpenStack-Infra] Updating gerrit jenkins-job-builder-release group members

[Sorin Ionuț Sbârnea]

Hi

Can someone help updating the jenkins-job-builder-release group on gerrit to 
add two persons to it? Apparently there is no way to raise a CR to update group 
membership, or if it is I am unaware about it.
https://review.openstack.org/#/admin/groups/321,members 


Sorin Sbarnea sorin.sbar...@gmail.com
Darragh Bailey daragh.bai...@gmail.com

We are both already members of python-jenkins-release 
 but not this one and 
it would be very useful to be able to release much easier.

This change would bring the two gerrit groups in-sync, which makes sense to me.

Thanks
Sorin Sbarnea


signature.asc
Description: Message signed with OpenPGP
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] How to take over a project? (tap-as-a-service)

[Anil Rao]

Hi,

Sure. I’ll give it a try and get back to you guys.

Regards,
Anil

Sent from my iPhone

> On May 31, 2018, at 1:07 AM, Thierry Carrez  wrote:
> 
> Delivered-To: arao...@gmail.com
> Received: by 2002:a19:e21a:0:0:0:0:0 with SMTP id z26-v6csp5901938lfg; Thu, 31
> May 2018 01:07:30 -0700 (PDT)
> X-Google-Smtp-Source: 
> ADUXVKKQOqQkg/OK3B0quxkdCcVZjBMNtHZCom1G9aIxUS5xa+BuOM817LVG2bYdhSq7h3dXhX3a
> X-Received: by 2002:a0c:8b4b:: with SMTP id
> d11-v6mr1529025qvc.241.1527754050657; Thu, 31 May 2018 01:07:30 -0700 (PDT)
> ARC-Seal: i=1; a=rsa-sha256; t=1527754050; cv=none; d=google.com;
> s=arc-20160816;
> b=xO4W1y0PjqXxJDsVA80BzHl4UZllHdNeJEwEJlkCu7sCvO/fTQ/en0OfeUO5NxwMF1
> D7yOPvc3rGwfWtvg80lvrpDmweigmLhXwMbHH0J0wT3kjIkwbRwj6tH6nLYfeWgLvKDy
> HksPO8upTUWDrneEuZjZ9YDLSBTiPcxu4bo4nyFEXmxQ+wyuNXPykB+teatCGGQNdh2K
> 1A346RyPgHiWIlbYlI/9dd4wuLBpVUqcORtRDqJCQWwFFd6eY8RkE4/9NEFjpCkxNydy
> yzzV1sONBzi4Gsp/e1XRI9O8P3Z1MR52SQZevGLpov2T4DKr+e+raJCT9oFdXBZem1k9 7aYA==
> ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com;
> s=arc-20160816;
> h=content-transfer-encoding:content-language:in-reply-to:mime-version
> :user-agent:date:message-id:organization:from:references:cc:to
> :subject:arc-authentication-results;
> bh=/buaa2NmAYjG3+0fppj3fIUX8lWhDsIpVeBp7YZBZtw=;
> b=ZMeGxd7gYz1o/BmdrMmryG18F0yAT98ZhXtRFqDkhmeLtBnmCpaAqIeYVbCBJd40c7
> FA2sYWm/3SYrbAWlY9XRGxIPuhbXzMV9JOgwrEL3U2Kd1Y/Vp7eqC6Qlzp4z+jE0+M0C
> ePvp3eB4q8pMTgdvVt/vjHh8gCdxa7AYRH1MRh5dVrNP0gCrG23/SIYHnX9XDWE7I5pY
> bNszkCu/ymZGIIXHS2LDBt18vLZvJ0sSfIoeg0OYISGn0cV/W+x1jSB8ESEmxK2p2sOY
> bePq2U4tFZChMlLQSxBxh2ZHMGob+DB2IwfYrT89Nx4FNAJnNN/EqTK9fs9BCVARJkS0 yW4Q==
> ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain
> of thie...@openstack.org designates 173.203.187.79 as permitted sender)
> smtp.mailfrom=thie...@openstack.org
> Return-Path: 
> Received: from smtp79.iad3a.emailsrvr.com (smtp79.iad3a.emailsrvr.com.
> [173.203.187.79]) by mx.google.com with ESMTPS id
> r5-v6si360921qvc.200.2018.05.31.01.07.28 for 
> (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 31 May
> 2018 01:07:29 -0700 (PDT)
> Received-SPF: pass (google.com: domain of thie...@openstack.org designates
> 173.203.187.79 as permitted sender) client-ip=173.203.187.79;
> Authentication-Results: mx.google.com; spf=pass (google.com: domain of
> thie...@openstack.org designates 173.203.187.79 as permitted sender)
> smtp.mailfrom=thie...@openstack.org
> Received: from smtp34.relay.iad3a.emailsrvr.com (localhost [127.0.0.1]) by
> smtp34.relay.iad3a.emailsrvr.com (SMTP Server) with ESMTP id 6367924BC5; Thu,
> 31 May 2018 04:07:27 -0400 (EDT)
> X-Auth-ID: thierry.car...@openstack.org
> Received: by smtp34.relay.iad3a.emailsrvr.com (Authenticated sender:
> thierry.carrez-AT-openstack.org) with ESMTPSA id D22C824BC6; Thu, 31 May 2018
> 04:07:26 -0400 (EDT)
> X-Sender-Id: thierry.car...@openstack.org
> Received: from [192.168.0.113] (LFbn-1-13111-9.w109-209.abo.wanadoo.fr
> [109.209.34.9]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA) by 0.0.0.0:465
> (trex/5.7.12); Thu, 31 May 2018 04:07:27 -0400
> Subject: Re: [OpenStack-Infra] How to take over a project? (tap-as-a-service)
> To: openstack-infra@lists.openstack.org
> Cc: Takashi Yamamoto , Anil Rao ,
> Vinay Yadhav 
> References: 
> 
> From: Thierry Carrez 
> Organization: OpenStack
> Message-ID: <4f4046b9-ca01-a420-278a-abee04d36...@openstack.org>
> Date: Thu, 31 May 2018 10:07:26 +0200
> User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
> Thunderbird/52.8.0
> MIME-Version: 1.0
> In-Reply-To: 
> 
> Content-Type: text/plain; charset=utf-8; format=flowed
> Content-Language: en-US
> Content-Transfer-Encoding: 7bit
> 
> Takashi Yamamoto wrote:
>> i want to take over tap-as-a-service project.
>> originally the project had two leaders. but one of them (vinay) has not been
>> responding these days. another (anil) recently told us he might not work
>> on it anymore. this email has cc: to them.
>> i think gerrit is ok as i'm already a member of tap-as-a-service-core group.
>> however, for launchpad, i'm not sure who can change the permissions.
>> can the infra team help?
>> the LP project is this one: https://launchpad.net/tap-as-a-service
> 
> The infra team can't help much with that. The Launchpad project is owned by 
> Anil Rao (although Vinay Yadhav might be able to change that as the original 
> registrant), and the OpenStack Administrators was never added as a co-owner.
> 
> Anil: any chance you could update the maintainer ?
> Thanks in advance,
> 
> -- 
> Thierry Carrez (ttx)
> Takashi Yamamoto wrote:
>> i want to take over tap-as-a-service project.
>> originally the project had two leaders. but one of them (vinay) has not been
>> responding these days. another (anil) recently told us he might not work
>> on it anymore. this email has cc: to them.
>> i think gerrit is ok as i'm already a member of tap-as-a-service-core group.
>> however, for launchpad, i'm not sure who 

Re: [OpenStack-Infra] Updating gerrit jenkins-job-builder-release group members

[Thanh Ha]

On 23 May 2018 at 03:17, Sorin Ionuț Sbârnea 
wrote:

> Hi
>
> Can someone help updating the *jenkins-job-builder-release* group on
> gerrit to add two persons to it? Apparently there is no way to raise a CR
> to update group membership, or if it is I am unaware about it.
> https://review.openstack.org/#/admin/groups/321,members
>
> Sorin Sbarnea sorin.sbar...@gmail.com
> Darragh Bailey daragh.bai...@gmail.com
>
> We are both already members of python-jenkins-release
>  but not this
> one and it would be very useful to be able to release much easier.
>
> This change would bring the two gerrit groups in-sync, which makes sense
> to me.
>
> Thanks
> Sorin Sbarnea
>

I talked to Clark and we got this resolved. The both of you should have the
ability to push tags now. Remember to be careful about what tags you push
as it's policy that tags never get deleted.

Regards,
Thanh
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [openstack-dev] [tc] Organizational diversity tag

[Chris Dent]

On Tue, 29 May 2018, Samuel Cassiba wrote:


The moniker of 'low-activity' does give the very real, negative perception
that things are just barely hanging on. It conveys the subconscious,
officiated statement (!!!whether or not this was intended!!!) that nobody
in their right mind should consider using the subproject, let alone develop
on or against it, for fear that it wind up some poor end-user's support
nightmare.


Yeah. Which is really unfortunate because to some extent all
projects ought to be striving to be low activity in the sense of
mature, stable, (nearly) bug-free.

If our metrics are biased towards always committing then we are
encouraging unfettered growth which means we can never have any
sense of complete-ness or done-ness in any domains. It should be
okay to say a sub-domain of activity is done and move on to
improving the wider domain.


--
Chris Dent   ٩◔̯◔۶   https://anticdent.org/
freenode: cdent tw: @anticdent__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[Openstack-operators] OpenLab Cross-community Impact

[Melvin Hillsman]

Hi everyone,

I know we have sent out quite a bit of information over the past few days
with the OpenStack Summit and other updates recently. Additionally there
are plenty of meetings we all attend. I just want to take time to point to
something very significant in my opinion and again give big thanks to
Chris, Dims, Liusheng, Chenrui, Zhuli, Joe (gophercloud), and anyone else
contributing to OpenLab.

A member of the release team working on the testing infrastructure for
Kubernetes did a shoutout to the team for the following:

(AishSundar)
Shoutout to @dims and OpenStack team for quickly getting their 1.11
Conformance results piped to CI runs and contributing results to
Conformance dashboard !
https://k8s-testgrid.appspot.com/sig-release-1.11-all#Conformance%20-%20OpenStack=

Here is why this is significant and those working on this who I previously
mentioned should get recognition:

(hogepodge)
OpenStack and GCE are the first two clouds that will release block on
conformance testing failures. Thanks @dims for building out the test
pipeline and @mrhillsman for leading the OpenLab efforts that are reporting
back to the test grid. @RuiChen for his contributions to the testing
effort. Amazing work for the last six months.

In other words, if the external cloud provider ci conformance tests we do
in OpenLab are not passing, it will be one of the signals used for blocking
the release. OpenStack and GCE are the first two clouds to achieve this and
it is a significant accomplishment for the OpenLab team and the OpenStack
community overall regarding our relationship with the Kubernetes community.
Thanks again Chris, Dims, Joe, Liusheng, Chenrui, and Zhuli for the work
you have done and continue to do in this space.

Personally I hope we take a moment to really consider this milestone and
work to ensure OpenLab's continued success as we embark on working on other
integrations. We started OpenLab hoping we could make substantial impact
specifically for the ecosystem that builds on top of OpenStack and this is
evidence we can and should do more.

-- 
Kind regards,

Melvin Hillsman
mrhills...@gmail.com
mobile: (832) 264-2646
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

[openstack-dev] OpenLab Cross-community Impact

[Melvin Hillsman]

Hi everyone,

I know we have sent out quite a bit of information over the past few days
with the OpenStack Summit and other updates recently. Additionally there
are plenty of meetings we all attend. I just want to take time to point to
something very significant in my opinion and again give big thanks to
Chris, Dims, Liusheng, Chenrui, Zhuli, Joe (gophercloud), and anyone else
contributing to OpenLab.

A member of the release team working on the testing infrastructure for
Kubernetes did a shoutout to the team for the following:

(AishSundar)
Shoutout to @dims and OpenStack team for quickly getting their 1.11
Conformance results piped to CI runs and contributing results to
Conformance dashboard !
https://k8s-testgrid.appspot.com/sig-release-1.11-all#Conformance%20-%20OpenStack=

Here is why this is significant and those working on this who I previously
mentioned should get recognition:

(hogepodge)
OpenStack and GCE are the first two clouds that will release block on
conformance testing failures. Thanks @dims for building out the test
pipeline and @mrhillsman for leading the OpenLab efforts that are reporting
back to the test grid. @RuiChen for his contributions to the testing
effort. Amazing work for the last six months.

In other words, if the external cloud provider ci conformance tests we do
in OpenLab are not passing, it will be one of the signals used for blocking
the release. OpenStack and GCE are the first two clouds to achieve this and
it is a significant accomplishment for the OpenLab team and the OpenStack
community overall regarding our relationship with the Kubernetes community.
Thanks again Chris, Dims, Joe, Liusheng, Chenrui, and Zhuli for the work
you have done and continue to do in this space.

Personally I hope we take a moment to really consider this milestone and
work to ensure OpenLab's continued success as we embark on working on other
integrations. We started OpenLab hoping we could make substantial impact
specifically for the ecosystem that builds on top of OpenStack and this is
evidence we can and should do more.

-- 
Kind regards,

Melvin Hillsman
mrhills...@gmail.com
mobile: (832) 264-2646
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Ceph multiattach support

[Tom Barron]

On 31/05/18 10:00 +0800, fengyd wrote:

Hi,

I'm using Ceph for cinder backend.
Do you have any plan to support multiattach for Ceph backend?

Thanks

Yafeng


Yafeng,

Would you describe your use case for cinder multi-attach with ceph 
backend?


I'd like to understand better whether manila (file share 
infrastructure as a service) with  CephFS native or CephFS-NFS 
backends would (as Erik McCormick also suggested) meet your needs.


-- Tom Barron


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

[Chris Dent]

On Thu, 31 May 2018, Dan Smith wrote:


I kinda think we need to either:

1. Make everything perform the pivot on compute node start (which can be
  re-used by a CLI tool for the offline case)


This sounds effectively like: validate my inventory and allocations
at compute node start, correcting them as required (including the
kind of migration stuff related to nested). Is that right?

That's something I'd like to be the norm. It takes us back to a sort
of self-healing compute node.

Or am I missing something (forgive me, I've been on holiday).


I just think that forcing people to take down their data plane to work
around our own data model is kinda evil and something we should be
avoiding at this level of project maturity. What we're really saying is
"we know how to translate A into B, but we require you to move many GBs
of data over the network and take some downtime because it's easier for
*us* than making it seamless."


If we can do it, I agree that being not evil is good.

--
Chris Dent   ٩◔̯◔۶   https://anticdent.org/
freenode: cdent tw: @anticdent__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla][vote] Nominating Steve Noyes for kolla-cli core reviewer

[Mark Giles]

+1

On May 31, 2018 at 1:06:43 PM, Borne Mace (borne.m...@oracle.com) wrote:

Greetings all,

I would like to propose the addition of Steve Noyes to the kolla-cli 
core reviewer team. Consider this nomination as my personal +1.

Steve has a long history with the kolla-cli and should be considered its 
co-creator as probably half or more of the existing code was due to his 
efforts. He has now been working diligently since it was pushed 
upstream to improve the stability and testability of the cli and has the 
second most commits on the project.

The kolla core team consists of 19 people, and the kolla-cli team of 2, 
for a total of 21. Steve therefore requires a minimum of 11 votes (so 
just 10 more after my +1), with no veto -2 votes within a 7 day voting 
window to end on June 6th. Voting will be closed immediately on a veto 
or in the case of a unanimous vote.

As I'm not sure how active all of the 19 kolla cores are, your attention 
and timely vote is much appreciated.

Thanks!

-- Borne


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

[Dan Smith]

> My feeling is that we should not attempt to "migrate" any allocations
> or inventories between root or child providers within a compute node,
> period.

While I agree this is the simplest approach, it does put a lot of
responsibility on the operators to do work to sidestep this issue, which
might not even apply to them (and knowing if it does might be
difficult).

> The virt drivers should simply error out of update_provider_tree() if
> there are ANY existing VMs on the host AND the virt driver wishes to
> begin tracking resources with nested providers.
>
> The upgrade operation should look like this:
>
> 1) Upgrade placement
> 2) Upgrade nova-scheduler
> 3) start loop on compute nodes. for each compute node:
>  3a) disable nova-compute service on node (to take it out of scheduling)
>  3b) evacuate all existing VMs off of node

You mean s/evacuate/cold migrate/ of course... :)

>  3c) upgrade compute node (on restart, the compute node will see no
>  VMs running on the node and will construct the provider tree inside
>  update_provider_tree() with an appropriate set of child providers
>  and inventories on those child providers)
>  3d) enable nova-compute service on node
>
> Which is virtually identical to the "normal" upgrade process whenever
> there are significant changes to the compute node -- such as upgrading
> libvirt or the kernel.

Not necessarily. It's totally legit (and I expect quite common) to just
reboot the host to take kernel changes, bringing back all the instances
that were there when it resumes. The "normal" case of moving things
around slide-puzzle-style applies to live migration (which isn't an
option here). I think people that can take downtime for the instances
would rather not have to move things around for no reason if the
instance has to get shut off anyway.

> Nested resource tracking is another such significant change and should
> be dealt with in a similar way, IMHO.

This basically says that for anyone to move to rocky, they will have to
cold migrate every single instance in order to do that upgrade right? I
mean, anyone with two socket machines or SRIOV NICs would end up with at
least one level of nesting, correct? Forcing everyone to move everything
to do an upgrade seems like a non-starter to me.

We also need to consider the case where people would be FFU'ing past
rocky (i.e. never running rocky computes). We've previously said that
we'd provide a way to push any needed transitions with everything
offline to facilitate that case, so I think we need to implement that
method anyway.

I kinda think we need to either:

1. Make everything perform the pivot on compute node start (which can be
   re-used by a CLI tool for the offline case)
2. Make everything default to non-nested inventory at first, and provide
   a way to migrate a compute node and its instances one at a time (in
   place) to roll through.

We can also document "or do the cold-migration slide puzzle thing" as an
alternative for people that feel that's more reasonable.

I just think that forcing people to take down their data plane to work
around our own data model is kinda evil and something we should be
avoiding at this level of project maturity. What we're really saying is
"we know how to translate A into B, but we require you to move many GBs
of data over the network and take some downtime because it's easier for
*us* than making it seamless."

--Dan

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [kolla][vote] Nominating Steve Noyes for kolla-cli core reviewer

[Borne Mace]

Greetings all,

I would like to propose the addition of Steve Noyes to the kolla-cli 
core reviewer team.  Consider this nomination as my personal +1.


Steve has a long history with the kolla-cli and should be considered its 
co-creator as probably half or more of the existing code was due to his 
efforts.  He has now been working diligently since it was pushed 
upstream to improve the stability and testability of the cli and has the 
second most commits on the project.


The kolla core team consists of 19 people, and the kolla-cli team of 2, 
for a total of 21.  Steve therefore requires a minimum of 11 votes (so 
just 10 more after my +1), with no veto -2 votes within a 7 day voting 
window to end on June 6th.  Voting will be closed immediately on a veto 
or in the case of a unanimous vote.


As I'm not sure how active all of the 19 kolla cores are, your attention 
and timely vote is much appreciated.


Thanks!

-- Borne


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [tc][all] CD tangent - was: A culture change (nitpicking)

[Sean McGinnis]

On 05/31/2018 03:50 AM, Thierry Carrez wrote:
Right... There might be a reasonable middle ground between "every 
commit on master must be backward-compatible" and "rip out all 
testing" that allows us to routinely revert broken feature commits (as 
long as they don't cross a release boundary).


To be fair, I'm pretty sure that's already the case: we did revert 
feature commits on master in the past, therefore breaking backward 
compatibility if someone started to use that feature right away. It's 
the issue with implicit rules: everyone interprets them the way they 
want... So I think that could use some explicit clarification.


[ This tangent should probably gets its own thread to not disrupt the 
no-nitpicking discussion ]



Just one last one on this, then I'm hoping this tangent ends.

I think what Thierry said is exactly what Dims and I were saying. I'm 
not sure how that turned into
the idea of supporting committing broken code. The point (at least mine) 
was just that we should
not have the mindset that HEAD~4 committed something that we realize was 
not right, so we
should not have the mindset that "someone might have deployed that 
broken behavior so we
need to make sure we don't break them." HEAD should always be 
deployable, just not treated like

an official release that needs to be maintained.


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Help required to install devstack with GBP

[Sumit Naiksatam]

Hi, Sure we can help you. Could you please take a look at the neutron logs
and let me know what exception you are seeing? Also, please let me know
which branch you are trying to install.

Thanks,
~Sumit.

On Thu, May 31, 2018 at 1:52 AM, ., Alex Dominic Savio <
alex.will...@microfocus.com> wrote:

>
>
> Hi Experts,
>
>
>
> I have been trying to install devstack with gbp as per the instruction
> given in the  GitHub https://github.com/openstack/group-based-policy
>
> This I am running on Ubuntu 16.x as well as 14.x but both the attempts
> were not successful.
>
> It fails stating “neutron is not started”
>
>
>
> Can you please help me with this issue to get pass ?
>
>
>
>
>
> Thanks & Regards,
>
> *Alex Dominic Savio*
> Product Manager, ITOM-HCM
> Micro Focus
> Bagmane Tech Park
>
> Bangalore, India.
>
> (M)+91 9880634388
> alex.will...@microfocus.com
> --
>
> [image: cid:image003.jpg@01D3ED74.F4E1E2F0]
>
>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo][puppet] Hello all, puppet modules

[Tim Bell]

CERN use these puppet modules too and contributes any missing functionality we 
need upstream.

Tim

From: Alex Schultz 
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 

Date: Thursday, 31 May 2018 at 16:24
To: "OpenStack Development Mailing List (not for usage questions)" 

Subject: Re: [openstack-dev] [tripleo][puppet] Hello all, puppet modules



On Wed, May 30, 2018 at 3:18 PM, Remo Mattei mailto:r...@rm.ht>> 
wrote:
Hello all,
I have talked to several people about this and I would love to get this 
finalized once and for all. I have checked the OpenStack puppet modules which 
are mostly developed by the Red Hat team, as of right now, TripleO is using a 
combo of Ansible and puppet to deploy but in the next couple of releases, the 
plan is to move away from the puppet option.


So the OpenStack puppet modules are maintained by others other than Red Hat, 
however we have been a major contributor since TripleO has relied on them for 
some time.  That being said, as TripleO has migrated to containers built with 
Kolla, we've adapted our deployment mechanism to include Ansible and we really 
only use puppet for configuration generation.  Our goal for TripleO is to 
eventually be fully containerized which isn't something the puppet modules 
support today and I'm not sure is on the road map.


So consequently, what will be the plan of TripleO and the puppet modules?


As TripleO moves forward, we may continue to support deployments via puppet 
modules but the amount of testing that we'll be including upstream will mostly 
exercise external Ansible integrations (example, ceph-ansible, 
openshift-ansible, etc) and Kolla containers.  As of Queens, most of the 
services deployed via TripleO are deployed via containers and not on baremetal 
via puppet. We no longer support deploying OpenStack services on baremetal via 
the puppet modules and will likely be removing this support in the code in 
Stein.  The end goal will likely be moving away from puppet modules within 
TripleO if we can solve the backwards compatibility and configuration 
generation via other mechanism.  We will likely recommend leveraging external 
Ansible role calls rather than including puppet modules and using those to 
deploy services that are not inherently supported by TripleO.  I can't really 
give a time frame as we are still working out the details, but it is likely 
that over the next several cycles we'll see a reduction in the dependence of 
puppet in TripleO and an increase in leveraging available Ansible roles.


From the Puppet OpenStack standpoint, others are stepping up to continue to 
ensure the modules are available and I know I'll keep an eye on them for as 
long as TripleO leverages some of the functionality.  The Puppet OpenStack 
modules are very stable but I'm not sure without additional community folks 
stepping up that there will be support for newer functionality being added by 
the various OpenStack projects.  I'm sure others can chime in here on their 
usage/plans for the Puppet OpenStack modules.


Hope that helps.


Thanks,
-Alex


Thanks

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

[Jay Pipes]

On 05/31/2018 05:10 AM, Sylvain Bauza wrote:
After considering the whole approach, discussing with a couple of folks 
over IRC, here is what I feel the best approach for a seamless upgrade :
  - VGPU inventory will be kept on root RP (for the first type) in 
Queens so that a compute service upgrade won't impact the DB
  - during Queens, operators can run a DB online migration script (like 
the ones we currently have in 
https://github.com/openstack/nova/blob/c2f42b0/nova/cmd/manage.py#L375) 
that will create a new resource provider for the first type and move the 
inventory and allocations to it.
  - it's the responsibility of the virt driver code to check whether a 
child RP with its name being the first type name already exists to know 
whether to update the inventory against the root RP or the child RP.


Does it work for folks ?


No, sorry, that doesn't work for me. It seems overly complex and 
fragile, especially considering that VGPUs are not moveable anyway (no 
support for live migrating them). Same goes for CPU pinning, NUMA 
topologies, PCI passthrough devices, SR-IOV PF/VFs and all the other 
"must have" features that have been added to the virt driver over the 
last 5 years.


My feeling is that we should not attempt to "migrate" any allocations or 
inventories between root or child providers within a compute node, period.


The virt drivers should simply error out of update_provider_tree() if 
there are ANY existing VMs on the host AND the virt driver wishes to 
begin tracking resources with nested providers.


The upgrade operation should look like this:

1) Upgrade placement
2) Upgrade nova-scheduler
3) start loop on compute nodes. for each compute node:
 3a) disable nova-compute service on node (to take it out of scheduling)
 3b) evacuate all existing VMs off of node
 3c) upgrade compute node (on restart, the compute node will see no
 VMs running on the node and will construct the provider tree inside
 update_provider_tree() with an appropriate set of child providers
 and inventories on those child providers)
 3d) enable nova-compute service on node

Which is virtually identical to the "normal" upgrade process whenever 
there are significant changes to the compute node -- such as upgrading 
libvirt or the kernel. Nested resource tracking is another such 
significant change and should be dealt with in a similar way, IMHO.


Best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [release] Release countdown for week R-12, June 4-8

[Sean McGinnis]

Welcome back to our weekly countdown email.

Development Focus
-

The Rocky-2 milestone deadline is June 7th. Teams should be focused on
implementing priority features.

General Information
---

Membership freeze coincides with milestone 2 [0]. This means projects that have
not done a release yet must do so for the next two milestones to be included in
the Rocky release.

[0] https://releases.openstack.org/rocky/schedule.html#r-mf

The following libraries have not done a release yet in the rocky cycle:

automaton
blazar-nova
ceilometermiddleware
debtcollector
glance-store
heat-translator
kuryr
oslo.context
pycadf
requestsexceptions
stevedore
taskflow
python-aodhclient
python-barbicanclient
python-blazarclient
python-brick-cinderclient-ext
python-cinderclient
python-cloudkittyclient
python-congressclient
python-cyborgclient
python-designateclient
python-karborclient
python-magnumclient
python-masakariclient
python-muranoclient
python-octaviaclient
python-pankoclient
python-searchlightclient
python-senlinclient
python-solumclient
python-swiftclient
python-tricircleclient
python-vitrageclient
python-zaqarclient

For library-only projects, please be aware of the membership freeze mentioned
above. I believe all of these use the cycle-with-intermediary release model,
but especially for clients, it is good to get pending changes released
early/often in the cycle to make sure there is enough time to address issues
found by those that only use the released libraries.

Remember that there are client and non-client library freezes for the release
starting mid-July.

If there are any questions about preparing a release by the 7th, please come
talk to us in #openstack-releases.

**Note for projects that publish to PyPi**
There was a recent change with PyPi where they now enforce valid RST formatting
for package long descriptions. In most cases, the repo's README.rst gets pulled
in as this long description.

This means that there is now a need to ensure these README files are properly
formatted and do not have errors that will prevent the upload of a package.
This would fail after all of the other release automation was complete, so to
prevent this from happening we now have validation performed against repos when
new releases are proposed to the openstack/releases repo. If you see the
openstack-tox-validate job fail, this is a likely culprit.

See the note added to the project-team-guide for a recommendation on how to
protect against this before release time:

https://docs.openstack.org/project-team-guide/project-setup/python.html#running-the-style-checks

Unfortunately the error message isn't too helpful, so if you see a failure due
to this, the next step to help in identifying the cause may be to run doc8
against the README.rst file locally.

Upcoming Deadlines & Dates
--

Rocky-2 Milestone: June 7
Final non-client library release deadline: July 19
Final client library release deadline: July 26
Rocky-3 Milestone: July 26

-- 
Sean McGinnis (smcginnis)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

[Jay Pipes]

On 05/30/2018 07:06 AM, Balázs Gibizer wrote:
The nova-manage is another possible way similar to my idea #c) but there 
I imagined the logic in placement-manage instead of nova-manage.


Please note there is no placement-manage CLI tool.

Best,
-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

[Jay Pipes]

On 05/29/2018 09:12 AM, Sylvain Bauza wrote:
We could keep the old inventory in the root RP for the previous vGPU 
type already supported in Queens and just add other inventories for 
other vGPU types now supported. That looks possibly the simpliest option 
as the virt driver knows that.


What do you mean by "vGPU type"? Are you referring to the multiple GPU 
types stuff where specific virt drivers know how to handle different 
vGPU vendor types? Or are you referring to a "non-nested VGPU inventory 
on the compute node provider" versus a "VGPU inventory on multiple child 
providers, each representing a different physical GPU (or physical GPU 
group in the case of Xen)"?


-jay

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Questions about token scopes

[Lance Bragstad]

On 05/31/2018 12:09 AM, Ghanshyam Mann wrote:
> On Wed, May 30, 2018 at 11:53 PM, Lance Bragstad  wrote:
>>
>> On 05/30/2018 08:47 AM, Matt Riedemann wrote:
>>> I know the keystone team has been doing a lot of work on scoped tokens
>>> and Lance has been trying to roll that out to other projects (like nova).
>>>
>>> In Rocky the nova team is adding granular policy rules to the
>>> placement API [1] which is a good opportunity to set scope on those
>>> rules as well.
>>>
>>> For now, we've just said everything is system scope since resources in
>>> placement, for the most part, are managed by "the system". But we do
>>> have some resources in placement which have project/user information
>>> in them, so could theoretically also be scoped to a project, like GET
>>> /usages [2].
> Just adding that this is same for nova policy also. As you might know
> spec[1] try to make nova policy more granular but on hold because of
> default roles things. We will do policy rule split with more better
> defaults values like read-only for GET APIs.
>
> Along with that, like you mentioned about scope setting for placement
> policy rules, we need to do same for nova policy also. That can be
> done later or together with nova policy granular. spec.
>
> [1] https://review.openstack.org/#/c/547850/
>
>>> While going through this, I've been hammering Lance with questions but
>>> I had some more this morning and wanted to send them to the list to
>>> help spread the load and share the knowledge on working with scoped
>>> tokens in the other projects.
>> ++ good idea
>>
>>> So here goes with the random questions:
>>>
>>> * devstack has the admin project/user - does that by default get
>>> system scope tokens? I see the scope is part of the token create
>>> request [3] but it's optional, so is there a default value if not
>>> specified?
>> No, not necessarily. The keystone-manage bootstrap command is what
>> bootstraps new deployments with the admin user, an admin role, a project
>> to work in, etc. It also grants the newly created admin user the admin
>> role on a project and the system. This functionality was added in Queens
>> [0]. This should be backwards compatible and allow the admin user to get
>> tokens scoped to whatever they had authorization on previously. The only
>> thing they should notice is that they have another role assignment on
>> something called the "system". That being said, they can start
>> requesting system-scoped tokens from keystone. We have a document that
>> tries to explain the differences in scopes and what they mean [1].
> Another related question is, does scope setting will impact existing
> operator? I mean when policy rule start setting scope, that might
> break the existing operator as their current token (say project
> scoped) might not be able to authorize the policy modified with
> setting the system scope.
>
> In that case, how we are going to avoid the upgrade break. One way can
> be to soft enforcement scope things for a cycle with warning and then
> start enforcing that after one cycle (like we do for any policy rule
> change)? but not sure at this point.

Good question. This was the primary driver behind adding a new
configuration option to the oslo.policy library called `enforce_scope`
[0]. This let's operators turn off scope checking while they do a few
things.

They'll need to audit their users and give administrators of the
deployment access to the system via a system role assignment (as opposed
to the 'admin' role on some random project). They also need to ensure
those people understand the concept of system scope. They might also
send emails or notifications explaining the incoming changes and why
they're being done, et cetera. Ideally, this should buy operators time
to clean things up by reassessing their policy situation with the new
defaults and scope types before enforcing those constraints. If
`enforce_scope` is False, then a warning is logged during the
enforcement check saying something along the lines of "someone used a
token scoped to X to do something in Y".

[0]
https://docs.openstack.org/oslo.policy/latest/configuration/index.html#oslo_policy.enforce_scope

>
>> [0] https://review.openstack.org/#/c/530410/
>> [1] https://docs.openstack.org/keystone/latest/admin/identity-tokens.html
>>
>>> * Why don't the token create and show APIs return the scope?
>> Good question. In a way, they do. If you look at a response when you
>> authenticate for a token or validate a token, you should see an object
>> contained within the token reference for the purpose of scope. For
>> example, a project-scoped token will have a project object in the
>> response [2]. A domain-scoped token will have a domain object in the
>> response [3]. The same is true for system scoped tokens [4]. Unscoped
>> tokens do not have any of these objects present and do not contain a
>> service catalog [5]. While scope isn't explicitly denoted by an
>> attribute, it can be derived from the attributes of the 

Re: [openstack-dev] [tripleo][puppet] Hello all, puppet modules

[Alex Schultz]

On Wed, May 30, 2018 at 3:18 PM, Remo Mattei  wrote:

> Hello all,
> I have talked to several people about this and I would love to get this
> finalized once and for all. I have checked the OpenStack puppet modules
> which are mostly developed by the Red Hat team, as of right now, TripleO is
> using a combo of Ansible and puppet to deploy but in the next couple of
> releases, the plan is to move away from the puppet option.
>
>
So the OpenStack puppet modules are maintained by others other than Red
Hat, however we have been a major contributor since TripleO has relied on
them for some time.  That being said, as TripleO has migrated to containers
built with Kolla, we've adapted our deployment mechanism to include Ansible
and we really only use puppet for configuration generation.  Our goal for
TripleO is to eventually be fully containerized which isn't something the
puppet modules support today and I'm not sure is on the road map.


>
> So consequently, what will be the plan of TripleO and the puppet modules?
>


As TripleO moves forward, we may continue to support deployments via puppet
modules but the amount of testing that we'll be including upstream will
mostly exercise external Ansible integrations (example, ceph-ansible,
openshift-ansible, etc) and Kolla containers.  As of Queens, most of the
services deployed via TripleO are deployed via containers and not on
baremetal via puppet. We no longer support deploying OpenStack services on
baremetal via the puppet modules and will likely be removing this support
in the code in Stein.  The end goal will likely be moving away from puppet
modules within TripleO if we can solve the backwards compatibility and
configuration generation via other mechanism.  We will likely recommend
leveraging external Ansible role calls rather than including puppet modules
and using those to deploy services that are not inherently supported by
TripleO.  I can't really give a time frame as we are still working out the
details, but it is likely that over the next several cycles we'll see a
reduction in the dependence of puppet in TripleO and an increase in
leveraging available Ansible roles.

>From the Puppet OpenStack standpoint, others are stepping up to continue to
ensure the modules are available and I know I'll keep an eye on them for as
long as TripleO leverages some of the functionality.  The Puppet OpenStack
modules are very stable but I'm not sure without additional community folks
stepping up that there will be support for newer functionality being added
by the various OpenStack projects.  I'm sure others can chime in here on
their usage/plans for the Puppet OpenStack modules.

Hope that helps.

Thanks,
-Alex


>
> Thanks
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Questions about token scopes

[Lance Bragstad]

On 05/30/2018 03:37 PM, Matt Riedemann wrote:
> On 5/30/2018 9:53 AM, Lance Bragstad wrote:
>> While scope isn't explicitly denoted by an
>> attribute, it can be derived from the attributes of the token response.
>>
>
> Yeah, this was confusing to me, which is why I reported it as a bug in
> the API reference documentation:
>
> https://bugs.launchpad.net/keystone/+bug/1774229
>
>>> * It looks like python-openstackclient doesn't allow specifying a
>>> scope when issuing a token, is that going to be added?
>> Yes, I have a patch up for it [6]. I wanted to get this in during
>> Queens, but it missed the boat. I believe this and a new release of
>> oslo.context are the only bits left in order for services to have
>> everything they need to easily consume system-scoped tokens.
>> Keystonemiddleware should know how to handle system-scoped tokens in
>> front of each service [7]. The oslo.context library should be smart
>> enough to handle system scope set by keystonemiddleware if context is
>> built from environment variables [8]. Both keystoneauth [9] and
>> python-keystoneclient [10] should have what they need to generate
>> system-scoped tokens.
>>
>> That should be enough to allow the service to pass a request environment
>> to oslo.context and use the context object to reason about the scope of
>> the request. As opposed to trying to understand different token scope
>> responses from keystone. We attempted to abstract that away in to the
>> context object.
>>
>> [6]https://review.openstack.org/#/c/524416/
>> [7]https://review.openstack.org/#/c/564072/
>> [8]https://review.openstack.org/#/c/530509/
>> [9]https://review.openstack.org/#/c/529665/
>> [10]https://review.openstack.org/#/c/524415/
>
> I think your reply in IRC was more what I was looking for:
>
> lbragstad    mriedem: if you install
> https://review.openstack.org/#/c/524416/5 locally with devstack and
> setup a clouds.yaml, ``openstack token issue --os-cloud
> devstack-system-admin`` should work    15:39
> lbragstad    http://paste.openstack.org/raw/722357/    15:39
>
> So users with the system role will need to create a token using that
> role to get the system-scoped token, as far as I understand. There is
> no --scope option on the 'openstack token issue' CLI.
>
>> Uhm, if I understand your question, it depends on how you define the
>> scope types for those APIs. If you set them to system-scope, then an
>> operator will need to use a system-scoped token in order to access those
>> APIs iff the placement configuration file contains placement.conf
>> [oslo.policy] enforce_scope = True. Otherwise, setting that option to
>> false will log a warning to operators saying that someone is accessing a
>> system-scoped API with a project-scoped token (e.g. education needs to
>> happen).
>>
>
> All placement APIs will be system scoped for now, so yeah I guess if
> operators enable scope enforcement they'll just have to learn how to
> deal with system-scope enforced APIs.
>
> Here is another random question:
>
> Do we have any CI jobs running devstack/tempest with scope enforcement
> enabled to see what blows up?
>

Yes and no. There is an effort to include CI testing of some sort,
building on devstack, tempest, and patrole [0]. We actually have a
specification that details how we plan to start testing these changes
with an experimental job, once we get the correct RBAC behavior that we
want [1].

If anyone has cycles or is interested in test coverage for this type of
stuff, please don't hesitate to reach out. We could really use some help
in this area and we have a pretty good plan in place.

[0] https://github.com/openstack/patrole
[1] https://review.openstack.org/#/c/464678/





signature.asc
Description: OpenPGP digital signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

[Naichuan Sun]

I can do it on xenserver side, although keep old inv in compute node rp looks 
weird to me(it just work for one case: upgrade)...

-Original Message-
From: Eric Fried [mailto:openst...@fried.cc] 
Sent: Thursday, May 31, 2018 9:54 PM
To: OpenStack Development Mailing List (not for usage questions) 

Subject: Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested 
Resource Providers

This seems reasonable, but...

On 05/31/2018 04:34 AM, Balázs Gibizer wrote:
> 
> 
> On Thu, May 31, 2018 at 11:10 AM, Sylvain Bauza  wrote:
>>>
>>
>> After considering the whole approach, discussing with a couple of 
>> folks over IRC, here is what I feel the best approach for a seamless 
>> upgrade :
>>  - VGPU inventory will be kept on root RP (for the first type) in 
>> Queens so that a compute service upgrade won't impact the DB
>>  - during Queens, operators can run a DB online migration script 
>> (like
-^^
Did you mean Rocky?

>> the ones we currently have in
>> https://github.com/openstack/nova/blob/c2f42b0/nova/cmd/manage.py#L37
>> 5) that will create a new resource provider for the first type and 
>> move the inventory and allocations to it.
>>  - it's the responsibility of the virt driver code to check whether a 
>> child RP with its name being the first type name already exists to 
>> know whether to update the inventory against the root RP or the child RP.
>>
>> Does it work for folks ?
> 
> +1 works for me
> gibi
> 
>> PS : we already have the plumbing in place in nova-manage and we're 
>> still managing full Nova resources. I know we plan to move Placement 
>> out of the nova tree, but for the Rocky timeframe, I feel we can 
>> consider nova-manage as the best and quickiest approach for the data 
>> upgrade.
>>
>> -Sylvain
>>
>>
> 
> 
> __
>  OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: 
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

[Eric Fried]

This seems reasonable, but...

On 05/31/2018 04:34 AM, Balázs Gibizer wrote:
> 
> 
> On Thu, May 31, 2018 at 11:10 AM, Sylvain Bauza  wrote:
>>>
>>
>> After considering the whole approach, discussing with a couple of
>> folks over IRC, here is what I feel the best approach for a seamless
>> upgrade :
>>  - VGPU inventory will be kept on root RP (for the first type) in
>> Queens so that a compute service upgrade won't impact the DB
>>  - during Queens, operators can run a DB online migration script (like
-^^
Did you mean Rocky?

>> the ones we currently have in
>> https://github.com/openstack/nova/blob/c2f42b0/nova/cmd/manage.py#L375) that
>> will create a new resource provider for the first type and move the
>> inventory and allocations to it.
>>  - it's the responsibility of the virt driver code to check whether a
>> child RP with its name being the first type name already exists to
>> know whether to update the inventory against the root RP or the child RP.
>>
>> Does it work for folks ?
> 
> +1 works for me
> gibi
> 
>> PS : we already have the plumbing in place in nova-manage and we're
>> still managing full Nova resources. I know we plan to move Placement
>> out of the nova tree, but for the Rocky timeframe, I feel we can
>> consider nova-manage as the best and quickiest approach for the data
>> upgrade.
>>
>> -Sylvain
>>
>>
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Cyborg] [Nova] Cyborg traits

[Eric Fried]

Yup.  I'm sure reviewers will bikeshed the names, but the review is the
appropriate place for that to happen.

A couple of test changes will also be required.  You can have a look at
[1] as an example to follow.

-efried

[1] https://review.openstack.org/#/c/511180/

On 05/31/2018 01:02 AM, Nadathur, Sundar wrote:
> On 5/30/2018 1:18 PM, Eric Fried wrote:
>> This all sounds fully reasonable to me.  One thing, though...
>>
    * There is a resource class per device category e.g.
  CUSTOM_ACCELERATOR_GPU, CUSTOM_ACCELERATOR_FPGA.
>> Let's propose standard resource classes for these ASAP.
>>
>> https://github.com/openstack/nova/blob/d741f624c81baf89fc8b6b94a2bc20eb5355a818/nova/rc_fields.py
>>
>>
>> -efried
> Makes sense, Eric. The obvious names would be ACCELERATOR_GPU and
> ACCELERATOR_FPGA. Do we just submit a patch to rc_fields.py?
> 
> Thanks,
> Sundar
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][forum] TC Retrospective for Queens/Rocky

[Doug Hellmann]

Excerpts from Thierry Carrez's message of 2018-05-31 10:31:12 +0200:
> Doug Hellmann wrote:
> > [...]
> > I'm missing details and/or whole topics. Please review the list and
> > make any updates you think are necessary.
> 
> One thing that was raised at the Board+TC+UC meeting is the idea of 
> creating a group to help with wording and communication of "help most 
> needed" list items, so that they contain more business-value explanation 
> and get more regular status updates at the Board...
> 
> If I remember correctly, Chris Price, dims and you volunteered :) I'm 
> happy to help too.
> 
> Is that something you would like to track on this document as well ?
> 

Yes, that would be a good thing to add. I also still plan to send
a summary of the meeting from my perspective.

Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] [barbican] [tc] key store in base services

[Jeremy Stanley]

On 2018-05-31 10:33:51 +0200 (+0200), Thierry Carrez wrote:
> Ade Lee wrote:
> > [...]
> > So it seems that the two blockers above have been resolved. So is it
> > time to ad a castellan compatible secret store to the base services?
> 
> It's definitely time to start a discussion about it, at least :)
> 
> Would you be interested in starting a ML thread about it ? If not, that's
> probably something I can do :)

That was, in fact, the entire reason I started this subthread,
changed the subject and added the [tc] tag. ;)

http://lists.openstack.org/pipermail/openstack-dev/2018-May/130567.html

I figured I'd let it run through the summit to garner feedback
before proposing the corresponding Gerrit change.
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack] Struggling to get the s3 api interface to work with swift.

[Shyam Prasad N]

Hi Yuxin,

Thank you for sharing your configs.
So I've managed to get past the Signature not matching error.

Now the error is different. InvalidbucketName:

eightkpc@objectstore1:~/s3curl$ ./s3curl.pl --debug --id=testerks
--createBucket --
http://20.20.20.229:8080/v1/AUTH_dc86f7d8787b46158268bd77098b6578/testBucket
s3curl: Found the url: host=20.20.20.229; port=8080;
uri=/v1/AUTH_dc86f7d8787b46158268bd77098b6578/testBucket; query=;
s3curl: cname endpoint signing case
s3curl: StringToSign='PUT\n\n\nThu, 31 May 2018 12:02:57 +\n/
20.20.20.229/v1/AUTH_dc86f7d8787b46158268bd77098b6578/testBucket'
s3curl: exec curl -v -H 'Date: Thu, 31 May 2018 12:02:57 +' -H
'Authorization: AWS
76498e1413284b9d961d452db608dff4:jj/kaAEuX/vK+WUTvZyDQUUEGV0=' -L -H
'content-type: ' --data-binary  -X PUT
http://20.20.20.229:8080/v1/AUTH_dc86f7d8787b46158268bd77098b6578/testBucket
*   Trying 20.20.20.229...
* TCP_NODELAY set
* Connected to 20.20.20.229 (20.20.20.229) port 8080 (#0)
> PUT /v1/AUTH_dc86f7d8787b46158268bd77098b6578/testBucket HTTP/1.1
> Host: 20.20.20.229:8080
> User-Agent: curl/7.58.0
> Accept: */*
> Date: Thu, 31 May 2018 12:02:57 +
> Authorization: AWS
76498e1413284b9d961d452db608dff4:jj/kaAEuX/vK+WUTvZyDQUUEGV0=
> Content-Length: 0
>
< HTTP/1.1 400 Bad Request
< x-amz-id-2: tx18266052d5044eb2a3bc7-005b0fe471
< x-amz-request-id: tx18266052d5044eb2a3bc7-005b0fe471
< Content-Type: application/xml
< X-Trans-Id: tx18266052d5044eb2a3bc7-005b0fe471
< X-Openstack-Request-Id: tx18266052d5044eb2a3bc7-005b0fe471
< Date: Thu, 31 May 2018 12:02:57 GMT
< Transfer-Encoding: chunked
* HTTP error before end of send, stop sending
<

* Closing connection 0
InvalidBucketNameThe specified bucket is not
valid.tx18266052d5044eb2a3bc7-005b0fe471v1eightkpc@objectstore1:~/s3curl$


My specified endpoint is
http://20.20.20.229:8080/v1/AUTH_dc86f7d8787b46158268bd77098b6578
What am I doing wrong?

Regards,
Shyam

On Wed, May 30, 2018 at 7:32 PM, Yuxin Wang 
wrote:

> Hi Shyam,
>
> No problem. The output of the commands is attached.
>
> And my test cluster is on Swift v2.15.1 with Swift3 v1.12
>
> Also, here is the common process when I'm creating an S3 credential and
> using in s3curl. Hope it helps.
>
> 1. Create a user and a project, and assign a proper role.
>
> openstack project create testproject
> openstack user create testuser --password 123
> openstack role add --project testproject --user testuser _member_
>
> 2. Check accessibility to swift
>
> create a test-openrc file with above info
> source test-openrc
> swift list
>
> 3.Create a credential
>
> openstack credential create --type ec2 --project testproject testuser
>  '{"access": "testaccess", "secret": "testsecret"}'
>
> 4. Use it in s3curl
>
> add the endpoint url to `my @endpoints` in s3curl.pl
> add the credential to .s3curl config file
>
> do `s3curl.pl -i cred_name --debug -- http://endpoint -X GET`
>
>
>
>
> 在 2018年5月25日，18:17，Shyam Prasad N  写道：
>
> Hi Yuxin,
>
> If you don't mind, can you share the output of the following commands in
> your running swift3 setup?
>
> openstack credential list
> openstack ec2 credentials list
> cat /etc/swift/proxy-server.conf
>
> Also, what are the access keys and secret keys that you use?
> I want to make sure that I'm not missing anything in configuration.
>
> Regards,
> Shyam
>
> On Fri, May 25, 2018 at 3:05 PM, Shyam Prasad N 
> wrote:
>
>> Tried that. Unfortunately same error.
>> Is there anything I can do to troubleshoot this?
>>
>> On Fri, May 25, 2018 at 2:56 PM, Yuxin Wang 
>> wrote:
>>
>>> They can be any strings.
>>>
>>> Replace them with whatever you want.
>>>
>>> - Yuxin
>>>
>>> 在 2018年5月25日，14:57，Shyam Prasad N  写道：
>>>
>>> Thanks. I'll try this.
>>> But what values do I use in place of ak and sk? I want to use some
>>> command to get those values, right?
>>>
>>> On Fri, May 25, 2018 at 9:52 AM, Yuxin Wang 
>>>  wrote:
>>>
 I created ec2 credentials using command `openstack credential create`.

 i.e.

 openstack credential create --type ec2 --project proj user '{"access":
 "ak", "secret": "sk”}'


 It seems the two credentials are not the same thing.

 Ref:

 https://www.ibm.com/support/knowledgecenter/en/STXKQY_4.1.1/
 com.ibm.spectrum.scale.v4r11.adv.doc/bl1adv_ConfigureOpensta
 ckEC2credentials.htm
 

 在 2018年5月25日，10:32，Shyam Prasad N  写道：

 Yes, I did.
 I don't think this is s3curl related issue, because I tried with python
 AWS SDK, and got the same error.

 On Fri, May 25, 2018, 07:42 Yuxin Wang 
 wrote:

> Did you add 127.0.0.1 

[openstack-dev] [sahara] Canceling today's meeting

[Telles Nobrega]

Hi saharans and interested folks, we won't be having meeting today since at
least half of our team is on PTO today.

We will be back next Thursday.

See you all.
-- 

TELLES NOBREGA

SOFTWARE ENGINEER

Red Hat Brasil  

Av. Brg. Faria Lima, 3900 - 8º andar - Itaim Bibi, São Paulo

tenob...@redhat.com

TRIED. TESTED. TRUSTED. 
 Red Hat é reconhecida entre as melhores empresas para trabalhar no Brasil
pelo Great Place to Work.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

[Balázs Gibizer]

On Thu, May 31, 2018 at 11:10 AM, Sylvain Bauza  
wrote:




After considering the whole approach, discussing with a couple of 
folks over IRC, here is what I feel the best approach for a seamless 
upgrade :
 - VGPU inventory will be kept on root RP (for the first type) in 
Queens so that a compute service upgrade won't impact the DB
 - during Queens, operators can run a DB online migration script 
(like the ones we currently have in 
https://github.com/openstack/nova/blob/c2f42b0/nova/cmd/manage.py#L375) 
that will create a new resource provider for the first type and move 
the inventory and allocations to it.
 - it's the responsibility of the virt driver code to check whether a 
child RP with its name being the first type name already exists to 
know whether to update the inventory against the root RP or the child 
RP.


Does it work for folks ?


+1 works for me
gibi

PS : we already have the plumbing in place in nova-manage and we're 
still managing full Nova resources. I know we plan to move Placement 
out of the nova tree, but for the Rocky timeframe, I feel we can 
consider nova-manage as the best and quickiest approach for the data 
upgrade.


-Sylvain





__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo] Summit onboarding and project update slides

[ChangBo Guo]

Thanks Ben

2018-05-31 6:48 GMT+08:00 Ben Nemec :

> As promised in the sessions, here are the slides that were presented:
>
> https://www.slideshare.net/BenNemec1/oslo-vancouver-onboarding
>
> https://www.slideshare.net/BenNemec1/oslo-vancouver-project-update
>
> The font in the onboarding one got a little funny in the conversion, so if
> you want to see the original that is more readable let me know and I can
> send it to you.
>
> -Ben
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
ChangBo Guo(gcb)
Community Director @EasyStack
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [nova] [placement] Upgrade concerns with nested Resource Providers

[Sylvain Bauza]

On Wed, May 30, 2018 at 1:06 PM, Balázs Gibizer  wrote:

>
>
> On Tue, May 29, 2018 at 3:12 PM, Sylvain Bauza  wrote:
>
>>
>>
>> On Tue, May 29, 2018 at 2:21 PM, Balázs Gibizer <
>> balazs.gibi...@ericsson.com> wrote:
>>
>>>
>>>
>>> On Tue, May 29, 2018 at 1:47 PM, Sylvain Bauza 
>>> wrote:
>>>


 Le mar. 29 mai 2018 à 11:02, Balázs Gibizer <
 balazs.gibi...@ericsson.com> a écrit :

>
>
> On Tue, May 29, 2018 at 9:38 AM, Sylvain Bauza 
> wrote:
> >
> >
> > On Tue, May 29, 2018 at 3:08 AM, TETSURO NAKAMURA
> >  wrote
> >
> >> > In that situation, say for example with VGPU inventories, that
> >> would mean
> >> > that the compute node would stop reporting inventories for its
> >> root RP, but
> >> > would rather report inventories for at least one single child RP.
> >> > In that model, do we reconcile the allocations that were already
> >> made
> >> > against the "root RP" inventory ?
> >>
> >> It would be nice to see Eric and Jay comment on this,
> >> but if I'm not mistaken, when the virt driver stops reporting
> >> inventories for its root RP, placement would try to delete that
> >> inventory inside and raise InventoryInUse exception if any
> >> allocations still exist on that resource.
> >>
> >> ```
> >> update_from_provider_tree() (nova/compute/resource_tracker.py)
> >>   + _set_inventory_for_provider() (nova/scheduler/client/report.py)
> >>   + put() - PUT /resource_providers//inventories with
> >> new inventories (scheduler/client/report.py)
> >>   + set_inventories() (placement/handler/inventory.py)
> >>   + _set_inventory()
> >> (placement/objects/resource_proveider.py)
> >>   + _delete_inventory_from_provider()
> >> (placement/objects/resource_proveider.py)
> >>   -> raise exception.InventoryInUse
> >> ```
> >>
> >> So we need some trick something like deleting VGPU allocations
> >> before upgrading and set the allocation again for the created new
> >> child after upgrading?
> >>
> >
> > I wonder if we should keep the existing inventory in the root RP, and
> > somehow just reserve the left resources (so Placement wouldn't pass
> > that root RP for queries, but would still have allocations). But
> > then, where and how to do this ? By the resource tracker ?
> >
>
> AFAIK it is the virt driver that decides to model the VGU resource at a
> different place in the RP tree so I think it is the responsibility of
> the same virt driver to move any existing allocation from the old place
> to the new place during this change.
>
> Cheers,
> gibi
>

 Why not instead not move the allocation but rather have the virt driver
 updating the root RP by modifying the reserved value to the total size?

 That way, the virt driver wouldn't need to ask for an allocation but
 rather continue to provide inventories...

 Thoughts?

>>>
>>> Keeping the old allocaton at the old RP and adding a similar sized
>>> reservation in the new RP feels hackis as those are not really reserved
>>> GPUs but used GPUs just from the old RP. If somebody sums up the total
>>> reported GPUs in this setup via the placement API then she will get more
>>> GPUs in total that what is physically visible for the hypervisor as the
>>> GPUs part of the old allocation reported twice in two different total
>>> value. Could we just report less GPU inventories to the new RP until the
>>> old RP has GPU allocations?
>>>
>>>
>>
>> We could keep the old inventory in the root RP for the previous vGPU type
>> already supported in Queens and just add other inventories for other vGPU
>> types now supported. That looks possibly the simpliest option as the virt
>> driver knows that.
>>
>
> That works for me. Can we somehow deprecate the previous, already
> supported vGPU types to eventually get rid of the splitted inventory?
>
>
>>
>> Some alternatives from my jetlagged brain:
>>>
>>> a) Implement a move inventory/allocation API in placement. Given a
>>> resource class and a source RP uuid and a destination RP uuid placement
>>> moves the inventory and allocations of that resource class from the source
>>> RP to the destination RP. Then the virt drive can call this API to move the
>>> allocation. This has an impact on the fast forward upgrade as it needs
>>> running virt driver to do the allocation move.
>>>
>>>
>> Instead of having the virt driver doing that (TBH, I don't like that
>> given both Xen and libvirt drivers have the same problem), we could write a
>> nova-manage upgrade call for that that would call the Placement API, sure.
>>
>
> The nova-manage is another possible way similar to my idea #c) but there I
> imagined the logic in placement-manage instead of nova-manage.
>
>
>> b) For this I assume that live migrating an 

[openstack-dev] Help required to install devstack with GBP

[., Alex Dominic Savio]

Hi Experts,

I have been trying to install devstack with gbp as per the instruction given in 
the  GitHub https://github.com/openstack/group-based-policy
This I am running on Ubuntu 16.x as well as 14.x but both the attempts were not 
successful.
It fails stating "neutron is not started"

Can you please help me with this issue to get pass ?


Thanks & Regards,
Alex Dominic Savio
Product Manager, ITOM-HCM
Micro Focus
Bagmane Tech Park
Bangalore, India.
(M)+91 9880634388
alex.will...@microfocus.com

[cid:image003.jpg@01D3ED74.F4E1E2F0]

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][all] A culture change (nitpicking)

[Thierry Carrez]

Davanum Srinivas wrote:

"master should be always deployable and fully backward compatible and
so we cant let anything in anytime that could possibly regress anyone"

Should we change that attitude too? Anyone agree? disagree?

Thanks,
Dims


I'll definitely jump at this one.

I've always thought (and shared on the ML several times now) that our
implied
but not explicit support for CD from any random commit was a bad thing.

While I think it's good to support the idea that master is always
deployable, I
do not think it is a good mindset to think that every commit is a
"release" and
therefore should be supported until the end of time. We have a coordinated
release for a reason, and I think design decisions and fixes should be
based on
the assumption that a release is a release and the point at which we
need to be
cognizant and caring about keeping backward compatibility. Doing that for
every single commit is not ideal for the overall health of the product, IMO.



It's more than just a CD guarantee, while from a quick glance it would seem like
that's the only value it goes much deeper than that. Ensuring that every commit
works, is deployable, and maintains backwards compatibility is what enables us
to have such a high quality end result at release time. Quite frankly it's
looking at every commit as always being a working unit that enables us to manage
a project this size at this velocity. Even if people assume no one is actually
CDing the projects(which we shouldn't), it's a flawed assumption to think that
everyone is running strictly the same code as what's in the release tarballs. I
can't think of any production cloud out there that doesn't carry patches to fix
things encountered in the real world. Or look at stable maint we regularly need
to backport fixes to fix bugs found after release. If we can't rely on these to
always work this makes our life much more difficult, both as upstream
maintainers but also as downstream consumers of OpenStack.

The other aspect to look at here is just the review mindset, supporting every
every commit is useable puts reviewers in the mindset to consider things like
backwards compatibility and deployability when looking at proposed changes. If
we stop looking for these potential issues, we t will also cause many more bugs
to be in our released code. To simply discount this as only a release concern
and punt this kind of scrutiny until it's time to release is not only going to
make release time much more stressful. Also, our testing is built to try and
ensure every commit works **before** we merge it. If we decided to take this
stance as a community then we should really just rip out all the testing,
because that's what it's there to verify and help us make sure we don't land a
change that doesn't work. If we don't actually care about that making sure every
commit is deployable we are wasting quite a lot of resources on it.


"rip out all testing" is probably taking it too far Matt.

  Instead of perfection when merging, we should look for iteration and
reverts. That's what i would like to see. I am not asking for a
"Commit-Then-Review" like the ASF. I want us to be just be practical
and have some leeway to iterate / update / experiment instead of
absolute perfection from all angles. We should move the needle at
least a bit away from it.


Right... There might be a reasonable middle ground between "every commit 
on master must be backward-compatible" and "rip out all testing" that 
allows us to routinely revert broken feature commits (as long as they 
don't cross a release boundary).


To be fair, I'm pretty sure that's already the case: we did revert 
feature commits on master in the past, therefore breaking backward 
compatibility if someone started to use that feature right away. It's 
the issue with implicit rules: everyone interprets them the way they 
want... So I think that could use some explicit clarification.


[ This tangent should probably gets its own thread to not disrupt the 
no-nitpicking discussion ]


--
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] [barbican] [tc] key store in base services

[Thierry Carrez]

Ade Lee wrote:

[...]
So it seems that the two blockers above have been resolved. So is it
time to ad a castellan compatible secret store to the base services?


It's definitely time to start a discussion about it, at least :)

Would you be interested in starting a ML thread about it ? If not, 
that's probably something I can do :)


--
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tc][forum] TC Retrospective for Queens/Rocky

[Thierry Carrez]

Doug Hellmann wrote:

[...]
I'm missing details and/or whole topics. Please review the list and
make any updates you think are necessary.


One thing that was raised at the Board+TC+UC meeting is the idea of 
creating a group to help with wording and communication of "help most 
needed" list items, so that they contain more business-value explanation 
and get more regular status updates at the Board...


If I remember correctly, Chris Price, dims and you volunteered :) I'm 
happy to help too.


Is that something you would like to track on this document as well ?

--
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [OpenStack-Infra] How to take over a project? (tap-as-a-service)

[Thierry Carrez]

Takashi Yamamoto wrote:

i want to take over tap-as-a-service project.

originally the project had two leaders. but one of them (vinay) has not been
responding these days. another (anil) recently told us he might not work
on it anymore. this email has cc: to them.

i think gerrit is ok as i'm already a member of tap-as-a-service-core group.

however, for launchpad, i'm not sure who can change the permissions.
can the infra team help?
the LP project is this one: https://launchpad.net/tap-as-a-service


The infra team can't help much with that. The Launchpad project is owned 
by Anil Rao (although Vinay Yadhav might be able to change that as the 
original registrant), and the OpenStack Administrators was never added 
as a co-owner.


Anil: any chance you could update the maintainer ?
Thanks in advance,

--
Thierry Carrez (ttx)

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [openstack-dev] Questions about token scopes

[Ghanshyam Mann]

On Thu, May 31, 2018 at 2:09 PM, Ghanshyam Mann  wrote:
> On Wed, May 30, 2018 at 11:53 PM, Lance Bragstad  wrote:
>>
>>
>> On 05/30/2018 08:47 AM, Matt Riedemann wrote:
>>> I know the keystone team has been doing a lot of work on scoped tokens
>>> and Lance has been trying to roll that out to other projects (like nova).
>>>
>>> In Rocky the nova team is adding granular policy rules to the
>>> placement API [1] which is a good opportunity to set scope on those
>>> rules as well.
>>>
>>> For now, we've just said everything is system scope since resources in
>>> placement, for the most part, are managed by "the system". But we do
>>> have some resources in placement which have project/user information
>>> in them, so could theoretically also be scoped to a project, like GET
>>> /usages [2].
>
> Just adding that this is same for nova policy also. As you might know
> spec[1] try to make nova policy more granular but on hold because of
> default roles things. We will do policy rule split with more better
> defaults values like read-only for GET APIs.
>
> Along with that, like you mentioned about scope setting for placement
> policy rules, we need to do same for nova policy also. That can be
> done later or together with nova policy granular. spec.
>
> [1] https://review.openstack.org/#/c/547850/
>
>>>
>>> While going through this, I've been hammering Lance with questions but
>>> I had some more this morning and wanted to send them to the list to
>>> help spread the load and share the knowledge on working with scoped
>>> tokens in the other projects.
>>
>> ++ good idea
>>
>>>
>>> So here goes with the random questions:
>>>
>>> * devstack has the admin project/user - does that by default get
>>> system scope tokens? I see the scope is part of the token create
>>> request [3] but it's optional, so is there a default value if not
>>> specified?
>>
>> No, not necessarily. The keystone-manage bootstrap command is what
>> bootstraps new deployments with the admin user, an admin role, a project
>> to work in, etc. It also grants the newly created admin user the admin
>> role on a project and the system. This functionality was added in Queens
>> [0]. This should be backwards compatible and allow the admin user to get
>> tokens scoped to whatever they had authorization on previously. The only
>> thing they should notice is that they have another role assignment on
>> something called the "system". That being said, they can start
>> requesting system-scoped tokens from keystone. We have a document that
>> tries to explain the differences in scopes and what they mean [1].
>
> Another related question is, does scope setting will impact existing
> operator? I mean when policy rule start setting scope, that might
> break the existing operator as their current token (say project
> scoped) might not be able to authorize the policy modified with
> setting the system scope.
>
> In that case, how we are going to avoid the upgrade break. One way can
> be to soft enforcement scope things for a cycle with warning and then
> start enforcing that after one cycle (like we do for any policy rule
> change)? but not sure at this point.

^^ this is basically the same question i got while this review
-https://review.openstack.org/#/c/570621/1/nova/api/openstack/placement/policies/aggregate.py
Checking how scope_type will affect existing operator(token) so that
we can evaluate the upgrade impact.

>
>>
>> [0] https://review.openstack.org/#/c/530410/
>> [1] https://docs.openstack.org/keystone/latest/admin/identity-tokens.html
>>
>>>
>>> * Why don't the token create and show APIs return the scope?
>>
>> Good question. In a way, they do. If you look at a response when you
>> authenticate for a token or validate a token, you should see an object
>> contained within the token reference for the purpose of scope. For
>> example, a project-scoped token will have a project object in the
>> response [2]. A domain-scoped token will have a domain object in the
>> response [3]. The same is true for system scoped tokens [4]. Unscoped
>> tokens do not have any of these objects present and do not contain a
>> service catalog [5]. While scope isn't explicitly denoted by an
>> attribute, it can be derived from the attributes of the token response.
>>
>> [2] http://paste.openstack.org/raw/722349/
>> [3] http://paste.openstack.org/raw/722351/
>> [4] http://paste.openstack.org/raw/722348/
>> [5] http://paste.openstack.org/raw/722350/
>>
>>
>>>
>>> * It looks like python-openstackclient doesn't allow specifying a
>>> scope when issuing a token, is that going to be added?
>>
>> Yes, I have a patch up for it [6]. I wanted to get this in during
>> Queens, but it missed the boat. I believe this and a new release of
>> oslo.context are the only bits left in order for services to have
>> everything they need to easily consume system-scoped tokens.
>> Keystonemiddleware should know how to handle system-scoped tokens in
>> front of each service [7]. The oslo.context 

Re: [openstack-dev] [neutron][api][grapql] Proof of Concept

[Flint WALRUS]

Hi Gilles, Ed,

I’m really glad and thrilled to read such good news!

At this point it’s cool to see that many initiatives have the same
convergent needs regarding GraphQL as it will give us a good traction from
the beginning if our PoC manage to sufficiently convince our peers.

Let me know as soon as the branch have been made, I’ll work on it.

Regards,
Fl1nt.
Le jeu. 31 mai 2018 à 09:17, Gilles Dubreuil  a écrit :

> Hi Flint,
>
> I wish it was "my" summit ;)
> In the latter case I'd make the sessions an hour and not 20 or 40 minutes,
> well at least for the Forum part. And I will also make only one summit a
> year instead of two (which is also a feed back I got from the Market
> place). I've passed that during the user feedback session.
> Sorry for not responding earlier, @elmiko is going to send the minutes of
> the API SIG forum session we had.
>
> We confirmed Neutron to be the PoC.
> We are going to use a feature branch, waiting for Miguel Lavalle to
> confirm the request has been acknowledge by the Infra group.
> The PoC goal is to show GraphQL efficiency.
> So we're going to make something straightforward, use Neutron existing
> server by  adding the graphQL endpoint and cover few core items such as
> network, subnets and ports (for example).
>
> Also the idea of having a central point of access for OpenStack APIs using
> GrahpQL stitching and delegation is exciting for everyone (and I had
> obviously same feedback off the session) and that's something that could
> happen once the PoC has convinced.
>
> During the meeting, Jiri Tomasek explained how GraphQL could help TripleO
> UI. Effectively they struggle with APIs requests and had to create a
> middle(ware) module in JS to do API work and reconstruction before the
> Javascript client can use it. GraphQL would simplify the process and allow
> to get rid of the module. He also explained, after the meeting, how Horizon
> could benefit as well, allowing to use only JS and avoid Django altogether!
>
> I've also been told that Zuul nees GraphQL.
>
> Well basically the question is who doesn't need it?
>
> Cheers,
> Gilles
>
>
>
> On 31/05/18 03:34, Flint WALRUS wrote:
>
> Hi Gilles, I hope you enjoyed your Summit!?
>
> Did you had any interesting talk to report about our little initiative ?
> Le dim. 6 mai 2018 à 15:01, Gilles Dubreuil  a
> écrit :
>
>>
>> Akihiro, thank you for your precious help!
>>
>> Regarding the choice of Neutron as PoC, I'm sorry for not providing much
>> details when I said "because of its specific data model",
>> effectively the original mention was  "its API exposes things at an
>> individual table level, requiring the client to join that information to
>> get the answers they need".
>> I realize now such description probably applies to many OpenStack APIs.
>> So I'm not sure what was the reason for choosing Neutron.
>> I suppose Nova is also a good candidate because API is quite complex too,
>> in a different way, and need to expose the data API and the control API
>> plane as we discussed.
>>
>> After all Neutron is maybe not the best candidate but it seems good
>> enough.
>>
>> And as Flint say the extension mechanism shouldn't be an issue.
>>
>> So if someone believe there is a better candidate for the PoC, please
>> speak now.
>>
>> Thanks,
>> Gilles
>>
>> PS: Flint,  Thank you for offering to be the advocate for Berlin. That's
>> great!
>>
>>
>> On 06/05/18 02:23, Flint WALRUS wrote:
>>
>> Hi Akihiro,
>>
>> Thanks a lot for this insight on how neutron behave.
>>
>> We would love to get support and backing from the neutron team in order
>> to be able to get the best PoC possible.
>>
>> Someone suggested neutron as a good choice because of it simple database
>> model. As GraphQL can manage your behavior of an extension declaring its
>> own schemes I don’t think it would take that much time to implement it.
>>
>> @Gilles, if I goes to the berlin summitt I could definitely do the
>> networking and relationship work needed to get support on our PoC from
>> different teams members. This would help to spread the world multiple time
>> and don’t have a long time before someone come to talk about this subject
>> as what happens with the 2015 talk of the Facebook speaker.
>>
>> Le sam. 5 mai 2018 à 18:05, Akihiro Motoki  a écrit :
>>
>>> Hi,
>>>
>>> I am happy to see the effort to explore a new API mechanism.
>>> I would like to see good progress and help effort as API liaison from
>>> the neutron team.
>>>
>>> > Neutron has been selected for the PoC because of its specific data
>>> model
>>>
>>> On the other hand, I am not sure this is the right reason to choose
>>> 'neutron' only from this reason. I would like to note "its specific data
>>> model" is not the reason that makes the progress of API versioning slowest
>>> in the OpenStack community. I believe it is worth recognized as I would
>>> like not to block the effort due to the neutron-specific reasons.
>>> The most complicated point in the neutron API is that the 

Re: [openstack-dev] [neutron][api][grapql] Proof of Concept

[Gilles Dubreuil]

On 31/05/18 13:08, Ed Leafe wrote:

On May 6, 2018, at 8:01 AM, Gilles Dubreuil  wrote:


Regarding the choice of Neutron as PoC, I'm sorry for not providing much details when I 
said "because of its specific data model",
effectively the original mention was  "its API exposes things at an individual table 
level, requiring the client to join that information to get the answers they need".
I realize now such description probably applies to many OpenStack APIs.
So I'm not sure what was the reason for choosing Neutron.

Blame Monty!

It was Monty who suggested Neutron due to his particular experience working 
with the API. Since none of the rest of us in the API-SIG had any better 
suggestions, that was what we passed on to you. But I think that any target 
that demonstrates the advantages to be had by adopting GraphQL is fine. So if 
the team working on this think they can create a more impressive PoC with 
another API, the API-SIG will support that.


-- Ed Leafe





Well after being explained the story of the duck versus the duck parts 
(liver, heart, etc) it makes sense. With Neutron the API provides lots 
of parts but consumers have to put the part together to get the whole.


So Neutron is a good candidate as GraphQL will be able to show how it 
can fetch several parts at once (maybe not the whole beast since the PoC 
will cover only a fraction of the API).


And as you said as any API it should allow for GraphQL to show it's 
performance anyway.


So I believe we're good.

Cheers,
Gilles

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [neutron][api][grapql] Proof of Concept

[Gilles Dubreuil]

Hi Flint,

I wish it was "my" summit ;)
In the latter case I'd make the sessions an hour and not 20 or 40 
minutes, well at least for the Forum part. And I will also make only one 
summit a year instead of two (which is also a feed back I got from the 
Market place). I've passed that during the user feedback session.


Sorry for not responding earlier, @elmiko is going to send the minutes 
of the API SIG forum session we had.


We confirmed Neutron to be the PoC.
We are going to use a feature branch, waiting for Miguel Lavalle to 
confirm the request has been acknowledge by the Infra group.

The PoC goal is to show GraphQL efficiency.
So we're going to make something straightforward, use Neutron existing 
server by  adding the graphQL endpoint and cover few core items such as 
network, subnets and ports (for example).


Also the idea of having a central point of access for OpenStack APIs 
using GrahpQL stitching and delegation is exciting for everyone (and I 
had obviously same feedback off the session) and that's something that 
could happen once the PoC has convinced.


During the meeting, Jiri Tomasek explained how GraphQL could help 
TripleO UI. Effectively they struggle with APIs requests and had to 
create a middle(ware) module in JS to do API work and reconstruction 
before the Javascript client can use it. GraphQL would simplify the 
process and allow to get rid of the module. He also explained, after the 
meeting, how Horizon could benefit as well, allowing to use only JS and 
avoid Django altogether!


I've also been told that Zuul nees GraphQL.

Well basically the question is who doesn't need it?

Cheers,
Gilles


On 31/05/18 03:34, Flint WALRUS wrote:

Hi Gilles, I hope you enjoyed your Summit!?

Did you had any interesting talk to report about our little initiative ?
Le dim. 6 mai 2018 à 15:01, Gilles Dubreuil > a écrit :



Akihiro, thank you for your precious help!

Regarding the choice of Neutron as PoC, I'm sorry for not
providing much details when I said "because of its specific data
model",
effectively the original mention was  "its API exposes things at
an individual table level, requiring the client to join that
information to get the answers they need".
I realize now such description probably applies to many OpenStack
APIs.
So I'm not sure what was the reason for choosing Neutron.
I suppose Nova is also a good candidate because API is quite
complex too, in a different way, and need to expose the data API
and the control API plane as we discussed.

After all Neutron is maybe not the best candidate but it seems
good enough.

And as Flint say the extension mechanism shouldn't be an issue.

So if someone believe there is a better candidate for the PoC,
please speak now.

Thanks,
Gilles

PS: Flint,  Thank you for offering to be the advocate for Berlin.
That's great!


On 06/05/18 02:23, Flint WALRUS wrote:

Hi Akihiro,

Thanks a lot for this insight on how neutron behave.

We would love to get support and backing from the neutron team in
order to be able to get the best PoC possible.

Someone suggested neutron as a good choice because of it simple
database model. As GraphQL can manage your behavior of an
extension declaring its own schemes I don’t think it would take
that much time to implement it.

@Gilles, if I goes to the berlin summitt I could definitely do
the networking and relationship work needed to get support on our
PoC from different teams members. This would help to spread the
world multiple time and don’t have a long time before someone
come to talk about this subject as what happens with the 2015
talk of the Facebook speaker.

Le sam. 5 mai 2018 à 18:05, Akihiro Motoki mailto:amot...@gmail.com>> a écrit :

Hi,

I am happy to see the effort to explore a new API mechanism.
I would like to see good progress and help effort as API
liaison from the neutron team.

> Neutron has been selected for the PoC because of its
specific data model

On the other hand, I am not sure this is the right reason to
choose 'neutron' only from this reason. I would like to note
"its specific data model" is not the reason that makes the
progress of API versioning slowest in the OpenStack
community. I believe it is worth recognized as I would like
not to block the effort due to the neutron-specific reasons.
The most complicated point in the neutron API is that the
neutron API layer allows neutron plugins to declare which
features are supported. The neutron API is a collection of
API extensions defined in the neutron-lib repo and each
neutron plugin can declare which subset(s) of the neutron
APIs are supported. (For more detail, you can check how the
neutron API extension 

Re: [openstack-dev] [tap-as-a-service] core reviewer update

[Soichi Shigeta]

+1

Soichi


On 2018/05/31 14:36, Takashi Yamamoto wrote:

hi,

i plan to add Kazuhiro Suzuki to tap-as-a-service-core group. [1]
he is one of active members of the project.
he is also the original author of tap-as-a-service-dashboard.
i'll make the change after a week unless i hear any objections/concerns.

[1] https://review.openstack.org/#/admin/groups/957,members
http://stackalytics.com/report/contribution/tap-as-a-service/120

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [Openstack-operators] Problems with AggregateMultiTenancyIsolation while migrating an instance

[Massimo Sgaravatto]

Thanks a lot !!

On Wed, May 30, 2018 at 8:06 PM, Matt Riedemann  wrote:

> On 5/30/2018 9:41 AM, Matt Riedemann wrote:
>
>> Thanks for your patience in debugging this Massimo! I'll get a bug
>> reported and patch posted to fix it.
>>
>
> I'm tracking the problem with this bug:
>
> https://bugs.launchpad.net/nova/+bug/1774205
>
> I found that this has actually been fixed since Pike:
>
> https://review.openstack.org/#/c/449640/
>
> But I've got a patch up for another related issue, and a functional test
> to avoid regressions which I can also use when backporting the fix to
> stable/ocata.
>
> --
>
> Thanks,
>
> Matt
>
___
OpenStack-operators mailing list
OpenStack-operators@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-operators

Re: [openstack-dev] [stable][kolla] tagging newton EOL

[Jeffrey Zhang]

have no idea about this too.
and it looks as expected.

Thanks tony
On Thu, May 31, 2018 at 8:17 AM Tony Breeds  wrote:
>
> On Sat, Apr 14, 2018 at 11:02:54AM +0800, Jeffrey Zhang wrote:
> > hi stable team,
> >
> > Kolla project is ready for Newton EOL.  Since kolla-ansible is split from
> > kolla since ocata cycle, so there is not newton branch in kolla-ansible.
> > please make following repo EOL
> >
> > openstack/kolla
>
> Okay I did this today but to be perfectly frank I suspect I've done it
> wrong.
>
> There was already an existing tag for newton-eol pointing at
> 3.0.3-20'ish so I tagged what was the HEAD of the existing newton branch
> which was 3.0.0.0rc1-335'ish:
>
> About to delete the branch stable/newton from openstack/kolla (rev 
> 40e768ec2a370dc010be773af37e2ce417adda80)
>
> I'm not really sure about the history there.  I apologise if I've made a
> mistake.
>
> but at least as we have everything in git we can recover the branches
> and retag if required.
>
> Yours Tony.
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Regards,
Jeffrey Zhang
Blog: http://xcodest.me

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Cyborg] [Nova] Cyborg traits

[Nadathur, Sundar]

On 5/30/2018 1:18 PM, Eric Fried wrote:

This all sounds fully reasonable to me.  One thing, though...


   * There is a resource class per device category e.g.
 CUSTOM_ACCELERATOR_GPU, CUSTOM_ACCELERATOR_FPGA.

Let's propose standard resource classes for these ASAP.

https://github.com/openstack/nova/blob/d741f624c81baf89fc8b6b94a2bc20eb5355a818/nova/rc_fields.py

-efried
Makes sense, Eric. The obvious names would be ACCELERATOR_GPU and 
ACCELERATOR_FPGA. Do we just submit a patch to rc_fields.py?


Thanks,
Sundar

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev