[openstack-dev] [kolla][osic] Ubuntu scale testing completed - now need CentOS deployed

[Steven Dake (stdake)]

Britt, Paul, and Michal,

We are in need of centos deployed on the osic cluster (but not the deployment 
node).  I'm not sure if we have time to get through all of the scenarios we 
have but I'd like to have data on centos as well.

When you wake up, if you could sort out getting centos deployed (and kolla 
master on top of it) that would be super helpful.

I have updated the review with the ubuntu data.

Regards
-steve

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [heat][yaql] Deep merge map of lists?

[Thomas Herve]

On Sun, Aug 28, 2016 at 11:58 PM, Steven Hardy  wrote:
> Hi all,
>
> I have a need to merge a list of maps of lists:
>
> heat_template_version: 2016-10-14
>
> outputs:
>   debug:
> value:
>   yaql:
> # dict(vms=>dict($.vms.select([$.name, $])))
> expression: dict($.data.l.select([$.keys().toList()[0],
> $.values().toList()[0]]))
> data:
>   l:
> - a: [123]
> - b: [123]
> - a: [456]
>
>
>
> I want to end up with debug as:
>
>   a: [123, 456]
>   b: [123]
>
> Perhaps we need a map_deep_merge function, but can this be done with yaql?
>
> I suspect it can, but can't currently figure out how the assignment to the
> intermediate "a" value is supposed to work, any ideas on the cleanest
> approach appreciated!

I believe you don't need the intermediate value, and can rely on what
you'd do in Python with setdefault:

dict($.groupBy($.keys().toList()[0], $.values().toList()[0][0]))

ought to work, I believe?

-- 
Thomas

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all][massively distributed][architecture]Coordination between actions/WGs

[joehuang]

Hello, Bin,

Understand your expectation. In Tricircle big-tent application: 
https://review.openstack.org/#/c/338796/, a proposal was also given to add 
plugin mechnism in Nova/Cinder API layer, just like Neutron support plugin 
mechanism in API layer, that boosts innovation for different backend 
implementation to be supported, from ODL to OVN, Open Contrail

Mobile edging computing, NFV netwoking, distributed edge cloud etc are some new 
scneario for OpenStack, I suggest to have at least two successive dedicated 
design summit sessions to discuss about that f2f, the topics to be discussed 
could be:

1, Use cases
2, Requirements  in detail
3, Gaps in OpenStack
4, Proposal to be discussed

Arhietcture level proposal discussion
1, Proposals
2, Pros. and Cons. comparation
3, Chellenges
4, next step


Looking forward to your thoughts.


Best Regards
Chaoyi Huang(joehuang)


From: HU, BIN [bh5...@att.com]
Sent: 28 August 2016 2:16
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev][all][massively 
distributed][architecture]Coordination  between actions/WGs

The challenge in OpenStack is how to enable the innovation built on top of 
OpenStack.

So telco use cases is not only the innovation built on top of OpenStack. 
Instead, telco use cases, e.g. Gluon (NFV networking), vCPE Cloud, Mobile 
Cloud, Mobile Edge Cloud, brings the needed requirement for innovation in 
OpenStack itself. If OpenStack don't address those basic requirements, the 
innovation will never happen on top of OpenStack.

An example is - self-driving car is built on top of many technologies, such as 
sensor/camera, AI, maps, middleware etc. All innovations in each technology 
(sensor/camera, AI, map, etc.) bring together the innovation of self-driving 
car.

WE NEED INNOVATION IN OPENSTACK in order to enable the innovation built on top 
of OpenStack.

Thanks
Bin
-Original Message-
From: Edward Leafe [mailto:e...@leafe.com]
Sent: Saturday, August 27, 2016 10:49 AM
To: OpenStack Development Mailing List (not for usage questions) 

Subject: Re: [openstack-dev] [all][massively 
distributed][architecture]Coordination between actions/WGs

On Aug 27, 2016, at 12:18 PM, HU, BIN  wrote:

>> From telco perspective, those are the areas that allow innovation, and 
>> provide telco customers with new types of services.
>
> We need innovation, starting from not limiting ourselves from bringing new 
> idea and new use cases, and bringing those impossibility to reality.

There is innovation in OpenStack, and there is innovation in things built on 
top of OpenStack. We are simply trying to keep the two layers from getting 
confused.


-- Ed Leafe






__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] relationship_type in static_datasources

[Yujun Zhang]

Thanks for replying. See my comments inline.

On Sun, Aug 28, 2016 at 8:02 PM Afek, Ifat (Nokia - IL) 
wrote:

> Hi Yujun,
>
> Regarding the validation – I agree that we should implement it in another
> way, but as a first stage I think you can just remove it.
>

OK


> If you have some thoughts regarding the way to implement it, we will be
> happy to hear them. We thought about a yaml file per datasource, but didn’t
> design it yet.
>

No problem. I'll put it to open discuss once I have a proposal.


> BTW, did you notice the new API for template validation?
> https://github.com/openstack/vitrage/blob/master/doc/source/vitrage-api.rst#template-validate
>
>

I'll have a look at it.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla][oisc] scenario #8 running - need help deploying centos next

[Steven Dake (stdake)]

Scenario #8 was running about 12 hours (mitaka rally and tempest testing) and 
was only half way done.  Inspection of the control machines showed out of disk 
space errors on the control nodes (which only have a 50gb total partition for 
where /var/lib/docker was stored).  Elasticsearch produces about 25gb/day under 
our rally benchmarks, so a cleanup was necessary.  Note during the out of disk 
space problem, OpenStack made forward progress, it was just extremely slow and 
many tests that would have passed with sufficient disk space did not.

Expect scenario #8 to complete and 4-6 hours and scenario #9 to take 2 hours.

Then the next step is a reload of the base os on al target deployment nodes to 
centos.  Ping me before undertaking this activity such that all data is 
captured for ubuntu.

Regards,
-steve

From: Steven Dake >
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
>
Date: Saturday, August 27, 2016 at 1:20 PM
To: "OpenStack Development Mailing List (not for usage questions)" 
>
Subject: Re: [openstack-dev] [kolla][oisc] scenario #5 running

Scenario #6 is running – should finish in 4-6 hours.

Regards
-steve

From: Steven Dake >
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
>
Date: Saturday, August 27, 2016 at 4:34 AM
To: "OpenStack Development Mailing List (not for usage questions)" 
>
Subject: [openstack-dev] [kolla][oisc] scenario #5 running

Hey folks,

I didn't undo any of what Dave did to the cluster.  I wrote a simple shell 
script to run time on all of the operations.  We will be doing the same thing 
for liberty and mitaka.  I expect the test of all operations will take about 40 
minutes to 2 hours.

Full instructions are on the tmux screen for scenario 6 when scenario 5 
finishes.  Note you will need to undo the things Dave has done.  Ping me if you 
have questions on irc if you get to it before I do.

Regards
-steve

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [heat][yaql] Deep merge map of lists?

[Steven Hardy]

Hi all,

I have a need to merge a list of maps of lists:

heat_template_version: 2016-10-14

outputs:
  debug:
value:
  yaql:
# dict(vms=>dict($.vms.select([$.name, $])))
expression: dict($.data.l.select([$.keys().toList()[0],
$.values().toList()[0]]))
data:
  l:
- a: [123]
- b: [123]
- a: [456]



I want to end up with debug as:

  a: [123, 456]
  b: [123]

Perhaps we need a map_deep_merge function, but can this be done with yaql?

I suspect it can, but can't currently figure out how the assignment to the
intermediate "a" value is supposed to work, any ideas on the cleanest
approach appreciated!

Thanks,

Steve

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [OpenStack-docs] [cinder] [api] [doc] API status report

[Sean McGinnis]

On Sat, Aug 27, 2016 at 07:46:48PM +0200, Andreas Jaeger wrote:
> On 08/26/2016 11:33 PM, Anne Gentle wrote:
> > Hi cinder block storage peeps: 
> > 
> > I haven't heard from you on your comfort level with publishing so I went
> > ahead and made the publishing job myself with this review:
> > 
> > https://review.openstack.org/361475
> > 
> > Please let me know your thoughts there. Is the document ready to
> > publish? Need anything else to get comfy? Let me know.
> > 
> 
> The current api-ref does not build at all, let's not merge 361475 yet.
> 
> I've rebased https://review.openstack.org/#/c/322489 and added
> https://review.openstack.org/#/c/361616 so that the cinder api-ref
> follows the same patterns as other repositories - including building and
> review on docs-draft.
> 
> Once those two are in, we can merge 361475.
> 
> Cinder team, could you prioritize these reviews, please?

Merged.

Thanks Andreas and Anne!

I'm sure there will be some things we will need to fix or adjust, but I
think this is a good time to get it published. That will help as well
with getting visibility on it and start identifying those issues.

Sorry for the delay getting back to you Anne. Thanks for moving that
forward.

> 
> Andreas
> -- 
>  Andreas Jaeger aj@{suse.com,opensuse.org} Twitter/Identica: jaegerandi
>   SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
>GF: Felix Imendörffer, Jane Smithard, Graham Norton,
>HRB 21284 (AG Nürnberg)
> GPG fingerprint = 93A3 365E CE47 B889 DF7F  FED1 389A 563C C272 A126
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [kolla][ironic][bifrost][osic] future of bifrost & Ironic for Kolla

[Steven Dake (stdake)]

Hey Stackers & Sean :),

I just wanted to provide a quick update on where we think Kolla will be when 
ironic and bare metal config are fully fleshed out (landing in Newton).

3 controller, 20 storage, 100 compute with 
ceph+ovs+cinder+enable_central_logging on OSIC 130 node cluster

Currently openstack-only deployment takes 20 minutes
A bare metal dependency install and configuration takes about 10-15 minutes 
(the current playbook craters so this is just a rough estimate)
We would like to see the bifrost deployment complete in 35 minutes (we were 
unable to test this at 123 node scale because the patches haven't landed and we 
are out of osic time).

The goal is to keep a total deploy of 1 hour for 123 nodes from nothing to 
fully operational OpenStack cloud.

As far as subject, I feel pretty happy with the bifrost implementation as it 
exists today .  The idea of turning biforst into thin containers isn't hot on 
my list of things to do.  I am pretty happy with treating bifrost as a fat 
container and using containers here solely for dependency management to keep 
the host os tidy.

That isn't to say we won't ask for further changes to bifrost (and do the work 
to make them so).  I just don't see a strong  rationale for thin-containerizing 
bifrost.  We could change our minds in the future especially if the bifrost 
team were supportive of this idea.  For now I think its fruit at the top of the 
tree without much benefit.  The dependency management and integration with 
kolla will land in newton.

We also have a full ironic implementation that is thin-containerized that is 
nearing completion (tech preview for Newton) to serve the bare metal as a 
service use case.  We plan to use this for bare metal deployment via nova – 
much as is done with virtual machines by selecting a flavor and using nova 
boot.  We are not going to use bifrost for this portion of the job.  Most of 
the work here is getting ironic to cooperate with Vms in the nova scheduler via 
flavors.  I suspect we will have a design session at summit around this very 
topic but its really up to the next PTL.

Hope to see the Ironic  and Kolla community there in this critical (atleast to 
Kolla) cross-project integration effort.

Regards
-seve



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [kolla] NOTICE: Deadline of 31st for milestone #3 - PLEASE UPDATE BLUEPRINT STATUS

[Steven Dake (stdake)]

Please take heed of the subject line.  It would be immensely helpful if people 
working on the 35 blueprints would get them into he correct state (good 
progress or NEEDS REVIEW) so we can best select which features land in Newton.  
Ping any of the cores on #openstack-kolla if you don’t have permissions to 
change the blueprint status.

Regards
-steve

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Suspected SPAM - Re: [vitrage] relationship_type in static_datasources

[Afek, Ifat (Nokia - IL)]

Hi Yujun,

Regarding the validation – I agree that we should implement it in another way, 
but as a first stage I think you can just remove it. If you have some thoughts 
regarding the way to implement it, we will be happy to hear them. We thought 
about a yaml file per datasource, but didn’t design it yet.
BTW, did you notice the new API for template validation? 
https://github.com/openstack/vitrage/blob/master/doc/source/vitrage-api.rst#template-validate

Best regards,
Ifat.

From: "Weyl, Alexey (Nokia - IL)"
Date: Sunday, 28 August 2016 at 11:05


Hi Yujun,

In order for the static_physical to work for different datasources without the 
restrictions, you need to do the following changes:
Go to the static_physical transformer:

1.   Remove the methods: _register_relations_direction, 
_find_relation_direction_source.

2.   Add to the static_physical.yaml for each definition also a field for 
direction which will indicate the source and the destination between the 
datasources.

3.   In method: _create_neighbor, remove the usage of method 
_find_relation_direction_source, and use the new definition from the yaml file 
here to decide the edge direction.

Is it ok?


From: Yujun Zhang [mailto:zhangyujun+...@gmail.com]
Sent: Friday, August 26, 2016 4:22 AM

Lost in the code...It seems the datasource just construct the entities and send 
them over event bus to entity graph processor. I need to dig further to find 
out the exact point the "backup" relationship is filtered.

I think we should some how keep the validation of relationship type. It is so 
easy to make typo when creating the template manually (I did this quite 
often...).

My idea is to delegate the validation to datasource instead of enumerating all 
constants it in evaluator. I think this will introduce better extensibility. 
Any comments?

On Thu, Aug 25, 2016 at 1:32 PM Weyl, Alexey (Nokia - IL) 
> wrote:
Hi Yujun,

You can find the names of the lables in the constants.py file.

In addition, the restriction on the physical_static datasource is done in it’s 
driver.py.

Alexey

From: Yujun Zhang 
[mailto:zhangyujun+...@gmail.com]
Sent: Thursday, August 25, 2016 4:50 AM

Hi, Ifat,

I searched for edge_labels in the project. It seems it is validated only in 
`vitrage/evaluator/template_validation/template_syntax_validator.py`. Where is 
such restriction applied in static_datasources?

--
Yujun

On Wed, Aug 24, 2016 at 3:19 PM Afek, Ifat (Nokia - IL) 
> wrote:
Hi Yujun,

Indeed, we have some restrictions on the relationship types that can be used in 
the static datasources. I think we should remove these restrictions, and allow 
any kind of relationship type.

Best regards,
Ifat.

From: Yujun Zhang
Date: Monday, 22 August 2016 at 08:37
I'm following the sample configuration in docs [1] to verify how static 
datasources works.

It seems `backup` relationship is not displayed in the entity graph view and 
neither is it included in topology show.

There is an enumeration for edge labels [2]. Should relationship in static 
datasource be limited to it?

[1] 
https://github.com/openstack/vitrage/blob/master/doc/source/static-physical-config.rst
[2] 
https://github.com/openstack/vitrage/blob/master/vitrage/common/constants.py#L49

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [kolla][vote] Core nomination for Dave Walker (Daviey on irc)

[Steven Dake (stdake)]

This vote has passed unanimously prior to August 30th, therefore, closing 
voting early.

Welcome to the core reviewer team Dave!  If you have any questions, feel free 
to ask any core reviewer or myself.  I have made the necessary changes in 
gerrit.

Regards
-steve

From: Steven Dake >
Reply-To: "OpenStack Development Mailing List (not for usage questions)" 
>
Date: Tuesday, August 23, 2016 at 1:45 PM
To: "OpenStack Development Mailing List (not for usage questions)" 
>
Subject: [openstack-dev] [kolla][vote] Core nomination for Dave Walker (Daviey 
on irc)

Kolla core reviewers,

I am nominating Dave Walker for the Kolla core reviewer team.  His 30 day 
review stats [1] place him in the middle of the pack for reviewers and his 60 
day stats[2] are about equivalent.  Dave participates heavily in IRC and has 
done some good technical work including the Watcher playbook and container.  He 
also worked on Sensu, but since we are unclear if we are choosing Sensu or Tig, 
that work is on hold.  He will also be helping us sort out how to execute with 
PBR going into the future on our stable and master branches.  Dave has proven 
to me his reviews are well thought out and he understands the Kolla 
Architecture.  Dave is part time like many Kolla core reviewers and is 
independent of any particular affiliation.

Consider this nomination as a +1 from me.

As a reminder, a +1 vote indicates you approve of the candidate, an abstain 
vote indicates you don't care or don't know for certain, and a –1 vote 
indicates a veto.  If a veto occurs or a unanimous response is reached prior to 
our 7 day voting window which concludes on August 30th, voting will be closed 
early.

[1] http://stackalytics.com/report/contribution/kolla/30
[2] http://stackalytics.com/report/contribution/kolla/60
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] relationship_type in static_datasources

[Weyl, Alexey (Nokia - IL)]

Hi Yujun,

In order for the static_physical to work for different datasources without the 
restrictions, you need to do the following changes:
Go to the static_physical transformer:

1.   Remove the methods: _register_relations_direction, 
_find_relation_direction_source.

2.   Add to the static_physical.yaml for each definition also a field for 
direction which will indicate the source and the destination between the 
datasources.

3.   In method: _create_neighbor, remove the usage of method 
_find_relation_direction_source, and use the new definition from the yaml file 
here to decide the edge direction.

Is it ok?


From: Yujun Zhang [mailto:zhangyujun+...@gmail.com]
Sent: Friday, August 26, 2016 4:22 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [vitrage] relationship_type in static_datasources

Lost in the code...It seems the datasource just construct the entities and send 
them over event bus to entity graph processor. I need to dig further to find 
out the exact point the "backup" relationship is filtered.

I think we should some how keep the validation of relationship type. It is so 
easy to make typo when creating the template manually (I did this quite 
often...).

My idea is to delegate the validation to datasource instead of enumerating all 
constants it in evaluator. I think this will introduce better extensibility. 
Any comments?

On Thu, Aug 25, 2016 at 1:32 PM Weyl, Alexey (Nokia - IL) 
> wrote:
Hi Yujun,

You can find the names of the lables in the constants.py file.

In addition, the restriction on the physical_static datasource is done in it’s 
driver.py.

Alexey

From: Yujun Zhang 
[mailto:zhangyujun+...@gmail.com]
Sent: Thursday, August 25, 2016 4:50 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [vitrage] relationship_type in static_datasources

Hi, Ifat,

I searched for edge_labels in the project. It seems it is validated only in 
`vitrage/evaluator/template_validation/template_syntax_validator.py`. Where is 
such restriction applied in static_datasources?

--
Yujun

On Wed, Aug 24, 2016 at 3:19 PM Afek, Ifat (Nokia - IL) 
> wrote:
Hi Yujun,

Indeed, we have some restrictions on the relationship types that can be used in 
the static datasources. I think we should remove these restrictions, and allow 
any kind of relationship type.

Best regards,
Ifat.

From: Yujun Zhang
Date: Monday, 22 August 2016 at 08:37
I'm following the sample configuration in docs [1] to verify how static 
datasources works.

It seems `backup` relationship is not displayed in the entity graph view and 
neither is it included in topology show.

There is an enumeration for edge labels [2]. Should relationship in static 
datasource be limited to it?

[1] 
https://github.com/openstack/vitrage/blob/master/doc/source/static-physical-config.rst
[2] 
https://github.com/openstack/vitrage/blob/master/vitrage/common/constants.py#L49
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: 
openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev