Re: [openstack-dev] Storyboard python script

[Jeremy Stanley]

On 2018-10-31 22:39:01 + (+), Krishna Jain wrote:
> I’m an animator with some coding experience picking up Python. I
> came across your python-storyboardclient library, which would be
> very helpful for automating our pipeline in Toon Boom Storyboard
> Pro.
[...]

The python-storyboardclient library is for use with the free/libre
open source StoryBoard task and defect tracker project described by
the documentation located at
https://docs.openstack.org/infra/storyboard/ and not the commercial
"Toon Boom Storyboard Pro" animation software to which you seem to
be referring. Sorry for the confusion!
-- 
Jeremy Stanley


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [ironic] Team gathering at the Forum?

[Stig Telfer]

Hello Ironicers -

We’ve booked the same venue for the Scientific SIG for Wednesday evening, and 
hopefully we’ll see you there.  There’s plenty of cross-over between our 
groups, particularly at an operator level.

Cheers,
Stig


> On 29 Oct 2018, at 14:58, Dmitry Tantsur  wrote:
> 
> Hi folks!
> 
> This is your friendly reminder to vote on the day. Even if you're fine with 
> all days, please leave a vote, so that we know how many people are coming. We 
> will need to make a reservation, and we may not be able to accommodate more 
> people than voted!
> 
> Dmitry
> 
> On 10/22/18 6:06 PM, Dmitry Tantsur wrote:
>> Hi ironicers! :)
>> We are trying to plan an informal Ironic team gathering in Berlin. If you 
>> care about Ironic and would like to participate, please fill in 
>> https://doodle.com/poll/iw5992px765nthde. Note that the location is 
>> tentative, also depending on how many people sign up.
>> Dmitry
> 
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Storyboard python script

[Mohammed Naser]

I believe this project is a client for Storyboard, an OpenStack project and not 
the commercial product you’re mentioning

Sent from my iPhone

> On Oct 31, 2018, at 11:39 PM, Krishna Jain  wrote:
> 
> Hi,
> I’m an animator with some coding experience picking up Python. I came across 
> your python-storyboardclient library, which would be very helpful for 
> automating our pipeline in Toon Boom Storyboard Pro. I’d like to have Python 
> call some of the Javascript scripts I’ve written to extend SBPro. Or at least 
> make it possible to rewrite the scripts in Python if need be. Unfortunately, 
> when I try to install it, I get:
>  
> Command ""c:\program files\python37\python.exe" -u -c "import setuptools, 
> tokeni
> ze;__file__='C:\\Users\\kjain\\AppData\\Local\\Temp\\pip-install-gli0gz3z\\netif
> aces\\setup.py';f=getattr(tokenize, 'open', 
> open)(__file__);code=f.read().replac
> e('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install 
> --recor
> d C:\Users\kjain\AppData\Local\Temp\pip-record-1qhmhrv5\install-record.txt 
> --sin
> gle-version-externally-managed --compile" failed with error code 1 in 
> C:\Users\k
> jain\AppData\Local\Temp\pip-install-gli0gz3z\netifaces\
>  
> Do you know what might be going wrong here?
> Thanks!
> -Krishna Jain
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Storyboard python script

[Krishna Jain]

Hi,
I’m an animator with some coding experience picking up Python. I came across 
your python-storyboardclient library, which would be very helpful for 
automating our pipeline in Toon Boom Storyboard Pro. I’d like to have Python 
call some of the Javascript scripts I’ve written to extend SBPro. Or at least 
make it possible to rewrite the scripts in Python if need be. Unfortunately, 
when I try to install it, I get:

Command ""c:\program files\python37\python.exe" -u -c "import setuptools, tokeni
ze;__file__='C:\\Users\\kjain\\AppData\\Local\\Temp\\pip-install-gli0gz3z\\netif
aces\\setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replac
e('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --recor
d C:\Users\kjain\AppData\Local\Temp\pip-record-1qhmhrv5\install-record.txt --sin
gle-version-externally-managed --compile" failed with error code 1 in C:\Users\k
jain\AppData\Local\Temp\pip-install-gli0gz3z\netifaces\

Do you know what might be going wrong here?
Thanks!
-Krishna Jain
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] reducing our upstream CI footprint

[Ben Nemec]

On 10/31/18 4:59 PM, Harald Jensås wrote:

On Wed, 2018-10-31 at 11:39 -0600, Wesley Hayutin wrote:



On Wed, Oct 31, 2018 at 11:21 AM Alex Schultz 
wrote:

Hey everyone,

Based on previous emails around this[0][1], I have proposed a
possible
reducing in our usage by switching the scenario001--011 jobs to
non-voting and removing them from the gate[2]. This will reduce the
likelihood of causing gate resets and hopefully allow us to land
corrective patches sooner.  In terms of risks, there is a risk that
we
might introduce breaking changes in the scenarios because they are
officially non-voting, and we will still be gating promotions on
these
scenarios.  This means that if they are broken, they will need the
same attention and care to fix them so we should be vigilant when
the
jobs are failing.

The hope is that we can switch these scenarios out with voting
standalone versions in the next few weeks, but until that I think
we
should proceed by removing them from the gate.  I know this is less
than ideal but as most failures with these jobs in the gate are
either
timeouts or unrelated to the changes (or gate queue), they are more
of
hindrance than a help at this point.

Thanks,
-Alex


I think I also have to agree.
Having to deploy with containers, update containers and run with two
nodes is no longer a very viable option upstream.  It's not
impossible but it should be the exception and not the rule for all
our jobs.


afaict in my local environment, the container prep stuff takes ages
when adding the playbooks to update them with yum. We will still have
to do this for every standalone job right?



Also, I enabled profiling for ansible tasks on the undercloud and
noticed that the UndercloudPostDeploy was high on the list, actually
the longest running task when re-running the undercloud install ...

Moving from shell script using openstack cli to python reduced the time
for this task dramatically in my environment, see:
https://review.openstack.org/614540. 6 and half minutes reduced to 40
seconds ...


Everything old is new again: 
https://github.com/openstack/instack-undercloud/commit/0eb1b59926c7dc46e321c56db29af95b3d755f34#diff-5602f1b710e86ca1eb7334cb0632f9ee


:-)




How much time would we save in the gates if we converted some of the
shell scripting to python, or if we want to stay in shell script we can
use the interactive shell or use the client-as-a-service[2]?

Interactive shell:
time openstack <<-EOC
server list
workflow list
workflow execution list
EOC

real0m2.852s
time (openstack server list; \
   openstack workflow list; \
   openstack workflow execution list)

real0m7.119s

The difference is significant.

We could cache a token[1], and specify the end-point on each command,
but doing so is still far from as effective as using the interactive.


There is an old thread[2] on the mailing list, which contain a
server/client solution. If we run this service in CI nodes and drop in
the replacement openstack command in /usr/local/bin/openstack we would
use ~1/5 of the time for each command.

(undercloud) [stack@leafs ~]$ time (/usr/bin/openstack network list -f
value -c ID; /usr/bin/openstack network segment list -f value -c ID;
/usr/bin/openstack subnet list -f value -c ID)


real0m6.443s
user0m2.171s
sys 0m0.366s

(undercloud) [stack@leafs ~]$ time (/usr/local/bin/openstack network
list -f value -c ID; /usr/local/bin/openstack network segment list -f
value -c ID; /usr/local/bin/openstack subnet list -f value -c ID)

real0m1.698s
user0m0.042s
sys 0m0.018s



I relize this is a kind of hacky approch, but it does seem to work and
it should be fairly quick to get in there. (With the Undercloud post
script I see 6 minutes returned, what can we get in CI, 10-15 minutes?
Then we could look at moving these scripts to python or use ansible
openstack modules which hopefully does'nt share the same issues with
loading as the python clients?


I'm personally a fan of using Python as then it is unit-testable, but 
I'm not sure how that works with the tht-based code so maybe it's not a 
factor.






[1] https://wiki.openstack.org/wiki/OpenStackClient/Authentication
[2]
http://lists.openstack.org/pipermail/openstack-dev/2016-April/092546.html



Thanks Alex

  

[0]
http://lists.openstack.org/pipermail/openstack-dev/2018-October/136141.html
[1]
http://lists.openstack.org/pipermail/openstack-dev/2018-October/135396.html
[2]
https://review.openstack.org/#/q/topic:reduce-tripleo-usage+(status:open+OR+status:merged
)

___
___
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsu
bscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


--
WES HAYUTIN
ASSOCIATE MANAGER
Red Hat

whayu...@redhat.comT: +19194232509 IRC:  weshay


View my calendar and check my availability for meetings HERE
__

[openstack-dev] [tripleo] shutting down 3rd party TripleO CI for measurements

[Wesley Hayutin]

Greetings,

The TripleO-CI team would like to consider shutting down all the third
party check jobs running against TripleO projects in order to measure
results with and without load on the cloud for some amount of time.  I
suspect we would want to shut things down for roughly 24-48 hours.

If there are any strong objects please let us know.
Thank you
-- 

Wes Hayutin

Associate MANAGER

Red Hat



whayu...@redhat.comT: +1919 <+19197544114>4232509 IRC:  weshay


View my calendar and check my availability for meetings HERE

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] reducing our upstream CI footprint

[Harald Jensås]

On Wed, 2018-10-31 at 11:39 -0600, Wesley Hayutin wrote:
> 
> 
> On Wed, Oct 31, 2018 at 11:21 AM Alex Schultz 
> wrote:
> > Hey everyone,
> > 
> > Based on previous emails around this[0][1], I have proposed a
> > possible
> > reducing in our usage by switching the scenario001--011 jobs to
> > non-voting and removing them from the gate[2]. This will reduce the
> > likelihood of causing gate resets and hopefully allow us to land
> > corrective patches sooner.  In terms of risks, there is a risk that
> > we
> > might introduce breaking changes in the scenarios because they are
> > officially non-voting, and we will still be gating promotions on
> > these
> > scenarios.  This means that if they are broken, they will need the
> > same attention and care to fix them so we should be vigilant when
> > the
> > jobs are failing.
> > 
> > The hope is that we can switch these scenarios out with voting
> > standalone versions in the next few weeks, but until that I think
> > we
> > should proceed by removing them from the gate.  I know this is less
> > than ideal but as most failures with these jobs in the gate are
> > either
> > timeouts or unrelated to the changes (or gate queue), they are more
> > of
> > hindrance than a help at this point.
> > 
> > Thanks,
> > -Alex
> 
> I think I also have to agree.
> Having to deploy with containers, update containers and run with two
> nodes is no longer a very viable option upstream.  It's not
> impossible but it should be the exception and not the rule for all
> our jobs.
> 
afaict in my local environment, the container prep stuff takes ages
when adding the playbooks to update them with yum. We will still have
to do this for every standalone job right?



Also, I enabled profiling for ansible tasks on the undercloud and
noticed that the UndercloudPostDeploy was high on the list, actually
the longest running task when re-running the undercloud install ...

Moving from shell script using openstack cli to python reduced the time
for this task dramatically in my environment, see: 
https://review.openstack.org/614540. 6 and half minutes reduced to 40
seconds ...


How much time would we save in the gates if we converted some of the
shell scripting to python, or if we want to stay in shell script we can
use the interactive shell or use the client-as-a-service[2]?

Interactive shell:
time openstack <<-EOC
server list
workflow list
workflow execution list
EOC

real0m2.852s
time (openstack server list; \
  openstack workflow list; \
  openstack workflow execution list)

real0m7.119s

The difference is significant.

We could cache a token[1], and specify the end-point on each command,
but doing so is still far from as effective as using the interactive.


There is an old thread[2] on the mailing list, which contain a
server/client solution. If we run this service in CI nodes and drop in
the replacement openstack command in /usr/local/bin/openstack we would
use ~1/5 of the time for each command.

(undercloud) [stack@leafs ~]$ time (/usr/bin/openstack network list -f
value -c ID; /usr/bin/openstack network segment list -f value -c ID;
/usr/bin/openstack subnet list -f value -c ID)


real0m6.443s
user0m2.171s
sys 0m0.366s

(undercloud) [stack@leafs ~]$ time (/usr/local/bin/openstack network
list -f value -c ID; /usr/local/bin/openstack network segment list -f
value -c ID; /usr/local/bin/openstack subnet list -f value -c ID)

real0m1.698s
user0m0.042s
sys 0m0.018s



I relize this is a kind of hacky approch, but it does seem to work and
it should be fairly quick to get in there. (With the Undercloud post
script I see 6 minutes returned, what can we get in CI, 10-15 minutes?
Then we could look at moving these scripts to python or use ansible
openstack modules which hopefully does'nt share the same issues with
loading as the python clients?



[1] https://wiki.openstack.org/wiki/OpenStackClient/Authentication
[2] 
http://lists.openstack.org/pipermail/openstack-dev/2016-April/092546.html


> Thanks Alex
> 
>  
> > [0] 
> > http://lists.openstack.org/pipermail/openstack-dev/2018-October/136141.html
> > [1] 
> > http://lists.openstack.org/pipermail/openstack-dev/2018-October/135396.html
> > [2] 
> > https://review.openstack.org/#/q/topic:reduce-tripleo-usage+(status:open+OR+status:merged
> > )
> > 
> > ___
> > ___
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsu
> > bscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 
> -- 
> WES HAYUTIN
> ASSOCIATE MANAGER
> Red Hat 
> 
> whayu...@redhat.comT: +19194232509 IRC:  weshay
> 
> 
> View my calendar and check my availability for meetings HERE
> _
> _
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@li

Re: [openstack-dev] [manila][tc] Seeking feedback on the OpenStack cloud vision

[Tom Barron]

On 24/10/18 11:14 -0400, Zane Bitter wrote:

Greetings, Manila team!
As you may be aware, I've been working with other folks in the 
community on documenting a vision for OpenStack clouds (formerly known 
as the 'Technical Vision') - essentially to interpret the mission 
statement in long-form, in a way that we can use to actually help 
guide decisions. You can read the latest draft here: 
https://review.openstack.org/592205


We're trying to get feedback from as many people as possible - in many 
ways the value is in the process of coming together to figure out what 
we're trying to achieve as a community with OpenStack and how we can 
work together to build it. The document is there to help us remember 
what we decided so we don't have to do it all again over and over.


The vision is structured with two sections that apply broadly to every 
project in OpenStack - describing the principles that we believe are 
essential to every cloud, and the ones that make OpenStack different 
from some other clouds. The third section is a list of design goals 
that we want OpenStack as a whole to be able to meet - ideally each 
project would be contributing toward one or more of these design 
goals.


I think that, like Cinder, Manila would qualify as contributing to the 
'Basic Physical Data Center Management' goal, since it also allows 
users to access external storage providers through a standardised API.


If you would like me or another TC member to join one of your team IRC 
meetings to discuss further what the vision means for your team, 
please reply to this thread to set it up. You are also welcome to 
bring up any questions in the TC IRC channel, #openstack-tc - there's 
more of us around during Office Hours 
(https://governance.openstack.org/tc/#office-hours), but you can talk 
to us at any time.


Feedback can also happen either in this thread or on the review 
https://review.openstack.org/592205


If the team is generally happy with the vision as it is and doesn't 
have any specific feedback, that's cool but I'd like to request that 
at least the PTL leave a vote on the review. It's important to know 
whether we are actually developing a consensus in the community or 
just talking to ourselves :)


many thanks,
Zane.


Zane and I chatted on IRC and he is going to attend the manila 
community meeting tomorrow, November 1, at 1500 UTC in 
#openstack-meeting-alt to follow up and solicit feedback.  If you are 
unable to attend the meeting and have points to make or questions 
please follow up in this thread or in the review mentioned above.


Cheers,

-- Tom


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [goal][python3] week 12 update

[Doug Hellmann]

This is week 12 of the "Run under Python 3 by default" goal
(https://governance.openstack.org/tc/goals/stein/python3-first.html).

Observant readers will notice that the last update email was during week
9. I've been out for a couple of weeks, but you've all been busy in that
time!

== What we learned last week ==

I'm still working on an upgrade script for heat to allow us to rename it
and publish releases.

* https://review.openstack.org/#/c/606160/

== Ongoing and Completed Work ==

We are very very close to finishing the phase of work that updates the
tox settings and documentation build jobs.

Those documentation updates should be relatively quick to review because
they're very minimal patches. Please take a few minutes to look for them
and let's try to get them merged before the first milestone. The tox
patches may require a bit more work to update pylint and the goal
champions could use your help there (see below).

+-+--+-+--+-++---++
| Team| tox defaults | Docs| 3.6 unit | Failing | 
Unreviewed | Total | Champion   |
+-+--+-+--+-++---++
| adjutant|   1/  1  | -   | +|   0 |  
1 | 2 | Doug Hellmann  |
| barbican| +|   1/  3 | +|   1 |  
1 | 7 | Doug Hellmann  |
| blazar  | +| +   | +|   0 |  
0 | 9 | Nguyen Hai |
| Chef OpenStack  | +| -   | -|   0 |  
0 | 2 | Doug Hellmann  |
| cinder  | +| +   | +|   0 |  
0 |11 | Doug Hellmann  |
| cloudkitty  | +| +   | +|   0 |  
0 | 9 | Doug Hellmann  |
| congress| +| +   | +|   0 |  
0 | 9 | Nguyen Hai |
| cyborg  | +| +   | +|   0 |  
0 | 7 | Nguyen Hai |
| designate   | +| +   | +|   0 |  
0 | 9 | Nguyen Hai |
| Documentation   | +| +   | +|   0 |  
0 |10 | Doug Hellmann  |
| dragonflow  | -| +   | +|   0 |  
0 | 2 | Nguyen Hai |
| ec2-api |   2/  2  | +   | +|   2 |  
2 | 7 ||
| freezer | +| +   | +|   0 |  
0 |11 ||
| glance  | +| +   | +|   0 |  
0 |10 | Nguyen Hai |
| heat|   3/  8  | +   |   1/  7  |   2 |  
0 |21 | Doug Hellmann  |
| horizon | +| +   | +|   0 |  
0 |34 | Nguyen Hai |
| I18n| +| -   | -|   0 |  
0 | 1 | Doug Hellmann  |
| InteropWG   |   3/  4  | +   |   1/  3  |   2 |  
2 |10 | Doug Hellmann  |
| ironic  |   1/ 10  | +   | +|   0 |  
0 |35 | Doug Hellmann  |
| karbor  | +| +   | +|   0 |  
0 | 7 | Nguyen Hai |
| keystone| +| +   | +|   0 |  
0 |18 | Doug Hellmann  |
| kolla   | +| +   | +|   0 |  
0 | 5 ||
| kuryr   | +| +   | +|   0 |  
0 | 9 | Doug Hellmann  |
| magnum  |   2/  5  | +   | +|   0 |  
1 |10 ||
| manila  | +| +   | +|   0 |  
0 |13 | Goutham Pacha Ravi |
| masakari|   2/  5  | +   | -|   0 |  
2 | 6 | Nguyen Hai |
| mistral | +| +   | +|   0 |  
0 |13 | Nguyen Hai |
| monasca |   1/ 17  | +   | +|   1 |  
1 |34 | Doug Hellmann  |
| murano  | +| +   | +|   0 |  
0 |14 ||
| neutron |   7/ 19  |   1/ 14 |   1/ 13  |   6 |  
3 |46 | Doug Hellmann  |
| nova| +| +   | +|   0 |  
0 |14 ||
| octavia | +| +   | +|   0 |  
0 |12 | Nguyen Hai |
| OpenStack Charms|  36/ 73  | -   | -|  36 | 
15 |73 | Doug Hellmann

Re: [openstack-dev] [all] Zuul job backlog

[Abhishek Kekane]

Hi All,

I have fixed the glance functional test issue, patch [1] is merged in
master. I hope the mentioned issue is now resolved.

Kindly let me know.

[1] https://review.openstack.org/#/c/608856/

Thank you,
Abhishek

On Mon, 8 Oct 2018 at 11:37 PM, Doug Hellmann  wrote:

> Abhishek Kekane  writes:
>
> > Hi Doug,
> >
> > Should I use something like SimpleHttpServer to upload a file and
> download
> > the same, or there are other efficient ways to handle it,
> > Kindly let me know if you are having any suggestions for the same.
>
> Sure, that would work, especially if your tests are running in the unit
> test jobs. If you're running a functional test, it seems like it would
> also be OK to just copy a file into the directory Apache is serving from
> and then download it from there.
>
> Doug
>
> >
> > Thanks & Best Regards,
> >
> > Abhishek Kekane
> >
> >
> > On Fri, Oct 5, 2018 at 4:57 PM Doug Hellmann 
> wrote:
> >
> >> Abhishek Kekane  writes:
> >>
> >> > Hi Matt,
> >> >
> >> > Thanks for the input, I guess I should use '
> >> > http://git.openstack.org/static/openstack.png' which will definitely
> >> work.
> >> > Clark, Matt, Kindly let me know your opinion about the same.
> >>
> >> That URL would not be on the local node running the test, and would
> >> eventually exhibit the same problems. In fact we have seen issues
> >> cloning git repositories as part of the tests in the past.
> >>
> >> You need to use a localhost URL to ensure that the download doesn't have
> >> to go off of the node. That may mean placing something into the
> directory
> >> where Apache is serving files as part of the test setup.
> >>
> >> Doug
> >>
>
-- 
Thanks & Best Regards,

Abhishek Kekane
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Edge-computing] [tripleo][FEMDC] IEEE Fog Computing: Call for Contributions - Deadline Approaching

[Ildiko Vancsa]

> On 2018. Oct 31., at 19:11, Mike Bayer  wrote:
> 
> On Wed, Oct 31, 2018 at 10:57 AM Bogdan Dobrelya  wrote:
>> 
>> (cross-posting openstack-dev)
>> 
>> Hello.
>> [tl;dr] I'm looking for co-author(s) to come up with "Edge clouds data
>> consistency requirements and challenges" a position paper [0] (papers
>> submitting deadline is Nov 8).
>> 
>> The problem scope is synchronizing control plane and/or
>> deployments-specific data (not necessary limited to OpenStack) across
>> remote Edges and central Edge and management site(s). Including the same
>> aspects for overclouds and undercloud(s), in terms of TripleO; and other
>> deployment tools of your choice.
>> 
>> Another problem is to not go into different solutions for Edge
>> deployments management and control planes of edges. And for tenants as
>> well, if we think of tenants also doing Edge deployments based on Edge
>> Data Replication as a Service, say for Kubernetes/OpenShift on top of
>> OpenStack.
>> 
>> So the paper should name the outstanding problems, define data
>> consistency requirements and pose possible solutions for synchronization
>> and conflicts resolving. Having maximum autonomy cases supported for
>> isolated sites, with a capability to eventually catch up its distributed
>> state. Like global database [1], or something different perhaps (see
>> causal-real-time consistency model [2],[3]), or even using git. And
>> probably more than that?.. (looking for ideas)
> 
> 
> I can offer detail on whatever aspects of the "shared  / global
> database" idea.  The way we're doing it with Galera for now is all
> about something simple and modestly effective for the moment, but it
> doesn't have any of the hallmarks of a long-term, canonical solution,
> because Galera is not well suited towards being present on many
> (dozens) of endpoints. The concept that the StarlingX folks were
> talking about, that of independent databases that are synchronized
> using some kind of middleware is potentially more scalable, however I
> think the best approach would be API-level replication, that is, you
> have a bunch of Keystone services and there is a process that is
> regularly accessing the APIs of these keystone services and
> cross-publishing state amongst all of them.   Clearly the big
> challenge with that is how to resolve conflicts, I think the answer
> would lie in the fact that the data being replicated would be of
> limited scope and potentially consist of mostly or fully
> non-overlapping records.
> 
> That is, I think "global database" is a cheap way to get what would be
> more effective as asynchronous state synchronization between identity
> services.

Recently we’ve been also exploring federation with an IdP (Identity Provider) 
master: 
https://wiki.openstack.org/wiki/Keystone_edge_architectures#Identity_Provider_.28IdP.29_Master_with_shadow_users

One of the pros is that it removes the need for synchronization and potentially 
increases scalability.

Thanks,
Ildikó


> 
>> 
>> See also the "check" list in-line, which I think also meets the data
>> consistency topics well - it would be always nice to have some
>> theoretical foundations at hand, when repairing some
>> 1000-edges-spread-off and fully broken global database, by hand :)
>> 
>> PS. I must admit I have yet any experience with those IEEE et al
>> academic things and looking for someone who has it, to team and
>> co-author that positioning paper by. That's as a start, then we can
>> think of presenting it and expanding into work items for OpenStack Edge
>> WG and future development plans.
>> 
>> [0] http://conferences.computer.org/ICFC/2019/Paper_Submission.html
>> [1] https://review.openstack.org/600555
>> [2] https://jepsen.io/consistency
>> [3] http://www.cs.cornell.edu/lorenzo/papers/cac-tr.pdf
>> 
>> On 10/22/18 3:44 PM, Flavia Delicato wrote:
>>> =
>>> IEEE International Conference on Fog Computing (ICFC 2019)
>>> June 24-26, 2019
>>> Prague, Czech Republic
>>> http://conferences.computer.org/ICFC/2019/
>>> Co-located with the IEEE International Conference on Cloud Engineering
>>> (IC2E 2019)
>>> ==
>>> 
>>> Important Dates
>>> ---
>>> Paper registration and abstract: Nov 1st, 2018
>>> Full paper submission due: Nov 8th, 2018
>>> Notification of paper acceptance: Jan. 20th, 2019
>>> Workshop and tutorial proposals due: Nov 11, 2018
>>> Notification of proposal acceptance: Nov 18, 2018
>>> 
>>> Call for Contributions
>>> --
>>> Fog computing is the extension of cloud computing into its edge and
>>> the physical world to meet the data volume and decision velocity
>>> requirements in many emerging applications, such as augmented and
>>> virtual realities (AR/VR), cyber-physical systems (CPS), intelligent
>>> and autonomous systems, and mission-critical systems. The boundary
>>> betw

Re: [openstack-dev] [tripleo] Zuul Queue backlogs and resource usage

[Harald Jensås]

On Wed, 2018-10-31 at 11:35 -0600, Alex Schultz wrote:
> On Wed, Oct 31, 2018 at 11:16 AM Harald Jensås 
> wrote:
> > 
> > On Tue, 2018-10-30 at 15:00 -0600, Alex Schultz wrote:
> > > On Tue, Oct 30, 2018 at 12:25 PM Clark Boylan <
> > > cboy...@sapwetik.org>
> > > wrote:
> > > > 
> > > > On Tue, Oct 30, 2018, at 10:42 AM, Alex Schultz wrote:
> > > > > On Tue, Oct 30, 2018 at 11:36 AM Ben Nemec <
> > > > > openst...@nemebean.com> wrote:
> > > > > > 
> > > > > > Tagging with tripleo since my suggestion below is specific
> > > > > > to
> > > > > > that project.
> > > > > > 
> > > > > > On 10/30/18 11:03 AM, Clark Boylan wrote:
> > > > > > > Hello everyone,
> > > > > > > 
> > > > > > > A little while back I sent email explaining how the gate
> > > > > > > queues work and how fixing bugs helps us test and merge
> > > > > > > more
> > > > > > > code. All of this still is still true and we should keep
> > > > > > > pushing to improve our testing to avoid gate resets.
> > > > > > > 
> > > > > > > Last week we migrated Zuul and Nodepool to a new
> > > > > > > Zookeeper
> > > > > > > cluster. In the process of doing this we had to restart
> > > > > > > Zuul
> > > > > > > which brought in a new logging feature that exposes node
> > > > > > > resource usage by jobs. Using this data I've been able to
> > > > > > > generate some report information on where our node demand
> > > > > > > is
> > > > > > > going. This change [0] produces this report [1].
> > > > > > > 
> > > > > > > As with optimizing software we want to identify which
> > > > > > > changes
> > > > > > > will have the biggest impact and to be able to measure
> > > > > > > whether or not changes have had an impact once we have
> > > > > > > made
> > > > > > > them. Hopefully this information is a start at doing
> > > > > > > that.
> > > > > > > Currently we can only look back to the point Zuul was
> > > > > > > restarted, but we have a thirty day log rotation for this
> > > > > > > service and should be able to look at a month's worth of
> > > > > > > data
> > > > > > > going forward.
> > > > > > > 
> > > > > > > Looking at the data you might notice that Tripleo is
> > > > > > > using
> > > > > > > many more node resources than our other projects. They
> > > > > > > are
> > > > > > > aware of this and have a plan [2] to reduce their
> > > > > > > resource
> > > > > > > consumption. We'll likely be using this report generator
> > > > > > > to
> > > > > > > check progress of this plan over time.
> > > > > > 
> > > > > > I know at one point we had discussed reducing the
> > > > > > concurrency
> > > > > > of the
> > > > > > tripleo gate to help with this. Since tripleo is still
> > > > > > using
> > > > > > > 50% of the
> > > > > > 
> > > > > > resources it seems like maybe we should revisit that, at
> > > > > > least
> > > > > > for the
> > > > > > short-term until the more major changes can be made?
> > > > > > Looking
> > > > > > through the
> > > > > > merge history for tripleo projects I don't see a lot of
> > > > > > cases
> > > > > > (any, in
> > > > > > fact) where more than a dozen patches made it through
> > > > > > anyway*,
> > > > > > so I
> > > > > > suspect it wouldn't have a significant impact on gate
> > > > > > throughput, but it
> > > > > > would free up quite a few nodes for other uses.
> > > > > > 
> > > > > 
> > > > > It's the failures in gate and resets.  At this point I think
> > > > > it
> > > > > would
> > > > > be a good idea to turn down the concurrency of the tripleo
> > > > > queue
> > > > > in
> > > > > the gate if possible. As of late it's been timeouts but we've
> > > > > been
> > > > > unable to track down why it's timing out specifically.  I
> > > > > personally
> > > > > have a feeling it's the container download times since we do
> > > > > not
> > > > > have
> > > > > a local registry available and are only able to leverage the
> > > > > mirrors
> > > > > for some levels of caching. Unfortunately we don't get the
> > > > > best
> > > > > information about this out of docker (or the mirrors) and
> > > > > it's
> > > > > really
> > > > > hard to determine what exactly makes things run a bit slower.
> > > > 
> > > > We actually tried this not too long ago
> > > > 
https://git.openstack.org/cgit/openstack-infra/project-config/commit/?id=22d98f7aab0fb23849f715a8796384cffa84600b
> > > >  but decided to revert it because it didn't decrease the check
> > > > queue backlog significantly. We were still running at several
> > > > hours
> > > > behind most of the time.
> > > > 
> > > > If we want to set up better monitoring and measuring and try it
> > > > again we can do that. But we probably want to measure queue
> > > > sizes
> > > > with and without the change like that to better understand if
> > > > it
> > > > helps.
> > > > 
> > > > As for container image download times can we quantify that via
> > > > docker logs? Basically sum up the amount of time spent by a job
> > > > downloading images so that we can see what the impact is 

Re: [openstack-dev] [tripleo] reducing our upstream CI footprint

[Doug Hellmann]

Alex Schultz  writes:

> Hey everyone,
>
> Based on previous emails around this[0][1], I have proposed a possible
> reducing in our usage by switching the scenario001--011 jobs to
> non-voting and removing them from the gate[2]. This will reduce the
> likelihood of causing gate resets and hopefully allow us to land
> corrective patches sooner.  In terms of risks, there is a risk that we
> might introduce breaking changes in the scenarios because they are
> officially non-voting, and we will still be gating promotions on these
> scenarios.  This means that if they are broken, they will need the
> same attention and care to fix them so we should be vigilant when the
> jobs are failing.
>
> The hope is that we can switch these scenarios out with voting
> standalone versions in the next few weeks, but until that I think we
> should proceed by removing them from the gate.  I know this is less
> than ideal but as most failures with these jobs in the gate are either
> timeouts or unrelated to the changes (or gate queue), they are more of
> hindrance than a help at this point.
>
> Thanks,
> -Alex
>
> [0] 
> http://lists.openstack.org/pipermail/openstack-dev/2018-October/136141.html
> [1] 
> http://lists.openstack.org/pipermail/openstack-dev/2018-October/135396.html
> [2] 
> https://review.openstack.org/#/q/topic:reduce-tripleo-usage+(status:open+OR+status:merged)
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

This makes a lot of sense as a temporary measure. Thanks for continuing
to drive these changes!

Doug


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Edge-computing] [tripleo][FEMDC] IEEE Fog Computing: Call for Contributions - Deadline Approaching

[Ildiko Vancsa]

Hi,

Thank you for sharing your proposal.

I think this is a very interesting topic with a list of possible solutions some 
of which this group is also discussing. It would also be great to learn more 
about the IEEE activities and have experience about the process in this group 
on the way forward.

I personally do not have experience with IEEE conferences, but I’m happy to 
help with the paper if I can.

Thanks,
Ildikó


> On 2018. Oct 31., at 17:53, Bogdan Dobrelya  wrote:
> 
> I forgot to mention the submission registration and abstract has to be 
> submitted today. I've created it as #1570506394, and the paper itself can be 
> uploaded until the Nov 8 (or Nov 9 perhaps as the registration system shows 
> to me). I'm not sure that paper number is searchable publicly, so here is the 
> paper name and abstract for your kind review please:
> 
> name: "Edge clouds control plane and management data consistency challenges"
> abstract: "Fog computing is emerging Cloud of (Edge) Clouds technology. Its 
> control plane and deployments data synchronization is a major challenge. 
> Autonomy requirements expect even the most distant edge sites always 
> manageable, available for monitoring and alerting, scaling up/down, upgrading 
> and applying security fixes. Whenever temporary disconnected sites are 
> managed locally or centrally, some changes and data need to be eventually 
> synchronized back to the central site(s) with having its merge-conflicts 
> resolved for the central data hub(s). While some data needs to be pushed from 
> the central site(s) to the Edge, which might require resolving data 
> collisions at the remote sites as well. In this paper, we position the 
> outstanding data synchronization problems for OpenStack cloud platform 
> becoming a solution number one for fog computing. We outline the data 
> consistency requirements and design approaches to meet the AA (Always 
> Available) autonomy expectations. Finally, the paper brings the vision of 
> unified tooling, which solves the data synchronization problems the same way 
> for infrastructure owners, IaaS cloud operators and tenants running workloads 
> for PaaS like OpenShift or Kubernetes deployed on top of OpenStack. The 
> secondary goal of this work is to help cloud architects and developers to 
> federate stateful cloud components over reliable distributed data backends 
> and having its failure modes known."
> Thank you  for your time, if still reading this.
> 
> On 10/31/18 3:57 PM, Bogdan Dobrelya wrote:
>> (cross-posting openstack-dev)
>> Hello.
>> [tl;dr] I'm looking for co-author(s) to come up with "Edge clouds data 
>> consistency requirements and challenges" a position paper [0] (papers 
>> submitting deadline is Nov 8).
>> The problem scope is synchronizing control plane and/or deployments-specific 
>> data (not necessary limited to OpenStack) across remote Edges and central 
>> Edge and management site(s). Including the same aspects for overclouds and 
>> undercloud(s), in terms of TripleO; and other deployment tools of your 
>> choice.
>> Another problem is to not go into different solutions for Edge deployments 
>> management and control planes of edges. And for tenants as well, if we think 
>> of tenants also doing Edge deployments based on Edge Data Replication as a 
>> Service, say for Kubernetes/OpenShift on top of OpenStack.
>> So the paper should name the outstanding problems, define data consistency 
>> requirements and pose possible solutions for synchronization and conflicts 
>> resolving. Having maximum autonomy cases supported for isolated sites, with 
>> a capability to eventually catch up its distributed state. Like global 
>> database [1], or something different perhaps (see causal-real-time 
>> consistency model [2],[3]), or even using git. And probably more than 
>> that?.. (looking for ideas)
>> See also the "check" list in-line, which I think also meets the data 
>> consistency topics well - it would be always nice to have some theoretical 
>> foundations at hand, when repairing some 1000-edges-spread-off and fully 
>> broken global database, by hand :)
>> PS. I must admit I have yet any experience with those IEEE et al academic 
>> things and looking for someone who has it, to team and co-author that 
>> positioning paper by. That's as a start, then we can think of presenting it 
>> and expanding into work items for OpenStack Edge WG and future development 
>> plans.
>> [0] http://conferences.computer.org/ICFC/2019/Paper_Submission.html
>> [1] https://review.openstack.org/600555
>> [2] https://jepsen.io/consistency
>> [3] http://www.cs.cornell.edu/lorenzo/papers/cac-tr.pdf
>> On 10/22/18 3:44 PM, Flavia Delicato wrote:
>>> =
>>>  
>>> IEEE International Conference on Fog Computing (ICFC 2019)
>>> June 24-26, 2019
>>> Prague, Czech Republic
>>> http://conferences.computer.org/ICFC/2019/
>>> Co-located with the IEEE Internationa

Re: [openstack-dev] [tripleo] reducing our upstream CI footprint

[Wesley Hayutin]

On Wed, Oct 31, 2018 at 11:21 AM Alex Schultz  wrote:

> Hey everyone,
>
> Based on previous emails around this[0][1], I have proposed a possible
> reducing in our usage by switching the scenario001--011 jobs to
> non-voting and removing them from the gate[2]. This will reduce the
> likelihood of causing gate resets and hopefully allow us to land
> corrective patches sooner.  In terms of risks, there is a risk that we
> might introduce breaking changes in the scenarios because they are
> officially non-voting, and we will still be gating promotions on these
> scenarios.  This means that if they are broken, they will need the
> same attention and care to fix them so we should be vigilant when the
> jobs are failing.
>
> The hope is that we can switch these scenarios out with voting
> standalone versions in the next few weeks, but until that I think we
> should proceed by removing them from the gate.  I know this is less
> than ideal but as most failures with these jobs in the gate are either
> timeouts or unrelated to the changes (or gate queue), they are more of
> hindrance than a help at this point.
>
> Thanks,
> -Alex
>

I think I also have to agree.
Having to deploy with containers, update containers and run with two nodes
is no longer a very viable option upstream.  It's not impossible but it
should be the exception and not the rule for all our jobs.

Thanks Alex



>
> [0]
> http://lists.openstack.org/pipermail/openstack-dev/2018-October/136141.html
> [1]
> http://lists.openstack.org/pipermail/openstack-dev/2018-October/135396.html
> [2]
> https://review.openstack.org/#/q/topic:reduce-tripleo-usage+(status:open+OR+status:merged)
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
-- 

Wes Hayutin

Associate MANAGER

Red Hat



whayu...@redhat.comT: +1919 <+19197544114>4232509 IRC:  weshay


View my calendar and check my availability for meetings HERE

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] Zuul Queue backlogs and resource usage

[Alex Schultz]

On Wed, Oct 31, 2018 at 11:16 AM Harald Jensås  wrote:
>
> On Tue, 2018-10-30 at 15:00 -0600, Alex Schultz wrote:
> > On Tue, Oct 30, 2018 at 12:25 PM Clark Boylan 
> > wrote:
> > >
> > > On Tue, Oct 30, 2018, at 10:42 AM, Alex Schultz wrote:
> > > > On Tue, Oct 30, 2018 at 11:36 AM Ben Nemec <
> > > > openst...@nemebean.com> wrote:
> > > > >
> > > > > Tagging with tripleo since my suggestion below is specific to
> > > > > that project.
> > > > >
> > > > > On 10/30/18 11:03 AM, Clark Boylan wrote:
> > > > > > Hello everyone,
> > > > > >
> > > > > > A little while back I sent email explaining how the gate
> > > > > > queues work and how fixing bugs helps us test and merge more
> > > > > > code. All of this still is still true and we should keep
> > > > > > pushing to improve our testing to avoid gate resets.
> > > > > >
> > > > > > Last week we migrated Zuul and Nodepool to a new Zookeeper
> > > > > > cluster. In the process of doing this we had to restart Zuul
> > > > > > which brought in a new logging feature that exposes node
> > > > > > resource usage by jobs. Using this data I've been able to
> > > > > > generate some report information on where our node demand is
> > > > > > going. This change [0] produces this report [1].
> > > > > >
> > > > > > As with optimizing software we want to identify which changes
> > > > > > will have the biggest impact and to be able to measure
> > > > > > whether or not changes have had an impact once we have made
> > > > > > them. Hopefully this information is a start at doing that.
> > > > > > Currently we can only look back to the point Zuul was
> > > > > > restarted, but we have a thirty day log rotation for this
> > > > > > service and should be able to look at a month's worth of data
> > > > > > going forward.
> > > > > >
> > > > > > Looking at the data you might notice that Tripleo is using
> > > > > > many more node resources than our other projects. They are
> > > > > > aware of this and have a plan [2] to reduce their resource
> > > > > > consumption. We'll likely be using this report generator to
> > > > > > check progress of this plan over time.
> > > > >
> > > > > I know at one point we had discussed reducing the concurrency
> > > > > of the
> > > > > tripleo gate to help with this. Since tripleo is still using
> > > > > >50% of the
> > > > > resources it seems like maybe we should revisit that, at least
> > > > > for the
> > > > > short-term until the more major changes can be made? Looking
> > > > > through the
> > > > > merge history for tripleo projects I don't see a lot of cases
> > > > > (any, in
> > > > > fact) where more than a dozen patches made it through anyway*,
> > > > > so I
> > > > > suspect it wouldn't have a significant impact on gate
> > > > > throughput, but it
> > > > > would free up quite a few nodes for other uses.
> > > > >
> > > >
> > > > It's the failures in gate and resets.  At this point I think it
> > > > would
> > > > be a good idea to turn down the concurrency of the tripleo queue
> > > > in
> > > > the gate if possible. As of late it's been timeouts but we've
> > > > been
> > > > unable to track down why it's timing out specifically.  I
> > > > personally
> > > > have a feeling it's the container download times since we do not
> > > > have
> > > > a local registry available and are only able to leverage the
> > > > mirrors
> > > > for some levels of caching. Unfortunately we don't get the best
> > > > information about this out of docker (or the mirrors) and it's
> > > > really
> > > > hard to determine what exactly makes things run a bit slower.
> > >
> > > We actually tried this not too long ago
> > > https://git.openstack.org/cgit/openstack-infra/project-config/commit/?id=22d98f7aab0fb23849f715a8796384cffa84600b
> > >  but decided to revert it because it didn't decrease the check
> > > queue backlog significantly. We were still running at several hours
> > > behind most of the time.
> > >
> > > If we want to set up better monitoring and measuring and try it
> > > again we can do that. But we probably want to measure queue sizes
> > > with and without the change like that to better understand if it
> > > helps.
> > >
> > > As for container image download times can we quantify that via
> > > docker logs? Basically sum up the amount of time spent by a job
> > > downloading images so that we can see what the impact is but also
> > > measure if changes improve that? As for other ideas improving
> > > things seems like many of the images that tripleo use are quite
> > > large. I recall seeing a > 600MB image just for rsyslog. Wouldn't
> > > it be advantageous for both the gate and tripleo in the real world
> > > to trim the size of those images (which should improve download
> > > times). In any case quantifying the size of the downloads and
> > > trimming those if possible is likely also worthwhile.
> > >
> >
> > So it's not that simple as we don't just download all the images in a
> > distinct task and there isn't any information

[openstack-dev] [tripleo] reducing our upstream CI footprint

[Alex Schultz]

Hey everyone,

Based on previous emails around this[0][1], I have proposed a possible
reducing in our usage by switching the scenario001--011 jobs to
non-voting and removing them from the gate[2]. This will reduce the
likelihood of causing gate resets and hopefully allow us to land
corrective patches sooner.  In terms of risks, there is a risk that we
might introduce breaking changes in the scenarios because they are
officially non-voting, and we will still be gating promotions on these
scenarios.  This means that if they are broken, they will need the
same attention and care to fix them so we should be vigilant when the
jobs are failing.

The hope is that we can switch these scenarios out with voting
standalone versions in the next few weeks, but until that I think we
should proceed by removing them from the gate.  I know this is less
than ideal but as most failures with these jobs in the gate are either
timeouts or unrelated to the changes (or gate queue), they are more of
hindrance than a help at this point.

Thanks,
-Alex

[0] http://lists.openstack.org/pipermail/openstack-dev/2018-October/136141.html
[1] http://lists.openstack.org/pipermail/openstack-dev/2018-October/135396.html
[2] 
https://review.openstack.org/#/q/topic:reduce-tripleo-usage+(status:open+OR+status:merged)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] Zuul Queue backlogs and resource usage

[Harald Jensås]

On Tue, 2018-10-30 at 15:00 -0600, Alex Schultz wrote:
> On Tue, Oct 30, 2018 at 12:25 PM Clark Boylan 
> wrote:
> > 
> > On Tue, Oct 30, 2018, at 10:42 AM, Alex Schultz wrote:
> > > On Tue, Oct 30, 2018 at 11:36 AM Ben Nemec <
> > > openst...@nemebean.com> wrote:
> > > > 
> > > > Tagging with tripleo since my suggestion below is specific to
> > > > that project.
> > > > 
> > > > On 10/30/18 11:03 AM, Clark Boylan wrote:
> > > > > Hello everyone,
> > > > > 
> > > > > A little while back I sent email explaining how the gate
> > > > > queues work and how fixing bugs helps us test and merge more
> > > > > code. All of this still is still true and we should keep
> > > > > pushing to improve our testing to avoid gate resets.
> > > > > 
> > > > > Last week we migrated Zuul and Nodepool to a new Zookeeper
> > > > > cluster. In the process of doing this we had to restart Zuul
> > > > > which brought in a new logging feature that exposes node
> > > > > resource usage by jobs. Using this data I've been able to
> > > > > generate some report information on where our node demand is
> > > > > going. This change [0] produces this report [1].
> > > > > 
> > > > > As with optimizing software we want to identify which changes
> > > > > will have the biggest impact and to be able to measure
> > > > > whether or not changes have had an impact once we have made
> > > > > them. Hopefully this information is a start at doing that.
> > > > > Currently we can only look back to the point Zuul was
> > > > > restarted, but we have a thirty day log rotation for this
> > > > > service and should be able to look at a month's worth of data
> > > > > going forward.
> > > > > 
> > > > > Looking at the data you might notice that Tripleo is using
> > > > > many more node resources than our other projects. They are
> > > > > aware of this and have a plan [2] to reduce their resource
> > > > > consumption. We'll likely be using this report generator to
> > > > > check progress of this plan over time.
> > > > 
> > > > I know at one point we had discussed reducing the concurrency
> > > > of the
> > > > tripleo gate to help with this. Since tripleo is still using
> > > > >50% of the
> > > > resources it seems like maybe we should revisit that, at least
> > > > for the
> > > > short-term until the more major changes can be made? Looking
> > > > through the
> > > > merge history for tripleo projects I don't see a lot of cases
> > > > (any, in
> > > > fact) where more than a dozen patches made it through anyway*,
> > > > so I
> > > > suspect it wouldn't have a significant impact on gate
> > > > throughput, but it
> > > > would free up quite a few nodes for other uses.
> > > > 
> > > 
> > > It's the failures in gate and resets.  At this point I think it
> > > would
> > > be a good idea to turn down the concurrency of the tripleo queue
> > > in
> > > the gate if possible. As of late it's been timeouts but we've
> > > been
> > > unable to track down why it's timing out specifically.  I
> > > personally
> > > have a feeling it's the container download times since we do not
> > > have
> > > a local registry available and are only able to leverage the
> > > mirrors
> > > for some levels of caching. Unfortunately we don't get the best
> > > information about this out of docker (or the mirrors) and it's
> > > really
> > > hard to determine what exactly makes things run a bit slower.
> > 
> > We actually tried this not too long ago 
> > https://git.openstack.org/cgit/openstack-infra/project-config/commit/?id=22d98f7aab0fb23849f715a8796384cffa84600b
> >  but decided to revert it because it didn't decrease the check
> > queue backlog significantly. We were still running at several hours
> > behind most of the time.
> > 
> > If we want to set up better monitoring and measuring and try it
> > again we can do that. But we probably want to measure queue sizes
> > with and without the change like that to better understand if it
> > helps.
> > 
> > As for container image download times can we quantify that via
> > docker logs? Basically sum up the amount of time spent by a job
> > downloading images so that we can see what the impact is but also
> > measure if changes improve that? As for other ideas improving
> > things seems like many of the images that tripleo use are quite
> > large. I recall seeing a > 600MB image just for rsyslog. Wouldn't
> > it be advantageous for both the gate and tripleo in the real world
> > to trim the size of those images (which should improve download
> > times). In any case quantifying the size of the downloads and
> > trimming those if possible is likely also worthwhile.
> > 
> 
> So it's not that simple as we don't just download all the images in a
> distinct task and there isn't any information provided around
> size/speed AFAIK.  Additionally we aren't doing anything special with
> the images (it's mostly kolla built containers with a handful of
> tweaks) so that's just the size of the containers.  I am currently
> working on red

Re: [openstack-dev] [tripleo] request for feedback/review on docker2podman upgrade

[Sofer Athlan-Guyot]

Emilien Macchi  writes:

> A bit of an update here:
>
> - We merged the patch in openstack/paunch that stop the Docker container if
> we try to start a Podman container.
> - We switched the undercloud upgrade job to test upgrades from Docker to
> Podman (for now containers are stopped in Docker and then started in
> Podman).
> - We are now looking how and where to remove the Docker containers once the
> upgrade finished. For that work, I started with the Undercloud and patched
> tripleoclient to run the post_upgrade_tasks which to me is a good candidate
> to run docker rm.

+1

>
> Please look:
> - tripleoclient / run post_upgrade_tasks when upgrading
> standalone/undercloud: https://review.openstack.org/614349
> - THT: prototype on how we would remove the Docker containers:
> https://review.openstack.org/611092
>

reviewed.

> Note: for now we assume that Docker is still available on the host after
> the upgrade as we are testing things under centos7. I'm aware that this
> assumption can change in the future but we'll probably re-iterate.
>
> What I need from the upgrade team is feedback on this workflow, and see if
> we can re-use these bits originally tested on Undercloud / Standalone, for
> the Overcloud as well.
>

So that workflow won't break in any case for the overcloud.  For an
inplace upgrade then we need that clean up anyway given how paunch
detect the need for an upgrade. For other upgrade scenario this won't do
anything bad.

So +1 for me.

> Thanks for the feedback,
>
>
> On Fri, Oct 19, 2018 at 8:00 AM Emilien Macchi  wrote:
>
>> On Fri, Oct 19, 2018 at 4:24 AM Giulio Fidente 
>> wrote:
>>
>>> 1) create the podman systemd unit
>>> 2) delete the docker container
>>>
>>
>> We finally went with "stop the docker container"
>>
>> 3) start the podman container
>>>
>>
>> and 4) delete the docker container later in THT upgrade_tasks.
>>
>> And yes +1 to do the same in ceph-ansible if possible.
>> --
>> Emilien Macchi
>>
>
>
> -- 
> Emilien Macchi
-- 
Sofer Athlan-Guyot

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Edge-computing][tripleo][FEMDC] IEEE Fog Computing: Call for Contributions - Deadline Approaching

[Bogdan Dobrelya]

I forgot to mention the submission registration and abstract has to be 
submitted today. I've created it as #1570506394, and the paper itself 
can be uploaded until the Nov 8 (or Nov 9 perhaps as the registration 
system shows to me). I'm not sure that paper number is searchable 
publicly, so here is the paper name and abstract for your kind review 
please:


name: "Edge clouds control plane and management data consistency challenges"
abstract: "Fog computing is emerging Cloud of (Edge) Clouds technology. 
Its control plane and deployments data synchronization is a major 
challenge. Autonomy requirements expect even the most distant edge sites 
always manageable, available for monitoring and alerting, scaling 
up/down, upgrading and applying security fixes. Whenever temporary 
disconnected sites are managed locally or centrally, some changes and 
data need to be eventually synchronized back to the central site(s) with 
having its merge-conflicts resolved for the central data hub(s). While 
some data needs to be pushed from the central site(s) to the Edge, which 
might require resolving data collisions at the remote sites as well. In 
this paper, we position the outstanding data synchronization problems 
for OpenStack cloud platform becoming a solution number one for fog 
computing. We outline the data consistency requirements and design 
approaches to meet the AA (Always Available) autonomy expectations. 
Finally, the paper brings the vision of unified tooling, which solves 
the data synchronization problems the same way for infrastructure 
owners, IaaS cloud operators and tenants running workloads for PaaS like 
OpenShift or Kubernetes deployed on top of OpenStack. The secondary goal 
of this work is to help cloud architects and developers to federate 
stateful cloud components over reliable distributed data backends and 
having its failure modes known."

Thank you  for your time, if still reading this.

On 10/31/18 3:57 PM, Bogdan Dobrelya wrote:

(cross-posting openstack-dev)

Hello.
[tl;dr] I'm looking for co-author(s) to come up with "Edge clouds data 
consistency requirements and challenges" a position paper [0] (papers 
submitting deadline is Nov 8).


The problem scope is synchronizing control plane and/or 
deployments-specific data (not necessary limited to OpenStack) across 
remote Edges and central Edge and management site(s). Including the same 
aspects for overclouds and undercloud(s), in terms of TripleO; and other 
deployment tools of your choice.


Another problem is to not go into different solutions for Edge 
deployments management and control planes of edges. And for tenants as 
well, if we think of tenants also doing Edge deployments based on Edge 
Data Replication as a Service, say for Kubernetes/OpenShift on top of 
OpenStack.


So the paper should name the outstanding problems, define data 
consistency requirements and pose possible solutions for synchronization 
and conflicts resolving. Having maximum autonomy cases supported for 
isolated sites, with a capability to eventually catch up its distributed 
state. Like global database [1], or something different perhaps (see 
causal-real-time consistency model [2],[3]), or even using git. And 
probably more than that?.. (looking for ideas)


See also the "check" list in-line, which I think also meets the data 
consistency topics well - it would be always nice to have some 
theoretical foundations at hand, when repairing some 
1000-edges-spread-off and fully broken global database, by hand :)


PS. I must admit I have yet any experience with those IEEE et al 
academic things and looking for someone who has it, to team and 
co-author that positioning paper by. That's as a start, then we can 
think of presenting it and expanding into work items for OpenStack Edge 
WG and future development plans.


[0] http://conferences.computer.org/ICFC/2019/Paper_Submission.html
[1] https://review.openstack.org/600555
[2] https://jepsen.io/consistency
[3] http://www.cs.cornell.edu/lorenzo/papers/cac-tr.pdf

On 10/22/18 3:44 PM, Flavia Delicato wrote:
= 


IEEE International Conference on Fog Computing (ICFC 2019)
June 24-26, 2019
Prague, Czech Republic
http://conferences.computer.org/ICFC/2019/
Co-located with the IEEE International Conference on Cloud Engineering
(IC2E 2019)
== 



Important Dates
---
Paper registration and abstract: Nov 1st, 2018
Full paper submission due: Nov 8th, 2018
Notification of paper acceptance: Jan. 20th, 2019
Workshop and tutorial proposals due: Nov 11, 2018
Notification of proposal acceptance: Nov 18, 2018

Call for Contributions
--
Fog computing is the extension of cloud computing into its edge and
the physical world to meet the data volume and decision velocity
requirements in many emerging applications, such as augmented and
virtual

[openstack-dev] [tripleo] gate issues please do not approve/recheck

[Alex Schultz]

Hey folks,

So we have identified an issue that has been causing a bunch of
failures and proposed a revert of our podman testing[0].  We have
cleared the gate and are asking that you not approve or recheck any
patches at this time.  We will let you know when it is safe to start
approving things.

Thanks,
-Alex

[0] https://review.openstack.org/#/c/614537/

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Ironic integration CI jobs

[Julia Kreger]

On Wed, Oct 31, 2018 at 7:38 AM Dmitry Tantsur  wrote:


> [trim]
>


> > Ditto, grenade jobs do not cover our tests at all. Also this is the
> very job we
> > run on other projects (nova, neutron, maybe more), so it will be a
> bit painful
> > to remove it.
> >
> >
> > We run the basic baremetal ops test, which tests deploy. If we're
> already
> > covering the same code paths in other tests (which I feel we are), then
> the test
> > feels redundant to me. I'm not worried about the effort to change the
> job in
> > other gates. We really need to pull agent_ipmitool out of the name if we
> keep it
> > anyway... which still means going through zuul configs.
>
> Do not smoke tests cover rescue with bare metal? Because our jobs do.
>
>
Smoke tests do not as far as I can tell, but I believe we run rescue by
default when our test scenarios execute on our other tempest executing jobs
as well as it is a superset of the main scenario.
Random example testr results:
http://logs.openstack.org/72/614472/1/check/ironic-tempest-dsvm-ipa-partition-redfish-tinyipa/7537b02/testr_results.html.gz
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Edge-computing][tripleo][FEMDC] IEEE Fog Computing: Call for Contributions - Deadline Approaching

[Bogdan Dobrelya]

(cross-posting openstack-dev)

Hello.
[tl;dr] I'm looking for co-author(s) to come up with "Edge clouds data 
consistency requirements and challenges" a position paper [0] (papers 
submitting deadline is Nov 8).


The problem scope is synchronizing control plane and/or 
deployments-specific data (not necessary limited to OpenStack) across 
remote Edges and central Edge and management site(s). Including the same 
aspects for overclouds and undercloud(s), in terms of TripleO; and other 
deployment tools of your choice.


Another problem is to not go into different solutions for Edge 
deployments management and control planes of edges. And for tenants as 
well, if we think of tenants also doing Edge deployments based on Edge 
Data Replication as a Service, say for Kubernetes/OpenShift on top of 
OpenStack.


So the paper should name the outstanding problems, define data 
consistency requirements and pose possible solutions for synchronization 
and conflicts resolving. Having maximum autonomy cases supported for 
isolated sites, with a capability to eventually catch up its distributed 
state. Like global database [1], or something different perhaps (see 
causal-real-time consistency model [2],[3]), or even using git. And 
probably more than that?.. (looking for ideas)


See also the "check" list in-line, which I think also meets the data 
consistency topics well - it would be always nice to have some 
theoretical foundations at hand, when repairing some 
1000-edges-spread-off and fully broken global database, by hand :)


PS. I must admit I have yet any experience with those IEEE et al 
academic things and looking for someone who has it, to team and 
co-author that positioning paper by. That's as a start, then we can 
think of presenting it and expanding into work items for OpenStack Edge 
WG and future development plans.


[0] http://conferences.computer.org/ICFC/2019/Paper_Submission.html
[1] https://review.openstack.org/600555
[2] https://jepsen.io/consistency
[3] http://www.cs.cornell.edu/lorenzo/papers/cac-tr.pdf

On 10/22/18 3:44 PM, Flavia Delicato wrote:

=
IEEE International Conference on Fog Computing (ICFC 2019)
June 24-26, 2019
Prague, Czech Republic
http://conferences.computer.org/ICFC/2019/
Co-located with the IEEE International Conference on Cloud Engineering
(IC2E 2019)
==

Important Dates
---
Paper registration and abstract: Nov 1st, 2018
Full paper submission due: Nov 8th, 2018
Notification of paper acceptance: Jan. 20th, 2019
Workshop and tutorial proposals due: Nov 11, 2018
Notification of proposal acceptance: Nov 18, 2018

Call for Contributions
--
Fog computing is the extension of cloud computing into its edge and
the physical world to meet the data volume and decision velocity
requirements in many emerging applications, such as augmented and
virtual realities (AR/VR), cyber-physical systems (CPS), intelligent
and autonomous systems, and mission-critical systems. The boundary
between centralized, powerful computing cloud and massively
distributed, Internet connected sensors, actuators, and things is
blurred in this new computing paradigm.

The ICFC 2019 technical program will feature tutorials, workshops, and
research paper sessions. We solicit high-quality contributions in the
above categories. Details of submission is available on the conference
Web site. Topics of interest include but are not limited to:

* System architecture for fog computing


(check)


* Coordination between cloud, fog, and sensing/actuation endpoints
* Connectivity, storage, and computation in the edge
* Data processing and management for fog computing


(check)


* Efficient and embedded AI in the fog
* System and network manageability
* Middleware and coordination platforms
* Power, energy, and resource management
* Device and hardware support for fog computing
* Programming models, abstractions, and software engineering for fog computing


(check)


* Security, privacy, and ethics issues related to fog computing
* Theoretical foundations and formal methods for fog computing systems


(check)


* Applications and experiences

Organizing Committee

General Chairs:
Hui Lei, IBM
Albert Zomaya, The University of Sydney

PC Co-chairs:
Erol Gelenbe, Imperial College London
Jie Liu, Microsoft Research

Tutorials and Workshops Chair:
David Bermbach, TU Berlin

Publicity Co-chairs:
Flavia Delicato,Federal University of Rio de Janeiro
Mathias Fischer, University Hamburg

Publication Chair
Javid Taheri, Karlstad University

Webmaster
Wei Li, The University of Sydney

Steering Committee
--
Mung Chiang, Purdue University
Erol Gelenbe, Imperial College London
Christos Kozarakis, Stanford University
Hui Lei, IBM
Chenyang Lu, Washington University in St Louis
Beng Chin Ooi, National University of Singap

Re: [openstack-dev] Ironic integration CI jobs

[Dmitry Tantsur]

On 10/31/18 2:57 PM, Julia Kreger wrote:



On Wed, Oct 31, 2018 at 5:44 AM Dmitry Tantsur > wrote:


Hi,

On 10/31/18 1:36 AM, Julia Kreger wrote:
[trim]
 >
 > ironic-tempest-dsvm-ipa-wholedisk-agent_ipmitool-tinyipa-multinode - This
job is
 > essentially the same as our grenade mutlinode job, the only difference 
being
 > grenade.

Nope, not the same. Grenade jobs run only smoke tests, this job runs

https://github.com/openstack/ironic-tempest-plugin/blob/master/ironic_tempest_plugin/tests/scenario/test_baremetal_multitenancy.py


Ugh, Looking closer, we still end up deploying when the smoke tests run. It 
feels like the only real difference between what is being exercised is that one 
our explicit test scenario of putting two instances on two separate networks and 
validating connectivity is not present between the two. I guess I'm failing to 
see why we need all of the setup and infrastructure when we're just testing 
pluggable network bits and settings their upon. Maybe it is a good cantidate for 
looking at evolving how we handle scenario testing so we reduce our gate load 
and resulting wait for test results.


 > ironic-tempest-dsvm-ipa-wholedisk-bios-agent_ipmitool-tinyipa - This job
 > essentially just duplicates the functionality already covered in other 
jobs,
 > including the grenade job.

Ditto, grenade jobs do not cover our tests at all. Also this is the very 
job we
run on other projects (nova, neutron, maybe more), so it will be a bit 
painful
to remove it.


We run the basic baremetal ops test, which tests deploy. If we're already 
covering the same code paths in other tests (which I feel we are), then the test 
feels redundant to me. I'm not worried about the effort to change the job in 
other gates. We really need to pull agent_ipmitool out of the name if we keep it 
anyway... which still means going through zuul configs.


Do not smoke tests cover rescue with bare metal? Because our jobs do.



[trim]


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Sharing upstream contribution mentoring result with Korea user group

[Ghanshyam Mann]

That's great job Ian and team. It is really great when local user groups doing 
so much effort for upstream contribution mentoring. 

From FirstContact SIG point of view, feel free to let us know any help you need 
in term of engaging new contributors with their interesting projects team and 
working items.

-gmann


  On Tue, 30 Oct 2018 23:10:42 +0900 Ian Y. Choi  
wrote  
 > Hello,
 > 
 > I got involved organizing & mentoring Korean people for OpenStack 
 > upstream contribution for about last two months,
 > and would like to share with community members.
 > 
 > Total nine mentees had started to learn OpenStack, contributed, and 
 > finally survived as volunteers for
 >   1) developing OpenStack mobile app for better mobile user interfaces 
 > and experiences
 >  (inspired from https://github.com/stackerz/app which worked on Juno 
 > release), and
 >   2) translating OpenStack official project artifacts including documents,
 >   and Container Whitepaper ( 
 > https://www.openstack.org/containers/leveraging-containers-and-openstack/ ).
 > 
 > Korea user group organizers (Seongsoo Cho, Taehee Jang, Hocheol Shin, 
 > Sungjin Kang, and Andrew Yongjoon Kong)
 > all helped to organize total 8 offline meetups + one mini-hackathon and 
 > mentored to attendees.
 > 
 > The followings are brief summary:
 >   - "OpenStack Controller" Android app is available on Play Store
 >: 
 > https://play.google.com/store/apps/details?id=openstack.contributhon.com.openstackcontroller
 > (GitHub: https://github.com/kosslab-kr/openstack-controller )
 > 
 >   - Most high-priority projects (although it is not during string freeze 
 > period) and documents are
 > 100% translated into Korean: Horizon, OpenStack-Helm, I18n Guide, 
 > and Container Whitepaper.
 > 
 >   - Total 18,695 words were translated into Korean by four contributors
 >(confirmed through Zanata API: 
 > https://translate.openstack.org/rest/stats/user/[Zanata 
 > ID]/2018-08-16..2018-10-25 ):
 > 
 > ++---+-+
 > | Zanata ID  | Name  | Number of words |
 > ++---+-+
 > | ardentpark | Soonyeul Park | 12517   |
 > ++---+-+
 > | bnitech| Dongbim Im| 693 |
 > ++---+-+
 > | csucom | Sungwook Choi | 4397|
 > ++---+-+
 > | jaeho93| Jaeho Cho | 1088|
 > ++---+-+
 > 
 >   - The list of projects translated into Korean are described as:
 > 
 > +-+-+
 > | Project | Number of words |
 > +-+-+
 > | api-site| 20  |
 > +-+-+
 > | cinder  | 405 |
 > +-+-+
 > | designate-dashboard | 4   |
 > +-+-+
 > | horizon | 3226|
 > +-+-+
 > | i18n| 434 |
 > +-+-+
 > | ironic  | 4   |
 > +-+-+
 > | Leveraging Containers and OpenStack | 5480|
 > +-+-+
 > | neutron-lbaas-dashboard | 5   |
 > +-+-+
 > | openstack-helm  | 8835|
 > +-+-+
 > | trove-dashboard | 89  |
 > +-+-+
 > | zun-ui  | 193 |
 > +-+-+
 > 
 > I would like to really appreciate all co-mentors and participants on 
 > such a big event for promoting OpenStack contribution.
 > The venue and food were supported by Korea Open Source Software 
 > Development Center ( https://kosslab.kr/ ).
 > 
 > 
 > With many thanks,
 > 
 > /Ian
 > 
 > __
 > OpenStack Development Mailing List (not for usage questions)
 > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 > 



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openst

Re: [openstack-dev] Ironic integration CI jobs

[Julia Kreger]

On Wed, Oct 31, 2018 at 5:44 AM Dmitry Tantsur  wrote:

> Hi,
>
> On 10/31/18 1:36 AM, Julia Kreger wrote:
> [trim]
> >
> > ironic-tempest-dsvm-ipa-wholedisk-agent_ipmitool-tinyipa-multinode -
> This job is
> > essentially the same as our grenade mutlinode job, the only difference
> being
> > grenade.
>
> Nope, not the same. Grenade jobs run only smoke tests, this job runs
>
> https://github.com/openstack/ironic-tempest-plugin/blob/master/ironic_tempest_plugin/tests/scenario/test_baremetal_multitenancy.py
>

Ugh, Looking closer, we still end up deploying when the smoke tests run. It
feels like the only real difference between what is being exercised is that
one our explicit test scenario of putting two instances on two separate
networks and validating connectivity is not present between the two. I
guess I'm failing to see why we need all of the setup and infrastructure
when we're just testing pluggable network bits and settings their upon.
Maybe it is a good cantidate for looking at evolving how we handle scenario
testing so we reduce our gate load and resulting wait for test results.

> ironic-tempest-dsvm-ipa-wholedisk-bios-agent_ipmitool-tinyipa - This job
> > essentially just duplicates the functionality already covered in other
> jobs,
> > including the grenade job.
>
> Ditto, grenade jobs do not cover our tests at all. Also this is the very
> job we
> run on other projects (nova, neutron, maybe more), so it will be a bit
> painful
> to remove it.
>

We run the basic baremetal ops test, which tests deploy. If we're already
covering the same code paths in other tests (which I feel we are), then the
test feels redundant to me. I'm not worried about the effort to change the
job in other gates. We really need to pull agent_ipmitool out of the name
if we keep it anyway... which still means going through zuul configs.

> [trim]
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [NOVA] nova GPU suppot and find GPU type

[Sylvain Bauza]

On Tue, Oct 30, 2018 at 12:21 AM Manuel Sopena Ballesteros <
manuel...@garvan.org.au> wrote:

> Dear Nova community,
>
>
>
> This is the first time I work with GPUs.
>
>
>
> I have a Dell C4140 with x4 Nvidia Tesla V100 SXM2 16GB I would like to
> setup on Openstack Rocky.
>
>
>
> I checked the documentation and I have 2 questions I would like to ask:
>
>
>
> 1.   Docs (1) says *As of the Queens release, there is no upstream
> continuous integration testing with a hardware environment that has virtual
> GPUs and therefore this feature is considered experimental*. Does it
> means nova will stop supporting GPUs? Is GPU support being transferred to a
> different project?
>

 No. We told about "experimental" because given we didn't have a CI
verifying the features, we were not sure operators were not having a lot of
bugs. After 2 cycles, we don't have a lot of bugs and some operators use
it, so we could remove the "experimental" situation.

2.   I installed
> cuda-repo-rhel7-10-0-local-10.0.130-410.48-1.0-1.x86_64 on the physical
> host but I can’t find the type of GPUs installed (2) (/sys/class/mdev_bus
> doesn’t exists). What should I do then? What should I put in
> devices.enabled_vgpu_types
> 
> ?
>
>
Make sure you use the correct grid server driver from nvidia and then
follow those steps :
https://docs.nvidia.com/grid/6.0/grid-vgpu-user-guide/index.html#install-vgpu-package-generic-linux-kvm

Once the driver will be installed (make sure tho you remove first the
nouveau driver as said above) and the system rebooted, you should see the
above sysfs directory.
HTH,
-Sylvain


>
> (1) - https://docs.openstack.org/nova/rocky/admin/virtual-gpu.html
>
> (2)-
> https://docs.openstack.org/nova/rocky/admin/virtual-gpu.html#how-to-discover-a-gpu-type
>
>
>
> Thank you very much
>
>
>
> *Manuel Sopena Ballesteros *| Big data Engineer
> *Garvan Institute of Medical Research *
> The Kinghorn Cancer Centre, 370 Victoria Street, Darlinghurst, NSW 2010
> *T:* + 61 (0)2 9355 5760 | *F:* +61 (0)2 9295 8507 | *E:*
> manuel...@garvan.org.au
>
>
> NOTICE
> Please consider the environment before printing this email. This message
> and any attachments are intended for the addressee named and may contain
> legally privileged/confidential/copyright information. If you are not the
> intended recipient, you should not read, use, disclose, copy or distribute
> this communication. If you have received this message in error please
> notify us at once by return email and then delete both messages. We accept
> no liability for the distribution of viruses or similar in electronic
> communications. This notice should not be removed.
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [oslo] No complains about rabbitmq SSL problems: could we have this in the logs?

[Mohammed Naser]

For what it’s worth: I ran into the same issue.  I think the problem lies a bit 
deeper because it’s a problem with kombu as when debugging I saw that Oslo 
messaging tried to connect and hung after. 

Sent from my iPhone

> On Oct 31, 2018, at 2:29 PM, Thomas Goirand  wrote:
> 
> Hi,
> 
> It took me a long long time to figure out that my SSL setup was wrong
> when trying to connect Heat to rabbitmq over SSL. Unfortunately, Oslo
> (or heat itself) never warn me that something was wrong, I just got
> nothing working, and no log at all.
> 
> I'm sure I wouldn't be the only one happy about having this type of
> problems being yelled out loud in the logs. Right now, it does work if I
> turn off SSL, though I'm still not sure what's wrong in my setup, and
> I'm given no clue if the issue is on rabbitmq-server or on the client
> side (ie: heat, in my current case).
> 
> Just a wishlist... :)
> Cheers,
> 
> Thomas Goirand (zigo)
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Neutron stadium project Tempest plugins

[Ghanshyam Mann]

  On Wed, 24 Oct 2018 05:08:11 +0900 Slawomir Kaplonski 
 wrote  
 > Hi,
 > 
 > Thx Miguel for raising this.
 > List of tempest plugins is on 
 > https://docs.openstack.org/tempest/latest/plugin-registry.html - if URL for 
 > Your plugin is the same as Your main repo, You should move Your tempest 
 > plugin code.

Thanks mlavalle, slaweq for bringing up this discussion. 

Separating the Tempest plugin from service repo was Queens goal and that goal 
clearly state the benefit of having the separate plugins repo [1]. For Neturon, 
that goal was marked as Complete after creating the neutron-temepst-plugin[2] 
and work to separate the neutron stadium project's tempest plugins was left 
out. I think many of the projects did not all. 

This came up while discussing the tempest plugins CI setup [3]. If you need any 
help from QA team, feel free to ping us on #openstack-qa channel. 


[1] https://governance.openstack.org/tc/goals/queens/split-tempest-plugins.html
[2] https://review.openstack.org/#/c/524605/
[3] 
https://etherpad.openstack.org/p/tempest-plugins-ci-release-tagging-clarification


-gmann

 > 
 > 
 > > Wiadomość napisana przez Miguel Lavalle  w dniu 
 > > 23.10.2018, o godz. 16:59:
 > > 
 > > Dear Neutron Stadium projects,
 > > 
 > > In a QA session during the recent PTG in Denver, it was suggested that the 
 > > Stadium projects should move their Tempest plugins to a repository of 
 > > their own or added to the Neutron Tempest plugin repository 
 > > (https://github.com/openstack/neutron-tempest-plugin). The purpose of this 
 > > message is to start a conversation for the Stadium projects to indicate 
 > > what is their preference. Please respond to this thread indicating how do 
 > > you want to move forward.
 > > 
 > > Best regards
 > > 
 > > Miguel
 > > __
 > > OpenStack Development Mailing List (not for usage questions)
 > > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 > 
 > — 
 > Slawek Kaplonski
 > Senior software engineer
 > Red Hat
 > 
 > 
 > __
 > OpenStack Development Mailing List (not for usage questions)
 > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
 > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
 > 



__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [oslo] No complains about rabbitmq SSL problems: could we have this in the logs?

[Thomas Goirand]

Hi,

It took me a long long time to figure out that my SSL setup was wrong
when trying to connect Heat to rabbitmq over SSL. Unfortunately, Oslo
(or heat itself) never warn me that something was wrong, I just got
nothing working, and no log at all.

I'm sure I wouldn't be the only one happy about having this type of
problems being yelled out loud in the logs. Right now, it does work if I
turn off SSL, though I'm still not sure what's wrong in my setup, and
I'm given no clue if the issue is on rabbitmq-server or on the client
side (ie: heat, in my current case).

Just a wishlist... :)
Cheers,

Thomas Goirand (zigo)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Ironic integration CI jobs

[Dmitry Tantsur]

Hi,

On 10/31/18 1:36 AM, Julia Kreger wrote:
With the discussion of CI jobs and the fact that I have been finding myself 
checking job status several times a day so early in the cycle, I think it is 
time for ironic to revisit many of our CI jobs.


The bottom line is ironic is very resource intensive to test. A lot of that is 
because of the underlying way we enroll/manage nodes and then execute the 
integration scenarios emulating bare metal. I think we can improve that with 
some ansible.


In the mean time I created a quick chart[1] to try and make sense out overall 
integration coverage and I think it makes sense to remove three of the jobs.


ironic-tempest-dsvm-ipa-wholedisk-agent_ipmitool-tinyipa-multinode - This job is 
essentially the same as our grenade mutlinode job, the only difference being 
grenade.


Nope, not the same. Grenade jobs run only smoke tests, this job runs 
https://github.com/openstack/ironic-tempest-plugin/blob/master/ironic_tempest_plugin/tests/scenario/test_baremetal_multitenancy.py


ironic-tempest-dsvm-ipa-wholedisk-bios-agent_ipmitool-tinyipa - This job 
essentially just duplicates the functionality already covered in other jobs, 
including the grenade job.


Ditto, grenade jobs do not cover our tests at all. Also this is the very job we 
run on other projects (nova, neutron, maybe more), so it will be a bit painful 
to remove it.


ironic-tempest-dsvm-bfv - This presently non-voting job validates that the iPXE 
mode of the 'pxe' boot interface supports boot from volume. It was superseded by 
ironic-tempest-dsvm-ipxe-bfv which focuses on the use of the 'ipxe' boot 
interface. The underlying code is all the same deep down in all of the helper 
methods.


+1 to this.

Dmitry



I'll go ahead and put this up as a topic for our weekly meeting next week so we 
can discuss.


Thanks,

-Julia

[1]: https://ethercalc.openstack.org/ces0z3xjb1ir

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Ironic integration CI jobs

[Nate Johnston]

On Tue, Oct 30, 2018 at 05:36:09PM -0700, Julia Kreger wrote:

> ironic-tempest-dsvm-ipa-wholedisk-bios-agent_ipmitool-tinyipa - This job
> essentially just duplicates the functionality already covered in other
> jobs, including the grenade job.

FYI this job runs in the Neutron check queue (non-voting).  Would you
like to substitute a different Ironic job to run on all neutron changes,
or should that requirement be revisited?

Thanks!

Nate

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo][openstack-ansible][nova][placement] Owners needed for placement extraction upgrade deployment tooling

[Chris Dent]

On Wed, 31 Oct 2018, Eduardo Gonzalez wrote:


- Run db syncs as there is not command for that yet in the master branch
- Apply upgrade process for db changes


The placement-side pieces for this are nearly ready, see the stack
beginning at https://review.openstack.org/#/c/611441/

--
Chris Dent   ٩◔̯◔۶   https://anticdent.org/
freenode: cdent tw: @anticdent__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo][openstack-ansible][nova][placement] Owners needed for placement extraction upgrade deployment tooling

[Eduardo Gonzalez]

Hi, from kolla side I've started the work.

In kolla images [0] for now only placement is separated to an independent
image, only source is code is being installed, binary still uses
nova-placement packages until a binary package exists from the debian and
centos families.

In kolla-ansible [1] placement service has been moved into a separate role
applied just before nova.

Things missing for now:
- Binary packages from distributions
- Run db syncs as there is not command for that yet in the master branch
- Apply upgrade process for db changes


[0] https://review.openstack.org/#/c/613589/
[1] https://review.openstack.org/#/c/613629/

Regards

El mié., 31 oct. 2018 a las 10:19, Lee Yarwood ()
escribió:

> On 30-10-18 14:29:12, Emilien Macchi wrote:
> > On the TripleO side, it sounds like Lee Yarwood is taking the lead with a
> > first commit in puppet-placement:
> > https://review.openstack.org/#/c/604182/
> >
> > Lee, can you confirm that you and your team are working on it for Stein
> > cycle?
>
> ACK, just getting back online after being out for three weeks but still
> planning on getting everything in place by the original M2 goal we
> agreed to at PTG. I'll try to post more details by the end of the week.
>
> Cheers,
>
> Lee
>
> > On Thu, Oct 25, 2018 at 1:34 PM Matt Riedemann 
> wrote:
> >
> > > Hello OSA/TripleO people,
> > >
> > > A plan/checklist was put in place at the Stein PTG for extracting
> > > placement from nova [1]. The first item in that list is done in grenade
> > > [2], which is the devstack-based upgrade project in the integrated
> gate.
> > > That should serve as a template for the necessary upgrade steps in
> > > deployment projects. The related devstack change for extracted
> placement
> > > on the master branch (Stein) is [3]. Note that change has some
> > > dependencies.
> > >
> > > The second point in the plan from the PTG was getting extracted
> > > placement upgrade tooling support in a deployment project, notably
> > > TripleO (and/or OpenStackAnsible).
> > >
> > > Given the grenade change is done and passing tests, TripleO/OSA should
> > > be able to start coding up and testing an upgrade step when going from
> > > Rocky to Stein. My question is who can we name as an owner in either
> > > project to start this work? Because we really need to be starting this
> > > as soon as possible to flush out any issues before they are too late to
> > > correct in Stein.
> > >
> > > So if we have volunteers or better yet potential patches that I'm just
> > > not aware of, please speak up here so we know who to contact about
> > > status updates and if there are any questions with the upgrade.
> > >
> > > [1]
> http://lists.openstack.org/pipermail/openstack-dev/2018-September/134541.html
> > > [2] https://review.openstack.org/#/c/604454/
> > > [3] https://review.openstack.org/#/c/600162/
>
> --
> Lee Yarwood A5D1 9385 88CB 7E5F BE64  6618 BCA6 6E33 F672
> 2D76
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] Announcing OpenStack Cluster Installer (OCI)

[Thomas Goirand]

Hi,

After about 6 months of development, I'm proud to announce the first
fully working version of OCI. Here's the description:

OCI (OpenStack Cluster Installer) is a software to provision an
OpenStack cluster automatically. This package install a provisioning
machine, which consists of a DHCP server, a PXE boot server, a web
server, and a puppet-master.

Once computers in the cluster boot for the first time, a Debian live
system is served by OCI, to act as a discovery image. This live system
then reports the hardware features back to OCI. The computers can then
be installed with Debian from that live system, configured with a
puppet-agent that will connect to the puppet-master of OCI. After Debian
is installed, the server reboots under it, and OpenStack services is
then provisionned in these machines, depending on their role in the cluster.

Currently, OCI can only install a highly available Swift cluster. In the
future, it will be able to deploy full compute clouds. Stay tuned, or
contribute. Now is the perfect time to influence OCI's design.

OCI has been deployed in production at Infomaniak and has been used for
deploying a cross data-center fully redondant swift cluster. We're
currently working on adding the compute feature to it.

OCI, internally, uses the Puppet modules of puppet-openstack, and is
fully packaged and tested in Debian Sid, Buster and Stretch (including
the Puppet modules). It is available from your closest Debian mirror in
Sid and Buster, and it is also available through the unofficial
stretch-rocky.debian.net backport repository. A simple "apt-get install
openstack-cluster-installer" is enough to install all needed modules.

For further information, see:

https://salsa.debian.org/openstack-team/debian/openstack-cluster-installer

To get in touch, contribute, or ask for support, please join the team's
IRC channel #debian-openstack on the OFTC network.

Cheers,

Thomas Goirand (zigo)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo][openstack-ansible][nova][placement] Owners needed for placement extraction upgrade deployment tooling

[Lee Yarwood]

On 30-10-18 14:29:12, Emilien Macchi wrote:
> On the TripleO side, it sounds like Lee Yarwood is taking the lead with a
> first commit in puppet-placement:
> https://review.openstack.org/#/c/604182/
> 
> Lee, can you confirm that you and your team are working on it for Stein
> cycle?

ACK, just getting back online after being out for three weeks but still
planning on getting everything in place by the original M2 goal we
agreed to at PTG. I'll try to post more details by the end of the week.

Cheers,

Lee
 
> On Thu, Oct 25, 2018 at 1:34 PM Matt Riedemann  wrote:
> 
> > Hello OSA/TripleO people,
> >
> > A plan/checklist was put in place at the Stein PTG for extracting
> > placement from nova [1]. The first item in that list is done in grenade
> > [2], which is the devstack-based upgrade project in the integrated gate.
> > That should serve as a template for the necessary upgrade steps in
> > deployment projects. The related devstack change for extracted placement
> > on the master branch (Stein) is [3]. Note that change has some
> > dependencies.
> >
> > The second point in the plan from the PTG was getting extracted
> > placement upgrade tooling support in a deployment project, notably
> > TripleO (and/or OpenStackAnsible).
> >
> > Given the grenade change is done and passing tests, TripleO/OSA should
> > be able to start coding up and testing an upgrade step when going from
> > Rocky to Stein. My question is who can we name as an owner in either
> > project to start this work? Because we really need to be starting this
> > as soon as possible to flush out any issues before they are too late to
> > correct in Stein.
> >
> > So if we have volunteers or better yet potential patches that I'm just
> > not aware of, please speak up here so we know who to contact about
> > status updates and if there are any questions with the upgrade.
> >
> > [1] 
> > http://lists.openstack.org/pipermail/openstack-dev/2018-September/134541.html
> > [2] https://review.openstack.org/#/c/604454/
> > [3] https://review.openstack.org/#/c/600162/

-- 
Lee Yarwood A5D1 9385 88CB 7E5F BE64  6618 BCA6 6E33 F672 2D76


signature.asc
Description: PGP signature
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [vitrage] I have some problems with Prometheus alarms in vitrage.

[Won]

Hi,

>
> This is strange. I would expect your original definition to work as well,
> since the alarm key in Vitrage is defined by a combination of the alert
> name and the instance. We will check it again.
> BTW,  we solved a different bug related to Prometheus alarms not being
> cleared [1]. Could it be related?
>

Using the original definition, no matter how different the instances are,
the alarm names are recognized as the same alarm in vitrage.
And I tried to install the rocky version and the master version on the new
server and retest but the problem was not solved. The latest bugfix seems
irrelevant.

Does the wrong timestamp appear if you run 'vitrage alarm list' cli
> command? please try running 'vitrage alarm list --debug' and send me the
> output.
>

I have attached 'vitrage-alarm-list.txt.'


> Please send me vitrage-collector.log and vitrage-graph.log from the time
> that the problematic vm was created and deleted. Please also create and
> delete a vm on your 'ubuntu' server, so I can check the differences in the
> log.
>

I have attached 'vitrage_log_on_compute1.zip' and
'vitrage_log_on_ubuntu.zip' files.
When creating a vm on computer1, a vitrage-collect log occurred, but no log
occurred when it was removed.

Br,
Won



2018년 10월 30일 (화) 오전 1:28, Ifat Afek 님이 작성:

> Hi,
>
> On Fri, Oct 26, 2018 at 10:34 AM Won  wrote:
>
>>
>> I solved the problem of not updating the Prometheus alarm.
>> Alarms with the same Prometheus alarm name are recognized as the same
>> alarm in vitrage.
>>
>> --- alert.rules.yml
>> groups:
>> - name: alert.rules
>>   rules:
>>   - alert: InstanceDown
>> expr: up == 0
>> for: 60s
>> labels:
>>   severity: warning
>> annotations:
>>   description: '{{ $labels.instance }} of job {{ $labels.job }} has
>> been down
>> for more than 30 seconds.'
>>   summary: Instance {{ $labels.instance }} down
>> --
>> This is the contents of the alert.rules.yml file before I modify it.
>> This is a yml file that generates an alarm when the cardvisor
>> stops(instance down). Alarm is triggered depending on which instance is
>> down, but all alarms have the same name as 'instance down'. Vitrage
>> recognizes all of these alarms as the same alarm. Thus, until all 'instance
>> down' alarms were cleared, the 'instance down' alarm was recognized as
>> unresolved and the alarm was not extinguished.
>>
>
> This is strange. I would expect your original definition to work as well,
> since the alarm key in Vitrage is defined by a combination of the alert
> name and the instance. We will check it again.
> BTW,  we solved a different bug related to Prometheus alarms not being
> cleared [1]. Could it be related?
>
>
>> Can you please show me where you saw the 2001 timestamp? I didn't find it
>>> in the log.
>>>
>>
>> [image: image.png]
>> The time stamp is recorded well in log(vitrage-graph,collect etc), but in
>> vitrage-dashboard it is marked 2001-01-01.
>> However, it seems that the time stamp is recognized well internally
>> because the alarm can be resolved and is recorded well in log.
>>
>
> Does the wrong timestamp appear if you run 'vitrage alarm list' cli
> command? please try running 'vitrage alarm list --debug' and send me the
> output.
>
>
>> [image: image.png]
>> Host name ubuntu is my main server. I install openstack all in one in
>> this server and i install compute node in host name compute1.
>> When i create a new vm in nova(compute1) it immediately appear in the
>> entity graph. But in does not disappear in the entity graph when i delete
>> the vm. No matter how long i wait, it doesn't disappear.
>> Afther i execute 'vitrage-purge-data' command and reboot the
>> Openstack(execute reboot command in openstack server(host name ubuntu)), it
>> disappear. Only execute 'vitrage-purge-data' does not work. It need a
>> reboot to disappear.
>> When i create a new vm in nova(ubuntu) there is no problem.
>>
> Please send me vitrage-collector.log and vitrage-graph.log from the time
> that the problematic vm was created and deleted. Please also create and
> delete a vm on your 'ubuntu' server, so I can check the differences in the
> log.
>
> I implemented the web service of the micro service architecture and
>> applied the RCA. Attached file picture shows the structure of the web
>> service I have implemented. I wonder what data I receive and what can i do
>> when I link vitrage with kubernetes.
>> As i know, the vitrage graph does not present information about
>> containers or pods inside the vm. If that is correct, I would like to make
>> the information of the pod level appear on the entity graph.
>>
>> I follow (
>> https://docs.openstack.org/vitrage/latest/contributor/k8s_datasource.html)
>> this step. I attached the vitage.conf file and the kubeconfig file. The
>> contents of the Kubeconconfig file are copied from the contents of the
>> admin.conf file on the master node.
>> I want to check my settings are right and connected, but I don't know
>> ho