Re: [openstack-dev] [tripleo] prototype with standalone mode and remote edge compute nodes

2018-07-20 Thread Ben Nemec 



On 07/20/2018 02:06 PM, Emilien Macchi wrote:



On Fri, Jul 20, 2018 at 2:55 PM Ben Nemec > wrote:


Okay, based on a private conversation this is coming off as way more
troll-ish than I intended.  I don't understand where this work is
going,
but I don't really need to so I'll step back from the discussion.
Apologies for any offense.


No offense here, Ben. In fact I hope we can still continue to have a 
productive discussion here.


I'm speaking on my own view now, and I'm happy to be wrong and learn but 
I wanted to explore how far we can bring the work around standalone 
architecture. If it was worth exploring making it "multi-node" somehow, 
what would be our technical challenges and more than anything else: what 
use-case we would enable.


I'm actually quite happy to see that people already looked at some of 
these challenges before (see what Giulio / James / Steve H. already 
worked on), so I guess it makes sense to continue the investigation.
We are not making any decision right now in what API we plan to use. The 
current production architecture is still undercloud + overcloud, and our 
day 2 operations are done by Mistral/Heat for now but as we transition 
more to Ansible I think we wanted to explore more options.


I hope this little background helped.


Yeah, I realize now that I invented some requirements that you didn't 
actually state in your original email.  Slap on the wrist to me for poor 
reading comprehension. :-)


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] prototype with standalone mode and remote edge compute nodes

2018-07-20 Thread Ben Nemec 



On 07/20/2018 02:53 PM, James Slagle wrote:

On Thu, Jul 19, 2018 at 7:13 PM, Ben Nemec  wrote:



On 07/19/2018 03:37 PM, Emilien Macchi wrote:


Today I played a little bit with Standalone deployment [1] to deploy a
single OpenStack cloud without the need of an undercloud and overcloud.
The use-case I am testing is the following:
"As an operator, I want to deploy a single node OpenStack, that I can
extend with remote compute nodes on the edge when needed."

We still have a bunch of things to figure out so it works out of the box,
but so far I was able to build something that worked, and I found useful to
share it early to gather some feedback:
https://gitlab.com/emacchi/tripleo-standalone-edge

Keep in mind this is a proof of concept, based on upstream documentation
and re-using 100% what is in TripleO today. The only thing I'm doing is to
change the environment and the roles for the remote compute node.
I plan to work on cleaning the manual steps that I had to do to make it
working, like hardcoding some hiera parameters and figure out how to
override ServiceNetmap.

Anyway, feel free to test / ask questions / provide feedback.



What is the benefit of doing this over just using deployed server to install
a remote server from the central management system?  You need to have
connectivity back to the central location anyway.  Won't this become
unwieldy with a large number of edge nodes?  I thought we told people not to
use Packstack for multi-node deployments for exactly that reason.

I guess my concern is that eliminating the undercloud makes sense for
single-node PoC's and development work, but for what sounds like a
production workload I feel like you're cutting off your nose to spite your
face.  In the interest of saving one VM's worth of resources, now all of
your day 2 operations have no built-in orchestration.  Every time you want
to change a configuration it's "copy new script to system, ssh to system,
run script, repeat for all systems.  So maybe this is a backdoor way to make
Ansible our API? ;-)


I believe Emilien was looking at this POC in part because of some
input from me, so I will attempt to address your questions
constructively.

What you're looking at here is exactly a POC. The deployment is a POC
using the experimental standalone code. I think the use case as
presented by Emilien is something worth considering:


"As an operator, I want to deploy a single node OpenStack, that I can
extend with remote compute nodes on the edge when needed."


I wouldn't interpret that to mean much of anything around eliminating
the undercloud, other than what is stated for the use case. I feel
that  jumping to eliminating the undercloud would be an over
simplification. The goal of the POC isn't packstack parity, or even
necessarily a packstack like architecture.


Okay, this was the main disconnect for me.  I got the impression from 
the discussion up til now that eliminating the undercloud was part of 
the requirements.  Looking back at Emilien's original email I think I 
conflated the standalone PoC description with the use-case description. 
My bad.




One of the goals is to see if we can deploy separate disconnected
stacks for Control and Compute. The standalone work happens to be a
good way to test out some of the work around that. The use case was
written to help describe and provide an overall picture of what is
going on with this specific POC, with a focus towards the edge use
case.

You make some points about centralized management and connectivity
back to the central location. Those are the exact sorts of things we
are thinking about when we consider how we will address edge
deployments. If you haven't had a chance yet, check out the Edge
Computing whitepaper from the foundation:

https://www.openstack.org/assets/edge/OpenStack-EdgeWhitepaper-v3-online.pdf

Particularly the challenges outlined around management and deployment
tooling. For lack of anything better I'm calling these the 3 D's:
- Decentralized
- Distributed
- Disconnected

How can TripleO address any of these?

For Decentralized, I'd like to see better separation between the
planning and application of the deployment in TripleO. TripleO has had
the concept of a plan for quite a while, and we've been using it very
effectively for our deployment, but it is somewhat hidden from the
operator. It's not entirely clear to the user that there is any
separation between the plan and the stack, and what benefit there even
is in the plan.


+1.  I was disappointed that we didn't adopt the plan as more of a 
first-class citizen for cli deployments after it was implemented.




I'd like to address some of that through API improvements around plan
management and making the plan the top level thing being managed
instead of a deployment. We're already moving in this direction with
config-download and a lot of the changes we've made during Queens.

For better or worse, some other tools like Terraform call this out as
one their main differentiat

Re: [openstack-dev] [tripleo] prototype with standalone mode and remote edge compute nodes

2018-07-20 Thread James Slagle 
On Thu, Jul 19, 2018 at 7:13 PM, Ben Nemec  wrote:
>
>
> On 07/19/2018 03:37 PM, Emilien Macchi wrote:
>>
>> Today I played a little bit with Standalone deployment [1] to deploy a
>> single OpenStack cloud without the need of an undercloud and overcloud.
>> The use-case I am testing is the following:
>> "As an operator, I want to deploy a single node OpenStack, that I can
>> extend with remote compute nodes on the edge when needed."
>>
>> We still have a bunch of things to figure out so it works out of the box,
>> but so far I was able to build something that worked, and I found useful to
>> share it early to gather some feedback:
>> https://gitlab.com/emacchi/tripleo-standalone-edge
>>
>> Keep in mind this is a proof of concept, based on upstream documentation
>> and re-using 100% what is in TripleO today. The only thing I'm doing is to
>> change the environment and the roles for the remote compute node.
>> I plan to work on cleaning the manual steps that I had to do to make it
>> working, like hardcoding some hiera parameters and figure out how to
>> override ServiceNetmap.
>>
>> Anyway, feel free to test / ask questions / provide feedback.
>
>
> What is the benefit of doing this over just using deployed server to install
> a remote server from the central management system?  You need to have
> connectivity back to the central location anyway.  Won't this become
> unwieldy with a large number of edge nodes?  I thought we told people not to
> use Packstack for multi-node deployments for exactly that reason.
>
> I guess my concern is that eliminating the undercloud makes sense for
> single-node PoC's and development work, but for what sounds like a
> production workload I feel like you're cutting off your nose to spite your
> face.  In the interest of saving one VM's worth of resources, now all of
> your day 2 operations have no built-in orchestration.  Every time you want
> to change a configuration it's "copy new script to system, ssh to system,
> run script, repeat for all systems.  So maybe this is a backdoor way to make
> Ansible our API? ;-)

I believe Emilien was looking at this POC in part because of some
input from me, so I will attempt to address your questions
constructively.

What you're looking at here is exactly a POC. The deployment is a POC
using the experimental standalone code. I think the use case as
presented by Emilien is something worth considering:

>> "As an operator, I want to deploy a single node OpenStack, that I can
>> extend with remote compute nodes on the edge when needed."

I wouldn't interpret that to mean much of anything around eliminating
the undercloud, other than what is stated for the use case. I feel
that  jumping to eliminating the undercloud would be an over
simplification. The goal of the POC isn't packstack parity, or even
necessarily a packstack like architecture.

One of the goals is to see if we can deploy separate disconnected
stacks for Control and Compute. The standalone work happens to be a
good way to test out some of the work around that. The use case was
written to help describe and provide an overall picture of what is
going on with this specific POC, with a focus towards the edge use
case.

You make some points about centralized management and connectivity
back to the central location. Those are the exact sorts of things we
are thinking about when we consider how we will address edge
deployments. If you haven't had a chance yet, check out the Edge
Computing whitepaper from the foundation:

https://www.openstack.org/assets/edge/OpenStack-EdgeWhitepaper-v3-online.pdf

Particularly the challenges outlined around management and deployment
tooling. For lack of anything better I'm calling these the 3 D's:
- Decentralized
- Distributed
- Disconnected

How can TripleO address any of these?

For Decentralized, I'd like to see better separation between the
planning and application of the deployment in TripleO. TripleO has had
the concept of a plan for quite a while, and we've been using it very
effectively for our deployment, but it is somewhat hidden from the
operator. It's not entirely clear to the user that there is any
separation between the plan and the stack, and what benefit there even
is in the plan.

I'd like to address some of that through API improvements around plan
management and making the plan the top level thing being managed
instead of a deployment. We're already moving in this direction with
config-download and a lot of the changes we've made during Queens.

For better or worse, some other tools like Terraform call this out as
one their main differentiators:

https://www.terraform.io/intro/vs/cloudformation.html (3rd paragraph).

TripleO has long separated the planning and application phases. We
just need to do a better job at developing useful features around that
work. The UI has been taking advantage of it more than anything else
at this point. I'd like to focus a bit more on what benefits we get
from the plan, and how we can turn these int

Re: [openstack-dev] [tripleo] prototype with standalone mode and remote edge compute nodes

2018-07-20 Thread Harald Jensås 
On Fri, 2018-07-20 at 15:06 -0400, Emilien Macchi wrote:
> 
> 
> On Fri, Jul 20, 2018 at 2:55 PM Ben Nemec 
> wrote:
> > Okay, based on a private conversation this is coming off as way
> > more 
> > troll-ish than I intended.  I don't understand where this work is
> > going, 
> > but I don't really need to so I'll step back from the discussion. 
> > Apologies for any offense.
> 
> No offense here, Ben. In fact I hope we can still continue to have a
> productive discussion here.
> 
> I'm speaking on my own view now, and I'm happy to be wrong and learn
> but I wanted to explore how far we can bring the work around
> standalone architecture. If it was worth exploring making it "multi-
> node" somehow, what would be our technical challenges and more than
> anything else: what use-case we would enable.
> 
> I'm actually quite happy to see that people already looked at some of
> these challenges before (see what Giulio / James / Steve H. already
> worked on), so I guess it makes sense to continue the investigation.
> We are not making any decision right now in what API we plan to use.
> The current production architecture is still undercloud + overcloud,
> and our day 2 operations are done by Mistral/Heat for now but as we
> transition more to Ansible I think we wanted to explore more options.
> 
> I hope this little background helped.
> Thanks,
> -- 
> Emilien Macchi
> 

The split-stack work is interesting.

I'm however not convinced driving the standalone for the edge use
cases. (Like Ben I don't have enough background ...) However, this is
the spec of an edge user: https://review.openstack.org/543936 - they
want ironic to deploy their nodes and I bet choosing os-net-config is a
tripleo influenced choice ... 

I think for these users an undercloud that can deploy multiple
overclouds would make more sense. I.e Deploy one undercloud and use
that to deploy a number of overclouds.

I imagine split stack being used to separate compute and controllers.
Something like: controlplane-overcloud-a + compute-overcloud-a0 +
compute-overcloud-a1 + compute-overcloud-a2. 3x stacks building one
cloud using split-stack. Then the undercloud must be able to deploy b,
c, d etc stack sets similar to the a stacks.

This keeps the central management system but we can manage multiple
clouds and for edge-cases we can do scale/update/upgrade operations on
the stack for compute/storage nodes in each edge individually etc.


--
Harald

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] prototype with standalone mode and remote edge compute nodes

2018-07-20 Thread Emilien Macchi 
On Fri, Jul 20, 2018 at 2:55 PM Ben Nemec  wrote:

> Okay, based on a private conversation this is coming off as way more
> troll-ish than I intended.  I don't understand where this work is going,
> but I don't really need to so I'll step back from the discussion.
> Apologies for any offense.
>

No offense here, Ben. In fact I hope we can still continue to have a
productive discussion here.

I'm speaking on my own view now, and I'm happy to be wrong and learn but I
wanted to explore how far we can bring the work around standalone
architecture. If it was worth exploring making it "multi-node" somehow,
what would be our technical challenges and more than anything else: what
use-case we would enable.

I'm actually quite happy to see that people already looked at some of these
challenges before (see what Giulio / James / Steve H. already worked on),
so I guess it makes sense to continue the investigation.
We are not making any decision right now in what API we plan to use. The
current production architecture is still undercloud + overcloud, and our
day 2 operations are done by Mistral/Heat for now but as we transition more
to Ansible I think we wanted to explore more options.

I hope this little background helped.
Thanks,
-- 
Emilien Macchi
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] prototype with standalone mode and remote edge compute nodes

2018-07-20 Thread Ben Nemec 
Okay, based on a private conversation this is coming off as way more 
troll-ish than I intended.  I don't understand where this work is going, 
but I don't really need to so I'll step back from the discussion. 
Apologies for any offense.


-Ben

On 07/20/2018 12:20 PM, Ben Nemec wrote:



On 07/20/2018 01:18 AM, Bogdan Dobrelya wrote:

On 7/20/18 2:13 AM, Ben Nemec wrote:



On 07/19/2018 03:37 PM, Emilien Macchi wrote:
Today I played a little bit with Standalone deployment [1] to deploy 
a single OpenStack cloud without the need of an undercloud and 
overcloud.

The use-case I am testing is the following:
"As an operator, I want to deploy a single node OpenStack, that I 
can extend with remote compute nodes on the edge when needed."


We still have a bunch of things to figure out so it works out of the 
box, but so far I was able to build something that worked, and I 
found useful to share it early to gather some feedback:

https://gitlab.com/emacchi/tripleo-standalone-edge

Keep in mind this is a proof of concept, based on upstream 
documentation and re-using 100% what is in TripleO today. The only 
thing I'm doing is to change the environment and the roles for the 
remote compute node.
I plan to work on cleaning the manual steps that I had to do to make 
it working, like hardcoding some hiera parameters and figure out how 
to override ServiceNetmap.


Anyway, feel free to test / ask questions / provide feedback.


What is the benefit of doing this over just using deployed server to 
install a remote server from the central management system?  You need 
to have connectivity back to the central location anyway.  Won't this 
become unwieldy with a large number of edge nodes?  I thought we told 
people not to use Packstack for multi-node deployments for exactly 
that reason.


I guess my concern is that eliminating the undercloud makes sense for 
single-node PoC's and development work, but for what sounds like a 
production workload I feel like you're cutting off your nose to spite 
your face.  In the interest of saving one VM's worth of resources, 
now all of your day 2 operations have no built-in orchestration.  
Every time you want to change a configuration it's "copy new script 
to system, ssh to system, run script, repeat for all systems.  So 
maybe this is a backdoor way to make Ansible our API? ;-)


Ansible may orchestrate that for day 2. Deploying Heat stacks is 
already made ephemeral for standalone/underclouds so only thing you'll 
need for day 2 is ansible really. Hence, the need of undercloud 
shrinks into having an ansible control node, like your laptop, to 
control all clouds via inventory.


So I guess the answer to my last question is yes. :-)

Are we planning to reimplement all of our API workflows in Ansible or 
are users expected to do that themselves?


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] prototype with standalone mode and remote edge compute nodes

2018-07-20 Thread Ben Nemec 



On 07/20/2018 01:18 AM, Bogdan Dobrelya wrote:

On 7/20/18 2:13 AM, Ben Nemec wrote:



On 07/19/2018 03:37 PM, Emilien Macchi wrote:
Today I played a little bit with Standalone deployment [1] to deploy 
a single OpenStack cloud without the need of an undercloud and 
overcloud.

The use-case I am testing is the following:
"As an operator, I want to deploy a single node OpenStack, that I can 
extend with remote compute nodes on the edge when needed."


We still have a bunch of things to figure out so it works out of the 
box, but so far I was able to build something that worked, and I 
found useful to share it early to gather some feedback:

https://gitlab.com/emacchi/tripleo-standalone-edge

Keep in mind this is a proof of concept, based on upstream 
documentation and re-using 100% what is in TripleO today. The only 
thing I'm doing is to change the environment and the roles for the 
remote compute node.
I plan to work on cleaning the manual steps that I had to do to make 
it working, like hardcoding some hiera parameters and figure out how 
to override ServiceNetmap.


Anyway, feel free to test / ask questions / provide feedback.


What is the benefit of doing this over just using deployed server to 
install a remote server from the central management system?  You need 
to have connectivity back to the central location anyway.  Won't this 
become unwieldy with a large number of edge nodes?  I thought we told 
people not to use Packstack for multi-node deployments for exactly 
that reason.


I guess my concern is that eliminating the undercloud makes sense for 
single-node PoC's and development work, but for what sounds like a 
production workload I feel like you're cutting off your nose to spite 
your face.  In the interest of saving one VM's worth of resources, now 
all of your day 2 operations have no built-in orchestration.  Every 
time you want to change a configuration it's "copy new script to 
system, ssh to system, run script, repeat for all systems.  So maybe 
this is a backdoor way to make Ansible our API? ;-)


Ansible may orchestrate that for day 2. Deploying Heat stacks is already 
made ephemeral for standalone/underclouds so only thing you'll need for 
day 2 is ansible really. Hence, the need of undercloud shrinks into 
having an ansible control node, like your laptop, to control all clouds 
via inventory.


So I guess the answer to my last question is yes. :-)

Are we planning to reimplement all of our API workflows in Ansible or 
are users expected to do that themselves?


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [self-healing] [ptg] PTG track schedule published

2018-07-20 Thread Thierry Carrez 

Adam Spiers wrote:

Apologies - I have had to change plans and leave on the Thursday
evening (old friend is getting married on Saturday morning).  Is there
any chance of swapping the self-healing slot with one of the others?


It's tricky, as you asked to avoid conflicts with API SIG, Watcher, 
Monasca, Masakari, and Mistral... Which day would be best for you given 
the current schedule (assuming we don't move anything else as it's too 
late for that).


--
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [self-healing] [ptg] PTG track schedule published

2018-07-20 Thread Adam Spiers 

Thierry Carrez  wrote:

Thierry Carrez wrote:

Hi everyone,

Last month we published the tentative schedule layout for the 5 days 
of PTG. There was no major complaint, so that was confirmed as the 
PTG event schedule and published on the PTG website:


https://www.openstack.org/ptg#tab_schedule


The tab temporarily disappeared, while it is being restored you can 
access the schedule at:


https://docs.google.com/spreadsheets/d/e/2PACX-1vRM2UIbpnL3PumLjRaso_9qpOfnyV9VrPqGbTXiMVNbVgjiR3SIdl8VSBefk339MhrbJO5RficKt2Rr/pubhtml?gid=1156322660&single=true


Apologies - I have had to change plans and leave on the Thursday
evening (old friend is getting married on Saturday morning).  Is there
any chance of swapping the self-healing slot with one of the others?

Sorry for having to ask!
Adam

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [all] [ptg] PTG track schedule published

2018-07-20 Thread Thierry Carrez 

Thierry Carrez wrote:

Hi everyone,

Last month we published the tentative schedule layout for the 5 days of 
PTG. There was no major complaint, so that was confirmed as the PTG 
event schedule and published on the PTG website:


https://www.openstack.org/ptg#tab_schedule


The tab temporarily disappeared, while it is being restored you can 
access the schedule at:


https://docs.google.com/spreadsheets/d/e/2PACX-1vRM2UIbpnL3PumLjRaso_9qpOfnyV9VrPqGbTXiMVNbVgjiR3SIdl8VSBefk339MhrbJO5RficKt2Rr/pubhtml?gid=1156322660&single=true

--
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [all] [ptg] PTG track schedule published

2018-07-20 Thread Thierry Carrez 

Hi everyone,

Last month we published the tentative schedule layout for the 5 days of 
PTG. There was no major complaint, so that was confirmed as the PTG 
event schedule and published on the PTG website:


https://www.openstack.org/ptg#tab_schedule

You'll notice that:

- The Ops meetup days were added.

- Keystone track is split in two: one day on Monday for cross-project 
discussions around identity management, and two days on Thursday/Friday 
for team discussions.


- The "Ask me anything" project helproom on Monday/Tuesday is for 
horizontal support teams (infrastructure, release management, stable 
maint, requirements...) to provide support for other teams, SIGs and 
workgroups and answer their questions. Goal champions should also be 
available there to help with Stein goal completion questions.


- Like in Dublin, a number of tracks do not get pre-allocated time, and 
will be scheduled on the spot in available rooms at the time that makes 
the most sense for the participants.


- Every track will be able to book extra time and space in available 
extra rooms at the event.


To find more information about the event, register or book a room at the 
event hotel, visit: https://www.openstack.org/ptg


Note that the second (and last) round of applications for travel support 
to the event is closing at the end of next week (July 29th) ! Apply if 
you need financial help attending the event:


https://openstackfoundation.formstack.com/forms/travelsupportptg_denver_2018

See you there !

--
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] New "validation" subcommand for "openstack undercloud"

2018-07-20 Thread Bogdan Dobrelya 

On 7/16/18 6:32 PM, Dan Prince wrote:

On Mon, Jul 16, 2018 at 11:27 AM Cédric Jeanneret  wrote:


Dear Stackers,

In order to let operators properly validate their undercloud node, I
propose to create a new subcommand in the "openstack undercloud" "tree":
`openstack undercloud validate'

This should only run the different validations we have in the
undercloud_preflight.py¹
That way, an operator will be able to ensure all is valid before
starting "for real" any other command like "install" or "upgrade".

Of course, this "validate" step is embedded in the "install" and
"upgrade" already, but having the capability to just validate without
any further action is something that can be interesting, for example:

- ensure the current undercloud hardware/vm is sufficient for an update
- ensure the allocated VM for the undercloud is sufficient for a deploy
- and so on

There are probably other possibilities, if we extend the "validation"
scope outside the "undercloud" (like, tripleo, allinone, even overcloud).

What do you think? Any pros/cons/thoughts?


I think this command could be very useful. I'm assuming the underlying
implementation would call a 'heat stack-validate' using an ephemeral
heat-all instance. If so way we implement it for the undercloud vs the


I think that should be just ansible commands triggered natively via 
tripleoclient. Why would we validate with heat deploying a throwaway 
one-time ephemeral stacks (for undercloud/standalon) each time a user 
runs that heat installer? We had to introduce the virtual stack state 
tracking system [0], for puppet manifests compatibility sakes only (it 
sometimes rely on states CREATE vs UPDATE), which added more "ephemeral 
complexity" in DF. I'm not following why would we validate ephemeral 
stacks or using it as an additional moving part?


[0] 
https://review.openstack.org/#/q/topic:bug/1778505+(status:open+OR+status:merged)



'standalone' use case would likely be a bit different. We can probably
subclass the implementations to share common code across the efforts
though.

For the undercloud you are likely to have a few extra 'local only'
validations. Perhaps extra checks for things on the client side.

For the all-in-one I had envisioned using the output from the 'heat
stack-validate' to create a sample config file for a custom set of
services. Similar to how tools like Packstack generate a config file
for example.

Dan



Cheers,

C.



¹
http://git.openstack.org/cgit/openstack/python-tripleoclient/tree/tripleoclient/v1/undercloud_preflight.py
--
Cédric Jeanneret
Software Engineer
DFG:DF

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




--
Best regards,
Bogdan Dobrelya,
Irc #bogdando

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] Proposing Jose Luis Franco for TripleO core reviewer on Upgrade bits

2018-07-20 Thread Jose Luis Franco Arza 
Thank you very much to all for the recognition.
I will use this power with responsibility, as Uncle Ben once said:
https://giphy.com/gifs/MCZ39lz83o5lC/fullscreen

Regards,
Jose Luis

On Fri, Jul 20, 2018 at 1:00 PM, Emilien Macchi  wrote:

>
>
> On Fri, Jul 20, 2018 at 4:09 AM Carlos Camacho Gonzalez <
> ccama...@redhat.com> wrote:
>
>> Hi!!!
>>
>> I'll like to propose Jose Luis Franco [1][2] for core reviewer in all the
>> TripleO upgrades bits. He shows a constant and active involvement in
>> improving and fixing our updates/upgrades workflows, he helps also trying
>> to develop/improve/fix our upstream support for testing the
>> updates/upgrades.
>>
>> Please vote -1/+1, and consider this my +1 vote :)
>>
>
> Nice work indeed, +1. Keep doing a good job and thanks for all your help!
> --
> Emilien Macchi
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [nova] [placement] placement update 18-29

2018-07-20 Thread Chris Dent 


HTML: https://anticdent.org/placement-update-18-29.html

This is placement update 18-28, a weekly update of ongoing development 
related to the [OpenStack](https://www.openstack.org/) [placement 
service](https://developer.openstack.org/api-ref/placement/).


Thanks to Jay for providing one of these last week when I was away:


# Most Important

Feature freeze is next week. We're racing now to get as much of
three styles of work done as possible:

* Effectively managing nested and shared resource providers when
  managing allocations (such as in migrations).
* Correctly handling resource provider and consumer generations in
  the nova-side report client.
* Supporting reshaping provider trees.

The latter two are actively in progress. Not sure about the first.
Anyone?

As ever, we continue to find bugs with existing features that
existing tests are not catching. These are being found by people
experimenting. So: experiment please.

# What's Changed

Most of the functionality and fixes related to consumer generations
is in place on the placement side.

We now enforce that consumer identifiers are uuids.

# Bugs

* Placement related [bugs not yet in progress](https://goo.gl/TgiPXb):
  15, no change from last week.
* [In progress placement bugs](https://goo.gl/vzGGDQ) 15, +1 on last
  week.

# Main Themes

## Documentation

This is a section for reminding us to document all the fun stuff we
are enabling. Open areas include:

* "How to deploy / model shared disk. Seems fairly straight-forward,
   and we could even maybe create a multi-node ceph job that does
   this - wouldn't that be awesome?!?!", says an enthusiastic Matt
   Riedemann.

* The whens and wheres of re-shaping and VGPUs.

## Consumer Generations

These are in place on the placement side. There's some pending work
on using them properly and addresssing some nits:

* 
  Address nits from consumer generation

* 
  return 404 when no consumer found in allocs

* 
  Use placement 1.28 in scheduler report client
  (1.28 is consumer gens)

* 
  Use consumer generation in _heal_allocations_for_instance

## Reshape Provider Trees

The work to support a /reshaper URI that allows moving inventory and
allocations between resource providers is in progress. The database
handling (at the bottom of the stack) is pretty much ready, the
HTTP API is close except for a [small issue with allocation
schema](https://review.openstack.org/#/c/583907/), and the nova side
is in active progress.

That's all at: 

## Mirror Host Aggregates

This needs a command line tool:

* 

## Extraction

I took some time yesterday to experiment with an alternative to the
os-resource-classes that [jay
created](https://github.com/jaypipes/os-resource-classes). [My
version](https://github.com/cdent/os-resource-classes) is, thus far,
just a simple spike that makes symbols pointing to strings, and
that's it. I've made a [proof of
concept](https://review.openstack.org/#/c/584084/) of integrating it
with placement.

Other extraction things that continue to need some thought are:

* infra and co-gating issues that are going to come up
* copying whatever nova-based test fixture we might like

# Other

20 entries two weeks ago. 29 now.

* 
   Purge comp_node and res_prvdr records during deletion of
   cells/hosts

* 
   Get resource provider by uuid or name (osc-placement)

* 
  Check provider generation and retry on conflict

* 
   Add unit test for non-placement resize

* 
   Move refresh time from report client to prov tree

* 
   PCPU resource class

* 
   rework how we pass candidate request information

* 
   add root parent NULL online migration

* 
   add resource_requests field to RequestSpec

* 
   Convert driver supported capabilities to compute node provider
   traits

* 
   Use placement.inventory.inuse in report client

* 
   ironic: Report resources as reserved when needed

* 
   Test for multiple limit/group_policy qparams

* 
   [placement] api-ref: add traits parameter

* 
   Convert 'place

Re: [openstack-dev] [tripleo] prototype with standalone mode and remote edge compute nodes

2018-07-20 Thread Emilien Macchi 
On Fri, Jul 20, 2018 at 7:43 AM Emilien Macchi  wrote:

> On Fri, Jul 20, 2018 at 7:09 AM Emilien Macchi  wrote:
>
>> Yeah I thought about this one too but I didn't have this challenge since
>> I just wanted nova-compute & neutron-ovs-agent running on the edge.
>>
>
> Actually I just faced it:
> Error: Failed to perform requested operation on instance "my-vm", the
> instance has an error status: Please try again later [Error: Host
> 'standalone-cpu-edge.localdomain' is not mapped to any cell].
>
> I had to manually add the edge compute on the central node, so yeah we
> need to figure that out for the compute as well (unless I missed something
> in the nova config).
>

Nevermind, I had to set NovaSchedulerDiscoverHostsInCellsInterval to 300,
so nova-schedule checks for new compute nodes every 300s and include them
in the cell.
-- 
Emilien Macchi
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Disk space requirement - any way to lower it a little?

2018-07-20 Thread Cédric Jeanneret 


On 07/20/2018 09:49 AM, Cédric Jeanneret wrote:
> 
> 
> On 07/19/2018 06:55 PM, Paul Belanger wrote:
>> On Thu, Jul 19, 2018 at 05:30:27PM +0200, Cédric Jeanneret wrote:
>>> Hello,
>>>
>>> While trying to get a new validation¹ in the undercloud preflight
>>> checks, I hit an (not so) unexpected issue with the CI:
>>> it doesn't provide flavors with the minimal requirements, at least
>>> regarding the disk space.
>>>
>>> A quick-fix is to disable the validations in the CI - Wes has already
>>> pushed a patch for that in the upstream CI:
>>> https://review.openstack.org/#/c/583275/
>>> We can consider this as a quick'n'temporary fix².
>>>
>>> The issue is on the RDO CI: apparently, they provide instances with
>>> "only" 55G of free space, making the checks fail:
>>> https://logs.rdoproject.org/17/582917/3/openstack-check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/c9cf398/logs/undercloud/home/zuul/undercloud_install.log.txt.gz#_2018-07-17_10_23_46
>>>
>>> So, the question is: would it be possible to lower the requirment to,
>>> let's say, 50G? Where does that 60G³ come from?
>>>
>>> Thanks for your help/feedback.
>>>
>>> Cheers,
>>>
>>> C.
>>>
>>>
>>>
>>> ¹ https://review.openstack.org/#/c/582917/
>>>
>>> ² as you might know, there's a BP for a unified validation framework,
>>> and it will allow to get injected configuration in CI env in order to
>>> lower the requirements if necessary:
>>> https://blueprints.launchpad.net/tripleo/+spec/validation-framework
>>>
>>> ³
>>> http://tripleo.org/install/environments/baremetal.html#minimum-system-requirements
>>>
>> Keep in mind, upstream we don't really have control over partitions of 
>> nodes, in
>> some case it is a single, other multiple. I'd suggest looking more at:
> 
> After some checks on y locally deployed containerized undercloud (hence,
> Rocky) without real activity, here's what I could get:
> - most data are located in /var - this explains the current check.
> 
> If we go a bit deeper, here are the "actually used" directory in /var/lib:
> 20K   alternatives
> 36K   certmonger
> 4.0K  chrony
> 1.2G  config-data
> 4.0K  dhclient
> 6.0G  docker
> 28K   docker-config-scripts
> 92K   docker-container-startup-configs.json
> 44K   docker-puppet
> 592K  heat-config
> 832K  ironic
> 4.0K  ironic-inspector
> 236K  kolla
> 4.0K  logrotate
> 286M  mysql
> 48K   neutron
> 4.0K  ntp
> 4.0K  postfix
> 872K  puppet
> 3.8M  rabbitmq
> 59M   rpm
> 4.0K  rsyslog
> 64K   systemd
> 20K   tripleo
> 236K  tripleo-config
> 9.8M  yum
> 7.5G  total
> 
> Most of the "default installer" partition schema don't go further than
> putting /var, /tmp, /home and /usr in dedicated volumes - of course,
> end-user can chose to ignore that and provide a custom schema.
> 
> That said, we can get the "used" paths. In addition to /var/lib, there's
> obviously /usr.
> 
> We might want to:
> - loop on known locations
> - check if they are on dedicated mount points
> - check the available disk space on those mount points.
> 
> An interesting thing in bash:
> df /var/lib/docker
> Filesystem 1K-blocks Used Available Use% Mounted on
> /dev/sda1  104846316 10188828  94657488  10% /
> 
> This allows to get:
> - actual volume
> - free space on the volume.
> 
> More than that, we might also try to figure out some pattern. For
> instance, "docker" seems to be a pretty good candidate for space, as it
> will get the images and container data. This is probably even the
> biggest eater, at least on the undercloud - as well as the logs (/var/logs).
> 
> We might do a check ensuring we can, at least, DEPLOY the app. This
> would require far less than the required 60G, and with a proper doc
> announcing that, we can get a functional test, aiming on its purpose:
> ensure we can deploy (so asking, let's say, 10G in /var/lib/docker, 5G
> in /var/lib/config-data, 5G in /usr, 1G in /var/log) and, later, upgrade
> (requiring the same amount of *free* space).
> 
> That would require some changes in the validation check of course. But
> at least, we might get a pretty nice covering, while allowing it to run
> smoothly in the CI.
> But, as said: proper documentation should be set, and the "60G minimum
> required" should be rephrased in order to point the locations needing
> space (with the appropriate warning about "none exhaustiveness" and the
> like).
> 
> Would that suit better the actual needs, and allow to get a proper disk
> space check/validation?
> 
> Cheers,
> 
> C.

Following those thoughts, here's a proposal, to be discussed, augmented,
enhanced:
https://review.openstack.org/#/c/584314/

This should allow to get a really nice space check, and in addition
allow ops to create a layout suited for the undercloud if they want -
getting dedicated volumes for specific uses, allowing to get a smart
monitoring of the disk usage per resources is always good.

It kind of also allow to sort out the issue of the CI, providing we
update the doc to reflect the "new" reality of this valida

Re: [openstack-dev] [tripleo] prototype with standalone mode and remote edge compute nodes

2018-07-20 Thread James Slagle 
On Fri, Jul 20, 2018 at 7:09 AM, Emilien Macchi  wrote:
>
>
> On Fri, Jul 20, 2018 at 4:20 AM Giulio Fidente  wrote:
> [...]
>>
>> I have started experimenting with edge deployments to help out on the
>> split-controplane spec [1], which Steven started addressing
>>
>> I was able to deploy multiple stacks and isolated Ceph clusters, there
>> are some bits missing to provision a working configuration for
>> nova-compute to the edge services, but we could probably collect/export
>> the necessary outputs from the parent stack (eg. rabbit connection
>> infos) and feed the edge stacks with those.
>
>
> Indeed, I faced the exact same problems. I could hardcode the rabbit
> password and memcache IP via hieradata extraconfig, James showed me
> AllNodesExtraMapData done via https://review.openstack.org/#/c/581080/ which
> I'll probably give a try.
> However I couldn't set keystone url for nova / neutron (they are taken from
> ServiceNetMap).
> James pointed out to me this patch: https://review.openstack.org/#/c/521929/

Emilien/Giulio:

These are 3 patches (2 are from shardy) that I've been testing with
split-controlplane:

https://review.openstack.org/#/c/521928/
https://review.openstack.org/#/c/521929/
https://review.openstack.org/#/c/581080/

I'll pull some docs together if I have some initial success.





-- 
-- James Slagle
--

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] prototype with standalone mode and remote edge compute nodes

2018-07-20 Thread Emilien Macchi 
On Fri, Jul 20, 2018 at 7:09 AM Emilien Macchi  wrote:

> Yeah I thought about this one too but I didn't have this challenge since I
> just wanted nova-compute & neutron-ovs-agent running on the edge.
>

Actually I just faced it:
Error: Failed to perform requested operation on instance "my-vm", the
instance has an error status: Please try again later [Error: Host
'standalone-cpu-edge.localdomain' is not mapped to any cell].

I had to manually add the edge compute on the central node, so yeah we need
to figure that out for the compute as well (unless I missed something in
the nova config).
-- 
Emilien Macchi
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [edge][glance]: Image handling in edge environment

2018-07-20 Thread Csatari, Gergely (Nokia - HU/Budapest) 
Hi,

We figured out with Jokke two timeslots what would be okay for both of us for 
this common meeting.

Please, other interested parties give your votes to here: 
https://doodle.com/poll/9rfcb8aavsmybzfu

I will evaluate the results and fix the time on 25.07.2018 12h CET.

Br,
Gerg0

From: Csatari, Gergely (Nokia - HU/Budapest)
Sent: Wednesday, July 18, 2018 10:02 AM
To: 'edge-computing' ; OpenStack 
Development Mailing List (not for usage questions) 

Subject: [edge][glance]: Image handling in edge environment

Hi,

We had a great Forum session about image handling in edge environment in 
Vancouver [1]. As one 
outcome of the session I've created a wiki with the mentioned architecture 
options 
[1]. During 
the Edge Working Group 
[3] discussions we 
identified some questions (some of them are in the wiki, some of them are in 
mails 
[4]) 
and also I would like to get some feedback on the analyzis in the wiki from 
people who know Glance.

I think the best would be to have some kind of meeting and I see two options to 
organize this:

  *   Organize a dedicated meeting for this
  *   Add this topic as an agenda point to the Glance weekly meeting

Please share your preference and/or opinion.

Thanks,
Gerg0

[1]: https://etherpad.openstack.org/p/yvr-edge-cloud-images
[2]: https://wiki.openstack.org/wiki/Image_handling_in_edge_environment
[3]: https://wiki.openstack.org/wiki/Edge_Computing_Group
[4]: http://lists.openstack.org/pipermail/edge-computing/2018-June/000239.html

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [cinder][nova] Proper behavior for os-force_detach

2018-07-20 Thread Erlon Cruz 
Nice, good to know. Thanks all for the feedback. We will fix that in our
drivers.

@Walter, so, in this case, if Cinder has the connector, it should not need
to call the driver passing a None object right?

Erlon

Em qua, 18 de jul de 2018 às 12:56, Walter Boring 
escreveu:

> The whole purpose of this test is to simulate the case where Nova doesn't
> know where the vm is anymore,
> or may simply not exist, but we need to clean up the cinder side of
> things.   That being said, with the new
> attach API, the connector is being saved in the cinder database for each
> volume attachment.
>
> Walt
>
> On Wed, Jul 18, 2018 at 5:02 AM, Gorka Eguileor 
> wrote:
>
>> On 17/07, Sean McGinnis wrote:
>> > On Tue, Jul 17, 2018 at 04:06:29PM -0300, Erlon Cruz wrote:
>> > > Hi Cinder and Nova folks,
>> > >
>> > > Working on some tests for our drivers, I stumbled upon this tempest
>> test
>> > > 'force_detach_volume'
>> > > that is calling Cinder API passing a 'None' connector. At the time
>> this was
>> > > added several CIs
>> > > went down, and people started discussing whether this
>> (accepting/sending a
>> > > None connector)
>> > > would be the proper behavior for what is expected to a driver to
>> do[1]. So,
>> > > some of CIs started
>> > > just skipping that test[2][3][4] and others implemented fixes that
>> made the
>> > > driver to disconnected
>> > > the volume from all hosts if a None connector was received[5][6][7].
>> >
>> > Right, it was determined the correct behavior for this was to
>> disconnect the
>> > volume from all hosts. The CIs that are skipping this test should stop
>> doing so
>> > (once their drivers are fixed of course).
>> >
>> > >
>> > > While implementing this fix seems to be straightforward, I feel that
>> just
>> > > removing the volume
>> > > from all hosts is not the correct thing to do mainly considering that
>> we
>> > > can have multi-attach.
>> > >
>> >
>> > I don't think multiattach makes a difference here. Someone is forcibly
>> > detaching the volume and not specifying an individual connection. So
>> based on
>> > that, Cinder should be removing any connections, whether that is to one
>> or
>> > several hosts.
>> >
>>
>> Hi,
>>
>> I agree with Sean, drivers should remove all connections for the volume.
>>
>> Even without multiattach there are cases where you'll have multiple
>> connections for the same volume, like in a Live Migration.
>>
>> It's also very useful when Nova and Cinder get out of sync and your
>> volume has leftover connections. In this case if you try to delete the
>> volume you get a "volume in use" error from some drivers.
>>
>> Cheers,
>> Gorka.
>>
>>
>> > > So, my questions are: What is the best way to fix this problem? Should
>> > > Cinder API continue to
>> > > accept detachments with None connectors? If, so, what would be the
>> effects
>> > > on other Nova
>> > > attachments for the same volume? Is there any side effect if the
>> volume is
>> > > not multi-attached?
>> > >
>> > > Additionally to this thread here, I should bring this topic to
>> tomorrow's
>> > > Cinder's meeting,
>> > > so please join if you have something to share.
>> > >
>> >
>> > +1 - good plan.
>> >
>> >
>> >
>> __
>> > OpenStack Development Mailing List (not for usage questions)
>> > Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] prototype with standalone mode and remote edge compute nodes

2018-07-20 Thread Emilien Macchi 
On Fri, Jul 20, 2018 at 4:20 AM Giulio Fidente  wrote:
[...]

> I have started experimenting with edge deployments to help out on the
> split-controplane spec [1], which Steven started addressing
>
> I was able to deploy multiple stacks and isolated Ceph clusters, there
> are some bits missing to provision a working configuration for
> nova-compute to the edge services, but we could probably collect/export
> the necessary outputs from the parent stack (eg. rabbit connection
> infos) and feed the edge stacks with those.
>

Indeed, I faced the exact same problems. I could hardcode the rabbit
password and memcache IP via hieradata extraconfig, James showed
me AllNodesExtraMapData done via https://review.openstack.org/#/c/581080/
which I'll probably give a try.
However I couldn't set keystone url for nova / neutron (they are taken from
ServiceNetMap).
James pointed out to me this patch: https://review.openstack.org/#/c/521929/
- Do you think we should re-use the service net map from the central node,
on the edge compute node?

A much bigger challenge seems to me that for some services (eg. glance
> or cinder) we need to "refresh" the configuration of the controlplane
> nodes to push the details of the newly deployed ceph clusters (backends)
> of the edge nodes as backends for the controlplane services.
>

Yeah I thought about this one too but I didn't have this challenge since I
just wanted nova-compute & neutron-ovs-agent running on the edge.

Alternatively, we could opt for the deployment of cinder-volume
> instances on the edge nodes, but we would still have the same problem
> for glance and possibly other services.
>

For now the only thing I see is to manually update the config on the
central node and run the deployment again, which should reconfigure the
containers.

I'd like to discuss further this topic at the PTG to gether more
> feedback so I added a bullet to the pad with the Stein PTG topics [2].


It would be awesome to spend time on this topic! Thanks for bringing this
blueprint up! Indeed I hope we'll make progress on this one at the PTG,
which is why I sent this email really early to groom some ideas.


> 1. https://blueprints.launchpad.net/tripleo/+spec/split-controlplane
> 2. https://etherpad.openstack.org/p/tripleo-ptg-stein


Thanks,
-- 
Emilien Macchi
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] Proposing Jose Luis Franco for TripleO core reviewer on Upgrade bits

2018-07-20 Thread Emilien Macchi 
On Fri, Jul 20, 2018 at 4:09 AM Carlos Camacho Gonzalez 
wrote:

> Hi!!!
>
> I'll like to propose Jose Luis Franco [1][2] for core reviewer in all the
> TripleO upgrades bits. He shows a constant and active involvement in
> improving and fixing our updates/upgrades workflows, he helps also trying
> to develop/improve/fix our upstream support for testing the
> updates/upgrades.
>
> Please vote -1/+1, and consider this my +1 vote :)
>

Nice work indeed, +1. Keep doing a good job and thanks for all your help!
-- 
Emilien Macchi
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [stable][meta] Proposing Retiring the Stable Branch project

2018-07-20 Thread Thierry Carrez 

Tony Breeds wrote:

On Fri, Jul 20, 2018 at 01:05:26PM +1000, Tony Breeds wrote:


Hello folks,
 So really the subject says it all.  I fell like at the time we
created the Stable branch project team that was the only option.  Since
then we have crated the SIG structure and in my opinion that's a better
fit.  We've also transition from 'Stable Branch Maintenance' to
'Extended Maintenance'

Being a SIG will make it explicit that we *need* operator, user and
developer contributions.


I meant to say I've created:
https://review.openstack.org/584205 and
https://review.openstack.org/584206

To make this transition.


I think it makes a lot of sense. Stable branch maintenance was always a 
bit of an odd duck in the project teams (owning no repository), and is 
technically a downstream activity (post-release) with lots of potential 
to get users involved.


--
Thierry Carrez (ttx)

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] Proposing Jose Luis Franco for TripleO core reviewer on Upgrade bits

2018-07-20 Thread Yurii Prokulevych 
On Fri, 2018-07-20 at 10:07 +0200, Carlos Camacho Gonzalez wrote:
> Hi!!!
> 
> I'll like to propose Jose Luis Franco [1][2] for core reviewer in all
> the TripleO upgrades bits. He shows a constant and active involvement
> in improving and fixing our updates/upgrades workflows, he helps also
> trying to develop/improve/fix our upstream support for testing the
> updates/upgrades.
> 
> Please vote -1/+1, and consider this my +1 vote :)

+1

> 
> [1]: https://review.openstack.org/#/q/owner:jfrancoa%2540redhat.com
> [2]: http://stackalytics.com/?release=all&metric=commits&user_id=jfra
> ncoa
> 
> Cheers,
> Carlos.
> _
> _
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubs
> cribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] prototype with standalone mode and remote edge compute nodes

2018-07-20 Thread Giulio Fidente 
On 07/19/2018 10:37 PM, Emilien Macchi wrote:
> Today I played a little bit with Standalone deployment [1] to deploy a
> single OpenStack cloud without the need of an undercloud and overcloud.
> The use-case I am testing is the following:
> "As an operator, I want to deploy a single node OpenStack, that I can
> extend with remote compute nodes on the edge when needed."
> 
> We still have a bunch of things to figure out so it works out of the
> box, but so far I was able to build something that worked, and I found
> useful to share it early to gather some feedback:
>   https://gitlab.com/emacchi/tripleo-standalone-edge
> 
> Keep in mind this is a proof of concept, based on upstream documentation
> and re-using 100% what is in TripleO today. The only thing I'm doing is
> to change the environment and the roles for the remote compute node.
> I plan to work on cleaning the manual steps that I had to do to make it
> working, like hardcoding some hiera parameters and figure out how to
> override ServiceNetmap.
> 
> Anyway, feel free to test / ask questions / provide feedback.

hi Emilien,

thanks for sharing this.

I have started experimenting with edge deployments to help out on the
split-controplane spec [1], which Steven started addressing

I was able to deploy multiple stacks and isolated Ceph clusters, there
are some bits missing to provision a working configuration for
nova-compute to the edge services, but we could probably collect/export
the necessary outputs from the parent stack (eg. rabbit connection
infos) and feed the edge stacks with those.

A much bigger challenge seems to me that for some services (eg. glance
or cinder) we need to "refresh" the configuration of the controlplane
nodes to push the details of the newly deployed ceph clusters (backends)
of the edge nodes as backends for the controlplane services.

Alternatively, we could opt for the deployment of cinder-volume
instances on the edge nodes, but we would still have the same problem
for glance and possibly other services.

I'd like to discuss further this topic at the PTG to gether more
feedback so I added a bullet to the pad with the Stein PTG topics [2].

1. https://blueprints.launchpad.net/tripleo/+spec/split-controlplane
2. https://etherpad.openstack.org/p/tripleo-ptg-stein

-- 
Giulio Fidente
GPG KEY: 08D733BA

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] Proposing Jose Luis Franco for TripleO core reviewer on Upgrade bits

2018-07-20 Thread Marios Andreou 
On Fri, Jul 20, 2018 at 11:07 AM, Carlos Camacho Gonzalez <
ccama...@redhat.com> wrote:

> Hi!!!
>
> I'll like to propose Jose Luis Franco [1][2] for core reviewer in all the
> TripleO upgrades bits. He shows a constant and active involvement in
> improving and fixing our updates/upgrades workflows, he helps also trying
> to develop/improve/fix our upstream support for testing the
> updates/upgrades.
>
> Please vote -1/+1, and consider this my +1 vote :)
>

+1  of course!




>
>
> [1]: https://review.openstack.org/#/q/owner:jfrancoa%2540redhat.com
> [2]: http://stackalytics.com/?release=all&metric=commits&user_id=jfrancoa
>
> Cheers,
> Carlos.
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [tripleo] Proposing Jose Luis Franco for TripleO core reviewer on Upgrade bits

2018-07-20 Thread Bogdan Dobrelya 

On 7/20/18 11:07 AM, Carlos Camacho Gonzalez wrote:

Hi!!!

I'll like to propose Jose Luis Franco [1][2] for core reviewer in all 
the TripleO upgrades bits. He shows a constant and active involvement in 
improving and fixing our updates/upgrades workflows, he helps also 
trying to develop/improve/fix our upstream support for testing the 
updates/upgrades.


Please vote -1/+1, and consider this my +1 vote :)


+1!



[1]: https://review.openstack.org/#/q/owner:jfrancoa%2540redhat.com
[2]: http://stackalytics.com/?release=all&metric=commits&user_id=jfrancoa

Cheers,
Carlos.

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




--
Best regards,
Bogdan Dobrelya,
Irc #bogdando

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [tripleo] Proposing Jose Luis Franco for TripleO core reviewer on Upgrade bits

2018-07-20 Thread Carlos Camacho Gonzalez 
Hi!!!

I'll like to propose Jose Luis Franco [1][2] for core reviewer in all the
TripleO upgrades bits. He shows a constant and active involvement in
improving and fixing our updates/upgrades workflows, he helps also trying
to develop/improve/fix our upstream support for testing the
updates/upgrades.

Please vote -1/+1, and consider this my +1 vote :)

[1]: https://review.openstack.org/#/q/owner:jfrancoa%2540redhat.com
[2]: http://stackalytics.com/?release=all&metric=commits&user_id=jfrancoa

Cheers,
Carlos.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Disk space requirement - any way to lower it a little?

2018-07-20 Thread Cédric Jeanneret 


On 07/19/2018 06:55 PM, Paul Belanger wrote:
> On Thu, Jul 19, 2018 at 05:30:27PM +0200, Cédric Jeanneret wrote:
>> Hello,
>>
>> While trying to get a new validation¹ in the undercloud preflight
>> checks, I hit an (not so) unexpected issue with the CI:
>> it doesn't provide flavors with the minimal requirements, at least
>> regarding the disk space.
>>
>> A quick-fix is to disable the validations in the CI - Wes has already
>> pushed a patch for that in the upstream CI:
>> https://review.openstack.org/#/c/583275/
>> We can consider this as a quick'n'temporary fix².
>>
>> The issue is on the RDO CI: apparently, they provide instances with
>> "only" 55G of free space, making the checks fail:
>> https://logs.rdoproject.org/17/582917/3/openstack-check/legacy-tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001-master/c9cf398/logs/undercloud/home/zuul/undercloud_install.log.txt.gz#_2018-07-17_10_23_46
>>
>> So, the question is: would it be possible to lower the requirment to,
>> let's say, 50G? Where does that 60G³ come from?
>>
>> Thanks for your help/feedback.
>>
>> Cheers,
>>
>> C.
>>
>>
>>
>> ¹ https://review.openstack.org/#/c/582917/
>>
>> ² as you might know, there's a BP for a unified validation framework,
>> and it will allow to get injected configuration in CI env in order to
>> lower the requirements if necessary:
>> https://blueprints.launchpad.net/tripleo/+spec/validation-framework
>>
>> ³
>> http://tripleo.org/install/environments/baremetal.html#minimum-system-requirements
>>
> Keep in mind, upstream we don't really have control over partitions of nodes, 
> in
> some case it is a single, other multiple. I'd suggest looking more at:

After some checks on y locally deployed containerized undercloud (hence,
Rocky) without real activity, here's what I could get:
- most data are located in /var - this explains the current check.

If we go a bit deeper, here are the "actually used" directory in /var/lib:
20K alternatives
36K certmonger
4.0Kchrony
1.2Gconfig-data
4.0Kdhclient
6.0Gdocker
28K docker-config-scripts
92K docker-container-startup-configs.json
44K docker-puppet
592Kheat-config
832Kironic
4.0Kironic-inspector
236Kkolla
4.0Klogrotate
286Mmysql
48K neutron
4.0Kntp
4.0Kpostfix
872Kpuppet
3.8Mrabbitmq
59M rpm
4.0Krsyslog
64K systemd
20K tripleo
236Ktripleo-config
9.8Myum
7.5Gtotal

Most of the "default installer" partition schema don't go further than
putting /var, /tmp, /home and /usr in dedicated volumes - of course,
end-user can chose to ignore that and provide a custom schema.

That said, we can get the "used" paths. In addition to /var/lib, there's
obviously /usr.

We might want to:
- loop on known locations
- check if they are on dedicated mount points
- check the available disk space on those mount points.

An interesting thing in bash:
df /var/lib/docker
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/sda1  104846316 10188828  94657488  10% /

This allows to get:
- actual volume
- free space on the volume.

More than that, we might also try to figure out some pattern. For
instance, "docker" seems to be a pretty good candidate for space, as it
will get the images and container data. This is probably even the
biggest eater, at least on the undercloud - as well as the logs (/var/logs).

We might do a check ensuring we can, at least, DEPLOY the app. This
would require far less than the required 60G, and with a proper doc
announcing that, we can get a functional test, aiming on its purpose:
ensure we can deploy (so asking, let's say, 10G in /var/lib/docker, 5G
in /var/lib/config-data, 5G in /usr, 1G in /var/log) and, later, upgrade
(requiring the same amount of *free* space).

That would require some changes in the validation check of course. But
at least, we might get a pretty nice covering, while allowing it to run
smoothly in the CI.
But, as said: proper documentation should be set, and the "60G minimum
required" should be rephrased in order to point the locations needing
space (with the appropriate warning about "none exhaustiveness" and the
like).

Would that suit better the actual needs, and allow to get a proper disk
space check/validation?

Cheers,

C.


> 
>   https://docs.openstack.org/infra/manual/testing.html
> 
> As for downstream RDO, the same is going to apply once we start adding more
> cloud providers. I would look to see if you actually need that much space for
> deployments, and make try to mock the testing of that logic.
> 
> - Paul
> 
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

-- 
Cédric Jeanneret
Software Engineer
DFG:DF



signature.asc
Description: OpenPGP digital signature