Re: [openstack-dev] [TripleO] Network Configuration in TripleO UI

2016-12-17 Thread Steven Hardy
On Thu, Dec 15, 2016 at 03:46:05PM -0600, Ben Nemec wrote:
> 
> 
> On 12/08/2016 08:05 AM, Jiri Tomasek wrote:
> > Hi all,
> > 
> > I've been investigating how to implement TripleO network configuration
> > in TripleO UI. Based on my findings I'd like to propose a solution.
> > 
> > tl;dr proposal: Slightly refactor Network environment files to match GUI
> > usage, Use Jinja Templating to generate dynamic parts of the
> > templates/environments
> > 
> > 
> > # Overview
> > 
> > I've used Ben Nemec's amazing Network template generator as a reference
> > to help me understand how the network configuration works [1]. In
> > general the process of configuring the network in TripleO is:
> > 
> > Define which Networks we intend to use -> Assign Roles to the Networks
> > (+ Assign Role Services to the Network) -> Generate NIC config templates
> > based on previous information
> > 
> > 
> [snip]
> > # The proposal
> > 
> > So having described previous, here is the approach I think we should use
> > to achieve network configuration using TripleO UI:
> > 
> > 1. Put networks definitions into separate environment for each network:
> > - this way GUI can provide a list of networks available to use and let
> > user select which of them he wants to use. These environments are not
> > dynamic and if user wants to add a new network, he does so by creating
> > new templates and environment for it. UI also provides means to
> > configure parameters for each network at this point (if needed).
> > 
> > For example the environment for a Storage Network looks like this:
> > 
> > resource_registry:
> >   OS::TripleO::Network::Storage: ../network/storage.yaml
> >   OS::TripleO::Network::Ports::StorageVipPort:
> > ../network/ports/storage.yaml
> 
> This seems like an obvious improvement, and is essentially how my tool works
> too except that it munges all of the individual environments back together
> at the end.
> 
> Definite +1 from me.
> 
> > 
> > 2. Assign Roles to Networks
> > Having the Networks selected as well as Roles defined, TripleO UI
> > provides user with means to assign Roles to Networks. This step involves
> > generating the network-environment.yaml file. So TripleO UI sends the
> > mapping of roles to network in json format to tripleo-common which in
> > turn uses network-isolation.j2.yaml Jinja template to generate the
> > environment file. I expect that pre-defined network-isolation.yaml will
> > be included in default plan so the user does not need to start from
> > scratch. Tripleo-common also provides an action to fetch network-roles
> > assignment data by parsing the network-isolation.yaml
> > 
> > In addition, user is able to assign individual Role Services to a
> > Network. ServiceNetMap parameter is currently used for this. GUI needs
> > to make sure that it represents Services-Networks assignment grouped by
> > Role so it is ensured that user assigns Services to only networks where
> > their Role is assigned.
> 
> This sounds reasonable to me, but I do want to note that assigning roles to
> networks feels a little backwards to me.  I tend to think of a role as kind
> of the top level thing here, to which we assign other things (services and
> networks, for example).  So my mental model kind of looks like:

I agree - we already have a list of roles (in roles_data.yaml), so the
approach I've been experimenting with is to also define a list of networks,
then we can assign those networks to each role in roles_data.yaml (or some
other file containing the same data).

> 
> roles:
>   Controller:
> networks:
>   - provision
>   - external
>   - internal
>   ...

+1 this is pretty much what I have proposed here:

https://review.openstack.org/#/c/409920/2/roles_data.yaml

Then the missing piece is a list of networks (so that in future we can have
fully composable or custom networks), I proposed a network_data.yaml here:

https://review.openstack.org/#/c/409921/

There is some discussion about if we do that, or instead have a single file
e.g "plan_data.yaml" or similar, but logically I think both of these fit
with your preferred mental model? :)

>   Compute:
> networks:
>   -provision
>   ...
> 
> as opposed to what I think you're describing, which is
> 
> networks:
>   Provision:
> roles:
>   - controller
>   - compute
>   External:
> roles:
>   - controller
>   ...
> 
> Maybe there are benefits to modeling it as the latter, but I think the
> former fits better with the composable roles architecture.

I agree, this second approach is awkward wrt the custom roles architecture,
and I'd prefer the former.

Thanks!

Steve

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [TripleO] Network Configuration in TripleO UI

2016-12-15 Thread Ben Nemec



On 12/08/2016 08:05 AM, Jiri Tomasek wrote:

Hi all,

I've been investigating how to implement TripleO network configuration
in TripleO UI. Based on my findings I'd like to propose a solution.

tl;dr proposal: Slightly refactor Network environment files to match GUI
usage, Use Jinja Templating to generate dynamic parts of the
templates/environments


# Overview

I've used Ben Nemec's amazing Network template generator as a reference
to help me understand how the network configuration works [1]. In
general the process of configuring the network in TripleO is:

Define which Networks we intend to use -> Assign Roles to the Networks
(+ Assign Role Services to the Network) -> Generate NIC config templates
based on previous information



[snip]

# The proposal

So having described previous, here is the approach I think we should use
to achieve network configuration using TripleO UI:

1. Put networks definitions into separate environment for each network:
- this way GUI can provide a list of networks available to use and let
user select which of them he wants to use. These environments are not
dynamic and if user wants to add a new network, he does so by creating
new templates and environment for it. UI also provides means to
configure parameters for each network at this point (if needed).

For example the environment for a Storage Network looks like this:

resource_registry:
  OS::TripleO::Network::Storage: ../network/storage.yaml
  OS::TripleO::Network::Ports::StorageVipPort:
../network/ports/storage.yaml


This seems like an obvious improvement, and is essentially how my tool 
works too except that it munges all of the individual environments back 
together at the end.


Definite +1 from me.



2. Assign Roles to Networks
Having the Networks selected as well as Roles defined, TripleO UI
provides user with means to assign Roles to Networks. This step involves
generating the network-environment.yaml file. So TripleO UI sends the
mapping of roles to network in json format to tripleo-common which in
turn uses network-isolation.j2.yaml Jinja template to generate the
environment file. I expect that pre-defined network-isolation.yaml will
be included in default plan so the user does not need to start from
scratch. Tripleo-common also provides an action to fetch network-roles
assignment data by parsing the network-isolation.yaml

In addition, user is able to assign individual Role Services to a
Network. ServiceNetMap parameter is currently used for this. GUI needs
to make sure that it represents Services-Networks assignment grouped by
Role so it is ensured that user assigns Services to only networks where
their Role is assigned.


This sounds reasonable to me, but I do want to note that assigning roles 
to networks feels a little backwards to me.  I tend to think of a role 
as kind of the top level thing here, to which we assign other things 
(services and networks, for example).  So my mental model kind of looks 
like:


roles:
  Controller:
networks:
  - provision
  - external
  - internal
  ...
  Compute:
networks:
  -provision
  ...

as opposed to what I think you're describing, which is

networks:
  Provision:
roles:
  - controller
  - compute
  External:
roles:
  - controller
  ...

Maybe there are benefits to modeling it as the latter, but I think the 
former fits better with the composable roles architecture.




3. Generate NIC Config templates
TripleO UI provides means to configure NICS, Bonds etc. for each Role,
using the information from previous steps. It sends the data in json
format to tripleo-common which then generates nic config templates for
each Role based on network/config/nic-configs/role.j2.yaml Jinja
template and generates network-environment.yaml based on
network-environment.j2.yaml which references those templates.

Note that network-environment.j2.yaml probably can't be combined with
network-isolation.j2.yaml as every time that environment would need to
get updated, all data the template needs would need to be provided.

There are wireframes made by Liz Blanchard currently available [5],
althought they are not exactly up to date to this proposal. Ideally
whole network configuration would happen on a screen based on the
graphical representation of network [6].


Any comments to this proposal are very welcome, please note that I am
not a networking expert so I might be missing something.

There is a spec [7] in progress aimed for Ocata, but the feature will
highly probably not land in Ocata, so we'll need to update the spec and
move it to next cycle.


[1]
http://blog.nemebean.com/content/tripleo-network-isolation-template-generator

[2] https://github.com/openstack/tripleo-heat-templates
[3]
https://github.com/openstack/tripleo-heat-templates/blob/master/environments/network-environment.yaml

[4]
https://github.com/openstack/tripleo-heat-templates/blob/master/environments/network-isolation.yaml

[5] 

Re: [openstack-dev] [TripleO] Network Configuration in TripleO UI

2016-12-13 Thread Dan Sneddon
On 12/08/2016 08:10 PM, Jason Rist wrote:
> On 12/08/2016 05:28 PM, Dan Sneddon wrote:
>> On 12/08/2016 06:05 AM, Jiri Tomasek wrote:
>>> Hi all,
>>>
>>> I've been investigating how to implement TripleO network configuration
>>> in TripleO UI. Based on my findings I'd like to propose a solution.
>>>
>>> tl;dr proposal: Slightly refactor Network environment files to match
>>> GUI usage, Use Jinja Templating to generate dynamic parts of the
>>> templates/environments
>>>
>>>
>>> # Overview
>>>
>>> I've used Ben Nemec's amazing Network template generator as a reference
>>> to help me understand how the network configuration works [1]. In
>>> general the process of configuring the network in TripleO is:
>>>
>>> Define which Networks we intend to use -> Assign Roles to the Networks
>>> (+ Assign Role Services to the Network) -> Generate NIC config
>>> templates based on previous information
>>>
>>>
>>> # Deeper dive into templates
>>>
>>> We currently have 2 environment files in THT [2] which define network
>>> configuration:
>>>
>>> network-environment.yaml [3] - holds the information on NIC
>>> configuration for each Role using
>>> OS::TripleONet::SoftwareConfig resource + related
>>> parameter configuration
>>>
>>> network-isolation.yaml [4]
>>> - defines the list of networks using
>>> OS::TripleO::Network:: resource
>>> - defines ports configuration for each network using
>>> OS::TripleO::Network::Ports::VipPort (note that both
>>> resources point to the static templates - those templates don't require
>>> any manual modification)
>>> - holds  Roles - Networks assignment using
>>> OS::TripleOPorts::Port for each role and
>>> storage (again, templates referenced by those resources don't require
>>> any modification)
>>>
>>> User is intended to go ahead and modify those environments and provide
>>> NIC config templates to achieve a network configuration that matches
>>> his needs.
>>>
>>>
>>> # How GUI works
>>>
>>> Before proceeding to proposed changes I need to describe briefly how
>>> TripleO UI works. TripleO UI is using THT as a source of truth, which
>>> means that it is trying not to add any additional business logic or
>>> manipulate templates. Rather it uses environment files as a 'features'
>>> which user can enable or disable depending on the needs of the
>>> deployment. The information about inter-environment relationships is
>>> tracked in capabilities-map.yaml which is also part of the THT. Based
>>> on these choices, UI allows user to configure parameters for those
>>> features. The parameter values and information about which environments
>>> are selected is stored in mistral environment. This approach leaves the
>>> plan templates intact. Huge benefit of this approach is that UI (or
>>> tripleo-common) does not need to hold explicit business logic related
>>> to certain deployment features as it is purely driven by THT. Also
>>> Adding a new feature involves only providing the templates/environments
>>> and it automatically appears as an option in UI.
>>>
>>> To achieve best user experience while using this approach, the
>>> environment files need to be defined in a granular manner, so they
>>> don't require user to modify them and each describe an isolated 'feature'.
>>>
>>> Roles and Network Configuration are exceptions to this concept as they
>>> require modification/generation of the templates/environments and
>>> therefore they use Jinja templating to achieve that.
>>>
>>>
>>> # The proposal
>>>
>>> So having described previous, here is the approach I think we should
>>> use to achieve network configuration using TripleO UI:
>>>
>>> 1. Put networks definitions into separate environment for each network:
>>> - this way GUI can provide a list of networks available to use and let
>>> user select which of them he wants to use. These environments are not
>>> dynamic and if user wants to add a new network, he does so by creating
>>> new templates and environment for it. UI also provides means to
>>> configure parameters for each network at this point (if needed).
>>>
>>> For example the environment for a Storage Network looks like this:
>>>
>>> resource_registry:
>>>   OS::TripleO::Network::Storage: ../network/storage.yaml
>>>   OS::TripleO::Network::Ports::StorageVipPort:
>>> ../network/ports/storage.yaml
>>>
>>> 2. Assign Roles to Networks
>>> Having the Networks selected as well as Roles defined, TripleO UI
>>> provides user with means to assign Roles to Networks. This step
>>> involves generating the network-environment.yaml file. So TripleO UI
>>> sends the mapping of roles to network in json format to tripleo-common
>>> which in turn uses network-isolation.j2.yaml Jinja template to generate
>>> the environment file. I expect that pre-defined network-isolation.yaml
>>> will be included in default plan so the user does not need to start
>>> from scratch. Tripleo-common also provides an action to fetch
>>> network-roles assignment data by parsing the network-isolation.yaml
>>>
>>> 

Re: [openstack-dev] [TripleO] Network Configuration in TripleO UI

2016-12-12 Thread Dan Sneddon
On 12/08/2016 06:05 AM, Jiri Tomasek wrote:
> Hi all,
> 
> I've been investigating how to implement TripleO network configuration
> in TripleO UI. Based on my findings I'd like to propose a solution.
> 
> tl;dr proposal: Slightly refactor Network environment files to match
> GUI usage, Use Jinja Templating to generate dynamic parts of the
> templates/environments
> 
> 
> # Overview
> 
> I've used Ben Nemec's amazing Network template generator as a reference
> to help me understand how the network configuration works [1]. In
> general the process of configuring the network in TripleO is:
> 
> Define which Networks we intend to use -> Assign Roles to the Networks
> (+ Assign Role Services to the Network) -> Generate NIC config
> templates based on previous information
> 
> 
> # Deeper dive into templates
> 
> We currently have 2 environment files in THT [2] which define network
> configuration:
> 
> network-environment.yaml [3] - holds the information on NIC
> configuration for each Role using
> OS::TripleONet::SoftwareConfig resource + related
> parameter configuration
> 
> network-isolation.yaml [4]
> - defines the list of networks using
> OS::TripleO::Network:: resource
> - defines ports configuration for each network using
> OS::TripleO::Network::Ports::VipPort (note that both
> resources point to the static templates - those templates don't require
> any manual modification)
> - holds  Roles - Networks assignment using
> OS::TripleOPorts::Port for each role and
> storage (again, templates referenced by those resources don't require
> any modification)
> 
> User is intended to go ahead and modify those environments and provide
> NIC config templates to achieve a network configuration that matches
> his needs.
> 
> 
> # How GUI works
> 
> Before proceeding to proposed changes I need to describe briefly how
> TripleO UI works. TripleO UI is using THT as a source of truth, which
> means that it is trying not to add any additional business logic or
> manipulate templates. Rather it uses environment files as a 'features'
> which user can enable or disable depending on the needs of the
> deployment. The information about inter-environment relationships is
> tracked in capabilities-map.yaml which is also part of the THT. Based
> on these choices, UI allows user to configure parameters for those
> features. The parameter values and information about which environments
> are selected is stored in mistral environment. This approach leaves the
> plan templates intact. Huge benefit of this approach is that UI (or
> tripleo-common) does not need to hold explicit business logic related
> to certain deployment features as it is purely driven by THT. Also
> Adding a new feature involves only providing the templates/environments
> and it automatically appears as an option in UI.
> 
> To achieve best user experience while using this approach, the
> environment files need to be defined in a granular manner, so they
> don't require user to modify them and each describe an isolated 'feature'.
> 
> Roles and Network Configuration are exceptions to this concept as they
> require modification/generation of the templates/environments and
> therefore they use Jinja templating to achieve that.
> 
> 
> # The proposal
> 
> So having described previous, here is the approach I think we should
> use to achieve network configuration using TripleO UI:
> 
> 1. Put networks definitions into separate environment for each network:
> - this way GUI can provide a list of networks available to use and let
> user select which of them he wants to use. These environments are not
> dynamic and if user wants to add a new network, he does so by creating
> new templates and environment for it. UI also provides means to
> configure parameters for each network at this point (if needed).
> 
> For example the environment for a Storage Network looks like this:
> 
> resource_registry:
>   OS::TripleO::Network::Storage: ../network/storage.yaml
>   OS::TripleO::Network::Ports::StorageVipPort:
> ../network/ports/storage.yaml
> 
> 2. Assign Roles to Networks
> Having the Networks selected as well as Roles defined, TripleO UI
> provides user with means to assign Roles to Networks. This step
> involves generating the network-environment.yaml file. So TripleO UI
> sends the mapping of roles to network in json format to tripleo-common
> which in turn uses network-isolation.j2.yaml Jinja template to generate
> the environment file. I expect that pre-defined network-isolation.yaml
> will be included in default plan so the user does not need to start
> from scratch. Tripleo-common also provides an action to fetch
> network-roles assignment data by parsing the network-isolation.yaml
> 
> In addition, user is able to assign individual Role Services to a
> Network. ServiceNetMap parameter is currently used for this. GUI needs
> to make sure that it represents Services-Networks assignment grouped by
> Role so it is ensured that user assigns Services to only 

Re: [openstack-dev] [TripleO] Network Configuration in TripleO UI

2016-12-08 Thread Jason Rist
On 12/08/2016 05:28 PM, Dan Sneddon wrote:
> On 12/08/2016 06:05 AM, Jiri Tomasek wrote:
> > Hi all,
> >
> > I've been investigating how to implement TripleO network configuration
> > in TripleO UI. Based on my findings I'd like to propose a solution.
> >
> > tl;dr proposal: Slightly refactor Network environment files to match
> > GUI usage, Use Jinja Templating to generate dynamic parts of the
> > templates/environments
> >
> >
> > # Overview
> >
> > I've used Ben Nemec's amazing Network template generator as a reference
> > to help me understand how the network configuration works [1]. In
> > general the process of configuring the network in TripleO is:
> >
> > Define which Networks we intend to use -> Assign Roles to the Networks
> > (+ Assign Role Services to the Network) -> Generate NIC config
> > templates based on previous information
> >
> >
> > # Deeper dive into templates
> >
> > We currently have 2 environment files in THT [2] which define network
> > configuration:
> >
> > network-environment.yaml [3] - holds the information on NIC
> > configuration for each Role using
> > OS::TripleONet::SoftwareConfig resource + related
> > parameter configuration
> >
> > network-isolation.yaml [4]
> > - defines the list of networks using
> > OS::TripleO::Network:: resource
> > - defines ports configuration for each network using
> > OS::TripleO::Network::Ports::VipPort (note that both
> > resources point to the static templates - those templates don't require
> > any manual modification)
> > - holds  Roles - Networks assignment using
> > OS::TripleOPorts::Port for each role and
> > storage (again, templates referenced by those resources don't require
> > any modification)
> >
> > User is intended to go ahead and modify those environments and provide
> > NIC config templates to achieve a network configuration that matches
> > his needs.
> >
> >
> > # How GUI works
> >
> > Before proceeding to proposed changes I need to describe briefly how
> > TripleO UI works. TripleO UI is using THT as a source of truth, which
> > means that it is trying not to add any additional business logic or
> > manipulate templates. Rather it uses environment files as a 'features'
> > which user can enable or disable depending on the needs of the
> > deployment. The information about inter-environment relationships is
> > tracked in capabilities-map.yaml which is also part of the THT. Based
> > on these choices, UI allows user to configure parameters for those
> > features. The parameter values and information about which environments
> > are selected is stored in mistral environment. This approach leaves the
> > plan templates intact. Huge benefit of this approach is that UI (or
> > tripleo-common) does not need to hold explicit business logic related
> > to certain deployment features as it is purely driven by THT. Also
> > Adding a new feature involves only providing the templates/environments
> > and it automatically appears as an option in UI.
> >
> > To achieve best user experience while using this approach, the
> > environment files need to be defined in a granular manner, so they
> > don't require user to modify them and each describe an isolated 'feature'.
> >
> > Roles and Network Configuration are exceptions to this concept as they
> > require modification/generation of the templates/environments and
> > therefore they use Jinja templating to achieve that.
> >
> >
> > # The proposal
> >
> > So having described previous, here is the approach I think we should
> > use to achieve network configuration using TripleO UI:
> >
> > 1. Put networks definitions into separate environment for each network:
> > - this way GUI can provide a list of networks available to use and let
> > user select which of them he wants to use. These environments are not
> > dynamic and if user wants to add a new network, he does so by creating
> > new templates and environment for it. UI also provides means to
> > configure parameters for each network at this point (if needed).
> >
> > For example the environment for a Storage Network looks like this:
> >
> > resource_registry:
> >   OS::TripleO::Network::Storage: ../network/storage.yaml
> >   OS::TripleO::Network::Ports::StorageVipPort:
> > ../network/ports/storage.yaml
> >
> > 2. Assign Roles to Networks
> > Having the Networks selected as well as Roles defined, TripleO UI
> > provides user with means to assign Roles to Networks. This step
> > involves generating the network-environment.yaml file. So TripleO UI
> > sends the mapping of roles to network in json format to tripleo-common
> > which in turn uses network-isolation.j2.yaml Jinja template to generate
> > the environment file. I expect that pre-defined network-isolation.yaml
> > will be included in default plan so the user does not need to start
> > from scratch. Tripleo-common also provides an action to fetch
> > network-roles assignment data by parsing the network-isolation.yaml
> >
> > In addition, user is able to assign 

Re: [openstack-dev] [TripleO] Network Configuration in TripleO UI

2016-12-08 Thread Dan Sneddon
On 12/08/2016 06:05 AM, Jiri Tomasek wrote:
> Hi all,
> 
> I've been investigating how to implement TripleO network configuration
> in TripleO UI. Based on my findings I'd like to propose a solution.
> 
> tl;dr proposal: Slightly refactor Network environment files to match
> GUI usage, Use Jinja Templating to generate dynamic parts of the
> templates/environments
> 
> 
> # Overview
> 
> I've used Ben Nemec's amazing Network template generator as a reference
> to help me understand how the network configuration works [1]. In
> general the process of configuring the network in TripleO is:
> 
> Define which Networks we intend to use -> Assign Roles to the Networks
> (+ Assign Role Services to the Network) -> Generate NIC config
> templates based on previous information
> 
> 
> # Deeper dive into templates
> 
> We currently have 2 environment files in THT [2] which define network
> configuration:
> 
> network-environment.yaml [3] - holds the information on NIC
> configuration for each Role using
> OS::TripleONet::SoftwareConfig resource + related
> parameter configuration
> 
> network-isolation.yaml [4]
> - defines the list of networks using
> OS::TripleO::Network:: resource
> - defines ports configuration for each network using
> OS::TripleO::Network::Ports::VipPort (note that both
> resources point to the static templates - those templates don't require
> any manual modification)
> - holds  Roles - Networks assignment using
> OS::TripleOPorts::Port for each role and
> storage (again, templates referenced by those resources don't require
> any modification)
> 
> User is intended to go ahead and modify those environments and provide
> NIC config templates to achieve a network configuration that matches
> his needs.
> 
> 
> # How GUI works
> 
> Before proceeding to proposed changes I need to describe briefly how
> TripleO UI works. TripleO UI is using THT as a source of truth, which
> means that it is trying not to add any additional business logic or
> manipulate templates. Rather it uses environment files as a 'features'
> which user can enable or disable depending on the needs of the
> deployment. The information about inter-environment relationships is
> tracked in capabilities-map.yaml which is also part of the THT. Based
> on these choices, UI allows user to configure parameters for those
> features. The parameter values and information about which environments
> are selected is stored in mistral environment. This approach leaves the
> plan templates intact. Huge benefit of this approach is that UI (or
> tripleo-common) does not need to hold explicit business logic related
> to certain deployment features as it is purely driven by THT. Also
> Adding a new feature involves only providing the templates/environments
> and it automatically appears as an option in UI.
> 
> To achieve best user experience while using this approach, the
> environment files need to be defined in a granular manner, so they
> don't require user to modify them and each describe an isolated 'feature'.
> 
> Roles and Network Configuration are exceptions to this concept as they
> require modification/generation of the templates/environments and
> therefore they use Jinja templating to achieve that.
> 
> 
> # The proposal
> 
> So having described previous, here is the approach I think we should
> use to achieve network configuration using TripleO UI:
> 
> 1. Put networks definitions into separate environment for each network:
> - this way GUI can provide a list of networks available to use and let
> user select which of them he wants to use. These environments are not
> dynamic and if user wants to add a new network, he does so by creating
> new templates and environment for it. UI also provides means to
> configure parameters for each network at this point (if needed).
> 
> For example the environment for a Storage Network looks like this:
> 
> resource_registry:
>   OS::TripleO::Network::Storage: ../network/storage.yaml
>   OS::TripleO::Network::Ports::StorageVipPort:
> ../network/ports/storage.yaml
> 
> 2. Assign Roles to Networks
> Having the Networks selected as well as Roles defined, TripleO UI
> provides user with means to assign Roles to Networks. This step
> involves generating the network-environment.yaml file. So TripleO UI
> sends the mapping of roles to network in json format to tripleo-common
> which in turn uses network-isolation.j2.yaml Jinja template to generate
> the environment file. I expect that pre-defined network-isolation.yaml
> will be included in default plan so the user does not need to start
> from scratch. Tripleo-common also provides an action to fetch
> network-roles assignment data by parsing the network-isolation.yaml
> 
> In addition, user is able to assign individual Role Services to a
> Network. ServiceNetMap parameter is currently used for this. GUI needs
> to make sure that it represents Services-Networks assignment grouped by
> Role so it is ensured that user assigns Services to only 

[openstack-dev] [TripleO] Network Configuration in TripleO UI

2016-12-08 Thread Jiri Tomasek

Hi all,

I've been investigating how to implement TripleO network configuration 
in TripleO UI. Based on my findings I'd like to propose a solution.


tl;dr proposal: Slightly refactor Network environment files to match GUI 
usage, Use Jinja Templating to generate dynamic parts of the 
templates/environments



# Overview

I've used Ben Nemec's amazing Network template generator as a reference 
to help me understand how the network configuration works [1]. In 
general the process of configuring the network in TripleO is:


Define which Networks we intend to use -> Assign Roles to the Networks 
(+ Assign Role Services to the Network) -> Generate NIC config templates 
based on previous information



# Deeper dive into templates

We currently have 2 environment files in THT [2] which define network 
configuration:


network-environment.yaml [3] - holds the information on NIC 
configuration for each Role using 
OS::TripleONet::SoftwareConfig resource + related 
parameter configuration


network-isolation.yaml [4]
- defines the list of networks using OS::TripleO::Network:: 
resource
- defines ports configuration for each network using 
OS::TripleO::Network::Ports::VipPort (note that both 
resources point to the static templates - those templates don't require 
any manual modification)
- holds  Roles - Networks assignment using 
OS::TripleOPorts::Port for each role and 
storage (again, templates referenced by those resources don't require 
any modification)


User is intended to go ahead and modify those environments and provide 
NIC config templates to achieve a network configuration that matches his 
needs.



# How GUI works

Before proceeding to proposed changes I need to describe briefly how 
TripleO UI works. TripleO UI is using THT as a source of truth, which 
means that it is trying not to add any additional business logic or 
manipulate templates. Rather it uses environment files as a 'features' 
which user can enable or disable depending on the needs of the 
deployment. The information about inter-environment relationships is 
tracked in capabilities-map.yaml which is also part of the THT. Based on 
these choices, UI allows user to configure parameters for those 
features. The parameter values and information about which environments 
are selected is stored in mistral environment. This approach leaves the 
plan templates intact. Huge benefit of this approach is that UI (or 
tripleo-common) does not need to hold explicit business logic related to 
certain deployment features as it is purely driven by THT. Also Adding a 
new feature involves only providing the templates/environments and it 
automatically appears as an option in UI.


To achieve best user experience while using this approach, the 
environment files need to be defined in a granular manner, so they don't 
require user to modify them and each describe an isolated 'feature'.


Roles and Network Configuration are exceptions to this concept as they 
require modification/generation of the templates/environments and 
therefore they use Jinja templating to achieve that.



# The proposal

So having described previous, here is the approach I think we should use 
to achieve network configuration using TripleO UI:


1. Put networks definitions into separate environment for each network:
- this way GUI can provide a list of networks available to use and let 
user select which of them he wants to use. These environments are not 
dynamic and if user wants to add a new network, he does so by creating 
new templates and environment for it. UI also provides means to 
configure parameters for each network at this point (if needed).


For example the environment for a Storage Network looks like this:

resource_registry:
  OS::TripleO::Network::Storage: ../network/storage.yaml
  OS::TripleO::Network::Ports::StorageVipPort: ../network/ports/storage.yaml

2. Assign Roles to Networks
Having the Networks selected as well as Roles defined, TripleO UI 
provides user with means to assign Roles to Networks. This step involves 
generating the network-environment.yaml file. So TripleO UI sends the 
mapping of roles to network in json format to tripleo-common which in 
turn uses network-isolation.j2.yaml Jinja template to generate the 
environment file. I expect that pre-defined network-isolation.yaml will 
be included in default plan so the user does not need to start from 
scratch. Tripleo-common also provides an action to fetch network-roles 
assignment data by parsing the network-isolation.yaml


In addition, user is able to assign individual Role Services to a 
Network. ServiceNetMap parameter is currently used for this. GUI needs 
to make sure that it represents Services-Networks assignment grouped by 
Role so it is ensured that user assigns Services to only networks where 
their Role is assigned.


3. Generate NIC Config templates
TripleO UI provides means to configure NICS, Bonds etc. for each Role, 
using the information from previous steps. It sends