Re: [openstack-dev] [openstack-ansible][kolla-ansible][tripleo] ansible roles: where they live and what do they do

2018-09-12 Thread Jiří Stránský

On 11.9.2018 18:53, Alex Schultz wrote:

Thanks everyone for coming and chatting.  From the meeting we've had a
few items where we can collaborate together.

Here are some specific bullet points:

- TripleO folks should feel free to propose some minor structural
changes if they make the integration easier.  TripleO is currently
investigating what it would look like to pull the keystone ansible
parts out of tripleo-heat-templates and put it into
ansible-role-tripleo-keystone.  It would be beneficial to use this
role as an example for how the os_keystone role can be consumed.


+1, please let's also focus on information flow towards Upgrades squad 
(and collab if needed) as ansible-role-tripleo-keystone is being 
implemented. It sounds like something that might potentially require 
rethinking how updates/upgrades work too. Maybe we could have a spec 
where the solution is described and we can assess/discuss the upgrades 
impact.


Thanks

Jirka


- The openstack-ansible-tests has some good examples of ansible-lint
rules that can be used to improve quality
- Tags could be used to limit the scope of OpenStack Ansible roles,
but it sounds like including tasks would be a better pattern.
- Need to establish a pattern for disabling packaging/service
configurations globally in OpenStack Ansible roles.
- Shared roles are open for reuse/replacement if something better is
available (upstream/elsewhere).

If anyone has any others, feel free to comment.

Thanks,
-Alex

On Mon, Sep 10, 2018 at 10:58 AM, Alex Schultz  wrote:

I just realized I booked the room and put it in the etherpad but
forgot to email out the time.

Time: Tuesday 09:00-10:45
Room: Big Thompson

https://etherpad.openstack.org/p/ansible-collaboration-denver-ptg

Thanks,
-Alex

On Tue, Sep 4, 2018 at 1:03 PM, Alex Schultz  wrote:

On Thu, Aug 9, 2018 at 2:43 PM, Mohammed Naser  wrote:

Hi Alex,

I am very much in favour of what you're bringing up.  We do have
multiple projects that leverage Ansible in different ways and we all
end up doing the same thing at the end.  The duplication of work is
not really beneficial for us as it takes away from our use-cases.

I believe that there is a certain number of steps that we all share
regardless of how we deploy, some of the things that come up to me
right away are:

- Configuring infrastructure services (i.e.: create vhosts for service
in rabbitmq, create databases for services, configure users for
rabbitmq, db, etc)
- Configuring inter-OpenStack services (i.e. keystone_authtoken
section, creating endpoints, etc and users for services)
- Configuring actual OpenStack services (i.e.
/etc//.conf file with the ability of extending
options)
- Running CI/integration on a cloud (i.e. common role that literally
gets an admin user, password and auth endpoint and creates all
resources and does CI)

This would deduplicate a lot of work, and especially the last one, it
might be beneficial for more than Ansible-based projects, I can
imagine Puppet OpenStack leveraging this as well inside Zuul CI
(optionally)... However, I think that this something which we should
discus further for the PTG.  I think that there will be a tiny bit
upfront work as we all standarize but then it's a win for all involved
communities.

I would like to propose that deployment tools maybe sit down together
at the PTG, all share how we use Ansible to accomplish these tasks and
then perhaps we can work all together on abstracting some of these
concepts together for us to all leverage.



I'm currently trying to get a spot on Tuesday morning to further
discuss some of this items.  In the mean time I've started an
etherpad[0] to start collecting ideas for things to discuss.  At the
moment I've got the tempest role collaboration and some basic ideas
for best practice items that we can discuss.  Feel free to add your
own and I'll update the etherpad with a time slot when I get one
nailed down.

Thanks,
-Alex

[0] https://etherpad.openstack.org/p/ansible-collaboration-denver-ptg


I'll let others chime in as well.

Regards,
Mohammed

On Thu, Aug 9, 2018 at 4:31 PM, Alex Schultz  wrote:

Ahoy folks,

I think it's time we come up with some basic rules/patterns on where
code lands when it comes to OpenStack related Ansible roles and as we
convert/export things. There was a recent proposal to create an
ansible-role-tempest[0] that would take what we use in
tripleo-quickstart-extras[1] and separate it for re-usability by
others.   So it was asked if we could work with the openstack-ansible
team and leverage the existing openstack-ansible-os_tempest[2].  It
turns out we have a few more already existing roles laying around as
well[3][4].

What I would like to propose is that we as a community come together
to agree on specific patterns so that we can leverage the same roles
for some of the core configuration/deployment functionality while
still allowing for specific project specific customization.  What I've
noticed between all the project is that we have a few 

Re: [openstack-dev] [openstack-ansible][kolla-ansible][tripleo] ansible roles: where they live and what do they do

2018-09-11 Thread Alex Schultz
Thanks everyone for coming and chatting.  From the meeting we've had a
few items where we can collaborate together.

Here are some specific bullet points:

- TripleO folks should feel free to propose some minor structural
changes if they make the integration easier.  TripleO is currently
investigating what it would look like to pull the keystone ansible
parts out of tripleo-heat-templates and put it into
ansible-role-tripleo-keystone.  It would be beneficial to use this
role as an example for how the os_keystone role can be consumed.
- The openstack-ansible-tests has some good examples of ansible-lint
rules that can be used to improve quality
- Tags could be used to limit the scope of OpenStack Ansible roles,
but it sounds like including tasks would be a better pattern.
- Need to establish a pattern for disabling packaging/service
configurations globally in OpenStack Ansible roles.
- Shared roles are open for reuse/replacement if something better is
available (upstream/elsewhere).

If anyone has any others, feel free to comment.

Thanks,
-Alex

On Mon, Sep 10, 2018 at 10:58 AM, Alex Schultz  wrote:
> I just realized I booked the room and put it in the etherpad but
> forgot to email out the time.
>
> Time: Tuesday 09:00-10:45
> Room: Big Thompson
>
> https://etherpad.openstack.org/p/ansible-collaboration-denver-ptg
>
> Thanks,
> -Alex
>
> On Tue, Sep 4, 2018 at 1:03 PM, Alex Schultz  wrote:
>> On Thu, Aug 9, 2018 at 2:43 PM, Mohammed Naser  wrote:
>>> Hi Alex,
>>>
>>> I am very much in favour of what you're bringing up.  We do have
>>> multiple projects that leverage Ansible in different ways and we all
>>> end up doing the same thing at the end.  The duplication of work is
>>> not really beneficial for us as it takes away from our use-cases.
>>>
>>> I believe that there is a certain number of steps that we all share
>>> regardless of how we deploy, some of the things that come up to me
>>> right away are:
>>>
>>> - Configuring infrastructure services (i.e.: create vhosts for service
>>> in rabbitmq, create databases for services, configure users for
>>> rabbitmq, db, etc)
>>> - Configuring inter-OpenStack services (i.e. keystone_authtoken
>>> section, creating endpoints, etc and users for services)
>>> - Configuring actual OpenStack services (i.e.
>>> /etc//.conf file with the ability of extending
>>> options)
>>> - Running CI/integration on a cloud (i.e. common role that literally
>>> gets an admin user, password and auth endpoint and creates all
>>> resources and does CI)
>>>
>>> This would deduplicate a lot of work, and especially the last one, it
>>> might be beneficial for more than Ansible-based projects, I can
>>> imagine Puppet OpenStack leveraging this as well inside Zuul CI
>>> (optionally)... However, I think that this something which we should
>>> discus further for the PTG.  I think that there will be a tiny bit
>>> upfront work as we all standarize but then it's a win for all involved
>>> communities.
>>>
>>> I would like to propose that deployment tools maybe sit down together
>>> at the PTG, all share how we use Ansible to accomplish these tasks and
>>> then perhaps we can work all together on abstracting some of these
>>> concepts together for us to all leverage.
>>>
>>
>> I'm currently trying to get a spot on Tuesday morning to further
>> discuss some of this items.  In the mean time I've started an
>> etherpad[0] to start collecting ideas for things to discuss.  At the
>> moment I've got the tempest role collaboration and some basic ideas
>> for best practice items that we can discuss.  Feel free to add your
>> own and I'll update the etherpad with a time slot when I get one
>> nailed down.
>>
>> Thanks,
>> -Alex
>>
>> [0] https://etherpad.openstack.org/p/ansible-collaboration-denver-ptg
>>
>>> I'll let others chime in as well.
>>>
>>> Regards,
>>> Mohammed
>>>
>>> On Thu, Aug 9, 2018 at 4:31 PM, Alex Schultz  wrote:
 Ahoy folks,

 I think it's time we come up with some basic rules/patterns on where
 code lands when it comes to OpenStack related Ansible roles and as we
 convert/export things. There was a recent proposal to create an
 ansible-role-tempest[0] that would take what we use in
 tripleo-quickstart-extras[1] and separate it for re-usability by
 others.   So it was asked if we could work with the openstack-ansible
 team and leverage the existing openstack-ansible-os_tempest[2].  It
 turns out we have a few more already existing roles laying around as
 well[3][4].

 What I would like to propose is that we as a community come together
 to agree on specific patterns so that we can leverage the same roles
 for some of the core configuration/deployment functionality while
 still allowing for specific project specific customization.  What I've
 noticed between all the project is that we have a few specific core
 pieces of functionality that needs to be handled (or skipped as it may
 be) for each service being 

Re: [openstack-dev] [openstack-ansible][kolla-ansible][tripleo] ansible roles: where they live and what do they do

2018-09-10 Thread Alex Schultz
I just realized I booked the room and put it in the etherpad but
forgot to email out the time.

Time: Tuesday 09:00-10:45
Room: Big Thompson

https://etherpad.openstack.org/p/ansible-collaboration-denver-ptg

Thanks,
-Alex

On Tue, Sep 4, 2018 at 1:03 PM, Alex Schultz  wrote:
> On Thu, Aug 9, 2018 at 2:43 PM, Mohammed Naser  wrote:
>> Hi Alex,
>>
>> I am very much in favour of what you're bringing up.  We do have
>> multiple projects that leverage Ansible in different ways and we all
>> end up doing the same thing at the end.  The duplication of work is
>> not really beneficial for us as it takes away from our use-cases.
>>
>> I believe that there is a certain number of steps that we all share
>> regardless of how we deploy, some of the things that come up to me
>> right away are:
>>
>> - Configuring infrastructure services (i.e.: create vhosts for service
>> in rabbitmq, create databases for services, configure users for
>> rabbitmq, db, etc)
>> - Configuring inter-OpenStack services (i.e. keystone_authtoken
>> section, creating endpoints, etc and users for services)
>> - Configuring actual OpenStack services (i.e.
>> /etc//.conf file with the ability of extending
>> options)
>> - Running CI/integration on a cloud (i.e. common role that literally
>> gets an admin user, password and auth endpoint and creates all
>> resources and does CI)
>>
>> This would deduplicate a lot of work, and especially the last one, it
>> might be beneficial for more than Ansible-based projects, I can
>> imagine Puppet OpenStack leveraging this as well inside Zuul CI
>> (optionally)... However, I think that this something which we should
>> discus further for the PTG.  I think that there will be a tiny bit
>> upfront work as we all standarize but then it's a win for all involved
>> communities.
>>
>> I would like to propose that deployment tools maybe sit down together
>> at the PTG, all share how we use Ansible to accomplish these tasks and
>> then perhaps we can work all together on abstracting some of these
>> concepts together for us to all leverage.
>>
>
> I'm currently trying to get a spot on Tuesday morning to further
> discuss some of this items.  In the mean time I've started an
> etherpad[0] to start collecting ideas for things to discuss.  At the
> moment I've got the tempest role collaboration and some basic ideas
> for best practice items that we can discuss.  Feel free to add your
> own and I'll update the etherpad with a time slot when I get one
> nailed down.
>
> Thanks,
> -Alex
>
> [0] https://etherpad.openstack.org/p/ansible-collaboration-denver-ptg
>
>> I'll let others chime in as well.
>>
>> Regards,
>> Mohammed
>>
>> On Thu, Aug 9, 2018 at 4:31 PM, Alex Schultz  wrote:
>>> Ahoy folks,
>>>
>>> I think it's time we come up with some basic rules/patterns on where
>>> code lands when it comes to OpenStack related Ansible roles and as we
>>> convert/export things. There was a recent proposal to create an
>>> ansible-role-tempest[0] that would take what we use in
>>> tripleo-quickstart-extras[1] and separate it for re-usability by
>>> others.   So it was asked if we could work with the openstack-ansible
>>> team and leverage the existing openstack-ansible-os_tempest[2].  It
>>> turns out we have a few more already existing roles laying around as
>>> well[3][4].
>>>
>>> What I would like to propose is that we as a community come together
>>> to agree on specific patterns so that we can leverage the same roles
>>> for some of the core configuration/deployment functionality while
>>> still allowing for specific project specific customization.  What I've
>>> noticed between all the project is that we have a few specific core
>>> pieces of functionality that needs to be handled (or skipped as it may
>>> be) for each service being deployed.
>>>
>>> 1) software installation
>>> 2) configuration management
>>> 3) service management
>>> 4) misc service actions
>>>
>>> Depending on which flavor of the deployment you're using, the content
>>> of each of these may be different.  Just about the only thing that is
>>> shared between them all would be the configuration management part.
>>> To that, I was wondering if there would be a benefit to establishing a
>>> pattern within say openstack-ansible where we can disable items #1 and
>>> #3 but reuse #2 in projects like kolla/tripleo where we need to do
>>> some configuration generation.  If we can't establish a similar
>>> pattern it'll make it harder to reuse and contribute between the
>>> various projects.
>>>
>>> In tripleo we've recently created a bunch of ansible-role-tripleo-*
>>> repositories which we were planning on moving the tripleo specific
>>> tasks (for upgrades, etc) to and were hoping that we might be able to
>>> reuse the upstream ansible roles similar to how we've previously
>>> leverage the puppet openstack work for configurations.  So for us, it
>>> would be beneficial if we could maybe help align/contribute/guide the
>>> configuration management and maybe 

Re: [openstack-dev] [openstack-ansible][kolla-ansible][tripleo] ansible roles: where they live and what do they do

2018-09-04 Thread Alex Schultz
On Thu, Aug 9, 2018 at 2:43 PM, Mohammed Naser  wrote:
> Hi Alex,
>
> I am very much in favour of what you're bringing up.  We do have
> multiple projects that leverage Ansible in different ways and we all
> end up doing the same thing at the end.  The duplication of work is
> not really beneficial for us as it takes away from our use-cases.
>
> I believe that there is a certain number of steps that we all share
> regardless of how we deploy, some of the things that come up to me
> right away are:
>
> - Configuring infrastructure services (i.e.: create vhosts for service
> in rabbitmq, create databases for services, configure users for
> rabbitmq, db, etc)
> - Configuring inter-OpenStack services (i.e. keystone_authtoken
> section, creating endpoints, etc and users for services)
> - Configuring actual OpenStack services (i.e.
> /etc//.conf file with the ability of extending
> options)
> - Running CI/integration on a cloud (i.e. common role that literally
> gets an admin user, password and auth endpoint and creates all
> resources and does CI)
>
> This would deduplicate a lot of work, and especially the last one, it
> might be beneficial for more than Ansible-based projects, I can
> imagine Puppet OpenStack leveraging this as well inside Zuul CI
> (optionally)... However, I think that this something which we should
> discus further for the PTG.  I think that there will be a tiny bit
> upfront work as we all standarize but then it's a win for all involved
> communities.
>
> I would like to propose that deployment tools maybe sit down together
> at the PTG, all share how we use Ansible to accomplish these tasks and
> then perhaps we can work all together on abstracting some of these
> concepts together for us to all leverage.
>

I'm currently trying to get a spot on Tuesday morning to further
discuss some of this items.  In the mean time I've started an
etherpad[0] to start collecting ideas for things to discuss.  At the
moment I've got the tempest role collaboration and some basic ideas
for best practice items that we can discuss.  Feel free to add your
own and I'll update the etherpad with a time slot when I get one
nailed down.

Thanks,
-Alex

[0] https://etherpad.openstack.org/p/ansible-collaboration-denver-ptg

> I'll let others chime in as well.
>
> Regards,
> Mohammed
>
> On Thu, Aug 9, 2018 at 4:31 PM, Alex Schultz  wrote:
>> Ahoy folks,
>>
>> I think it's time we come up with some basic rules/patterns on where
>> code lands when it comes to OpenStack related Ansible roles and as we
>> convert/export things. There was a recent proposal to create an
>> ansible-role-tempest[0] that would take what we use in
>> tripleo-quickstart-extras[1] and separate it for re-usability by
>> others.   So it was asked if we could work with the openstack-ansible
>> team and leverage the existing openstack-ansible-os_tempest[2].  It
>> turns out we have a few more already existing roles laying around as
>> well[3][4].
>>
>> What I would like to propose is that we as a community come together
>> to agree on specific patterns so that we can leverage the same roles
>> for some of the core configuration/deployment functionality while
>> still allowing for specific project specific customization.  What I've
>> noticed between all the project is that we have a few specific core
>> pieces of functionality that needs to be handled (or skipped as it may
>> be) for each service being deployed.
>>
>> 1) software installation
>> 2) configuration management
>> 3) service management
>> 4) misc service actions
>>
>> Depending on which flavor of the deployment you're using, the content
>> of each of these may be different.  Just about the only thing that is
>> shared between them all would be the configuration management part.
>> To that, I was wondering if there would be a benefit to establishing a
>> pattern within say openstack-ansible where we can disable items #1 and
>> #3 but reuse #2 in projects like kolla/tripleo where we need to do
>> some configuration generation.  If we can't establish a similar
>> pattern it'll make it harder to reuse and contribute between the
>> various projects.
>>
>> In tripleo we've recently created a bunch of ansible-role-tripleo-*
>> repositories which we were planning on moving the tripleo specific
>> tasks (for upgrades, etc) to and were hoping that we might be able to
>> reuse the upstream ansible roles similar to how we've previously
>> leverage the puppet openstack work for configurations.  So for us, it
>> would be beneficial if we could maybe help align/contribute/guide the
>> configuration management and maybe misc service action portions of the
>> openstack-ansible roles, but be able to disable the actual software
>> install/service management as that would be managed via our
>> ansible-role-tripleo-* roles.
>>
>> Is this something that would be beneficial to further discuss at the
>> PTG? Anyone have any additional suggestions/thoughts?
>>
>> My personal thoughts for tripleo would be that 

Re: [openstack-dev] [openstack-ansible][kolla-ansible][tripleo] ansible roles: where they live and what do they do

2018-08-10 Thread Sergii Golovatiuk
Hi,



On Fri, Aug 10, 2018 at 5:00 AM, Wesley Hayutin  wrote:

>
>
> On Thu, Aug 9, 2018 at 5:33 PM Alex Schultz  wrote:
>
>> On Thu, Aug 9, 2018 at 2:56 PM, Doug Hellmann 
>> wrote:
>> > Excerpts from Alex Schultz's message of 2018-08-09 14:31:34 -0600:
>> >> Ahoy folks,
>> >>
>> >> I think it's time we come up with some basic rules/patterns on where
>> >> code lands when it comes to OpenStack related Ansible roles and as we
>> >> convert/export things. There was a recent proposal to create an
>> >> ansible-role-tempest[0] that would take what we use in
>> >> tripleo-quickstart-extras[1] and separate it for re-usability by
>> >> others.   So it was asked if we could work with the openstack-ansible
>> >> team and leverage the existing openstack-ansible-os_tempest[2].  It
>> >> turns out we have a few more already existing roles laying around as
>> >> well[3][4].
>> >>
>> >> What I would like to propose is that we as a community come together
>> >> to agree on specific patterns so that we can leverage the same roles
>> >> for some of the core configuration/deployment functionality while
>> >> still allowing for specific project specific customization.  What I've
>> >> noticed between all the project is that we have a few specific core
>> >> pieces of functionality that needs to be handled (or skipped as it may
>> >> be) for each service being deployed.
>> >>
>> >> 1) software installation
>> >> 2) configuration management
>> >> 3) service management
>> >> 4) misc service actions
>> >>
>> >> Depending on which flavor of the deployment you're using, the content
>> >> of each of these may be different.  Just about the only thing that is
>> >> shared between them all would be the configuration management part.
>> >
>> > Does that make the 4 things separate roles, then? Isn't the role
>> > usually the unit of sharing between playbooks?
>> >
>>
>> It can be, but it doesn't have to be.  The problem comes in with the
>> granularity at which you are defining the concept of the overall
>> action.  If you want a role to encompass all that is "nova", you could
>> have a single nova role that you invoke 5 different times to do the
>> different actions during the overall deployment. Or you could create a
>> role for nova-install, nova-config, nova-service, nova-cells, etc etc.
>> I think splitting them out into their own role is a bit too much in
>> terms of management.   In my particular openstack-ansible is already
>> creating a role to manage "nova".  So is there a way that I can
>> leverage part of their process within mine without having to duplicate
>> it.  You can pull in the task files themselves from a different so
>> technically I think you could define a ansible-role-tripleo-nova that
>> does some include_tasks: ../../os_nova/tasks/install.yaml but then
>> we'd have to duplicate the variables in our playbook rather than
>> invoking a role with some parameters.
>>
>> IMHO this structure is an issue with the general sharing concepts of
>> roles/tasks within ansible.  It's not really well defined and there's
>> not really a concept of inheritance so I can't really extend your
>> tasks with mine in more of a programming sense. I have to duplicate it
>> or do something like include a specific task file from another role.
>> Since I can't really extend a role in the traditional OO programing
>> sense, I would like to figure out how I can leverage only part of it.
>> This can be done by establishing ansible variables to trigger specific
>> actions or just actually including the raw tasks themselves.   Either
>> of these concepts needs some sort of contract to be established to the
>> other won't get broken.   We had this in puppet via parameters which
>> are checked, there isn't really a similar concept in ansible so it
>> seems that we need to agree on some community established rules.
>>
>> For tripleo, I would like to just invoke the os_nova role and pass in
>> like install: false, service: false, config_dir:
>> /my/special/location/, config_data: {...} and it spit out the configs.
>> Then my roles would actually leverage these via containers/etc.  Of
>> course most of this goes away if we had a unified (not file based)
>> configuration method across all services (openstack and non-openstack)
>> but we don't. :D
>>
>
> I like your idea here Alex.
> So having a role for each of these steps is too much management I agree,
> however
> establishing a pattern of using tasks for each step may be a really good
> way to cleanly handle this.
>
> Are you saying something like the following?
>
> openstack-nova-role/
> * * /tasks/
> * * /tasks/install.yml
> * * /tasks/service.yml
> * */tasks/config.yml
> * */taks/main.yml
> ---
> # main.yml
>

I like the idea. We may also add upgrade tasks here also

>
> include: install.yml
> when: nova_install|bool
>
> include: service.yml
> when: nova_service|bool
>
> include: config.yml
> when: nova_config.yml
> --

Re: [openstack-dev] [openstack-ansible][kolla-ansible][tripleo] ansible roles: where they live and what do they do

2018-08-10 Thread Jesse Pretorius
> On 8/10/18, 3:20 AM, "Paul Belanger"  wrote:
>
>This is basically what I do with roles i write, allow the user to decide 
> to step
>over specific tasks. For example, I have created nodepool_task_manager 
> variable
>with the following:
>
>
> http://git.openstack.org/cgit/openstack/ansible-role-nodepool/tree/defaults/main.yaml#n16
>
> http://git.openstack.org/cgit/openstack/ansible-role-nodepool/tree/tasks/main.yaml
>
>Been using it for a few years now, works much better then tags for me.  The
>phases are pre, install, configure, service right now.

Hi folks,

I'm really happy that this conversation is happening. Thanks to Alex for 
raising it!

The task routing pattern is a reasonably good one, and is something which OSA 
is very likely to move towards in the future. Something else which I've always 
liked about the pattern used by the role's Paul has put together is that they 
clearly state the input expectation - for example, the role does not manage apt 
repositories, or implement any apt refreshes. This is the kind of thing that I 
think is going to be important - the role should be clear about what it does, 
clear about the inputs it expects, and the outputs of it. This will mean that 
the internals of the role can change, but those inputs are like an API - if you 
give the role that, it must always output the same result.

I can see it possibly being useful to include things like how to test the 
service in the service role, how to evaluate the health, how to enact an 
upgrade, how to enact a fast-forward upgrade, etc. But a good start initially 
would be to define some standards which inform the development of the roles.

Within OSA, for better or worse, we have a mix of role types - some are 
'utility', built for include_role usage. An example is 
http://git.openstack.org/cgit/openstack/ansible-role-systemd_service - give it 
the right vars, and it lays down the type of system service unit that you asked 
for in a standard way. We also make use of the 
http://git.openstack.org/cgit/openstack/ansible-config_template action plugin 
*everywhere* because it allows us not to be bothers with variables for every 
tunable under the sun - we only need variables for specific things that glue 
services together, or implement 'sensible defaults'.

We then have what I might call 'integration' roles - these are roles like 
http://git.openstack.org/cgit/openstack/openstack-ansible-os_nova which does 
all things nova, and uses include_role to bring in various utility roles. We 
try to follow the idea that a single role operates on a single host, and try to 
avoid orchestration across multiple hosts inside a role, but it's not always 
that simple for it to be cut and dried and I'm starting to think we might want 
to change that, for upgrades especially. Laying down something like keystone or 
nova, with all the options like federation or the nova drivers, is a complex 
thing to do. Putting it all in one role makes the role hard to understand, but 
at least it's obvious where you go to find it.

I guess what I'm trying to get across is that trying to build commonly used 
roles is not going to be a simple process. All projects have very different 
styles, and very different ways of putting a service like nova down. However, 
we should start somewhere - break it down into a set of utility roles we'd like 
to see available, figure out the standards that matter for input/output and 
Ansible version support, figure out the release and branching strategy for 
them, get them done and move to using them and retire any previously 
implemented roles which duplicate their purpose, then target the next set.

I think it would be beneficial to get in a room at the PTG and figure out where 
we start, and agree on some standards. I'd like to see a strong facilitator for 
the session who can ensure that we keep things on topic and productive.



Rackspace Limited is a company registered in England & Wales (company 
registered number 03897010) whose registered office is at 5 Millington Road, 
Hyde Park Hayes, Middlesex UB3 4AZ. Rackspace Limited privacy policy can be 
viewed at www.rackspace.co.uk/legal/privacy-policy - This e-mail message may 
contain confidential or privileged information intended for the recipient. Any 
dissemination, distribution or copying of the enclosed material is prohibited. 
If you receive this transmission in error, please notify us immediately by 
e-mail at ab...@rackspace.com and delete the original message. Your cooperation 
is appreciated.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [openstack-ansible][kolla-ansible][tripleo] ansible roles: where they live and what do they do

2018-08-10 Thread Mark Goddard
A couple of changes [1][2] that I proposed to kolla ansible recently as a
PoC could be related here. Kolla ansible is full of almost identical roles
for each service, with a lot of duplicated 'code' (YAML). The idea was to
try to factor some of that out into shared roles. This results in less code
and a more data-driven approach, which is inherently more configurable (for
better or worse). The two changes are for configuration and
user/service/endpoint registration. The same could easily be done with DB
management and various other tasks. These roles are quite specific to kolla
ansible, since they are tied to the configuration layout and the use of a
kolla_toolbox container for executing keystone/DB ansible modules.

[1] https://review.openstack.org/587591
[2] https://review.openstack.org/587590

Mark

On 10 August 2018 at 10:59, Chandan kumar  wrote:

> Hello,
>
> On Fri, Aug 10, 2018 at 7:47 AM Paul Belanger 
> wrote:
> >
> > On Thu, Aug 09, 2018 at 08:00:13PM -0600, Wesley Hayutin wrote:
> > > On Thu, Aug 9, 2018 at 5:33 PM Alex Schultz 
> wrote:
> > >
> > > > On Thu, Aug 9, 2018 at 2:56 PM, Doug Hellmann  >
> > > > wrote:
> > > > > Excerpts from Alex Schultz's message of 2018-08-09 14:31:34 -0600:
> > > > >> Ahoy folks,
> > > > >>
> > > > >> I think it's time we come up with some basic rules/patterns on
> where
> > > > >> code lands when it comes to OpenStack related Ansible roles and
> as we
> > > > >> convert/export things. There was a recent proposal to create an
> > > > >> ansible-role-tempest[0] that would take what we use in
> > > > >> tripleo-quickstart-extras[1] and separate it for re-usability by
> > > > >> others.   So it was asked if we could work with the
> openstack-ansible
> > > > >> team and leverage the existing openstack-ansible-os_tempest[2].
> It
> > > > >> turns out we have a few more already existing roles laying around
> as
> > > > >> well[3][4].
> > > > >>
> > > > >> What I would like to propose is that we as a community come
> together
> > > > >> to agree on specific patterns so that we can leverage the same
> roles
> > > > >> for some of the core configuration/deployment functionality while
> > > > >> still allowing for specific project specific customization.  What
> I've
> > > > >> noticed between all the project is that we have a few specific
> core
> > > > >> pieces of functionality that needs to be handled (or skipped as
> it may
> > > > >> be) for each service being deployed.
> > > > >>
> > > > >> 1) software installation
> > > > >> 2) configuration management
> > > > >> 3) service management
> > > > >> 4) misc service actions
> > > > >>
> > > > >> Depending on which flavor of the deployment you're using, the
> content
> > > > >> of each of these may be different.  Just about the only thing
> that is
> > > > >> shared between them all would be the configuration management
> part.
> > > > >
> > > > > Does that make the 4 things separate roles, then? Isn't the role
> > > > > usually the unit of sharing between playbooks?
> > > > >
> > > >
> > > > It can be, but it doesn't have to be.  The problem comes in with the
> > > > granularity at which you are defining the concept of the overall
> > > > action.  If you want a role to encompass all that is "nova", you
> could
> > > > have a single nova role that you invoke 5 different times to do the
> > > > different actions during the overall deployment. Or you could create
> a
> > > > role for nova-install, nova-config, nova-service, nova-cells, etc
> etc.
> > > > I think splitting them out into their own role is a bit too much in
> > > > terms of management.   In my particular openstack-ansible is already
> > > > creating a role to manage "nova".  So is there a way that I can
> > > > leverage part of their process within mine without having to
> duplicate
> > > > it.  You can pull in the task files themselves from a different so
> > > > technically I think you could define a ansible-role-tripleo-nova that
> > > > does some include_tasks: ../../os_nova/tasks/install.yaml but then
> > > > we'd have to duplicate the variables in our playbook rather than
> > > > invoking a role with some parameters.
> > > >
> > > > IMHO this structure is an issue with the general sharing concepts of
> > > > roles/tasks within ansible.  It's not really well defined and there's
> > > > not really a concept of inheritance so I can't really extend your
> > > > tasks with mine in more of a programming sense. I have to duplicate
> it
> > > > or do something like include a specific task file from another role.
> > > > Since I can't really extend a role in the traditional OO programing
> > > > sense, I would like to figure out how I can leverage only part of it.
> > > > This can be done by establishing ansible variables to trigger
> specific
> > > > actions or just actually including the raw tasks themselves.   Either
> > > > of these concepts needs some sort of contract to be established to
> the
> > > > other won't get broken.   We had this in puppet via 

Re: [openstack-dev] [openstack-ansible][kolla-ansible][tripleo] ansible roles: where they live and what do they do

2018-08-10 Thread Chandan kumar
Hello,

On Fri, Aug 10, 2018 at 7:47 AM Paul Belanger  wrote:
>
> On Thu, Aug 09, 2018 at 08:00:13PM -0600, Wesley Hayutin wrote:
> > On Thu, Aug 9, 2018 at 5:33 PM Alex Schultz  wrote:
> >
> > > On Thu, Aug 9, 2018 at 2:56 PM, Doug Hellmann 
> > > wrote:
> > > > Excerpts from Alex Schultz's message of 2018-08-09 14:31:34 -0600:
> > > >> Ahoy folks,
> > > >>
> > > >> I think it's time we come up with some basic rules/patterns on where
> > > >> code lands when it comes to OpenStack related Ansible roles and as we
> > > >> convert/export things. There was a recent proposal to create an
> > > >> ansible-role-tempest[0] that would take what we use in
> > > >> tripleo-quickstart-extras[1] and separate it for re-usability by
> > > >> others.   So it was asked if we could work with the openstack-ansible
> > > >> team and leverage the existing openstack-ansible-os_tempest[2].  It
> > > >> turns out we have a few more already existing roles laying around as
> > > >> well[3][4].
> > > >>
> > > >> What I would like to propose is that we as a community come together
> > > >> to agree on specific patterns so that we can leverage the same roles
> > > >> for some of the core configuration/deployment functionality while
> > > >> still allowing for specific project specific customization.  What I've
> > > >> noticed between all the project is that we have a few specific core
> > > >> pieces of functionality that needs to be handled (or skipped as it may
> > > >> be) for each service being deployed.
> > > >>
> > > >> 1) software installation
> > > >> 2) configuration management
> > > >> 3) service management
> > > >> 4) misc service actions
> > > >>
> > > >> Depending on which flavor of the deployment you're using, the content
> > > >> of each of these may be different.  Just about the only thing that is
> > > >> shared between them all would be the configuration management part.
> > > >
> > > > Does that make the 4 things separate roles, then? Isn't the role
> > > > usually the unit of sharing between playbooks?
> > > >
> > >
> > > It can be, but it doesn't have to be.  The problem comes in with the
> > > granularity at which you are defining the concept of the overall
> > > action.  If you want a role to encompass all that is "nova", you could
> > > have a single nova role that you invoke 5 different times to do the
> > > different actions during the overall deployment. Or you could create a
> > > role for nova-install, nova-config, nova-service, nova-cells, etc etc.
> > > I think splitting them out into their own role is a bit too much in
> > > terms of management.   In my particular openstack-ansible is already
> > > creating a role to manage "nova".  So is there a way that I can
> > > leverage part of their process within mine without having to duplicate
> > > it.  You can pull in the task files themselves from a different so
> > > technically I think you could define a ansible-role-tripleo-nova that
> > > does some include_tasks: ../../os_nova/tasks/install.yaml but then
> > > we'd have to duplicate the variables in our playbook rather than
> > > invoking a role with some parameters.
> > >
> > > IMHO this structure is an issue with the general sharing concepts of
> > > roles/tasks within ansible.  It's not really well defined and there's
> > > not really a concept of inheritance so I can't really extend your
> > > tasks with mine in more of a programming sense. I have to duplicate it
> > > or do something like include a specific task file from another role.
> > > Since I can't really extend a role in the traditional OO programing
> > > sense, I would like to figure out how I can leverage only part of it.
> > > This can be done by establishing ansible variables to trigger specific
> > > actions or just actually including the raw tasks themselves.   Either
> > > of these concepts needs some sort of contract to be established to the
> > > other won't get broken.   We had this in puppet via parameters which
> > > are checked, there isn't really a similar concept in ansible so it
> > > seems that we need to agree on some community established rules.
> > >
> > > For tripleo, I would like to just invoke the os_nova role and pass in
> > > like install: false, service: false, config_dir:
> > > /my/special/location/, config_data: {...} and it spit out the configs.
> > > Then my roles would actually leverage these via containers/etc.  Of
> > > course most of this goes away if we had a unified (not file based)
> > > configuration method across all services (openstack and non-openstack)
> > > but we don't. :D
> > >
> >
> > I like your idea here Alex.
> > So having a role for each of these steps is too much management I agree,
> > however
> > establishing a pattern of using tasks for each step may be a really good
> > way to cleanly handle this.
> >
> > Are you saying something like the following?
> >
> > openstack-nova-role/
> > * * /tasks/
> > * * /tasks/install.yml
> > * * /tasks/service.yml
> > * */tasks/config.yml
> > * 

Re: [openstack-dev] [openstack-ansible][kolla-ansible][tripleo] ansible roles: where they live and what do they do

2018-08-09 Thread Paul Belanger
On Thu, Aug 09, 2018 at 08:00:13PM -0600, Wesley Hayutin wrote:
> On Thu, Aug 9, 2018 at 5:33 PM Alex Schultz  wrote:
> 
> > On Thu, Aug 9, 2018 at 2:56 PM, Doug Hellmann 
> > wrote:
> > > Excerpts from Alex Schultz's message of 2018-08-09 14:31:34 -0600:
> > >> Ahoy folks,
> > >>
> > >> I think it's time we come up with some basic rules/patterns on where
> > >> code lands when it comes to OpenStack related Ansible roles and as we
> > >> convert/export things. There was a recent proposal to create an
> > >> ansible-role-tempest[0] that would take what we use in
> > >> tripleo-quickstart-extras[1] and separate it for re-usability by
> > >> others.   So it was asked if we could work with the openstack-ansible
> > >> team and leverage the existing openstack-ansible-os_tempest[2].  It
> > >> turns out we have a few more already existing roles laying around as
> > >> well[3][4].
> > >>
> > >> What I would like to propose is that we as a community come together
> > >> to agree on specific patterns so that we can leverage the same roles
> > >> for some of the core configuration/deployment functionality while
> > >> still allowing for specific project specific customization.  What I've
> > >> noticed between all the project is that we have a few specific core
> > >> pieces of functionality that needs to be handled (or skipped as it may
> > >> be) for each service being deployed.
> > >>
> > >> 1) software installation
> > >> 2) configuration management
> > >> 3) service management
> > >> 4) misc service actions
> > >>
> > >> Depending on which flavor of the deployment you're using, the content
> > >> of each of these may be different.  Just about the only thing that is
> > >> shared between them all would be the configuration management part.
> > >
> > > Does that make the 4 things separate roles, then? Isn't the role
> > > usually the unit of sharing between playbooks?
> > >
> >
> > It can be, but it doesn't have to be.  The problem comes in with the
> > granularity at which you are defining the concept of the overall
> > action.  If you want a role to encompass all that is "nova", you could
> > have a single nova role that you invoke 5 different times to do the
> > different actions during the overall deployment. Or you could create a
> > role for nova-install, nova-config, nova-service, nova-cells, etc etc.
> > I think splitting them out into their own role is a bit too much in
> > terms of management.   In my particular openstack-ansible is already
> > creating a role to manage "nova".  So is there a way that I can
> > leverage part of their process within mine without having to duplicate
> > it.  You can pull in the task files themselves from a different so
> > technically I think you could define a ansible-role-tripleo-nova that
> > does some include_tasks: ../../os_nova/tasks/install.yaml but then
> > we'd have to duplicate the variables in our playbook rather than
> > invoking a role with some parameters.
> >
> > IMHO this structure is an issue with the general sharing concepts of
> > roles/tasks within ansible.  It's not really well defined and there's
> > not really a concept of inheritance so I can't really extend your
> > tasks with mine in more of a programming sense. I have to duplicate it
> > or do something like include a specific task file from another role.
> > Since I can't really extend a role in the traditional OO programing
> > sense, I would like to figure out how I can leverage only part of it.
> > This can be done by establishing ansible variables to trigger specific
> > actions or just actually including the raw tasks themselves.   Either
> > of these concepts needs some sort of contract to be established to the
> > other won't get broken.   We had this in puppet via parameters which
> > are checked, there isn't really a similar concept in ansible so it
> > seems that we need to agree on some community established rules.
> >
> > For tripleo, I would like to just invoke the os_nova role and pass in
> > like install: false, service: false, config_dir:
> > /my/special/location/, config_data: {...} and it spit out the configs.
> > Then my roles would actually leverage these via containers/etc.  Of
> > course most of this goes away if we had a unified (not file based)
> > configuration method across all services (openstack and non-openstack)
> > but we don't. :D
> >
> 
> I like your idea here Alex.
> So having a role for each of these steps is too much management I agree,
> however
> establishing a pattern of using tasks for each step may be a really good
> way to cleanly handle this.
> 
> Are you saying something like the following?
> 
> openstack-nova-role/
> * * /tasks/
> * * /tasks/install.yml
> * * /tasks/service.yml
> * */tasks/config.yml
> * */taks/main.yml
> ---
> # main.yml
> 
> include: install.yml
> when: nova_install|bool
> 
> include: service.yml
> when: nova_service|bool
> 
> include: config.yml
> when: nova_config.yml
> 

Re: [openstack-dev] [openstack-ansible][kolla-ansible][tripleo] ansible roles: where they live and what do they do

2018-08-09 Thread Wesley Hayutin
On Thu, Aug 9, 2018 at 5:33 PM Alex Schultz  wrote:

> On Thu, Aug 9, 2018 at 2:56 PM, Doug Hellmann 
> wrote:
> > Excerpts from Alex Schultz's message of 2018-08-09 14:31:34 -0600:
> >> Ahoy folks,
> >>
> >> I think it's time we come up with some basic rules/patterns on where
> >> code lands when it comes to OpenStack related Ansible roles and as we
> >> convert/export things. There was a recent proposal to create an
> >> ansible-role-tempest[0] that would take what we use in
> >> tripleo-quickstart-extras[1] and separate it for re-usability by
> >> others.   So it was asked if we could work with the openstack-ansible
> >> team and leverage the existing openstack-ansible-os_tempest[2].  It
> >> turns out we have a few more already existing roles laying around as
> >> well[3][4].
> >>
> >> What I would like to propose is that we as a community come together
> >> to agree on specific patterns so that we can leverage the same roles
> >> for some of the core configuration/deployment functionality while
> >> still allowing for specific project specific customization.  What I've
> >> noticed between all the project is that we have a few specific core
> >> pieces of functionality that needs to be handled (or skipped as it may
> >> be) for each service being deployed.
> >>
> >> 1) software installation
> >> 2) configuration management
> >> 3) service management
> >> 4) misc service actions
> >>
> >> Depending on which flavor of the deployment you're using, the content
> >> of each of these may be different.  Just about the only thing that is
> >> shared between them all would be the configuration management part.
> >
> > Does that make the 4 things separate roles, then? Isn't the role
> > usually the unit of sharing between playbooks?
> >
>
> It can be, but it doesn't have to be.  The problem comes in with the
> granularity at which you are defining the concept of the overall
> action.  If you want a role to encompass all that is "nova", you could
> have a single nova role that you invoke 5 different times to do the
> different actions during the overall deployment. Or you could create a
> role for nova-install, nova-config, nova-service, nova-cells, etc etc.
> I think splitting them out into their own role is a bit too much in
> terms of management.   In my particular openstack-ansible is already
> creating a role to manage "nova".  So is there a way that I can
> leverage part of their process within mine without having to duplicate
> it.  You can pull in the task files themselves from a different so
> technically I think you could define a ansible-role-tripleo-nova that
> does some include_tasks: ../../os_nova/tasks/install.yaml but then
> we'd have to duplicate the variables in our playbook rather than
> invoking a role with some parameters.
>
> IMHO this structure is an issue with the general sharing concepts of
> roles/tasks within ansible.  It's not really well defined and there's
> not really a concept of inheritance so I can't really extend your
> tasks with mine in more of a programming sense. I have to duplicate it
> or do something like include a specific task file from another role.
> Since I can't really extend a role in the traditional OO programing
> sense, I would like to figure out how I can leverage only part of it.
> This can be done by establishing ansible variables to trigger specific
> actions or just actually including the raw tasks themselves.   Either
> of these concepts needs some sort of contract to be established to the
> other won't get broken.   We had this in puppet via parameters which
> are checked, there isn't really a similar concept in ansible so it
> seems that we need to agree on some community established rules.
>
> For tripleo, I would like to just invoke the os_nova role and pass in
> like install: false, service: false, config_dir:
> /my/special/location/, config_data: {...} and it spit out the configs.
> Then my roles would actually leverage these via containers/etc.  Of
> course most of this goes away if we had a unified (not file based)
> configuration method across all services (openstack and non-openstack)
> but we don't. :D
>

I like your idea here Alex.
So having a role for each of these steps is too much management I agree,
however
establishing a pattern of using tasks for each step may be a really good
way to cleanly handle this.

Are you saying something like the following?

openstack-nova-role/
* * /tasks/
* * /tasks/install.yml
* * /tasks/service.yml
* */tasks/config.yml
* */taks/main.yml
---
# main.yml

include: install.yml
when: nova_install|bool

include: service.yml
when: nova_service|bool

include: config.yml
when: nova_config.yml
--

Interested in anything other than tags :)
Thanks



>
> Thanks,
> -Alex
>
> > Doug
> >
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> 

Re: [openstack-dev] [openstack-ansible][kolla-ansible][tripleo] ansible roles: where they live and what do they do

2018-08-09 Thread Alex Schultz
On Thu, Aug 9, 2018 at 2:56 PM, Doug Hellmann  wrote:
> Excerpts from Alex Schultz's message of 2018-08-09 14:31:34 -0600:
>> Ahoy folks,
>>
>> I think it's time we come up with some basic rules/patterns on where
>> code lands when it comes to OpenStack related Ansible roles and as we
>> convert/export things. There was a recent proposal to create an
>> ansible-role-tempest[0] that would take what we use in
>> tripleo-quickstart-extras[1] and separate it for re-usability by
>> others.   So it was asked if we could work with the openstack-ansible
>> team and leverage the existing openstack-ansible-os_tempest[2].  It
>> turns out we have a few more already existing roles laying around as
>> well[3][4].
>>
>> What I would like to propose is that we as a community come together
>> to agree on specific patterns so that we can leverage the same roles
>> for some of the core configuration/deployment functionality while
>> still allowing for specific project specific customization.  What I've
>> noticed between all the project is that we have a few specific core
>> pieces of functionality that needs to be handled (or skipped as it may
>> be) for each service being deployed.
>>
>> 1) software installation
>> 2) configuration management
>> 3) service management
>> 4) misc service actions
>>
>> Depending on which flavor of the deployment you're using, the content
>> of each of these may be different.  Just about the only thing that is
>> shared between them all would be the configuration management part.
>
> Does that make the 4 things separate roles, then? Isn't the role
> usually the unit of sharing between playbooks?
>

It can be, but it doesn't have to be.  The problem comes in with the
granularity at which you are defining the concept of the overall
action.  If you want a role to encompass all that is "nova", you could
have a single nova role that you invoke 5 different times to do the
different actions during the overall deployment. Or you could create a
role for nova-install, nova-config, nova-service, nova-cells, etc etc.
I think splitting them out into their own role is a bit too much in
terms of management.   In my particular openstack-ansible is already
creating a role to manage "nova".  So is there a way that I can
leverage part of their process within mine without having to duplicate
it.  You can pull in the task files themselves from a different so
technically I think you could define a ansible-role-tripleo-nova that
does some include_tasks: ../../os_nova/tasks/install.yaml but then
we'd have to duplicate the variables in our playbook rather than
invoking a role with some parameters.

IMHO this structure is an issue with the general sharing concepts of
roles/tasks within ansible.  It's not really well defined and there's
not really a concept of inheritance so I can't really extend your
tasks with mine in more of a programming sense. I have to duplicate it
or do something like include a specific task file from another role.
Since I can't really extend a role in the traditional OO programing
sense, I would like to figure out how I can leverage only part of it.
This can be done by establishing ansible variables to trigger specific
actions or just actually including the raw tasks themselves.   Either
of these concepts needs some sort of contract to be established to the
other won't get broken.   We had this in puppet via parameters which
are checked, there isn't really a similar concept in ansible so it
seems that we need to agree on some community established rules.

For tripleo, I would like to just invoke the os_nova role and pass in
like install: false, service: false, config_dir:
/my/special/location/, config_data: {...} and it spit out the configs.
Then my roles would actually leverage these via containers/etc.  Of
course most of this goes away if we had a unified (not file based)
configuration method across all services (openstack and non-openstack)
but we don't. :D

Thanks,
-Alex

> Doug
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [openstack-ansible][kolla-ansible][tripleo] ansible roles: where they live and what do they do

2018-08-09 Thread Doug Hellmann
Excerpts from Alex Schultz's message of 2018-08-09 14:31:34 -0600:
> Ahoy folks,
> 
> I think it's time we come up with some basic rules/patterns on where
> code lands when it comes to OpenStack related Ansible roles and as we
> convert/export things. There was a recent proposal to create an
> ansible-role-tempest[0] that would take what we use in
> tripleo-quickstart-extras[1] and separate it for re-usability by
> others.   So it was asked if we could work with the openstack-ansible
> team and leverage the existing openstack-ansible-os_tempest[2].  It
> turns out we have a few more already existing roles laying around as
> well[3][4].
> 
> What I would like to propose is that we as a community come together
> to agree on specific patterns so that we can leverage the same roles
> for some of the core configuration/deployment functionality while
> still allowing for specific project specific customization.  What I've
> noticed between all the project is that we have a few specific core
> pieces of functionality that needs to be handled (or skipped as it may
> be) for each service being deployed.
> 
> 1) software installation
> 2) configuration management
> 3) service management
> 4) misc service actions
> 
> Depending on which flavor of the deployment you're using, the content
> of each of these may be different.  Just about the only thing that is
> shared between them all would be the configuration management part.

Does that make the 4 things separate roles, then? Isn't the role
usually the unit of sharing between playbooks?

Doug

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [openstack-ansible][kolla-ansible][tripleo] ansible roles: where they live and what do they do

2018-08-09 Thread Mohammed Naser
Hi Alex,

I am very much in favour of what you're bringing up.  We do have
multiple projects that leverage Ansible in different ways and we all
end up doing the same thing at the end.  The duplication of work is
not really beneficial for us as it takes away from our use-cases.

I believe that there is a certain number of steps that we all share
regardless of how we deploy, some of the things that come up to me
right away are:

- Configuring infrastructure services (i.e.: create vhosts for service
in rabbitmq, create databases for services, configure users for
rabbitmq, db, etc)
- Configuring inter-OpenStack services (i.e. keystone_authtoken
section, creating endpoints, etc and users for services)
- Configuring actual OpenStack services (i.e.
/etc//.conf file with the ability of extending
options)
- Running CI/integration on a cloud (i.e. common role that literally
gets an admin user, password and auth endpoint and creates all
resources and does CI)

This would deduplicate a lot of work, and especially the last one, it
might be beneficial for more than Ansible-based projects, I can
imagine Puppet OpenStack leveraging this as well inside Zuul CI
(optionally)... However, I think that this something which we should
discus further for the PTG.  I think that there will be a tiny bit
upfront work as we all standarize but then it's a win for all involved
communities.

I would like to propose that deployment tools maybe sit down together
at the PTG, all share how we use Ansible to accomplish these tasks and
then perhaps we can work all together on abstracting some of these
concepts together for us to all leverage.

I'll let others chime in as well.

Regards,
Mohammed

On Thu, Aug 9, 2018 at 4:31 PM, Alex Schultz  wrote:
> Ahoy folks,
>
> I think it's time we come up with some basic rules/patterns on where
> code lands when it comes to OpenStack related Ansible roles and as we
> convert/export things. There was a recent proposal to create an
> ansible-role-tempest[0] that would take what we use in
> tripleo-quickstart-extras[1] and separate it for re-usability by
> others.   So it was asked if we could work with the openstack-ansible
> team and leverage the existing openstack-ansible-os_tempest[2].  It
> turns out we have a few more already existing roles laying around as
> well[3][4].
>
> What I would like to propose is that we as a community come together
> to agree on specific patterns so that we can leverage the same roles
> for some of the core configuration/deployment functionality while
> still allowing for specific project specific customization.  What I've
> noticed between all the project is that we have a few specific core
> pieces of functionality that needs to be handled (or skipped as it may
> be) for each service being deployed.
>
> 1) software installation
> 2) configuration management
> 3) service management
> 4) misc service actions
>
> Depending on which flavor of the deployment you're using, the content
> of each of these may be different.  Just about the only thing that is
> shared between them all would be the configuration management part.
> To that, I was wondering if there would be a benefit to establishing a
> pattern within say openstack-ansible where we can disable items #1 and
> #3 but reuse #2 in projects like kolla/tripleo where we need to do
> some configuration generation.  If we can't establish a similar
> pattern it'll make it harder to reuse and contribute between the
> various projects.
>
> In tripleo we've recently created a bunch of ansible-role-tripleo-*
> repositories which we were planning on moving the tripleo specific
> tasks (for upgrades, etc) to and were hoping that we might be able to
> reuse the upstream ansible roles similar to how we've previously
> leverage the puppet openstack work for configurations.  So for us, it
> would be beneficial if we could maybe help align/contribute/guide the
> configuration management and maybe misc service action portions of the
> openstack-ansible roles, but be able to disable the actual software
> install/service management as that would be managed via our
> ansible-role-tripleo-* roles.
>
> Is this something that would be beneficial to further discuss at the
> PTG? Anyone have any additional suggestions/thoughts?
>
> My personal thoughts for tripleo would be that we'd have
> tripleo-ansible calls openstack-ansible- for core config but
> package/service installation disabled and calls
> ansible-role-tripleo- for tripleo specific actions such as
> opinionated packages/service configuration/upgrades.  Maybe this is
> too complex? But at the same time, do we need to come up with 3
> different ways to do this?
>
> Thanks,
> -Alex
>
> [0] https://review.openstack.org/#/c/589133/
> [1] 
> http://git.openstack.org/cgit/openstack/tripleo-quickstart-extras/tree/roles/validate-tempest
> [2] http://git.openstack.org/cgit/openstack/openstack-ansible-os_tempest/
> [3] 
>