Thanks Michael,
Tower allows you to upload a private key
Well the key only was an example. One that I just made up because there
might be the possibility one needs to change it. Another example would be
the ssh *timeout* or more generally the *ssh_args*. Probably this all could
be set in the ssh config. I just want to make sure there won't be a
showstopper in future when a user requires a config tweak and can't set it
because the cfg is in source control.
The storage of ssh keys in Tower for sure is a very nice feature which
might get interesting for us in future for security reasons.
It's recommended, to be consistent with the majority of the ansible
> community, that people don't adopt hash_behavior=merge. However there are
> some that really feel like they should use it.
I don't see a way around this setting in our case. That's because our
systems will be furnished with many different services which are provided
by different teams. And which config a service uses might depend on other
services. A good example for this would be Splunk. Splunk is a tool for
collecting and indexing logfiles. Which logs will be collected depends then
on which other services will run on a host. My idea was to organize this in
groups.
group_vars for group-A:
---
splunk-forwarder:
file-A: sourcetype-A
...
group_vars for group-B:
---
splunk-forwarder:
- file-B: sourcetype-B
...
When the host belongs to group A and B the content will be merged:
---
splunk-forwarder:
file-A: sourcetype-A
file-B: sourcetype-B
...
A playbook might look like this:
---
- name: Some playbook
hosts: [some-host-which-may-belong-to-A-and/or-B]
roles:
- { role: role-A, when: "'group-A' in group_names" }
- { role: role-B, when: "'group-B' in group_names" }
- { role: splunk-forwarder, when: "splunk-forwarder is defined and
splunk-forwarder | length > 0" }
...
So along with some other roles the splunk-forwarder role is applied which
then uses the config of the other groups.
Another use case - exceptionally one that is not made up and I really have
in my evaluation experiment - is a redis proxy (twemproxy by twitter) which
should forward connections to different redis clusters. Each cluster is
defined in a separate group. To get the relevant configuration of all
clusters I include the *group_vars* of all clusters in a loop. The proxy
config only holds references to the clusters, like so:
---
redisproxy:
- host: some-host
pools:
- cluster: A
pool: A
- cluster: B
pool: K
...
So this *group_vars* holds the config for all proxies on all hosts. Here we
have 2 pools each defined in a different cluster. In a loop in the proxy
role I then include all the group_vars of the clusters (redis-proxy-*A*,
redis-proxy-*B*) which again will be merged hashes.
I recommend you set a policy on what you use so that everyone can easily
> read playbooks and know what might be going on.
In case of the *hash_behavior* you are right. Since roles are services
provided by different teams we need consistent behavior which developers
can rely on. Though for callback plugins this still is a problem.
I'm not sure how you have callbacks and roles interlocking, as they are not
> related concepts.
It's just that you need to define the roles on the same level as the
ansible.cfg or the playbook won't find them.
The simple requirement "playbook specific callback" -> requires a specific
ansible.cfg -> requires a root folder for every playbook where the cfg can
be placed in along with the playbook -> requires the roles on the same
level inside this specific folder.
However, if your custom callback requires configuration, the common
> mechanism is for it to read an environment variable. This environment
> variable could even reference the path to a configuration file.
It's not that I need config (well, I do, but that's another topic ;-)) but
to enable or disable a callback per playbook. Some team might want to log
to a database like you say. Another team might want to send notifications
to their Hipchat channel. Who knows. That's up to them, I just try to find
a solution to give them the chance to do whatever they want. Environment
variables might be an option, but that's everything but convenient when a
user needs to manually set 15 variables before running the playbook and
then changing it when running another one.
>From the other mail thread I have seen how to access vars inside the
callback plugin and that might be a handy option. Then it would be possible
to enable/disable a callback per group_vars.
but I'd first like to step back and ask what your callbacks *do*
>
Nothing specific. I don't really have callbacks other than the Hipchat
plugin I'm playing with. I just want to find the best possible setup to
give our teams the most freedom in future. But as written before, roles
come from different teams and each team might want to get notified on
failure, log changes to a database or whatever comes to their mind. So I
need a flexible framework where things can be configured per playbook and
role.
With the settings available in the callback I believe I can work.
In the *all* group_vars we then can define notification settings:
---
notifications:
playbook-name:
role-name:
task-name:
fail:
- type: Hipchat
room: 12345
- type: Email
to: [email protected]
ok:
- type: Hipchat
room: 12345
...
Each element (playbook-name, role-name, task-name and the actual callback
type name could be a wildcard to match any value, so a role provider could
get notified of failures in-depended of the playbook name.
Each callback would then run through those definitions and either get
active or not. Then I can use a simple structure as we do not require a
custom cfg to define a separate set of callbacks.
As for user specific settings vs. company-wide settings: Is there a reasons
why multiple cfg's are not merged? Would you accept a PR for such a feature?
Cheers!
Daniel
Am Mittwoch, 30. Juli 2014 13:59:10 UTC+2 schrieb Michael DeHaan:
>
> Hi Daniel, replies are inline.
>
>
> On Wed, Jul 30, 2014 at 4:21 AM, Daniel Schroeder <[email protected]
> <javascript:>> wrote:
>
>> Heya,
>>
>> I'm currently evaluating Ansible for our company but I'm having problems
>> to understand how we would set up our global structure. We have like 10.000
>> hosts and hundreds of different projects with different teams and needs.
>> Still we need to use a common code base (roles, plugins) and not re-invent
>> the wheel in every project. Of course I understand roles and playbooks. My
>> problem is how settings are handled and from there it leads to more
>> problems.
>>
>> 1) User settings vs. project settings
>>
>> According to the docs, Ansible looks in different locations for the cfg
>> file and uses the first found, ignoring all others. So it is not possible
>> to have settings in different locations, which makes it hard to allow user
>> specific settings (e.g. *private_key_file*) and at the same time define
>> company wide or project specific settings, which should be stored in git.
>>
>
> With an infrastructure of this size, you should really consider Ansible
> Tower. Tower allows you to upload a private key into a concept called a
> *credential* and this credential is securely provided to those who have
> access to it (shared with specific teams) and they don't get to see the
> credential.
>
> Each job template can also be associated with specific credentials for
> launching.
>
> Alternatively, you could just store your ansible.cfg in git and set
> ANSIBLE_CFG to the ansible.cfg path.
>
>
>>
>> 2) Settings are global, not per playbook
>>
>> But different playbooks might have different needs, e.g. a playbook could
>> require *hash_behaviour=merge*. To archive this one would need to create
>> a folder specific for the playbook, place the cfg with the settings inside,
>> cd to that folder and run *ansible-playbook* from there.
>>
>
> It's recommended, to be consistent with the majority of the ansible
> community, that people don't adopt hash_behavior=merge. However there are
> some that really feel like they should use it.
>
> In this case, people can use it, it will continue to work, it may just be
> a little confusing. I recommend you set a policy on what you use so that
> everyone can easily read playbooks and know what might be going on.
>
> Most people don't need the complexity of hash_behavior=merge and I
> recommend people try to avoid it, since it starts to be fun figuring out
> where something added to a variable and you lose the ability to override
> hash variables completely.
>
>
>>
>> Another thing are callbacks. Those are fired as soon as they exists in
>> the per settings defined folder. So every playbook which has a specific
>> callback would require its own callback directory + settings. When
>> following this pattern, you'd need to create the roles within this
>> directory as well as those need to be located on the same level you execute
>> *ansible-playbook* on.
>>
>
> I'm not sure how you have callbacks and roles interlocking, as they are
> not related concepts.
>
> However, if your custom callback requires configuration, the common
> mechanism is for it to read an environment variable. This environment
> variable could even reference the path to a configuration file.
>
> Not too many people write custom callbacks, but if you're looking for
> something like logging to a database or making things API accessible,
> again, Tower provides this stock. For questions about that, feel free to
> ask [email protected] <javascript:>.
>
>
>>
>> This all makes it very hard to re-use roles and plugins. I worked on a
>> structure that might work but is a symlink mayhem:
>>
>> group_vars
>> all
>> group-1
>> group-N
>> host_vars
>> host-1
>> host-N
>> inventory
>> production
>> staging
>> uat
>> library
>> modules
>> mod-1.py
>> mod-N.py
>> plugins
>> callback
>> callback-1.py
>> callback-N.py
>> filter
>> filter-1.py
>> filter-N.py
>> lookup
>> lookup-1.py
>> lookup-N.py
>> playbooks
>> project-1
>> ansible.cfg
>> group_vars -> ../../group_vars
>> host_vars -> ../../host_vars
>> library
>> modules -> ../../../library/modules
>> plugins
>> callback
>> callback-1.py -> ../../../../../library/plugins/callback/
>> callback-1.py
>> callback-N.py -> ../../../../../library/plugins/callback/
>> callback-N.py
>> filter -> ../../../../library/plugins/filter
>> lookup -> ../../../../library/plugins/lookup
>> production -> ../../inventory/production
>> playbook-1.yml
>> playbook-N.yml
>> roles -> ../../roles
>> staging -> ../../inventory/staging
>> uat -> ../../inventory/uat
>> project-N
>> ...
>> roles
>> role-1
>> role-N
>>
>>
>
> I agree this looks horrible, but I don't think this is ansible's fault. I
> think you could use some best practices organization advice here, but I'd
> first like to step back and ask what your callbacks *do* to understand the
> problem, and also ask why callbacks are project specific. We can then
> work out the best way to organize this as well as write your callbacks.
>
>
>
>> This enables us to globally handle all re-usable components while every
>> project can define its own settings. A project would/could be a submodule
>> in the main git repo.
>>
>> So this seems to work. Only that user specific settings are still not
>> possible. Does this setup make sense from the PoV of more experienced
>> Ansible users? IMHO this looks quite complex and I wonder what I'm missing
>> here because things shouldn't be that complex.
>>
>
> It looks like it's quite an order of magnitude too complex. But see
> above, let's talk through the callback question and I think making your
> callbacks configurable may solve the problem. However I want to
> understand a bit more about what they do first, and I think we can help
> completely eliminate all the symlink fun.
>
>
>
>> Thanks in advance,
>> Daniel
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Ansible Project" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected] <javascript:>.
>> To post to this group, send email to [email protected]
>> <javascript:>.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/ansible-project/74ad0525-4925-45b5-9542-4c8de9bd270a%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/ansible-project/74ad0525-4925-45b5-9542-4c8de9bd270a%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
You received this message because you are subscribed to the Google Groups
"Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/ansible-project/82545a4a-7a65-474a-b545-1d3acc75d9d4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
