[ansible-project] Re: Proper handling of host specific credential files

[Jean Helou]

in a classical case of rubber duck debbuging, I found a possible answer 
seconds after posting my message 

- name: "Install client openvpn credentials"
  copy:
src: "{{ inventory_dir+'/host_vars/'+inventory_hostname+'/openvpn.'+ 
item }}"
dest: "/etc/openvpn/{{ hostname }}.{{ item }}"
mode: 0755
  with_items:
- crt
- key
  tags:
- foo

I am still interested to know if this is a proper solution and if not what 
is the idiomatic/recommended way to handle this case. 

thanks
jean

On Wednesday, April 11, 2018 at 11:52:08 AM UTC+2, Jean Helou wrote:
>
> Hi, 
>
> I am using ansible 2.5, I have gone through the best practices 
> <http://docs.ansible.com/ansible/latest/user_guide/playbooks_best_practices.html>
>  
> but I fail to understand how I am supposed to properly handle 
> *host-specific* files. 
>
> My exact use case is handling vpn certificates for computers embedded in 
> cars. the vpn lets the cars talk to our internal servers.
> Each deployed car has it's own vpn certificate and key allowing for 
> individual revocation. The .crt and the .key files really are host 
> specific. 
>
> Here is the tree I thought I could use : 
> .
> ├── inventories
> │   ├── group_vars
> │   │   └── mygroup
> │   └── host_vars
> │   └── demo-box
> │   ├── demo-box.crt
> │   ├── demo-box.key
> │   └── demo-box.yml
> ├── playbook.yml
> ├── playbooks
> │   └── setup.yml
> └── roles
> └── linux
> └── openvpn
> ├── defaults
> │   └── main.yml
> ├── files
> ├── meta
> │   └── main.yml
> └── tasks
> └── main.yml
>
> Unfortunately I can't seem to get the copy plugin to properly resolve the 
> .key and .crt files (the host vars are indeed processed) 
>
> my initial naive attempt for the tasks/main.yml was 
>
> - name: "Install client openvpn credentials"
>   copy:
> src: "openvpn.{{item}}"
> dest: "/etc/openvpn/{{ hostname }}.{{ item }}"
> mode: 0755
>   with_items:
> - crt
> - key
>   tags:
> - foo
>
> When running ansible-playbook -i inventories/test.lst playbook.yml I get 
> the follwing error
>
> AnsibleFileNotFound: Could not find or access 'openvpn.crt'
> Searched in:
> /home/ansible/ansible/roles/linux/openvpn/files/openvpn.crt
> /home/ansible/ansible/roles/linux/openvpn/openvpn.crt
> /home/ansible/ansible/roles/linux/openvpn/tasks/files/openvpn.crt
> /home/ansible/ansible/roles/linux/openvpn/tasks/openvpn.crt
> /home/ansible/ansible/playbooks/files/openvpn.crt
> /home/ansible/ansible/playbooks/openvpn.crt
>
> So it seems that unlike vars, default file lookup doesn't look into 
> inventory_dir at all. and also that using a top level file directory  like 
> in 
> https://stackoverflow.com/questions/32830428/where-should-i-be-organizing-host-specific-files-templates?lq=1
>  
> in not going to work. Adding playbook_vars_root = all to ansible.cfg 
> doesn't help either as it seem to only apply to vars.
>  
> Based on 
> https://www.reddit.com/r/ansible/comments/29mnp3/where_to_store_perhost_config_files/,
>  
> I tried using lookup like so 
>
> - name: "Install client openvpn credentials"
> src: "{{ lookup('file', inventory_dir + '/host_vars/' + 
> inventory_hostname + '/openvpn.'+ item ) }}"
> dest: "/etc/openvpn/{{ hostname }}.{{ item }}"
> mode: 0755
>   with_items:
> - crt
> - key
>
> but that doesn't seem to be valid anymore in 2.5 (I get an error about 
> quotes but can't spot a quoting error) and this syntax is totally absent 
> from 2.5 documentation.
>
> What is the "official" recommendation for this use case ? 
> I guess I could create 2 string vars for the key and crt files and use 
> ansible-vault encrypt-string instead of encrypt to secure the key. This is 
> a relatively painful process (as opposed to simply copying the files over 
> to the proper host folder) and we have tens of systems to retrofit into 
> ansible. I was hoping there would be a better solution for host-specfic 
> credential files. 
>
> thanks
> jean
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ansible-project+unsubscr...@googlegroups.com.
To post to this group, send email to ansible-project@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/ansible-project/2eae2c06-1520-4054-80e8-2bd322949b36%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ansible-project] Proper handling of host specific credential files

[Jean Helou]

Hi, 

I am using ansible 2.5, I have gone through the best practices 

 
but I fail to understand how I am supposed to properly handle 
*host-specific* files. 

My exact use case is handling vpn certificates for computers embedded in 
cars. the vpn lets the cars talk to our internal servers.
Each deployed car has it's own vpn certificate and key allowing for 
individual revocation. The .crt and the .key files really are host 
specific. 

Here is the tree I thought I could use : 
.
├── inventories
│   ├── group_vars
│   │   └── mygroup
│   └── host_vars
│   └── demo-box
│   ├── demo-box.crt
│   ├── demo-box.key
│   └── demo-box.yml
├── playbook.yml
├── playbooks
│   └── setup.yml
└── roles
└── linux
└── openvpn
├── defaults
│   └── main.yml
├── files
├── meta
│   └── main.yml
└── tasks
└── main.yml

Unfortunately I can't seem to get the copy plugin to properly resolve the 
.key and .crt files (the host vars are indeed processed) 

my initial naive attempt for the tasks/main.yml was 

- name: "Install client openvpn credentials"
  copy:
src: "openvpn.{{item}}"
dest: "/etc/openvpn/{{ hostname }}.{{ item }}"
mode: 0755
  with_items:
- crt
- key
  tags:
- foo

When running ansible-playbook -i inventories/test.lst playbook.yml I get 
the follwing error

AnsibleFileNotFound: Could not find or access 'openvpn.crt'
Searched in:
/home/ansible/ansible/roles/linux/openvpn/files/openvpn.crt
/home/ansible/ansible/roles/linux/openvpn/openvpn.crt
/home/ansible/ansible/roles/linux/openvpn/tasks/files/openvpn.crt
/home/ansible/ansible/roles/linux/openvpn/tasks/openvpn.crt
/home/ansible/ansible/playbooks/files/openvpn.crt
/home/ansible/ansible/playbooks/openvpn.crt

So it seems that unlike vars, default file lookup doesn't look into 
inventory_dir at all. and also that using a top level file directory  like 
in 
https://stackoverflow.com/questions/32830428/where-should-i-be-organizing-host-specific-files-templates?lq=1
 
in not going to work. Adding playbook_vars_root = all to ansible.cfg 
doesn't help either as it seem to only apply to vars.
 
Based on 
https://www.reddit.com/r/ansible/comments/29mnp3/where_to_store_perhost_config_files/,
 
I tried using lookup like so 

- name: "Install client openvpn credentials"
src: "{{ lookup('file', inventory_dir + '/host_vars/' + 
inventory_hostname + '/openvpn.'+ item ) }}"
dest: "/etc/openvpn/{{ hostname }}.{{ item }}"
mode: 0755
  with_items:
- crt
- key

but that doesn't seem to be valid anymore in 2.5 (I get an error about 
quotes but can't spot a quoting error) and this syntax is totally absent 
from 2.5 documentation.

What is the "official" recommendation for this use case ? 
I guess I could create 2 string vars for the key and crt files and use 
ansible-vault encrypt-string instead of encrypt to secure the key. This is 
a relatively painful process (as opposed to simply copying the files over 
to the proper host folder) and we have tens of systems to retrofit into 
ansible. I was hoping there would be a better solution for host-specfic 
credential files. 

thanks
jean

-- 
You received this message because you are subscribed to the Google Groups 
"Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ansible-project+unsubscr...@googlegroups.com.
To post to this group, send email to ansible-project@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/ansible-project/5c975118-37a4-42d4-aff6-e3fb9bc10852%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ansible-project] Re: Figuring out playbook_vars_root

[Jean Helou]

Looking further into this I find conflicting information (maybe I 
understand it all wrong)

In https://github.com/ansible/ansible/issues/33177#issuecomment-356976272 
("when including playbook, group_vars of original playbook are ignored") 
@sivel says : 

We have discussed this in the Core Team Meeting on Jan 11, 2018.

After looking at the code previous to 2.4, and discussing what our 
expectations are, we have decided that the previous behavior was a bug, and 
we will not be adding functionality to mimic the pre 2.4 behavior.

but in https://github.com/ansible/ansible/issues/29008#issuecomment-330558987 
("import_playbook from child directory break var scope") @bcoca says :

The fix we are planning does 2 things:

   - keep a 'stack' of playbook paths, starting with the initial playbook 
   and ending with the 'current' one
   - group/host_vars processing should got through each path in order

So 'initial playbook' and the 'current playbook' adjacent group/host_vars 
will all be read (as well as any intermediate ones). This is still a change 
from previous behaviour but we believe this is the 'most' correct way of 
dealing with the variables.

in https://github.com/ansible/ansible/issues/32936#issuecomment-352700117 
("Host variables adjacent to master playbook are unavailable to included 
tasks") @lufik also mentions

The PLAYBOOK_VARS_ROOT config doesn't change anything (it's only task 
related not playbook related). Shortly >=2.4.0 breaks the backward 
compatibility for including playbooks (maybe the caption of this issue 
should be changed also).

the code does seem to be in agreement 
(https://github.com/ansible/ansible/blob/stable-2.4/lib/ansible/vars/manager.py#L246)
 
though I am not familiar with the code base but I fail to see the 
differences between all 3 issues ... they all seem to have a structure 
similar to what I used in my experimentation.




-- 
You received this message because you are subscribed to the Google Groups 
"Ansible Project" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to ansible-project+unsubscr...@googlegroups.com.
To post to this group, send email to ansible-project@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/ansible-project/51d8fa94-ca65-4161-91c0-9396eada83bf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ansible-project] Figuring out playbook_vars_root

[Jean Helou]

I am trying to understand what playbook_vars_root does in Ansible 2.4.3, I 
have the feeling that it is totally ignored 

According to the documentation 
 : 

>   This sets which playbook dirs will be used as a root to process vars 
plugins, which includes finding host_vars/group_vars The top option follows 
the traditional behaviour of using the top playbook in the chain to find 
the root directory. The bottom option follows the 2.4.0 behaviour of using 
the current playbook to find the root directory. The all option examines 
from the first parent to the current playbook.  

Despite my efforts, I can't seem to get anything else than the "bottom" 
behaviour. 

I have the following tree

─ ansible.cfg
├── group_vars
│   └── foo
│   └── vars.yml
├── host_vars
│   └── localhost.yml
├── inventories
│   └── site.lst
├── main.yml
└── playbooks
└── sub.yml

ansible.cfg

[defaults]
playbook_vars_root = top

site.lst contains the following 

[foo]
localhost ansible_connection=local

group_vars/foo/vars.yml

---
_group_vars_msg: "host var message"

host_vars/localhost.yml

---
_host_vars_msg: "host var message"

main.yml

---
- hosts: foo
  tasks:
- debug: msg="main | {{ _group_vars_msg }}"
- debug: msg="main | {{ _host_vars_msg }}"

- import_playbook: "playbooks/sub.yml"

playbooks/sub.yml

---
- hosts: foo
  tasks:
- debug: msg="sub | {{ _group_vars_msg }}"
- debug: msg="sub | {{ _host_vars_msg }}"

Running the following command: 

ansible-playbook -i inventories/site.lst main.yml 

yields 


PLAY [foo] 
***

TASK [Gathering Facts] 
***
ok: [localhost]

TASK [debug] 
*
ok: [localhost] => {
"msg": "main | group var message"
}

TASK [debug] 
*
ok: [localhost] => {
"msg": "main | host var message"
}

PLAY [foo] 
***

TASK [Gathering Facts] 
***
ok: [localhost]

TASK [debug] 
*
fatal: [localhost]: FAILED! => {"msg": "The task includes an option with an 
undefined variable. The error was: '_group_vars_msg' is undefined\n\nThe 
error appears to have been in '/home/ansible/playbooks/sub.yml': line 4, 
column 7, but may\nbe elsewhere in the file depending on the exact syntax 
problem.\n\nThe offending line appears to be:\n\n  tasks:\n- debug: 
msg=\"sub | {{ _group_vars_msg }}\"\n  ^ here\nWe could be wrong, but 
this one looks like it might be an issue with\nmissing quotes.  Always 
quote template expression brackets when they\nstart a value. For 
instance:\n\nwith_items:\n  - {{ foo }}\n\nShould be written 
as:\n\nwith_items:\n  - \"{{ foo }}\"\n\nexception type: \nexception: '_group_vars_msg' is 
undefined"}
to retry, use: --limit @/home/ansible/main.retry

PLAY RECAP 
***
localhost  : ok=4changed=0unreachable=0failed=1


The imported playbook fails to resolve the variables. can someone explain 
in more details why my main.yml is not considered to be the "top" playbook 
in this case or is importing playbooks not what is meant by "chain" in the 
documentation ? 

> The top option follows the traditional behaviour of using the top 
playbook in the chain to find the root directory

I have of course tried to set the playbook_vars_root to