Re: [ansible-project] Complex process to generate template/lineinfile data?
On 18/05/21 2:24 am, Brian Coca wrote: For data manipulation you want to create a filter plugin, you still need to be explicit about the data you feed it. Thanks for that - this is what I've ended up with. There's probably some copy/paste boilerplate that's not needed in my case. Does it look reasonable? I've had a suggestion that I should avoid lineinfile, but I'm not sure what a good substitute is (other than rearranging my setup so I can template out individual files, rather than editing the existing one). Cheers, Richard --task - name: update dirvish-forced-command list lineinfile: dest: "/usr/local/etc/dirvish-forced-command/default.list" regexp: "^{{ item }}$" line: "{{ item }}" insertafter: EOF delegate_to: "{{ container_host }}" with_items: "{{ backup_paths | gen_dfc_lines(inventory_hostname, filesystems) }}" tags: - backup --plugin from __future__ import (absolute_import, division, print_function) __metaclass__ = type from ansible.errors import AnsibleError, AnsibleFilterError, AnsibleFilterTypeError from ansible.module_utils.six import string_types, integer_types, reraise, text_type def gen_dfc_lines(backup_paths, container_name, filesystems): '''Generate a list of lines for dirvish_forced_command''' if not isinstance(filesystems, list): raise AnsibleFilterTypeError('filesystems should be a list') if not isinstance(backup_paths, list): raise AnsibleFilterTypeError('backup_paths should be a list') fixed_backup_paths = [] for path_entry in backup_paths: found = False for fs_entry in filesystems: path = path_entry['path'] fs = fs_entry['container_mountpoint'] if path.startswith(fs): # remove prefix from path path = '/' + path[len(fs):] # use guestfs path fixed_path = "/guestfs/{cname}-{suffix}{bpath}".format( cname = container_name, suffix = fs_entry['suffix'], bpath = path ) found = True if not found: # use rootfs path fixed_path = "/var/lib/lxc/{cname}/rootfs{bpath}".format( cname = container_name, bpath = path ) fixed_backup_paths.append(fixed_path) return fixed_backup_paths class FilterModule(object): ''' backup helper filters ''' def filters(self): return { 'gen_dfc_lines': gen_dfc_lines } -- You received this message because you are subscribed to the Google Groups "Ansible Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to ansible-project+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/11476de0-de71-5542-2501-45b268aee55e%40walnut.gen.nz.
Re: [ansible-project] Complex process to generate template/lineinfile data?
plugins in general only get the variables they need, vars_plugins are supposed to generate them, not change them, so they get very little access to variables in general. For data manipulation you want to create a filter plugin, you still need to be explicit about the data you feed it. -- -- Brian Coca -- You received this message because you are subscribed to the Google Groups "Ansible Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to ansible-project+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CACVha7fwo08t6yLkZ%2BOSEQGiG9z-vAn1nEcxjX15JaYv%2B51cqQ%40mail.gmail.com.
Re: [ansible-project] Complex process to generate template/lineinfile data?
On 15/05/21 6:39 pm, Richard Hector wrote: Do I need to write a custom plugin of some kind? I'm experimenting with writing a vars plugin, referring to these: https://docs.ansible.com/ansible/latest/dev_guide/developing_plugins.html https://github.com/ansible/ansible/blob/devel/lib/ansible/plugins/vars/host_group_vars.py I've put it in the vars_plugins directory in my role. So far, I can create variables, and access them from my role, which is good. What I can't do, as far as I can see, is access existing facts/vars. Do they exist at this point? Or is my plugin running at a lower level? I was hoping they would be returned by the call to: super(VarsModule, self).get_vars(loader, path, entities) ... but that returns None. Is there a different kind of plugin that could do the job? Thanks, Richard -- You received this message because you are subscribed to the Google Groups "Ansible Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to ansible-project+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/300f5f57-fcce-abcd-7854-b4db6fbad655%40walnut.gen.nz.
Re: [ansible-project] Complex process to generate template/lineinfile data?
On 15/05/21 6:39 pm, Richard Hector wrote: Any suggestions? Is this stuff reasonable to do with ansible, and a templating language? Do I need to write a custom plugin of some kind? If I had a dynamic inventory, then I could probably generate extra variables at that stage, but that's further away too. I should give lots of credit to the nice people on irc who helped me get as far as I have. I've also thought that if I could pass data (as json perhaps?) to stdin of a local script, and capture the output (a list of strings, as json again?), that would probably work ok - I could then write the script in a language I'm more familiar with, such as perl. Is that doable? Thanks, Richard -- You received this message because you are subscribed to the Google Groups "Ansible Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to ansible-project+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/d72c0e64-5912-3369-0be6-69aea22a75d0%40walnut.gen.nz.
[ansible-project] Complex process to generate template/lineinfile data?
Hi all, Well, it's not really that complex, but seems complex to do within the constraints of ansible/jinja. My scenario is that I run dirvish to back up multiple containers on multiple hosts. The containers are backed up via the filesystems on the hosts. The root filesystem of the container lives under /var/lib/lxc//rootfs. Sometimes, extra filesystems are also used; they live under /guestfs/-, where 'name' is a possibly shortened version of the mountpoint. Relevant sections of host_vars files look like this: filesystems: - { index: 1, suffix: srv, size: 2, container_mountpoint: '/srv/' } backup_paths: - { shortname: 'ROOT', path: '/' } - { shortname: 'var', path: '/var/' } - { shortname: 'home', path: '/home/' } - { shortname: 'srv', path: '/srv/' } So when I'm setting up backups, I need to treat any backup path for this container starting with '/srv/' differently, in that I need to backup something under /guestfs/ rather than under /var/lib/lxc. I have a template that does this for the dirvish config: client: {{ hostvars[container_host].fqdn }} {% set ns = namespace(extra_fs='') %} {% for mountpoint in filesystems %} {% if item['path'] | regex_search(mountpoint['container_mountpoint']) %} {% set ns.extra_fs = mountpoint %} {% endif %} {% endfor %} {% if ns.extra_fs %} tree: /guestfs/{{ inventory_hostname + '-' + ns.extra_fs['suffix'] + '/' + item['path'] |regex_replace(ns.extra_fs['container_mountpoint'], '') }} {% else %} tree: /var/lib/lxc/{{ inventory_hostname }}/rootfs{{ item['path'] }} {% endif %} rsh: ssh -i /root/.ssh/id_rsa_dirvish {{ hostvars[container_host].fqdn }} {% for backup_path in (backup_paths | select('search', item['path'] + '.*')) if not (backup_path['shortname'] == item['shortname']) %} {% if loop.index == 1 %} excludes: {% endif %} {{ backup_path['path'] }}* {% endfor %} As you can see, that's quite complicated. Then in addition, I need to add to a list of acceptable paths on the host (for my ssh forced command script to read). That looks like this (for the same container described in the host_vars above): ... /var/lib/lxc/example-web1/rootfs/ /var/lib/lxc/example-web1/rootfs/var/ /var/lib/lxc/example-web1/rootfs/home/ /guestfs/example-web1-srv/ ... That isn't a template, because the file contains paths for all the containers. I suppose I could modify my script to read any files in a directory instead of a single file, but that would be a separate project, and the template would be just as nasty as the one above. I'm not even sure where to start with generating lines to add to the file. Any suggestions? Is this stuff reasonable to do with ansible, and a templating language? Do I need to write a custom plugin of some kind? If I had a dynamic inventory, then I could probably generate extra variables at that stage, but that's further away too. Cheers, Richard -- You received this message because you are subscribed to the Google Groups "Ansible Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to ansible-project+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/75a77a08-b175-d165-8bc9-a2f0656cd834%40walnut.gen.nz.