+1 would love to see this patch get prioritized as this severely limits the utility of vault files. As an example, a playbook running against one group with ~500 hosts which in turn references a single vault-encrypted file via vars_files, takes 6 1/2 minutes to run vs ~30 seconds when the file is decrypted.
On Friday, June 6, 2014 5:40:19 AM UTC-7, michael wrote: > > I'm open to the idea of parse_yaml_from_file caching small files in memory > if vault decoded. > > > > > > On Thu, Jun 5, 2014 at 2:33 PM, Serge van Ginderachter < > [email protected] <javascript:>> wrote: > >> I have a couple of patches in queue that might help on this (though not >> the core issue if the extra time is due to vault): >> >> https://github.com/ansible/ansible/pulls/sergevanginderachter >> >> especially this one: >> >> https://github.com/ansible/ansible/pull/6734 >> >> Can you test this patch set? You could also test with my 'integration' >> branch that has all those pathches merged in with 1.7 devel (currently last >> updated some weeks ago). >> >> https://github.com/sergevanginderachter/ansible/tree/INTEGRATION >> >> Given you load the encrypted file as a var file, it might not address >> your use case, but it might help loading 110 yaml files (assuming those are >> primarily group/host_vars files. >> >> >> Serge >> >> >> >> >> On 5 June 2014 19:07, <[email protected] <javascript:>> wrote: >> >>> >Are all of the files you're referencing vault-encrypted, or does the >>> slowness come from the initialization of the VaultLib? >>> >>> Only one file is vault encrypted, but (almost) every role references it >>> as a vars file. I don't know which part in particular is slow; I noticed >>> the delay after an open() on the vault file when stracing ansible and then >>> just confirmed it decrypted the file each time without digging any further. >>> >>> >>> > It might be better to cache that object based on the encryption method >>> and/or hashed password rather than the unencrypted contents of the YAML >>> files themselves. >>> >>> The choice to cache the parsed YAML rather than something lower down the >>> chain was made because that avoided the most work for ansible and, since I >>> know YAML files don't changes during execution, seemed safe. An initial >>> attempt only cached the decryption result, which already improved speeds >>> considerably but was still about three times as slow as caching the lot. >>> >>> I just tried caching only the VaultAES256 object instead of the above >>> approach, but that made no difference to the start up time. >>> >>> --J >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Ansible Project" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected] <javascript:>. >>> To post to this group, send email to [email protected] >>> <javascript:>. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/ansible-project/a252a080-a0e6-40da-9467-3359e91f93c3%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/ansible-project/a252a080-a0e6-40da-9467-3359e91f93c3%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Ansible Development" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "Ansible Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/2f2dda41-8146-4a08-9f51-f4d48c79eec2%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
