Chris, thank you *very* much for looking at that. I have tried your modified ec2.py but, unfortunately, even using it exactly as you describe, the user data is still being stored on AWS in its base64 form. From within the instance:
# fetch http://169.254.169.254/latest/user-data # cat user-data /Td6WFoAAATm1rRGAgAhARYAAAB … /sCAAAAAARZWg== # base64 -d user-data test.tar.xz # tar tvf test.tar.xz -rw-r--r-- 0 djn staff 2 Dec 26 10:40 1.txt -rw-r--r-- 0 djn staff 2 Dec 26 10:40 2.txt -rw-r--r-- 0 djn staff 322 Dec 24 12:03 configinit By the way, looking at line 930 of the boto connection.py source file you referenced, I don’t understand why boto is base64-encoding a parameter that its own documentation (on lines 774-776) indicates is apparently already to be base64-encoded. (Note, however, that I am not familiar with the boto source.) Still, since you are passing in a decoded version of the Ansible user_data_b64 parameter, I’d have thought what you’ve done ought to work. Empirically, though, it doesn’t seem to. Finally, perhaps I should note that to test this, I dropped your ec2.py over the top of the existing ec2.py in a stock Ansible 1.8.2 installation that was installed by Homebrew on my OS X machine. This installation of Ansible is using boto 2.34.0, I believe. Thank you again for your help! On Friday, 26 December 2014 07:55:09 UTC, Chris Church wrote: > > One issue may be that boto is encoding unicode user data into utf-8, then > base64-encoding it: > > https://github.com/boto/boto/blob/develop/boto/ec2/connection.py#L927 > > > So, if we can somehow pass a str() instance to boto instead of a unicode() > instance, it'll skip the utf-8 encoding step. > > I've implemented a version of the ec2 module that accepts a user_data_b64 > parameter as a base64-encoded string of the user data, and decodes it into > a str() instance before passing to boto: > > > https://github.com/cchurch/ansible-modules-core/blob/f6522309ffaedace2305fd67a5a5721213f59199/cloud/amazon/ec2.py > > > You can read the local binary file into a base64-encoded string using the > slurp module, then pass that blob to the ec2 module. The slurp module > needs an absolute path, so you can use role_path to specify a path > relative to a role, or playbook_dir relative to your top-level playbook. > > - local_action: slurp src="{{role_path}}/files/ec2_user_data" > register: ec2_user_data_b64 > - ec2: > user_data_b64: '{{ ec2_user_data_b64.content }}' > ... > > > > > On Wed, Dec 24, 2014 at 11:10 AM, Daniel Neades <[email protected] > <javascript:>> wrote: > >> Incidentally, I also tried using the file lookup plugin on a >> base64-encoded tar file, and then filtering that with Jinja2’s b64decode >> filter. That also fails, though I think it does so at a later stage than >> trying to use the file lookup plugin on a binary file. Using the b64decode >> filter on a looked-up base64-encoded file, one gets an error whose stack >> trace ends something like this: >> >> File >> "/usr/local/Cellar/ansible/1.8.2/libexec/lib/python2.7/site-packages/ansible/utils/template.py", >> >> line 362, in template_from_string >> res = jinja2.utils.concat(rf) >> File "<template>", line 7, in root >> UnicodeDecodeError: 'ascii' codec can't decode byte 0xfd in position 0: >> ordinal not in range(128) >> >> I then thought I’d do a little experiment, and provide the user data as a >> literal (copy-and-pasted) string of base64-encoded data, again filtered >> with the Jinga2 b64decode filter, like this: >> >> user_data: "{{ '/Td6WFoAAATm1rRGA … VZiascRn+wIAAAAABFla' | b64decode }}" >> >> That took the file lookup plugin out of the equation, but resulted in the >> same error. >> >> I next tried the pipe lookup plugin (with ‘cat my-binary-file’), but that >> also fails in the same way as the file lookup plugin. Looking at the source >> code, the pipe lookup plugin follows the file lookup one and >> unconditionally tries to decode the output from the command as a UTF-8 >> encoded string. >> >> I provisionally conclude from all this that Ansible generally doesn’t >> want to handle binary parameter values. That seems limiting. I suppose one >> solution to my specific problem (without fixing the wider issue) would be >> for the ec2 module to accept something like a user_data_file parameter (as >> an alternative to user_data), which would specify a local file whose >> content was to be used without any processing at all as an opaque user data >> binary object. In fact, that’s pretty much exactly what I want, as I could >> give it the FreeBSD configinit .tar file directly. >> >> I am still hoping that someone has a solution that will enable binary >> user data to be provided to an instance. >> >> Again, my thanks in advance for any assistance. >> >> -- >> Daniel >> >> On Tuesday, 23 December 2014 23:06:38 UTC, Daniel Neades wrote: >>> >>> On Saturday, 25 October 2014 20:33:32 UTC+1, Chris Church wrote: >>>> >>>> The file lookup plugin is your friend. >>>> >>> >>> It is – if one wishes to supply text-based user data. Unfortunately, I >>> have a requirement to provide a binary blob of user data, and the file >>> lookup plugin does not seem to handle that – looking at the source code, it >>> expects the file to be a UTF8-encoded text file. >>> >>> <snip> >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "Ansible Project" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To post to this group, send email to [email protected] >> <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/ansible-project/9708972a-8a15-4819-9837-4a172b9cf608%40googlegroups.com >> >> <https://groups.google.com/d/msgid/ansible-project/9708972a-8a15-4819-9837-4a172b9cf608%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "Ansible Project" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/4cd6aec1-9480-4dbb-8da3-832d13c12885%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
