[Bug 1404311] Re: gce metadata api doesn't properly stream binary data
** Description changed: - While implementing the GCE provider for Juju we found that the metadata - API breaks when trying to retrieve certain binary formats. In our case - the gz of user-data. The API only streams out the first 5 bytes, - encounters what it preceives as a EOF/nil character and truncates the - rest of the request. + [IMPACT] Due to limitations in GCE, binary user-data is mangled when + sent as user-data. + + [FIX] Allow user to declare binary encoding on user-data. + + [VERIFICATION] + 1. Create pristine image from -proposed + 2. For step 6 + 3. Boot GCE instance w/ normal user-data, i.e.: +$ cat user-data +#cloud-config +ssh_import_id: [utlemming] +$ gcloud compute instances create image from step 1 \ + --metadata-from-file user-data=user-data + 4. Confirm that user-data was parsed properly + 5. GZIP user-data, and encode using base64: +gzip -c user-data.txt | base64 user-data.b64 + 6. Boot GCE instance w/ user-data.b64 and character encoding meta-data +set: +$ gcloud compute instances create image from step 1 \ + --metadata-from-file user-data=user-data.b64 \ + --metadata user-data-encoding=base64 + 7. Confirm that user-data was consumed; attach /var/log/cloud-init.log +to report. + + [RISK] If a user sets the user-data-encoding to base64, but does not + provide base64 data the instance will fail to provision. However, since + the user has to explicitly setup the circumstances, it is low risk. + + [ORIGINAL REPORT] + The GCE datasource uses the long hostname. Hostnames longer than 64 characters can break several tools. + While implementing the GCE provider for Juju we found that the metadata API breaks when trying to retrieve certain binary formats. In our case the gz of user-data. The API only streams out the first 5 bytes, encounters what it preceives as a EOF/nil character and truncates the rest of the request. We've opened an issue with Google directly, but in the meantime a work around is to allow an explicit encoding to be set for the user-data field of the GCE metadata. This will allow use to base64 encode the binary blob, which the API returns the entire contents of without issue. ** Summary changed: - gce metadata api doesn't properly stream binary data + [SRU] gce metadata api doesn't properly stream binary data -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to cloud-init in Ubuntu. https://bugs.launchpad.net/bugs/1404311 Title: [SRU] gce metadata api doesn't properly stream binary data To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-init/+bug/1404311/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1404311] Re: gce metadata api doesn't properly stream binary data
This bug was fixed in the package cloud-init - 0.7.7~bzr1055-0ubuntu1 --- cloud-init (0.7.7~bzr1055-0ubuntu1) vivid; urgency=medium * New upstream snapshot. * move to python3 (LP: #1247132) * systemd: run cloud-init before systemd-user-sessions.service * Use the GCE short hostname. (LP: #1383794) * Enable user-data encoding support for GCE. (LP: #1404311) * Update to use a newer and better OMNIBUS_URL * Be more tolerant of 'ssh_authorized_keys' types * Fix parse_ssh_config failing in ssh_util.py * Increase the robustness/configurability of the chef module. * retain trailing newline from template files when using jinja2 (LP: #1355343) * fix broken output handling (LP: #1387340) * digital ocean datasource * update url in config drive documentation * freebsd: enable correct behavior on Ec2. * freebsd: Use the proper virtio FreeBSD network interface name. -- Scott Moser smo...@ubuntu.com Wed, 11 Feb 2015 15:55:58 -0500 ** Changed in: cloud-init (Ubuntu Vivid) Status: Confirmed = Fix Released -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to cloud-init in Ubuntu. https://bugs.launchpad.net/bugs/1404311 Title: gce metadata api doesn't properly stream binary data To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-init/+bug/1404311/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1404311] Re: gce metadata api doesn't properly stream binary data
** Branch linked: lp:ubuntu/cloud-init -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to cloud-init in Ubuntu. https://bugs.launchpad.net/bugs/1404311 Title: gce metadata api doesn't properly stream binary data To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-init/+bug/1404311/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1404311] Re: gce metadata api doesn't properly stream binary data
** Branch linked: lp:~smoser/ubuntu/vivid/cloud-init/2to3 -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to cloud-init in Ubuntu. https://bugs.launchpad.net/bugs/1404311 Title: gce metadata api doesn't properly stream binary data To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-init/+bug/1404311/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
[Bug 1404311] Re: gce metadata api doesn't properly stream binary data
** Also affects: cloud-init (Ubuntu) Importance: Undecided Status: New ** Changed in: cloud-init (Ubuntu) Status: New = Confirmed ** Changed in: cloud-init (Ubuntu) Importance: Undecided = High ** Also affects: cloud-init (Ubuntu Vivid) Importance: High Status: Confirmed ** Also affects: cloud-init (Ubuntu Utopic) Importance: Undecided Status: New ** Also affects: cloud-init (Ubuntu Trusty) Importance: Undecided Status: New ** Changed in: cloud-init (Ubuntu Utopic) Status: New = Confirmed ** Changed in: cloud-init (Ubuntu Trusty) Status: New = Confirmed ** Changed in: cloud-init (Ubuntu Trusty) Importance: Undecided = High ** Changed in: cloud-init (Ubuntu Utopic) Importance: Undecided = Medium -- You received this bug notification because you are a member of Ubuntu Server Team, which is subscribed to cloud-init in Ubuntu. https://bugs.launchpad.net/bugs/1404311 Title: gce metadata api doesn't properly stream binary data To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-init/+bug/1404311/+subscriptions -- Ubuntu-server-bugs mailing list Ubuntu-server-bugs@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-server-bugs
