[Bug 1885527] Re: cloud-init regenerating ssh-keys
This bug was fixed in the package cloud-init - 20.4-0ubuntu1 --- cloud-init (20.4-0ubuntu1) hirsute; urgency=medium * d/control: add gnupg to Recommends as cc_apt_configure requires it to be installed for some operations. * New upstream release. - Release 20.4 (#686) [James Falcon] (LP: #1905440) - tox: avoid tox testenv subsvars for xenial support (#684) - Ensure proper root permissions in integration tests (#664) [James Falcon] - LXD VM support in integration tests (#678) [James Falcon] - Integration test for fallocate falling back to dd (#681) [James Falcon] - .travis.yml: correctly integration test the built .deb (#683) - Ability to hot-attach NICs to preprovisioned VMs before reprovisioning (#613) [aswinrajamannar] - Support configuring SSH host certificates. (#660) [Jonathan Lung] - add integration test for LP: #1900837 (#679) - cc_resizefs on FreeBSD: Fix _can_skip_ufs_resize (#655) [Mina Galić] (LP: #1901958, #1901958) - DataSourceAzure: push dmesg log to KVP (#670) [Anh Vo] - Make mount in place for tests work (#667) [James Falcon] - integration_tests: restore emission of settings to log (#657) - DataSourceAzure: update password for defuser if exists (#671) [Anh Vo] - tox.ini: only select "ci" marked tests for CI runs (#677) - Azure helper: Increase Azure Endpoint HTTP retries (#619) [Johnson Shi] - DataSourceAzure: send failure signal on Azure datasource failure (#594) [Johnson Shi] - test_persistence: simplify VersionIsPoppedFromState (#674) - only run a subset of integration tests in CI (#672) - cli: add --system param to allow validating system user-data on a machine (#575) - test_persistence: add VersionIsPoppedFromState test (#673) - introduce an upgrade framework and related testing (#659) - add --no-tty option to gpg (#669) [Till Riedel] (LP: #1813396) - Pin pycloudlib to a working commit (#666) [James Falcon] - DataSourceOpenNebula: exclude SRANDOM from context output (#665) - cloud_tests: add hirsute release definition (#662) - split integration and cloud_tests requirements (#652) - faq.rst: add warning to answer that suggests running `clean` (#661) - Fix stacktrace in DataSourceRbxCloud if no metadata disk is found (#632) [Scott Moser] - Make wakeonlan Network Config v2 setting actually work (#626) [dermotbradley] - HACKING.md: unify network-refactoring namespace (#658) [Mina Galić] - replace usage of dmidecode with kenv on FreeBSD (#621) [Mina Galić] - Prevent timeout on travis integration tests. (#651) [James Falcon] - azure: enable pushing the log to KVP from the last pushed byte (#614) [Moustafa Moustafa] - Fix launch_kwargs bug in integration tests (#654) [James Falcon] - split read_fs_info into linux & freebsd parts (#625) [Mina Galić] - PULL_REQUEST_TEMPLATE.md: expand commit message section (#642) - Make some language improvements in growpart documentation (#649) [Shane Frasier] - Revert ".travis.yml: use a known-working version of lxd (#643)" (#650) - Fix not sourcing default 50-cloud-init ENI file on Debian (#598) [WebSpider] - remove unnecessary reboot from gpart resize (#646) [Mina Galić] - cloudinit: move dmi functions out of util (#622) [Scott Moser] - integration_tests: various launch improvements (#638) - test_lp1886531: don't assume /etc/fstab exists (#639) - Remove Ubuntu restriction from PR template (#648) [James Falcon] - util: fix mounting of vfat on *BSD (#637) [Mina Galić] - conftest: improve docstring for disable_subp_usage (#644) - doc: add example query commands to debug Jinja templates (#645) - Correct documentation and testcase data for some user-data YAML (#618) [dermotbradley] - Hetzner: Fix instance_id / SMBIOS serial comparison (#640) [Markus Schade] - .travis.yml: use a known-working version of lxd (#643) - tools/build-on-freebsd: fix comment explaining purpose of the script (#635) [Mina Galić] - Hetzner: initialize instance_id from system-serial-number (#630) [Markus Schade] (LP: #1885527) - Explicit set IPV6_AUTOCONF and IPV6_FORCE_ACCEPT_RA on static6 (#634) [Eduardo Otubo] - get_interfaces: don't exclude Open vSwitch bridge/bond members (#608) [Lukas Märdian] (LP: #1898997) - Add config modules for controlling IBM PowerVM RMC. (#584) [Aman306] (LP: #1895979) - Update network config docs to clarify MAC address quoting (#623) [dermotbradley] - gentoo: fix hostname rendering when value has a comment (#611) [Manuel Aguilera] - refactor integration testing infrastructure (#610) [James Falcon] - stages: don't reset permissions of cloud-init.log every boot (#624) (LP: #1900837) - docs: Add how to use cloud-localds to boot qemu (#617) [Joshua Powers] - Drop vestigial update_resolv
[Bug 1885527] Re: cloud-init regenerating ssh-keys
This bug is believed to be fixed in cloud-init in version 20.4. If this is still a problem for you, please make a comment and set the state back to New Thank you. ** Changed in: cloud-init Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1885527 Title: cloud-init regenerating ssh-keys To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-init/+bug/1885527/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1885527] Re: cloud-init regenerating ssh-keys
New images with the patched datasource have been rolled out on the platform. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1885527 Title: cloud-init regenerating ssh-keys To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-init/+bug/1885527/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1885527] Re: cloud-init regenerating ssh-keys
** Also affects: cloud-init Importance: Undecided Status: New ** Changed in: cloud-init Status: New => Fix Committed ** Changed in: cloud-init Importance: Undecided => Medium ** Changed in: cloud-init Assignee: (unassigned) => Markus Schade (lp-markusschade) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1885527 Title: cloud-init regenerating ssh-keys To manage notifications about this bug go to: https://bugs.launchpad.net/cloud-init/+bug/1885527/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1885527] Re: cloud-init regenerating ssh-keys
** Changed in: cloud-init (Ubuntu) Status: Confirmed => In Progress ** Changed in: cloud-init (Ubuntu) Assignee: (unassigned) => Markus Schade (lp-markusschade) -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1885527 Title: cloud-init regenerating ssh-keys To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1885527/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1885527] Re: cloud-init regenerating ssh-keys
https://github.com/canonical/cloud-init/pull/630 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1885527 Title: cloud-init regenerating ssh-keys To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1885527/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1885527] Re: cloud-init regenerating ssh-keys
I co-wrote the datasource. ;-) I will prepare a MR to add the following to our DS: def check_instance_id(self, sys_cfg): return sources.instance_id_matches_system_uuid( self.get_instance_id(), 'system-serial-number') That should take care of those cases and prevent an already configured instance from being reconfigured in case the metadata server does not respond properly. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1885527 Title: cloud-init regenerating ssh-keys To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1885527/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1885527] Re: cloud-init regenerating ssh-keys
@Markus, Can you please provide a link to documentation showing that? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1885527 Title: cloud-init regenerating ssh-keys To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1885527/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1885527] Re: cloud-init regenerating ssh-keys
Hetzner also provides instance ID via DMI information (system-serial- number). So this could be used as a fallback should the metadata service not respond -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1885527 Title: cloud-init regenerating ssh-keys To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1885527/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1885527] Re: cloud-init regenerating ssh-keys
Status changed to 'Confirmed' because the bug affects multiple users. ** Changed in: cloud-init (Ubuntu) Status: New => Confirmed -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1885527 Title: cloud-init regenerating ssh-keys To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1885527/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1885527] Re: cloud-init regenerating ssh-keys
I've found the problem. The cloud provider (in this case german Hetzner) provides a machine with a virtual ethernet interface # lspci | fgrep Ethernet 00:03.0 Ethernet controller: Red Hat, Inc. Virtio network device which is, following the naming standards, named ens3 But then, the provider gives an cloud-init-file, which cloud-init fetches by http (I have not yet fully understood the after-boot- behaviour of cloud-init), which gives the network-configuration as network-config: config: - mac_address: 96:00:00:70:f1:8f name: eth0 subnets: - dns_nameservers: - 213.133.99.99 - 213.133.100.100 - 213.133.98.98 ipv4: true type: dhcp - address: 2a01:4f9:c010:d0eb::1/64 gateway: fe80::1 ipv6: true type: static type: physical version: 1 thus renaming the device from ens3 to eth0 Therefore kern.log contains entries like Sep 23 14:13:44 worker-00 kernel: [1.372316] virtio_net virtio0 ens3: renamed from eth0 Sep 23 14:13:44 worker-00 kernel: [4.834687] virtio_net virtio0 eth0: renamed from ens3 proving that the network interface changes between ens3 and eth0. This then *sometimes* (logs taken from another machine where that problem occured) results in the cloud-init.log 2020-09-22 15:00:15,611 - __init__.py[DEBUG]: Attempting setup of ephemeral network on ens3 with 169.254.0.1/16 brd 169.254.255.255 2020-09-22 15:00:15,611 - subp.py[DEBUG]: Running command ['ip', '-family', 'inet', 'addr', 'add', '169.254.0.1/16', 'broadcast', '169.254.255.255', 'dev', 'ens3'] with allowed return codes [0] (shell=False, capture=True) 2020-09-22 15:00:15,637 - handlers.py[DEBUG]: finish: init-local/search-Hetzner: FAIL: no local data found from DataSourceHetzner 2020-09-22 15:00:15,637 - util.py[WARNING]: Getting data from failed 2020-09-22 15:00:15,639 - util.py[DEBUG]: Getting data from failed Traceback (most recent call last): File "/usr/lib/python3/dist-packages/cloudinit/sources/__init__.py", line 770, in find_source if s.update_metadata([EventType.BOOT_NEW_INSTANCE]): File "/usr/lib/python3/dist-packages/cloudinit/sources/__init__.py", line 659, in update_metadata result = self.get_data() File "/usr/lib/python3/dist-packages/cloudinit/sources/DataSourceHetzner.py", line 53, in get_data with cloudnet.EphemeralIPv4Network(nic, "169.254.0.1", 16, File "/usr/lib/python3/dist-packages/cloudinit/net/__init__.py", line 990, in __enter__ self._bringup_device() File "/usr/lib/python3/dist-packages/cloudinit/net/__init__.py", line 1027, in _bringup_device subp.subp( File "/usr/lib/python3/dist-packages/cloudinit/subp.py", line 291, in subp raise ProcessExecutionError(stdout=out, stderr=err, cloudinit.subp.ProcessExecutionError: Unexpected error while running command. Command: ['ip', '-family', 'inet', 'addr', 'add', '169.254.0.1/16', 'broadcast', '169.254.255.255', 'dev', 'ens3'] Exit code: 1 Reason: - Stdout: Stderr: Cannot find device "ens3" 2020-09-22 15:00:15,646 - main.py[DEBUG]: No local datasource found So it seems to be a timing problem. If the cloud-init-configuration contains a different interface name that the natural standard name (here: eth0 instead of ens3), then it can sometimes (rarely, but experienced several times) happen that cloud-init does not find the interface ens3, because it has been renamed eth0, thus can't initialize it, can't fetch the init file and then for some strange reason believes it should refresh itself. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1885527 Title: cloud-init regenerating ssh-keys To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1885527/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1885527] Re: cloud-init regenerating ssh-keys
** Changed in: cloud-init (Ubuntu) Status: Expired => New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1885527 Title: cloud-init regenerating ssh-keys To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1885527/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1885527] Re: cloud-init regenerating ssh-keys
[Expired for cloud-init (Ubuntu) because there has been no activity for 60 days.] ** Changed in: cloud-init (Ubuntu) Status: Incomplete => Expired -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1885527 Title: cloud-init regenerating ssh-keys To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1885527/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1885527] Re: cloud-init regenerating ssh-keys
I opened bug 158 to request better documentation on this feature. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1885527 Title: cloud-init regenerating ssh-keys To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1885527/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1885527] Re: cloud-init regenerating ssh-keys
@Daniel,
> One convenience we could potentially provide: if cloud-init had a way
> for image creators to express "when next launched, cloud-init should
> treat that instance ID as immutable and permanent" (in a way that could
> be undone on subsequent boots, if a user wants to "unfreeze" an instance
> for image capture) then we might be able to avoid some of this pain, but
> that idea would need more fleshing out before it's clear if it even
> makes sense.
I think you're basically describing "manual_cache_clean".
The intent (testing is needed) for manual_cache_clean is that
a.) user-data and system config (/etc/cloud/*.cfg) can set
manual_cache_clean to true or false. As always user-data overrides system
config. vendor-data should also be able to provide the setting.
b.) cloud-init renders /var/lib/cloud/instance/manual-clean
(path_helper.get_ipath_cur("manual_clean_marker")) if
c.) on boot, both ds-identify and cloud-init will check
and respect existance of /var/lib/cloud/instance/manual-clean
So... "unfreeze", if manual_cache_clean was set is just:
rm -Rf /var/lib/cloud/instance /var/lib/cloud/instance/
I think it would be good to both test that my intent/understanding are
correct, and document it.
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1885527
Title:
cloud-init regenerating ssh-keys
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1885527/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1885527] Re: cloud-init regenerating ssh-keys
** Changed in: cloud-init (Ubuntu) Status: New => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1885527 Title: cloud-init regenerating ssh-keys To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1885527/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1885527] Re: cloud-init regenerating ssh-keys
> It is definitely not Hetzner's task to fix Ubuntu. To be clear, cloud-init is not used only on Ubuntu; I believe that Hetzner's outage would have this effect across the majority of Linux distributions. And, that aside, I don't think this characterisation is fair: we're suggesting that if Hetzner are going to allow their internal services to go down, then they should provide a more reliable way for instances to determine their identity. (This is generally done via DMI in other clouds that do it. The hypervisor stores the instance ID and provides it as a DMI value, and obviously instances can only boot if the hypervisor is up; therefore, the instance ID is always available.) To state this more glibly (and therefore less helpfully): it is not cloud- init's task to fix Hetzner. That said, perhaps there is something that the Hetzner data source could do to handle this Hetzner-specific case. We already perform 60 retries with a 2 second wait between them, and a 2 second timeout. So we allow at least 2 minutes for the services to respond with something before we give up; we could bump that but I don't think it addresses the underlying issue. Any thoughts would be appreciated. Alternatively (or perhaps additionally), this may need a change in the instance ID model that cloud-init uses to handle an explicit "we are not currently able to determine instance ID, so assume it hasn't changed". I think, however, that this would lead to a converse problem: instances launched from instance-capture images which boot for the first time during an "instance ID outage" would not detect that they were new instances, and so would not perform their first boot customisation. This would result in potentially-inaccessible instances (if any credentials remaining in the image are not available to the user launching instances) with SSH host keys not rotated (meaning that they would all have the same host keys as the image; a security issue). Of course, if users are also relying on their cloud-init user-data to perform any actions, that also won't occur; depending on their threat model, some users might also consider this a security issue. The ultimate problem is that cloud-init cannot determine when it runs within an instance whether or not this is a "first boot": the cloud needs to indicate to us one way or the other, which is done via instance ID. If the cloud cannot do that, then there is no way to determine the correct behaviour. If you are certain that you will never be capturing instances as images (i.e. you can categorically say that the root filesystem in this instance will _never_ first boot again) and you aren't using any of cloud-init's functionality after first boot (e.g. per-boot scripts), then you can disable cloud-init in the ways described by Scott earlier in this bug. One convenience we could potentially provide: if cloud-init had a way for image creators to express "when next launched, cloud-init should treat that instance ID as immutable and permanent" (in a way that could be undone on subsequent boots, if a user wants to "unfreeze" an instance for image capture) then we might be able to avoid some of this pain, but that idea would need more fleshing out before it's clear if it even makes sense. > Especially since that process of re-initiialization of that instance ID is neither obvious nor documented. Agreed on both counts; cloud-init's documentation is lacking in many respects, including this one. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1885527 Title: cloud-init regenerating ssh-keys To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1885527/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1885527] Re: cloud-init regenerating ssh-keys
Yes, I do think this is unreasonable. It is definitely not Hetzner's task to fix Ubuntu. Especially since that process of re-initiialization of that instance ID is neither obvious nor documented. Looking at https://cloudinit.readthedocs.io/ I did not yet find an explanation of what is going on, and at https://cloudinit.readthedocs.io/en/latest/topics/instancedata.html it just says v1.instance_id Unique instance_id allocated by the cloud. Examples output: i- but does not give a hint that this is to be constantly provided by some internal service. On the contrary, it says "Cloud-init is the industry standard multi-distribution method for cross-platform cloud instance initialization. It is supported across all major public cloud providers, provisioning systems for private cloud infrastructure, and bare-metal installations." It says „instance initalization”. It does not say that is keeps modifying the living instance. So this is undocumented behaviour, and I am more and more thinking about the question, whether this is a backdoor. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1885527 Title: cloud-init regenerating ssh-keys To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1885527/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1885527] Re: cloud-init regenerating ssh-keys
** Changed in: cloud-init (Ubuntu) Status: Incomplete => New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1885527 Title: cloud-init regenerating ssh-keys To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1885527/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1885527] Re: cloud-init regenerating ssh-keys
If Hetzner has (or starts to provide) a way of determining instance ID without using the network, we'd be more than happy to accept patches to use that in cloud-init. However, as it sounds like the issue here is Hetzner's internal services being unreliable, rather than a cloud-init issue, I'm going to mark this Incomplete. If you think this is unreasonable, please comment and change the status back to New. Thanks! ** Changed in: cloud-init (Ubuntu) Status: New => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1885527 Title: cloud-init regenerating ssh-keys To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1885527/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1885527] Re: cloud-init regenerating ssh-keys
@Valery, Some cloud platforms provide the instance id via some non-network channel (dmi data is common). In those cases, cloud-init will check cached value versus the locally-available instance-id before looking for a network available datasource. So, if Hetzner provides that information in some way, cloud-init can use it. If not, the only options are to for the user to disable cloud-init (touch /etc/cloud/cloud-init.disabled) or set manual_cache_clean (https://bugs.launchpad.net/cloud-init/+bug/1712680/comments/11). I'm not really sold on "what if the metadata service is DOWN" argument. Your cloud should not have its important services just fail. If it does, things are going to break. You could make a similar argument "What if DNS server is down?". I'm not discounting "Design for failure", and cloud-init could definitely do better here, but we need some support from the platform (locally available instance-id) to do better without sacrificing design goals. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1885527 Title: cloud-init regenerating ssh-keys To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1885527/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1885527] Re: cloud-init regenerating ssh-keys
We have no problem about cloud-init still being active on the machine after the first boot init. My issue is: On first instance boot, the metadata service is successfully contacted, and the initialisation succeed (host ssh key generated, hostname set, ...) But on any reboot, if the metadata service is DOWN or not reachable for any reason, then cloud-init regenerates the host ssh keys. My understanding is that determining if it is a first boot, or not, is only based on the cached instance-id, compared to the data received from the metadata service. So if the metadata service is DOWN or not reachable, cloud-init will always think it is a first boot, right? Isn't it possible to make this test more robust? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1885527 Title: cloud-init regenerating ssh-keys To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1885527/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1885527] Re: cloud-init regenerating ssh-keys
"AGAIN: Why is cloud-init still manipulating the machine *after* initialization and first boot?" Because cloud-init thinks it is a "first boot". A supported use case for cloud-init is: * boot instance on cloud * ssh in * install some packages, prep this instance * stop instance * snapshot disk * register new image from disk * start new instances from this image cloud-init will recognize that these instances are new instances, and initialize them. It recognizes this by comparing the cached value of 'instance-id' versus the current value of 'instance-id'. If they have changed, then you have a new instance. The other reason for cloud-init to "remain active" is that it offers "per-boot" things. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1885527 Title: cloud-init regenerating ssh-keys To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1885527/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1885527] Re: cloud-init regenerating ssh-keys
** Changed in: cloud-init (Ubuntu) Status: Incomplete => New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1885527 Title: cloud-init regenerating ssh-keys To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1885527/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1885527] Re: cloud-init regenerating ssh-keys
I currently cannot give logs, since these were only temporary testing machines in a cloud, that existed only for tens of minutes to test installation procedures. I will supply logs as soon as a proceed with testing and the problem occurs again. However, I do not understand and did not find any documentation about why cloud-init even remains active after first boot. Descriptions like https://help.ubuntu.com/community/CloudInit or https://cloudinit.readthedocs.io/en/latest/ are just misleading as they suggest, that this is just about the initialization of the machine. They don't tell that cloud-init remains active and keeps manipulating the system. I found this to be a severy security issue (which I reported in an earlier bug report for 18.04) when I could not permanently change the hostname of a machine, since cloud-init was resetting it with every reboot, and the file, where this was stored, was hidden deeply somewhere in /var. I'm afraid I cannot even change a password, since cloud-init might reset it to it's initial state. I do consider it as a serious flaw and security problem just that cloud- init is behaving very differently from what's described in the documentation. AGAIN: Why is cloud-init still manipulating the machine *after* initialization and first boot? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1885527 Title: cloud-init regenerating ssh-keys To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1885527/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1885527] Re: cloud-init regenerating ssh-keys
We have a similar issue: 1. At first boot cloud-init generated the host ssh keys 2. The metadata service crashed and went down :( 3. At reboot, cloud-init can NOT reach the metadata service and regenerates the host ssh keys -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1885527 Title: cloud-init regenerating ssh-keys To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1885527/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1885527] Re: cloud-init regenerating ssh-keys
after replying with collected logs, please set the status back to 'new'. thanks for taking the time to file a bug. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1885527 Title: cloud-init regenerating ssh-keys To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1885527/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1885527] Re: cloud-init regenerating ssh-keys
Hi, please attach the output of 'cloud-init collect-logs' when run on a system that demonstrates the problem. cloud-init uses the "instance-id" from the metadata service to indicate a new instance. Some things run once per instance, some things run once per boot. ** Changed in: cloud-init (Ubuntu) Status: New => Incomplete -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1885527 Title: cloud-init regenerating ssh-keys To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1885527/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1885527] Re: cloud-init regenerating ssh-keys
BTW, docs at https://cloudinit.readthedocs.io completely fail to tell what cloud-init actually is or is supposed to do. It is not explaining that or why cloud-init survives the first boot and remains active for future boots, and what this is good for. There is no warning, no hint, no information that cloud-init keeps continuously twiddeling with the system. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1885527 Title: cloud-init regenerating ssh-keys To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1885527/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
