[Bug 1915345] Re: [SRU] eic_harvest_hostkeys fails in local zones
This bug was fixed in the package ec2-instance-connect - 1.1.12+dfsg1-0ubuntu3~16.04.2 --- ec2-instance-connect (1.1.12+dfsg1-0ubuntu3~16.04.2) xenial; urgency=medium * System startup now ignores failures on host key harvesting (LP: #1915345) -- Balint Reczey Wed, 31 Mar 2021 21:04:31 +0200 -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1915345 Title: [SRU] eic_harvest_hostkeys fails in local zones To manage notifications about this bug go to: https://bugs.launchpad.net/ec2-instance-connect/+bug/1915345/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1915345] Re: [SRU] eic_harvest_hostkeys fails in local zones
This bug was fixed in the package ec2-instance-connect - 1.1.12+dfsg1-0ubuntu3~18.04.2 --- ec2-instance-connect (1.1.12+dfsg1-0ubuntu3~18.04.2) bionic; urgency=medium * System startup now ignores failures on host key harvesting (LP: #1915345) -- Balint Reczey Wed, 31 Mar 2021 21:04:31 +0200 ** Changed in: ec2-instance-connect (Ubuntu Xenial) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1915345 Title: [SRU] eic_harvest_hostkeys fails in local zones To manage notifications about this bug go to: https://bugs.launchpad.net/ec2-instance-connect/+bug/1915345/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1915345] Re: [SRU] eic_harvest_hostkeys fails in local zones
This bug was fixed in the package ec2-instance-connect - 1.1.12+dfsg1-0ubuntu3.20.04.1 --- ec2-instance-connect (1.1.12+dfsg1-0ubuntu3.20.04.1) focal; urgency=medium * System startup now ignores failures on host key harvesting (LP: #1915345) -- Balint Reczey Wed, 31 Mar 2021 21:04:31 +0200 ** Changed in: ec2-instance-connect (Ubuntu Focal) Status: Fix Committed => Fix Released ** Changed in: ec2-instance-connect (Ubuntu Bionic) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1915345 Title: [SRU] eic_harvest_hostkeys fails in local zones To manage notifications about this bug go to: https://bugs.launchpad.net/ec2-instance-connect/+bug/1915345/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1915345] Re: [SRU] eic_harvest_hostkeys fails in local zones
This bug was fixed in the package ec2-instance-connect - 1.1.12+dfsg1-0ubuntu3.20.10.1 --- ec2-instance-connect (1.1.12+dfsg1-0ubuntu3.20.10.1) groovy; urgency=medium * System startup now ignores failures on host key harvesting (LP: #1915345) -- Balint Reczey Wed, 31 Mar 2021 21:04:31 +0200 ** Changed in: ec2-instance-connect (Ubuntu Groovy) Status: Fix Committed => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1915345 Title: [SRU] eic_harvest_hostkeys fails in local zones To manage notifications about this bug go to: https://bugs.launchpad.net/ec2-instance-connect/+bug/1915345/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1915345] Re: [SRU] eic_harvest_hostkeys fails in local zones
AWS did not respond on GitHub nor on internal channels to clarify if they are happy with their current fix. I agree with @rcj's concern that usability-wise the fix could be improved, but this improvement can take place later in a separate SRU. I'm marking the bug as verification-done because the fix does what it is expected to do and we received no further information about it. ** Tags removed: verification-failed verification-failed-focal verification-failed-groovy verification-failed-xenial verification-needed-bionic ** Tags added: verification-done verification-done-bionic verification-done-focal verification-done-groovy verification-done-xenial -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1915345 Title: [SRU] eic_harvest_hostkeys fails in local zones To manage notifications about this bug go to: https://bugs.launchpad.net/ec2-instance-connect/+bug/1915345/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1915345] Re: [SRU] eic_harvest_hostkeys fails in local zones
@rcj Thank you for testing the package. I realized that the "fix" is just ignoring the error and noted this regression in [Where problems could occur] section. I, possibly wrongly, assumed that this is upstream's intention and thank you for asking for clarification upstream. For now I don't change the uploaded packages, but wait for upstream's feedback if this is the solution they really want. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1915345 Title: [SRU] eic_harvest_hostkeys fails in local zones To manage notifications about this bug go to: https://bugs.launchpad.net/ec2-instance-connect/+bug/1915345/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1915345] Re: [SRU] eic_harvest_hostkeys fails in local zones
I've left a comment in the upstream bug https://github.com/aws/aws-ec2 -instance-connect-config/issues/28#issuecomment-816650597 regarding my SRU test feedback. -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1915345 Title: [SRU] eic_harvest_hostkeys fails in local zones To manage notifications about this bug go to: https://bugs.launchpad.net/ec2-instance-connect/+bug/1915345/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1915345] Re: [SRU] eic_harvest_hostkeys fails in local zones
I have tested in AWS local zone us-west-2-lax-1a
xenial:
image: ami-008b09448b998a562
build serial: 20201014
ec2-instance-connect 1.1.12+dfsg1-0ubuntu3~16.04.2
bionic:
image: ami-02701bcdc5509e57b
build serial: 20210224
ec2-instance-connect 1.1.12+dfsg1-0ubuntu3~18.04.2
focal:
image: ami-0ca5c3bd5a268e7db
build serial: 20210223
ec2-instance-connect 1.1.12+dfsg1-0ubuntu3.20.04.1
groovy:
image: ami-0c1204e0c5e73ef4c
build serial: 20210325
ec2-instance-connect 1.1.12+dfsg1-0ubuntu3.20.10.1
Are you sure this is right? Yes, the systemd unit no longer fails
because the patch ignored the script failure, but does it *work*? The
package update has no changes to /usr/share/ec2-instance-
connect/eic_harvest_hostkeys to match the string format for a local zone
still. So while the feature is available in local and wavelength zones
the package in -proposed fails to address the underlying failure and so
ec2-instance-connect still broken in those zones (just silently now).
Again, here is the failure:
$ sudo sh -x /usr/share/ec2-instance-connect/eic_harvest_hostkeys 2>&1 | tail -9
+ /usr/bin/curl -s -f -m 1 -H X-aws-ec2-metadata-token:
AQAEAF6AxckVUQFPqe3ivPjLa0b7dlvf4To2TaAReHD-lMpqgvuXBQ==
http://169.254.169.254/latest/meta-data/placement/availability-zone/
+ zone=us-west-2-lax-1b
+ zone_exit=0
+ [ 0 -ne 0 ]
+ /bin/echo us-west-2-lax-1b
+ /usr/bin/head -n 1
+ /bin/grep -Eq ^([a-z]+-){2,3}[0-9][a-z]$
+ exit 255
+ rm -rf /dev/shm/eic-hostkey-WZBt1Vck
Please look at the grep on line 101 of the script:
# Validate the zone
/bin/echo "${zone}" | /usr/bin/head -n 1 | /bin/grep -Eq
"^([a-z]+-){2,3}[0-9][a-z]$" || exit 255
The script needs to handle matches to the existing regex, but also local
zones like 'us-west-2-lax-1b' and wavelength zones like 'us-west-2-wl1
-den-wlz-1'
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1915345
Title:
[SRU] eic_harvest_hostkeys fails in local zones
To manage notifications about this bug go to:
https://bugs.launchpad.net/ec2-instance-connect/+bug/1915345/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1915345] Re: [SRU] eic_harvest_hostkeys fails in local zones
** Tags removed: verification-needed verification-needed-focal verification-needed-groovy verification-needed-xenial ** Tags added: verification-failed verification-failed-focal verification-failed-groovy verification-failed-xenial -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1915345 Title: [SRU] eic_harvest_hostkeys fails in local zones To manage notifications about this bug go to: https://bugs.launchpad.net/ec2-instance-connect/+bug/1915345/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1915345] Re: [SRU] eic_harvest_hostkeys fails in local zones
Hello Robert, or anyone else affected, Accepted ec2-instance-connect into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ec2 -instance-connect/1.1.12+dfsg1-0ubuntu3~16.04.2 in a few hours, and then in the -proposed repository. Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed- xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification- failed-xenial. In either case, without details of your testing we will not be able to proceed. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping! N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days. ** Changed in: ec2-instance-connect (Ubuntu Xenial) Status: New => Fix Committed ** Tags added: verification-needed-xenial -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1915345 Title: [SRU] eic_harvest_hostkeys fails in local zones To manage notifications about this bug go to: https://bugs.launchpad.net/ec2-instance-connect/+bug/1915345/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1915345] Re: [SRU] eic_harvest_hostkeys fails in local zones
Hello Robert, or anyone else affected, Accepted ec2-instance-connect into groovy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ec2 -instance-connect/1.1.12+dfsg1-0ubuntu3.20.10.1 in a few hours, and then in the -proposed repository. Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users. If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed- groovy to verification-done-groovy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification- failed-groovy. In either case, without details of your testing we will not be able to proceed. Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping! N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days. ** Changed in: ec2-instance-connect (Ubuntu Groovy) Status: New => Fix Committed ** Tags added: verification-needed verification-needed-groovy ** Changed in: ec2-instance-connect (Ubuntu Focal) Status: New => Fix Committed ** Tags added: verification-needed-focal -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1915345 Title: [SRU] eic_harvest_hostkeys fails in local zones To manage notifications about this bug go to: https://bugs.launchpad.net/ec2-instance-connect/+bug/1915345/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1915345] Re: [SRU] eic_harvest_hostkeys fails in local zones
** Description changed:
+ [Impact]
+
+ * ec2-instance-connect breaks during host key harvesting for instances
+ launched in local zones [1] making the system boot to degraded mode
+ only.
+
+ [Test Plan]
+
+ * Start a system with the the fixed ec2-instance-connect package in a
+ local zone [1] or break the the /usr/share/ec2-instance-
+ connect/eic_harvest_hostkeys script to exit with failure.
+
+ [Where problems could occur]
+
+ * The fix is ignoring the eic_harvest_hostkeys script's exit code which
+ may hide actual problems in the script or in the infrastructure
+ preventing connecting to the instance using Instance Connect. This is a
+ decision by upstream. There are no other expected issues.
+
+ [Original Bug Text]
+
ec2-instance-connect breaks during host key harvesting for instances
launched in local zones[1]. Here are is the relevant debug data:
- $ systemctl is-system-running
+ $ systemctl is-system-running
degraded
$ systemctl list-units --failed
- UNIT LOAD ACTIVE SUBDESCRIPTION
+ UNIT LOAD ACTIVE SUBDESCRIPTION
● ec2-instance-connect.service loaded failed failed EC2 Instance Connect Host
Key Harvesting
-
$ journalctl --unit ec2-instance-connect
-- Logs begin at Wed 2021-02-10 22:47:47 UTC, end at Wed 2021-02-10 22:55:46
UTC. --
Feb 10 22:48:16 ip-172-31-51-82 systemd[1]: Starting EC2 Instance Connect
Host Key Harvesting...
Feb 10 22:48:16 ip-172-31-51-82 systemd[1]: ec2-instance-connect.service:
Main process exited, code=exited, status=255/EXCEPTION
Feb 10 22:48:16 ip-172-31-51-82 systemd[1]: ec2-instance-connect.service:
Failed with result 'exit-code'.
Feb 10 22:48:16 ip-172-31-51-82 systemd[1]: Failed to start EC2 Instance
Connect Host Key Harvesting.
$ dpkg-query -l ec2-instance-connect
ii ec2-instance-connect 1.1.13-0ubuntu1 all Configures ssh daemon
to accept EC2 Instance Connect ssh keys
-
$ lsb_release -c
Codename: hirsute
- $ cat /etc/cloud/build.info
+ $ cat /etc/cloud/build.info
build_name: server
serial: 20210208
$ ec2metadata --availability-zone --ami-id
us-west-2-lax-1a
ami-098f71a7a25a0f1f2
-
$ bash -x /usr/share/ec2-instance-connect/eic_harvest_hostkeys
...
++ /usr/bin/curl -s -f -m 1 -H 'X-aws-ec2-metadata-token:
AQAEAEvStI0Ugwz1C3GQh7oubFTah7bXQllCmFU6BtMI6b6l5zMkVQ=='
http://169.254.169.254/latest/meta-data/placement/availability-zone/
+ zone=us-west-2-lax-1a
+ zone_exit=0
+ '[' 0 -ne 0 ']'
+ /bin/echo us-west-2-lax-1a
+ /bin/grep -Eq '^([a-z]+-){2,3}[0-9][a-z]$'
+ /usr/bin/head -n 1
+ exit 255
[1] https://aws.amazon.com/about-aws/global-infrastructure/localzones/
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1915345
Title:
[SRU] eic_harvest_hostkeys fails in local zones
To manage notifications about this bug go to:
https://bugs.launchpad.net/ec2-instance-connect/+bug/1915345/+subscriptions
--
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1915345] Re: [SRU] eic_harvest_hostkeys fails in local zones
This bug was fixed in the package ec2-instance-connect - 1.1.14-0ubuntu1 --- ec2-instance-connect (1.1.14-0ubuntu1) hirsute; urgency=medium * Bugfix only upload Also contains test improvements which don't affect the package in Ubuntu. * New upstream version 1.1.13 (LP: #1915345): - System startup now ignores failures on host key harvesting. -- Balint Reczey Mon, 29 Mar 2021 20:48:45 +0200 ** Changed in: ec2-instance-connect (Ubuntu) Status: In Progress => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1915345 Title: [SRU] eic_harvest_hostkeys fails in local zones To manage notifications about this bug go to: https://bugs.launchpad.net/ec2-instance-connect/+bug/1915345/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
