Public bug reported:
On AWS Graviton instances running Ubuntu 24.04's `linux-aws-7.0` HWE
kernel (`7.0.0-1010.10~24.04.1` and `7.0.0-1011.11~24.04.1`), the in-
tree `ena` driver fails to restore the NIC after an EC2 hibernation
resume: MSI-X interrupts never re-arm, every I/O queue creation times
out, and `pci_pm_restore` returns `-62` (ETIME). The kernel resumes
successfully otherwise — the box comes back alive but with no
functioning network. The identical instance/AMI/workload on `linux-
aws-6.17` (`6.17.0-1019.19~24.04.1`) resumes cleanly, repeatedly.
## Environment
Regression window: linux-aws-6.17 6.17.0-1019.19~24.04.1 (works) ->
linux-aws-7.0 7.0.0-1010.10~24.04.1 / 7.0.0-1011.11~24.04.1 (fails)
Related enablement bug: LP#2060992 (aws: Support hibernation on Graviton) —
this is a regression of that feature on the 7.0 HWE line.
Not yet tested: 7.0.0-1012.12~24.04.1 currently in noble-proposed — happy to
verify it on request.
- Instances: `t4g.medium` (controlled reproduction) and `t4g.xlarge` (2
production occurrences), arm64/Graviton, Nitro
- AMI: `ubuntu/images/hvm-ssd-gp3/ubuntu-noble-24.04-arm64-server-20260714`
- Hibernation: EC2 native (`HibernationOptions.Configured=true`,
`ec2-hibinit-agent`, encrypted gp3 root, swap file with resume offset via
`/sys/power/resume*`)
- Hibernate entry verified by EC2 (`StateReason: Client.UserInitiatedHibernate`)
- ENA device: "ENA device version: 0.10, controller version: 0.0.1
implementation version 1", LLQ not supported → host mode policy
## Reproduction (deterministic, 1/1 on 7.0; 0/2 failures on 6.17, same
box)
1. Launch hibernation-enabled t4g.medium from the AMI above (boots
`6.17.0-1019-aws`)
2. Hibernate via EC2 API, start again → clean resume (image restored, network
fine) — repeated twice
3. `apt-get install linux-aws` (pulls `7.0.0-1011-aws`), reboot
4. Hibernate via EC2 API (verified), start again
5. Kernel resumes (loader and restored kernel both 7.0.0-1011; image reads at
~490 MB/s), then ENA restore fails; instance is running but unreachable
indefinitely
## Serial console at the failure
```
[ 0.474314] PM: hibernation: resume from hibernation
[ 1.914485] PM: hibernation: Read 660360 kbytes in 1.34 seconds (492.80 MB/s)
[ 1.925796] Disabling non-boot CPUs ...
[ 1.926822] psci: CPU1 killed (polled 0 ms)
[ 347.764935] ena 0000:00:05.0 ens5: The ena device sent a completion but the
driver didn't receive a MSI-X interrupt (cmd 3)
[ 347.766398] ena 0000:00:05.0 ens5: Failed to create IO CQ. error: -62
[ 347.767242] ena 0000:00:05.0 ens5: Failed to create I/O TX queue num 0 rc:
-62
[ 347.768264] ena 0000:00:05.0 ens5: Queue creation failed with error code -62
[ 347.769355] ena 0000:00:05.0: Failed to create I/O queues
[ 347.771210] ena 0000:00:05.0: Reset attempt failed. Can not reset the device
[ 347.772070] ena 0000:00:05.0: PM: dpm_run_callback(): pci_pm_restore returns
-62
[ 347.773065] ena 0000:00:05.0: PM: failed to restore async: error -62
```
(The `[347s]` timestamps are the restored kernel's clock continuing from
the pre-hibernate session — the failure is immediately post-restore in
wall time.)
## Notes
- Two production occurrences on `7.0.0-1010` preceded the controlled repro: box
fully dark after resume (no journal entries post-`hibernation entry`; provider
reports instance running). Console was not captured for those; the controlled
7.0.0-1011 run above was instrumented with `get-console-output` polling.
- Possibly related non-AWS report of a 7.0 resume regression where the same
driver works on 6.17: NVIDIA/open-gpu-kernel-modules#1117 — may point at a
broader 7.0 PM/interrupt-restore ordering change rather than an ena-specific
fault.
- Also being filed against Ubuntu's `linux-aws` on Launchpad (the affected
build is Canonical's).
- Happy to test candidate fixes on the same setup — reproduction is cheap and
fully automated on our side.
** Affects: linux-aws-7.0 (Ubuntu)
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2164714
Title:
ena NIC fails PM restore after EC2 hibernation resume on
arm64/Graviton with 7.0.0-1010/1011 (noble HWE); 6.17.0-1019 resumes
cleanly
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux-aws-7.0/+bug/2164714/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
