Public bug reported: Since around 2024-05-24, we've been experiencing an error in the autopkgtest.ubuntu.com infrastructure. The symptom is all `lxc` command invokation returning `Error: Failed to begin transaction: context deadline exceeded`, whether from the machine usually running the commands through an LXD remote (like `lxc list lxd-armhf-10.123.123.123:` to list containers on the given remote), or from the machine itself where the containers run on (`lxc list` on the machine `10.123.123.123`).
This is happening quite randomly, sometime every few days, sometimes two times per day on the same machine. At one point, the 16 workers where in that same situation at the same time (around 2024-05-26, Sunday evening, when nobody took care of that). Those workers are all `arm64` Jammy machines, and the containers running on them are all `armhf` of all the supported Ubuntu releases. LXD version: 5.21.1 LTS, installed with snapd Kernel version: 5.15.0-107.117 Here are logs from around the issue on three machines: https://pastebin.ubuntu.com/p/ZMCbY2gHmX/ https://pastebin.ubuntu.com/p/kVBp7RQb2n/ https://pastebin.ubuntu.com/p/HyGsgdXkqb/ As we can see, the pattern is always the same, and has also been observed on other problematic machines: * First `kernel: physZlw57F: renamed from eth0` and following network related lines. These lines are common during normal operation, but also always happen before the kernel calltrace. That might still just be a coincidence. * Then `kernel: Unable to handle kernel paging request at virtual address` with the calltrace * Finally LXD starting to have issues with the `Failed to begin transaction: context deadline exceeded`. Sometimes these lines start to appear half an hour after the kernel issue, but we've never seen them before. One workaround we're experimenting right now is running the HWE kernel (version 6.5.0-35.35~22.04.1), and so far, the four machines running it haven't had the issue in two days, but it's still too early to conclude anything. ** Affects: linux (Ubuntu) Importance: Undecided Status: New ** Affects: lxd (Ubuntu) Importance: Undecided Status: New ** Attachment added: "`ubuntu-bug` report for one of those machines" https://bugs.launchpad.net/bugs/2067633/+attachment/5784269/+files/report-16 ** Also affects: lxd (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/2067633 Title: `lxc` commands returning `Error: Failed to begin transaction: context deadline exceeded` To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2067633/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
