Did the following: 1. launched a new VM from the custom build
lxc launch mantic-20231005 --vm --device root,size=20GiB mantic-cust-vm 2. pushed squashfs and lxc metadata from same custom build lxc file push build.output/livecd.ubuntu-cpc.squashfs mantic-cust-vm/root/ lxc file push vm/mantic-server-cloudimg-amd64-lxd.tar.xz mantic-cust-vm/root/ 3. initialized lxd lxd init --auto 3. imported into lxc lxc image import ./mantic-server-cloudimg-amd64-lxd.tar.xz ./livecd.ubuntu-cpc.squashfs --alias mantic-cust-con 4. launched the container lxc launch mantic-cust-con m-c-c 5. observed the DENIED ptrace Oct 06 12:27:01 mantic-cust-vm kernel: audit: type=1400 audit(1696595221.386:113): apparmor="DENIED" operation="ptrace" class="ptrace" profile="lxd-m-c-c_</var/snap/lxd/common/lxd>" pid=2420 comm="systemd" requested_mask="read" denied_mask="read" peer="lxd-m-c- c_</var/snap/lxd/common/lxd>//&unconfined" 6. deleted the container lxc stop m-c-c lxc delete m-c-c 6. disabled apparmor_restrict_unprivileged_unconfined sudo bash -c "echo 0 > /proc/sys/kernel/apparmor_restrict_unprivileged_unconfined" 7. launched container lxc launch mantic-cust-con m-c-c 8. see different apparmor denied messages: Oct 06 12:29:58 mantic-cust-vm kernel: audit: type=1400 audit(1696595398.722:905): apparmor="DENIED" operation="mount" class="mount" info="failed flags match" error=-13 profile="lxd-m-c-c_</var/snap/lxd/common/lxd>" name="/run/systemd/unit-root/proc/" pid=4055 comm="(d-logind)" fstype="proc" srcname="proc" flags="rw, nosuid, nodev, noexec" Oct 06 12:29:58 mantic-cust-vm kernel: audit: type=1400 audit(1696595398.766:906): apparmor="DENIED" operation="mount" class="mount" info="failed flags match" error=-13 profile="lxd-m-c-c_</var/snap/lxd/common/lxd>" name="/run/systemd/unit-root/proc/" pid=4048 comm="(polkitd)" fstype="proc" srcname="proc" flags="rw, nosuid, nodev, noexec" Oct 06 12:29:58 mantic-cust-vm kernel: audit: type=1400 audit(1696595398.818:907): apparmor="DENIED" operation="mount" class="mount" info="failed flags match" error=-13 profile="lxd-m-c-c_</var/snap/lxd/common/lxd>" name="/run/systemd/unit-root/proc/" pid=4071 comm="(ostnamed)" fstype="proc" srcname="proc" flags="rw, nosuid, nodev, noexec" Oct 06 12:29:58 mantic-cust-vm kernel: audit: type=1400 audit(1696595398.874:908): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" label="lxd-m-c-c_</var/snap/lxd/common/lxd>//&:lxd-m-c-c_<var-snap-lxd-common-lxd>:unconfined" name="rsyslogd" pid=4062 comm="apparmor_parser" Oct 06 12:29:59 mantic-cust-vm kernel: audit: type=1400 audit(1696595399.106:909): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" label="lxd-m-c-c_</var/snap/lxd/common/lxd>//&:lxd-m-c-c_<var-snap-lxd-common-lxd>:unconfined" name="/usr/lib/snapd/snap-confine" pid=4130 comm="apparmor_parser" Oct 06 12:29:59 mantic-cust-vm kernel: audit: type=1400 audit(1696595399.106:910): apparmor="STATUS" operation="profile_replace" info="same as current profile, skipping" label="lxd-m-c-c_</var/snap/lxd/common/lxd>//&:lxd-m-c-c_<var-snap-lxd-common-lxd>:unconfined" name="/usr/lib/snapd/snap-confine//mount-namespace-capture-helper" pid=4130 comm="apparmor_parser" Oct 06 12:29:59 mantic-cust-vm kernel: audit: type=1400 audit(1696595399.482:911): apparmor="DENIED" operation="file_inherit" class="net" namespace="root//lxd-m-c-c_<var-snap-lxd-common-lxd>" profile="/usr/lib/snapd/snap-confine" pid=4146 comm="snap-confine" family="netlink" sock_type="raw" protocol=15 requested_mask="send receive" denied_mask="send receive" Oct 06 12:29:59 mantic-cust-vm kernel: audit: type=1400 audit(1696595399.498:912): apparmor="DENIED" operation="file_inherit" class="file" namespace="root//lxd-m-c-c_<var-snap-lxd-common-lxd>" profile="snap-update-ns.lxd" name="/apparmor/.null" pid=4157 comm="6" requested_mask="wr" denied_mask="wr" fsuid=1000000 ouid=0 Oct 06 12:29:59 mantic-cust-vm kernel: audit: type=1400 audit(1696595399.558:913): apparmor="DENIED" operation="file_inherit" class="file" namespace="root//lxd-m-c-c_<var-snap-lxd-common-lxd>" profile="snap.lxd.hook.install" name="/apparmor/.null" pid=4146 comm="snap-exec" requested_mask="wr" denied_mask="wr" fsuid=1000000 ouid=0 9. However, these are not affecting the container in the same fashion. Snap finishes initializing, and cloud-init finishes root@mantic-cust-vm:~# lxc shell m-c-c root@m-c-c:~# snap changes ID Status Spawn Ready Summary 1 Done yesterday at 21:03 UTC today at 12:30 UTC Initialize system state 2 Done today at 12:29 UTC today at 12:30 UTC Initialize device root@m-c-c:~# cloud-init status status: done -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to linux in Ubuntu. https://bugs.launchpad.net/bugs/2038567 Title: Mantic 6.5.0-7 kernel causes regression in LXD container usage Status in Release Notes for Ubuntu: New Status in apparmor package in Ubuntu: New Status in linux package in Ubuntu: Incomplete Status in lxd package in Ubuntu: New Status in snapd package in Ubuntu: New Bug description: Following upgrade to 6.5.0-7 kernel in mantic cloud images we are seeing a regression in our cloud image tests. The test runs the following: ``` lxd init --auto --storage-backend dir lxc launch ubuntu-daily:mantic mantic lxc info mantic lxc exec mantic -- cloud-init status --wait ``` The `lxc exec mantic -- cloud-init status --wait` times out after 240s and will fail our test as a result. I have been able to replicate in a local VM ``` wget http://cloud-images.ubuntu.com/mantic/20231005/mantic-server-cloudimg-amd64.img wget --output-document=launch-qcow2-image-qemu.sh https://gist.githubusercontent.com/philroche/14c241c086a5730481e24178b654268f/raw/7af95cd4dfc8e1d0600e6118803d2c866765714e/gistfile1.txt chmod +x launch-qcow2-image-qemu.sh ./launch-qcow2-image-qemu.sh --password passw0rd --image ./mantic-server-cloudimg-amd64.img cat <<EOF > "./reproducer.sh" #!/bin/bash -eux lxd init --auto --storage-backend dir lxc launch ubuntu-daily:mantic mantic lxc info mantic lxc exec mantic -- cloud-init status --wait EOF chmod +x ./reproducer.sh sshpass -p passw0rd scp -o UserKnownHostsFile=/dev/null -o CheckHostIP=no -o StrictHostKeyChecking=no -P 2222 ./reproducer.sh ubuntu@127.0.0.1:~/ sshpass -p passw0rd ssh -o UserKnownHostsFile=/dev/null -o CheckHostIP=no -o StrictHostKeyChecking=no -p 2222 ubuntu@127.0.0.1 sudo apt-get update sshpass -p passw0rd ssh -o UserKnownHostsFile=/dev/null -o CheckHostIP=no -o StrictHostKeyChecking=no -p 2222 ubuntu@127.0.0.1 sudo apt-get upgrade --assume-yes sshpass -p passw0rd ssh -o UserKnownHostsFile=/dev/null -o CheckHostIP=no -o StrictHostKeyChecking=no -p 2222 ubuntu@127.0.0.1 ./reproducer.sh ``` The issue is not present with the 6.5.0-5 kernel and the issue is present regardless of the container launched. I tried the jammy container to test this. From my test VM ``` ubuntu@cloudimg:~$ uname --all Linux cloudimg 6.5.0-7-generic #7-Ubuntu SMP PREEMPT_DYNAMIC Fri Sep 29 09:14:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux ubuntu@cloudimg:~$ uname --kernel-release 6.5.0-7-generic ``` This is a regression in our test that will block 23.10 cloud image release next week. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu-release-notes/+bug/2038567/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
