[Touch-packages] [Bug 1912214] Re: qemu can't access files that are added as rules on hot-add
Next I was trying the same LXD setup that failed before on a different host (to check if it would be reproducible). Current LXD setup (Failing): - LXD is at 4.10 (most recent on latest/stable channel) - Kernel 5.4.0-60 Current Bare-Metal setup (working) - Kernel 5.10.6-051006 New LXD try #0 - LXD on other system (Failing) - same system that has the working bare metal - Same setup as the other LXD based tests - Kernel 5.10.6-051006 => Same issue, the access is blocked even if I add the paths as local override New LXD try #1 - older hirsute kernel (failing) - Same setup as the other tests with LXD - Kernel 5.8.0-36 New BareMetal #1 - older hirsute kernel (working): - Kernel 5.8.0-36 New LXD try #2 - Focal kernel (failing) - Same setup as the other tests with LXD - Kernel 5.4.0-54 New BareMetal #2 - Focal kernel (working): - Kernel 5.4.0-54 New LXD try #3 - recheck on 5.10 (failing) - Same setup as the other tests with LXD - Kernel 5.10.6-051006 New LXD try #4 - recheck on 5.8 (failing now) - Same setup as the other tests with LXD - Kernel 5.8.0-36 New LXD try #5 - Other 5.10 this time from H-proposed (TBD) - Same setup as the other tests with LXD - The former 5.10 I tried was a mainline build (https://kernel.ubuntu.com/~kernel-ppa/mainline/) - Kernel 5.10.6-051006 So there is no new kernel that makes it work. And the problem should be reproducible in many places. I'll add the steps to drive KVM in a container to the description to ease repro ** Summary changed: - qemu can't access files that are added as rules on hot-add + qemu can't access files - even with manual overrides (only an issue in Containers) -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to apparmor in Ubuntu. https://bugs.launchpad.net/bugs/1912214 Title: qemu can't access files - even with manual overrides (only an issue in Containers) Status in apparmor package in Ubuntu: New Bug description: Hi, to be clear I consider it quite likely that the error is on my side, but I'd appreciate guidance how to continue resolve. I have rubber ducked my setup with a coworker and talked to jjohansen if the profile would contain an obvious fault (it did not). So hereby I'm filing a bug hoping to get some help on this case. ## 1. What happened I was trying to add a feature to libvirt to support chained qcow files. That means instead of adding just one path libvirt has to process the chain of backing files and add all of those. That works already on guest start, but not on hot-add of devices. But while I see my code appending the rules I'd expect and issuing the apparmor_parser calls I'd expect it still fails. I'd love to get the profile at runtime to ensure things really are loaded as expected. But as discussed on IRC that won't work. So I tested and simplified and wonder for the test below why things fail. ## 2. how to reproduce I was debugging various cases (see below updates) and found that just one combination fails (also see comment #5). I'm rather sure now that libvirt adds the path correctly, but even adding the denied path that is needed to the local overrides does not help. Use a Hirsute Container and enable this PPA: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/4393 # First get a KVM capable container (the issue only triggers in containers) $ lxc profile create kvm $ cat > kvm_profile.yaml << EOF config: boot.autostart: "true" linux.kernel_modules: openvswitch,nbd,ip_tables,ip6_tables,kvm security.nesting: "true" security.privileged: "true" description: "" devices: eth0: mtu: "9000" name: eth0 nictype: bridged parent: lxdbr0 type: nic kvm: path: /dev/kvm type: unix-char mem: path: /dev/mem type: unix-char tun: path: /dev/net/tun type: unix-char name: kvm EOF $ lxc profile edit kvm < kvm_profile.yaml $ lxc launch ubuntu-daily:h h-libvirt-new --profile default --profile kvm All following steps are in the container $ sudo add-apt-repository ppa:ci-train-ppa-service/4393 $ apt update $ apt upgrade -y $ apt install uvtool-libvirt Then - to underline that the rules should be ok - also add it manually as local override (therefore we don't need to theorize or check if the rule was indeed added by libvirt - but since it works on bare metal we know it was added). # Allow the access that libvirt does not yet allow $ echo '"/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2" rwk,' > /etc/apparmor.d/local/abstractions/libvirt-qemu $ echo '"/var/lib/libvirt/images/testdisk-snap-base.qcow2" rwk,' >> /etc/apparmor.d/local/abstractions/libvirt-qemu # Check the profile can be loaded and contains these $ apparmor_parser -r /etc/apparmor.d/libvirt/libvirt-85410987-d91b-487a-8f2c-911f0136c877 $ apparmor_parser -p
[Touch-packages] [Bug 1912214] Re: qemu can't access files that are added as rules on hot-add
FYI: here the open that is denied from strace 6408 0.000148 openat(AT_FDCWD, "/var/lib/libvirt/images/testdisk-snap-base.qcow2", O_RDONLY|O_CLOEXEC) = -1 EACCES (Permission denied) <0.40> ^^ nothing special in there. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to apparmor in Ubuntu. https://bugs.launchpad.net/bugs/1912214 Title: qemu can't access files that are added as rules on hot-add Status in apparmor package in Ubuntu: New Bug description: Hi, to be clear I consider it quite likely that the error is on my side, but I'd appreciate guidance how to continue resolve. I have rubber ducked my setup with a coworker and talked to jjohansen if the profile would contain an obvious fault (it did not). So hereby I'm filing a bug hoping to get some help on this case. ## 1. What happened I was trying to add a feature to libvirt to support chained qcow files. That means instead of adding just one path libvirt has to process the chain of backing files and add all of those. That works already on guest start, but not on hot-add of devices. But while I see my code appending the rules I'd expect and issuing the apparmor_parser calls I'd expect it still fails. I'd love to get the profile at runtime to ensure things really are loaded as expected. But as discussed on IRC that won't work. So I tested and simplified and wonder for the test below why things fail. ## 2. how to reproduce I was debugging various cases (see below updates) and found that just one combination fails (also see comment #5) I'm rather sure now that libvirt adds the path correctly, but even adding the denied path that is needed to the local overrides does not help. Use a Hirsute Container and enable this PPA: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/4393 Then - to underline that the rule should be ok also add it manually as local override. # Allow the access that libvirt does not yet allow $ echo '"/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2" rwk,' > /etc/apparmor.d/local/abstractions/libvirt-qemu $ echo '"/var/lib/libvirt/images/testdisk-snap-base.qcow2" rwk,' >> /etc/apparmor.d/local/abstractions/libvirt-qem # Check the profile can be loaded and contains these $ apparmor_parser -r /etc/apparmor.d/libvirt/libvirt-85410987-d91b-487a-8f2c-911f0136c877 $ apparmor_parser -p /etc/apparmor.d/libvirt/libvirt-85410987-d91b-487a-8f2c-911f0136c877 | grep snap "/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2" rwk, "/var/lib/libvirt/images/testdisk-snap-base.qcow2" rwk, # prep the disk chain to be hot-added qemu-img create -f qcow2 /var/lib/libvirt/images/testdisk-snap-base.qcow2 100M qemu-img create -f qcow2 -b /var/lib/libvirt/images/testdisk-snap-base.qcow2 /var/lib/libvirt/images/testdisk-snap-snapshot.qcow2 cat > hot-add-test.xml << EOF EOF # prep a guest to attach the disk to uvt-simplestreams-libvirt --verbose sync --source http://cloud-images.ubuntu.com/daily arch=amd64 label=daily release=focal uvt-kvm create --host-passthrough --password=ubuntu f-test2 release=focal arch=amd64 label=daily We have checked the "effective" profile that is loaded. JJohanssen didn't see an obvious flaw in it. But for reference here: root@h-libvirt-orig:~# apparmor_parser -p /etc/apparmor.d/libvirt/libvirt-e033b910-be06-4975-826f-b6fba368d928 | pastebinit https://paste.ubuntu.com/p/yxdmMT6638/ # attach the disk virsh attach-device f-test2 hot-add-test.xml Libvirt (as usual) now adds rules for the image file. With the new code from the PPA it does so twice - once for the snapshot and once the base file. But in any case we allow that access through our override. You can see that on a little gimmick, since the flattened profile always has those rules - and when libvirt is adding them we will sometimes now see info="same as current profile, skipping". If the local override isn't in place that does not show up. So the rules libvirt adds are identical to what we have put into the local override and that should allow the access. But we get: [699826.573035] audit: type=1400 audit(1611044559.393:2047): apparmor="DENIED" operation="open" namespace="root//lxd-h-libvirt- new2_" profile="libvirt-85410987-d91b-487a- 8f2c-911f0136c877" name="/var/lib/libvirt/images/testdisk-snap- base.qcow2" pid=111701 comm="qemu-system-x86" requested_mask="r" denied_mask="r" fsuid=64055 ouid=64055 ## 3. tracking apparmor_parser I was using (for debug) a wrapper for apparmro_parser but all really looks as expected $ cat /usr/sbin/apparmor_parser.wrap #!/bin/bash echo "ARGS $@" | /usr/bin/systemd-cat echo "Content of ${2}:" | /usr/bin/systemd-cat /usr/bin/ls -laF "${2}" | /usr/bin/systemd-cat /usr/bin/cat "${2}" | /usr/bin/systemd-cat
[Touch-packages] [Bug 1912214] Re: qemu can't access files that are added as rules on hot-add
** Description changed: Hi, to be clear I consider it quite likely that the error is on my side, but I'd appreciate guidance how to continue resolve. I have rubber ducked my setup with a coworker and talked to jjohansen if the profile would contain an obvious fault (it did not). So hereby I'm filing a bug hoping to get some help on this case. ## 1. What happened I was trying to add a feature to libvirt to support chained qcow files. That means instead of adding just one path libvirt has to process the chain of backing files and add all of those. That works already on guest start, but not on hot-add of devices. But while I see my code appending the rules I'd expect and issuing the apparmor_parser calls I'd expect it still fails. I'd love to get the profile at runtime to ensure things really are loaded as expected. But as discussed on IRC that won't work. So I tested and simplified and wonder for the test below why things fail. ## 2. how to reproduce - I was taking my new code out of the equation - and to do so I was adding - the path that is needed to the local overrides. But it still is blocked. - So - right now - I'm assuming that my code worked in adding the lines, - but the access is still blocked. Therefore let us try to fix this one - first and only then I can debug into what might be failing on the new - code. + I was debugging various cases (see below updates) and found that just one + combination fails (also see comment #5) + I'm rather sure now that libvirt adds the path correctly, but even + adding the denied path that is needed to the local overrides does not + help. + + Use a Hirsute Container and enable this PPA: + https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/4393 + + Then - to underline that the rule should be ok also add it manually as local override. # Allow the access that libvirt does not yet allow $ echo '"/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2" rwk,' > /etc/apparmor.d/local/abstractions/libvirt-qemu + $ echo '"/var/lib/libvirt/images/testdisk-snap-base.qcow2" rwk,' >> /etc/apparmor.d/local/abstractions/libvirt-qem + + # Check the profile can be loaded and contains these + $ apparmor_parser -r /etc/apparmor.d/libvirt/libvirt-85410987-d91b-487a-8f2c-911f0136c877 + $ apparmor_parser -p /etc/apparmor.d/libvirt/libvirt-85410987-d91b-487a-8f2c-911f0136c877 | grep snap + "/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2" rwk, + "/var/lib/libvirt/images/testdisk-snap-base.qcow2" rwk, + # prep the disk chain to be hot-added qemu-img create -f qcow2 /var/lib/libvirt/images/testdisk-snap-base.qcow2 100M qemu-img create -f qcow2 -b /var/lib/libvirt/images/testdisk-snap-base.qcow2 /var/lib/libvirt/images/testdisk-snap-snapshot.qcow2 cat > hot-add-test.xml << EOF EOF - # prep a guest to attach to + # prep a guest to attach the disk to uvt-simplestreams-libvirt --verbose sync --source http://cloud-images.ubuntu.com/daily arch=amd64 label=daily release=focal uvt-kvm create --host-passthrough --password=ubuntu f-test2 release=focal arch=amd64 label=daily We have checked the "effective" profile that is loaded. JJohanssen didn't see an obvious flaw in it. But for reference here: root@h-libvirt-orig:~# apparmor_parser -p /etc/apparmor.d/libvirt/libvirt-e033b910-be06-4975-826f-b6fba368d928 | pastebinit https://paste.ubuntu.com/p/yxdmMT6638/ # attach the disk virsh attach-device f-test2 hot-add-test.xml - Libvirt/Qemu works as usual - except (known and expected) does not add a rule for "/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2". But for that we have added out local override. - Yet it is denied access. + Libvirt (as usual) now adds rules for the image file. + With the new code from the PPA it does so twice - once for the snapshot and once the base file. + But in any case we allow that access through our override. + You can see that on a little gimmick, since the flattened profile always has those rules - and when libvirt is adding them we will sometimes now see info="same as current profile, skipping". + If the local override isn't in place that does not show up. + So the rules libvirt adds are identical to what we have put into the local override and that should allow the access. - apparmor="DENIED" operation="open" namespace="root//lxd-h-libvirt-new_ - " profile="libvirt-a2caaf89-0682-464d-92ba- - 5295cb5f5128" name="/var/lib/libvirt/images/testdisk-snap- - snapshot.qcow2" pid=2889044 comm="qemu-system-x86" requested_mask="r" - denied_mask="r" fsuid=64055 ouid=64055 + But we get: - ## 3. The rule is generally working + [699826.573035] audit: type=1400 audit(1611044559.393:2047): + apparmor="DENIED" operation="open" namespace="root//lxd-h-libvirt-new2_ + " profile="libvirt-85410987-d91b-487a-8f2c- + 911f0136c877"
[Touch-packages] [Bug 1912214] Re: qemu can't access files that are added as rules on hot-add
I have the following alternations on the test now: A1) on bare metal hirsute A2) on LXD Hisute (on Focal Host) B1) attach a single disk B2) attach a disk with backing chain (profile is appended and reload twice) C1) old libvirt code (can only do one disk) C2) new libvirt code (iterates backing chain) D1) do the same actions manually in qemu-monitor+apparmor tools D2) run test through libvirt Only A2+B2+C2+D2 is failing. The rest I could clear by re-setting test environments to be sure old traces/experiments are gone. I'll update the description. P.S. I can stop libvirt with qemu at any point e.g. after adding one or both rules to the guest and before qemu is told to access the files. The profile on disk looks right and AFAICS it is loaded as that - yet the access from qemu is denied then. TL;DR: Still an odd fail, only in container environment and unclear why/what happens. Please guide me what you'd need next to get this any further. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to apparmor in Ubuntu. https://bugs.launchpad.net/bugs/1912214 Title: qemu can't access files that are added as rules on hot-add Status in apparmor package in Ubuntu: New Bug description: Hi, to be clear I consider it quite likely that the error is on my side, but I'd appreciate guidance how to continue resolve. I have rubber ducked my setup with a coworker and talked to jjohansen if the profile would contain an obvious fault (it did not). So hereby I'm filing a bug hoping to get some help on this case. ## 1. What happened I was trying to add a feature to libvirt to support chained qcow files. That means instead of adding just one path libvirt has to process the chain of backing files and add all of those. That works already on guest start, but not on hot-add of devices. But while I see my code appending the rules I'd expect and issuing the apparmor_parser calls I'd expect it still fails. I'd love to get the profile at runtime to ensure things really are loaded as expected. But as discussed on IRC that won't work. So I tested and simplified and wonder for the test below why things fail. ## 2. how to reproduce I was taking my new code out of the equation - and to do so I was adding the path that is needed to the local overrides. But it still is blocked. So - right now - I'm assuming that my code worked in adding the lines, but the access is still blocked. Therefore let us try to fix this one first and only then I can debug into what might be failing on the new code. # Allow the access that libvirt does not yet allow $ echo '"/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2" rwk,' > /etc/apparmor.d/local/abstractions/libvirt-qemu # prep the disk chain to be hot-added qemu-img create -f qcow2 /var/lib/libvirt/images/testdisk-snap-base.qcow2 100M qemu-img create -f qcow2 -b /var/lib/libvirt/images/testdisk-snap-base.qcow2 /var/lib/libvirt/images/testdisk-snap-snapshot.qcow2 cat > hot-add-test.xml << EOF EOF # prep a guest to attach to uvt-simplestreams-libvirt --verbose sync --source http://cloud-images.ubuntu.com/daily arch=amd64 label=daily release=focal uvt-kvm create --host-passthrough --password=ubuntu f-test2 release=focal arch=amd64 label=daily We have checked the "effective" profile that is loaded. JJohanssen didn't see an obvious flaw in it. But for reference here: root@h-libvirt-orig:~# apparmor_parser -p /etc/apparmor.d/libvirt/libvirt-e033b910-be06-4975-826f-b6fba368d928 | pastebinit https://paste.ubuntu.com/p/yxdmMT6638/ # attach the disk virsh attach-device f-test2 hot-add-test.xml Libvirt/Qemu works as usual - except (known and expected) does not add a rule for "/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2". But for that we have added out local override. Yet it is denied access. apparmor="DENIED" operation="open" namespace="root//lxd-h-libvirt-new_ " profile="libvirt-a2caaf89-0682-464d-92ba- 5295cb5f5128" name="/var/lib/libvirt/images/testdisk-snap- snapshot.qcow2" pid=2889044 comm="qemu-system-x86" requested_mask="r" denied_mask="r" fsuid=64055 ouid=64055 ## 3. The rule is generally working To be clear, on the same system if I just open up a new profile and allow this path to be accessed it works fine. See the working example below. It must be somewhat that is tied to the way KVM guests get their profile updates/assigned. root@h-libvirt-orig:~# cat /etc/apparmor.d/test #include profile test flags=(attach_disconnected) { #include "/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2" rwk, "/usr/bin/md5sum" rmix, } root@h-libvirt-orig:~# apparmor_parser -r /etc/apparmor.d/test; aa-exec -v -p test -- md5sum /var/lib/libvirt/images/testdisk-snap-snapshot.qcow2 [6939]
[Touch-packages] [Bug 1912214] Re: qemu can't access files that are added as rules on hot-add
New test eliminating libvirt from the equation: # Create one backing chain and one normal image $ qemu-img create -f qcow2 /var/lib/libvirt/images/testdisk-nosnap.qcow2 100M $ qemu-img create -f qcow2 /var/lib/libvirt/images/testdisk-snap-base.qcow2 100M $ qemu-img create -f qcow2 -b /var/lib/libvirt/images/testdisk-snap-base.qcow2 /var/lib/libvirt/images/testdisk-snap-snapshot.qcow2 # Use a profile similar to those created by libvirt - our guest doesn't do much, so he doesn't need much $ cat > /etc/apparmor.d/qemu-testprofile << EOF # Test profile for bug LP: #1912214 #include profile qemu-testprofile flags=(attach_disconnected) { #include #include } EOF $ touch /etc/apparmor.d/local/extrarules $ apparmor_parser -r /etc/apparmor.d/qemu-testprofile # Start qemu into the monitor under control of an apparmor profile $ aa-exec -p qemu-testprofile -- qemu-system-x86_64 -serial none -monitor stdio -m 64 -display none # Attach and detach the disk (qemu) drive_add 0 if=none,file=/var/lib/libvirt/images/testdisk-nosnap.qcow2,format=qcow2,id=disk1 # This can be detached (e.g. to iterate the test) via (qemu) drive_del disk1 As-is this fails (expected) since it gets the access to the file blocked: [58147.954482] audit: type=1400 audit(1611040852.319:86): apparmor="DENIED" operation="open" profile="qemu-testprofile" name="/var/lib/libvirt/images/testdisk-nosnap.qcow2" pid=23222 comm="qemu-system-x86" requested_mask="r" denied_mask="r" fsuid=0 ouid=0 Now we can "manually play to be libvirt" and append the rule and reload the profile. In another console we can do: $ echo '"/var/lib/libvirt/images/testdisk-nosnap.qcow2" rwk,' > /etc/apparmor.d/local/extrarules $ apparmor_parser -r /etc/apparmor.d/qemu-testprofile Once that is done the `drive_add` command above will work. Now if we allow the snapshot file but not the base image in apparmor. And then attach the snapshot, we expect qemu to parse the snapshot, identify the backing file and fail to access that. $ echo '"/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2" rwk,' > /etc/apparmor.d/local/extrarules $ apparmor_parser -r /etc/apparmor.d/qemu-testprofile (qemu) drive_add 0 if=none,file=/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2,format=qcow2,id=disk1 Could not open backing file: Could not open '/var/lib/libvirt/images/testdisk-snap-base.qcow2': Permission denied [58777.551226] audit: type=1400 audit(1611041481.906:96): apparmor="DENIED" operation="open" profile="qemu-testprofile" name="/var/lib/libvirt/images/testdisk-snap-base.qcow2" pid=23222 comm="qemu-system-x86" requested_mask="r" denied_mask="r" fsuid=0 ouid=0 Perfect - still as expected. Now if we add both rules we expect this to work. $ echo '"/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2" rwk,' > /etc/apparmor.d/local/extrarules $ echo '"/var/lib/libvirt/images/testdisk-snap-base.qcow2" rwk,' >> /etc/apparmor.d/local/extrarules $ apparmor_parser -r /etc/apparmor.d/qemu-testprofile (qemu) drive_add 0 if=none,file=/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2,format=qcow2,id=disk1 And indeed it does work fine. This also works fine in the containers ... good, but no help for debugging this. ** Description changed: Hi, to be clear I consider it quite likely that the error is on my side, but I'd appreciate guidance how to continue resolve. I have rubber ducked my setup with a coworker and talked to jjohansen if the profile would contain an obvious fault (it did not). So hereby I'm filing a bug hoping to get some help on this case. ## 1. What happened I was trying to add a feature to libvirt to support chained qcow files. That means instead of adding just one path libvirt has to process the chain of backing files and add all of those. That works already on guest start, but not on hot-add of devices. But while I see my code appending the rules I'd expect and issuing the apparmor_parser calls I'd expect it still fails. I'd love to get the profile at runtime to ensure things really are loaded as expected. But as discussed on IRC that won't work. So I tested and simplified and wonder for the test below why things fail. ## 2. how to reproduce I was taking my new code out of the equation - and to do so I was adding the path that is needed to the local overrides. But it still is blocked. So - right now - I'm assuming that my code worked in adding the lines, but the access is still blocked. Therefore let us try to fix this one first and only then I can debug into what might be failing on the new code. # Allow the access that libvirt does not yet allow $ echo '"/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2" rwk,' > /etc/apparmor.d/local/abstractions/libvirt-qemu # prep the disk chain to be hot-added qemu-img create -f qcow2 /var/lib/libvirt/images/testdisk-snap-base.qcow2 100M qemu-img create -f qcow2 -b /var/lib/libvirt/images/testdisk-snap-base.qcow2
[Touch-packages] [Bug 1912214] Re: qemu can't access files that are added as rules on hot-add
Also my new code that I was trying to finish for submission works on bare metal ... ? /me is puzzled -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to apparmor in Ubuntu. https://bugs.launchpad.net/bugs/1912214 Title: qemu can't access files that are added as rules on hot-add Status in apparmor package in Ubuntu: New Bug description: Hi, to be clear I consider it quite likely that the error is on my side, but I'd appreciate guidance how to continue resolve. I have rubber ducked my setup with a coworker and talked to jjohansen if the profile would contain an obvious fault (it did not). So hereby I'm filing a bug hoping to get some help on this case. ## 1. What happened I was trying to add a feature to libvirt to support chained qcow files. That means instead of adding just one path libvirt has to process the chain of backing files and add all of those. That works already on guest start, but not on hot-add of devices. But while I see my code appending the rules I'd expect and issuing the apparmor_parser calls I'd expect it still fails. I'd love to get the profile at runtime to ensure things really are loaded as expected. But as discussed on IRC that won't work. So I tested and simplified and wonder for the test below why things fail. ## 2. how to reproduce I was taking my new code out of the equation - and to do so I was adding the path that is needed to the local overrides. But it still is blocked. So - right now - I'm assuming that my code worked in adding the lines, but the access is still blocked. Therefore let us try to fix this one first and only then I can debug into what might be failing on the new code. # Allow the access that libvirt does not yet allow $ echo '"/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2" rwk,' > /etc/apparmor.d/local/abstractions/libvirt-qemu # prep the disk chain to be hot-added qemu-img create -f qcow2 /var/lib/libvirt/images/testdisk-snap-base.qcow2 100M qemu-img create -f qcow2 -b /var/lib/libvirt/images/testdisk-snap-base.qcow2 /var/lib/libvirt/images/testdisk-snap-snapshot.qcow2 cat > hot-add-test.xml << EOF EOF # prep a guest to attach to uvt-simplestreams-libvirt --verbose sync --source http://cloud-images.ubuntu.com/daily arch=amd64 label=daily release=focal uvt-kvm create --host-passthrough --password=ubuntu f-test2 release=focal arch=amd64 label=daily We have checked the "effective" profile that is loaded. JJohanssen didn't see an obvious flaw in it. But for reference here: root@h-libvirt-orig:~# apparmor_parser -p /etc/apparmor.d/libvirt/libvirt-e033b910-be06-4975-826f-b6fba368d928 | pastebinit https://paste.ubuntu.com/p/yxdmMT6638/ # attach the disk virsh attach-device f-test2 hot-add-test.xml Libvirt/Qemu works as usual - except (known and expected) does not add a rule for "/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2". But for that we have added out local override. Yet it is denied access. apparmor="DENIED" operation="open" namespace="root//lxd-h-libvirt-new_ " profile="libvirt-a2caaf89-0682-464d-92ba- 5295cb5f5128" name="/var/lib/libvirt/images/testdisk-snap- snapshot.qcow2" pid=2889044 comm="qemu-system-x86" requested_mask="r" denied_mask="r" fsuid=64055 ouid=64055 ## 3. The rule is generally working To be clear, on the same system if I just open up a new profile and allow this path to be accessed it works fine. See the working example below. It must be somewhat that is tied to the way KVM guests get their profile updates/assigned. root@h-libvirt-orig:~# cat /etc/apparmor.d/test #include profile test flags=(attach_disconnected) { #include "/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2" rwk, "/usr/bin/md5sum" rmix, } root@h-libvirt-orig:~# apparmor_parser -r /etc/apparmor.d/test; aa-exec -v -p test -- md5sum /var/lib/libvirt/images/testdisk-snap-snapshot.qcow2 [6939] aa_change_onexec("test") [6939] exec md5sum /var/lib/libvirt/images/testdisk-snap-snapshot.qcow2 0f7fc62d270b69ee1b51453fa614d3e4 /var/lib/libvirt/images/testdisk-snap-snapshot.qcow2 root@h-libvirt-orig:~# apparmor_parser -r /etc/apparmor.d/test; aa-exec -v -p test -- md5sum /var/lib/libvirt/images/testdisk-snap-base.qcow2 [6944] aa_change_onexec("test") [6944] exec md5sum /var/lib/libvirt/images/testdisk-snap-base.qcow2 md5sum: /var/lib/libvirt/images/testdisk-snap-base.qcow2: Permission denied ## 4. tracking apparmor_parser I was using (for debug) a wrapper for apparmro_parser but all really looks as expected $ cat /usr/sbin/apparmor_parser.wrap #!/bin/bash echo "ARGS $@" | /usr/bin/systemd-cat echo "Content of ${2}:" | /usr/bin/systemd-cat /usr/bin/ls -laF "${2}" | /usr/bin/systemd-cat /usr/bin/cat "${2}" | /usr/bin/systemd-cat
[Touch-packages] [Bug 1912214] Re: qemu can't access files that are added as rules on hot-add
On bare metal this (the simplified case that I described above with the rule in the local include file) seems to work just fine. I'll later (sorry sprint week) or tomorrow - re-setup the very same on a container again. ** Description changed: Hi, to be clear I consider it quite likely that the error is on my side, but I'd appreciate guidance how to continue resolve. I have rubber ducked my setup with a coworker and talked to jjohansen if the profile would contain an obvious fault (it did not). So hereby I'm filing a bug hoping to get some help on this case. ## 1. What happened I was trying to add a feature to libvirt to support chained qcow files. That means instead of adding just one path libvirt has to process the chain of backing files and add all of those. That works already on guest start, but not on hot-add of devices. But while I see my code appending the rules I'd expect and issuing the apparmor_parser calls I'd expect it still fails. I'd love to get the profile at runtime to ensure things really are loaded as expected. But as discussed on IRC that won't work. So I tested and simplified and wonder for the test below why things fail. - ## 2. how to reproduce I was taking my new code out of the equation - and to do so I was adding the path that is needed to the local overrides. But it still is blocked. So - right now - I'm assuming that my code worked in adding the lines, but the access is still blocked. Therefore let us try to fix this one first and only then I can debug into what might be failing on the new code. - # Allow the access that libvirt does not yet allow $ echo '"/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2" rwk,' > /etc/apparmor.d/local/abstractions/libvirt-qemu # prep the disk chain to be hot-added qemu-img create -f qcow2 /var/lib/libvirt/images/testdisk-snap-base.qcow2 100M qemu-img create -f qcow2 -b /var/lib/libvirt/images/testdisk-snap-base.qcow2 /var/lib/libvirt/images/testdisk-snap-snapshot.qcow2 cat > hot-add-test.xml << EOF - - - - - - - - + + + + + + + + EOF # prep a guest to attach to uvt-simplestreams-libvirt --verbose sync --source http://cloud-images.ubuntu.com/daily arch=amd64 label=daily release=focal uvt-kvm create --host-passthrough --password=ubuntu f-test2 release=focal arch=amd64 label=daily We have checked the "effective" profile that is loaded. JJohanssen didn't see an obvious flaw in it. But for reference here: - root@h-libvirt-orig:~# apparmor_parser -p /etc/apparmor.d/libvirt/libvirt-e033b910-be06-4975-826f-b6fba368d928 | pastebinit + root@h-libvirt-orig:~# apparmor_parser -p /etc/apparmor.d/libvirt/libvirt-e033b910-be06-4975-826f-b6fba368d928 | pastebinit https://paste.ubuntu.com/p/yxdmMT6638/ # attach the disk - virsh attach-device f-test2 hot-add-test.xm - + virsh attach-device f-test2 hot-add-test.xml Libvirt/Qemu works as usual - except (known and expected) does not add a rule for "/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2". But for that we have added out local override. Yet it is denied access. apparmor="DENIED" operation="open" namespace="root//lxd-h-libvirt-new_ " profile="libvirt-a2caaf89-0682-464d-92ba- 5295cb5f5128" name="/var/lib/libvirt/images/testdisk-snap- snapshot.qcow2" pid=2889044 comm="qemu-system-x86" requested_mask="r" denied_mask="r" fsuid=64055 ouid=64055 - ## 3. The rule is generally working To be clear, on the same system if I just open up a new profile and allow this path to be accessed it works fine. See the working example below. It must be somewhat that is tied to the way KVM guests get their profile updates/assigned. - root@h-libvirt-orig:~# cat /etc/apparmor.d/test #include profile test flags=(attach_disconnected) { - #include + #include - "/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2" rwk, - "/usr/bin/md5sum" rmix, + "/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2" rwk, + "/usr/bin/md5sum" rmix, } root@h-libvirt-orig:~# apparmor_parser -r /etc/apparmor.d/test; aa-exec -v -p test -- md5sum /var/lib/libvirt/images/testdisk-snap-snapshot.qcow2 [6939] aa_change_onexec("test") [6939] exec md5sum /var/lib/libvirt/images/testdisk-snap-snapshot.qcow2 0f7fc62d270b69ee1b51453fa614d3e4 /var/lib/libvirt/images/testdisk-snap-snapshot.qcow2 - root@h-libvirt-orig:~# apparmor_parser -r /etc/apparmor.d/test; aa-exec -v -p test -- md5sum /var/lib/libvirt/images/testdisk-snap-base.qcow2 + root@h-libvirt-orig:~# apparmor_parser -r /etc/apparmor.d/test; aa-exec -v -p test -- md5sum /var/lib/libvirt/images/testdisk-snap-base.qcow2 [6944] aa_change_onexec("test") [6944] exec md5sum /var/lib/libvirt/images/testdisk-snap-base.qcow2 md5sum:
[Touch-packages] [Bug 1912214] Re: qemu can't access files that are added as rules on hot-add
Note/TODO (to myself): I have run and failed with that in containers, try it on bare metal if it is any different. -- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to apparmor in Ubuntu. https://bugs.launchpad.net/bugs/1912214 Title: qemu can't access files that are added as rules on hot-add Status in apparmor package in Ubuntu: New Bug description: Hi, to be clear I consider it quite likely that the error is on my side, but I'd appreciate guidance how to continue resolve. I have rubber ducked my setup with a coworker and talked to jjohansen if the profile would contain an obvious fault (it did not). So hereby I'm filing a bug hoping to get some help on this case. ## 1. What happened I was trying to add a feature to libvirt to support chained qcow files. That means instead of adding just one path libvirt has to process the chain of backing files and add all of those. That works already on guest start, but not on hot-add of devices. But while I see my code appending the rules I'd expect and issuing the apparmor_parser calls I'd expect it still fails. I'd love to get the profile at runtime to ensure things really are loaded as expected. But as discussed on IRC that won't work. So I tested and simplified and wonder for the test below why things fail. ## 2. how to reproduce I was taking my new code out of the equation - and to do so I was adding the path that is needed to the local overrides. But it still is blocked. So - right now - I'm assuming that my code worked in adding the lines, but the access is still blocked. Therefore let us try to fix this one first and only then I can debug into what might be failing on the new code. # Allow the access that libvirt does not yet allow $ echo '"/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2" rwk,' > /etc/apparmor.d/local/abstractions/libvirt-qemu # prep the disk chain to be hot-added qemu-img create -f qcow2 /var/lib/libvirt/images/testdisk-snap-base.qcow2 100M qemu-img create -f qcow2 -b /var/lib/libvirt/images/testdisk-snap-base.qcow2 /var/lib/libvirt/images/testdisk-snap-snapshot.qcow2 cat > hot-add-test.xml << EOF EOF # prep a guest to attach to uvt-simplestreams-libvirt --verbose sync --source http://cloud-images.ubuntu.com/daily arch=amd64 label=daily release=focal uvt-kvm create --host-passthrough --password=ubuntu f-test2 release=focal arch=amd64 label=daily We have checked the "effective" profile that is loaded. JJohanssen didn't see an obvious flaw in it. But for reference here: root@h-libvirt-orig:~# apparmor_parser -p /etc/apparmor.d/libvirt/libvirt-e033b910-be06-4975-826f-b6fba368d928 | pastebinit https://paste.ubuntu.com/p/yxdmMT6638/ # attach the disk virsh attach-device f-test2 hot-add-test.xm Libvirt/Qemu works as usual - except (known and expected) does not add a rule for "/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2". But for that we have added out local override. Yet it is denied access. apparmor="DENIED" operation="open" namespace="root//lxd-h-libvirt-new_ " profile="libvirt-a2caaf89-0682-464d-92ba- 5295cb5f5128" name="/var/lib/libvirt/images/testdisk-snap- snapshot.qcow2" pid=2889044 comm="qemu-system-x86" requested_mask="r" denied_mask="r" fsuid=64055 ouid=64055 ## 3. The rule is generally working To be clear, on the same system if I just open up a new profile and allow this path to be accessed it works fine. See the working example below. It must be somewhat that is tied to the way KVM guests get their profile updates/assigned. root@h-libvirt-orig:~# cat /etc/apparmor.d/test #include profile test flags=(attach_disconnected) { #include "/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2" rwk, "/usr/bin/md5sum" rmix, } root@h-libvirt-orig:~# apparmor_parser -r /etc/apparmor.d/test; aa-exec -v -p test -- md5sum /var/lib/libvirt/images/testdisk-snap-snapshot.qcow2 [6939] aa_change_onexec("test") [6939] exec md5sum /var/lib/libvirt/images/testdisk-snap-snapshot.qcow2 0f7fc62d270b69ee1b51453fa614d3e4 /var/lib/libvirt/images/testdisk-snap-snapshot.qcow2 root@h-libvirt-orig:~# apparmor_parser -r /etc/apparmor.d/test; aa-exec -v -p test -- md5sum /var/lib/libvirt/images/testdisk-snap-base.qcow2 [6944] aa_change_onexec("test") [6944] exec md5sum /var/lib/libvirt/images/testdisk-snap-base.qcow2 md5sum: /var/lib/libvirt/images/testdisk-snap-base.qcow2: Permission denied ## 4. tracking apparmor_parser I was using (for debug) a wrapper for apparmro_parser but all really looks as expected $ cat /usr/sbin/apparmor_parser.wrap #!/bin/bash echo "ARGS $@" | /usr/bin/systemd-cat echo "Content of ${2}:" | /usr/bin/systemd-cat /usr/bin/ls -laF "${2}" | /usr/bin/systemd-cat /usr/bin/cat