** Description changed:

  Hi,
  to be clear I consider it quite likely that the error is on my side, but I'd 
appreciate guidance how to continue resolve. I have rubber ducked my setup with 
a coworker and talked to jjohansen if the profile would contain an obvious 
fault (it did not).
  
  So hereby I'm filing a bug hoping to get some help on this case.
  
  ## 1. What happened
  
  I was trying to add a feature to libvirt to support chained qcow files.
  That means instead of adding just one path libvirt has to process the chain 
of backing files and add all of those. That works already on guest start, but 
not on hot-add of devices.
  
  But while I see my code appending the rules I'd expect and issuing the
  apparmor_parser calls I'd expect it still fails.
  
  I'd love to get the profile at runtime to ensure things really are loaded as 
expected.
  But as discussed on IRC that won't work. So I tested and simplified and 
wonder for the test below why things fail.
  
  ## 2. how to reproduce
  
  I was debugging various cases (see below updates) and found that just one
- combination fails (also see comment #5)
+ combination fails (also see comment #5).
  
  I'm rather sure now that libvirt adds the path correctly, but even
  adding the denied path that is needed to the local overrides does not
  help.
  
  Use a Hirsute Container and enable this PPA:
  https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/4393
  
- Then - to underline that the rule should be ok also add it manually as local 
override.
+ 
+ # First get a KVM capable container (the issue only triggers in containers)
+ $ lxc profile create kvm
+ $ cat > kvm_profile.yaml << EOF
+ config:
+   boot.autostart: "true"
+   linux.kernel_modules: openvswitch,nbd,ip_tables,ip6_tables,kvm
+   security.nesting: "true"
+   security.privileged: "true"
+ description: ""
+ devices:
+   eth0:
+     mtu: "9000"
+     name: eth0
+     nictype: bridged
+     parent: lxdbr0
+     type: nic
+   kvm:
+     path: /dev/kvm
+     type: unix-char
+   mem:
+     path: /dev/mem
+     type: unix-char
+   tun:
+     path: /dev/net/tun
+     type: unix-char
+ name: kvm
+ EOF
+ $ lxc profile edit kvm < kvm_profile.yaml
+ $ lxc launch ubuntu-daily:h h-libvirt-new --profile default --profile kvm
+ 
+ 
+ All following steps are in the container
+ $ sudo add-apt-repository ppa:ci-train-ppa-service/4393
+ $ apt update
+ $ apt upgrade -y
+ $ apt install uvtool-libvirt
+ 
+ 
+ Then - to underline that the rules should be ok - also add it manually as 
local override (therefore we don't need to theorize or check if the rule was 
indeed added by libvirt - but since it works on bare metal we know it was 
added).
  # Allow the access that libvirt does not yet allow
  $ echo '"/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2" rwk,' > 
/etc/apparmor.d/local/abstractions/libvirt-qemu
- $ echo '"/var/lib/libvirt/images/testdisk-snap-base.qcow2" rwk,' >> 
/etc/apparmor.d/local/abstractions/libvirt-qem
+ $ echo '"/var/lib/libvirt/images/testdisk-snap-base.qcow2" rwk,' >> 
/etc/apparmor.d/local/abstractions/libvirt-qemu
+ 
  
  # Check the profile can be loaded and contains these
  $ apparmor_parser -r 
/etc/apparmor.d/libvirt/libvirt-85410987-d91b-487a-8f2c-911f0136c877
  $ apparmor_parser -p 
/etc/apparmor.d/libvirt/libvirt-85410987-d91b-487a-8f2c-911f0136c877 | grep snap
  "/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2" rwk,
  "/var/lib/libvirt/images/testdisk-snap-base.qcow2" rwk,
  
  
  # prep the disk chain to be hot-added
  qemu-img create -f qcow2 /var/lib/libvirt/images/testdisk-snap-base.qcow2 100M
  qemu-img create -f qcow2 -b /var/lib/libvirt/images/testdisk-snap-base.qcow2 
/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2
  cat > hot-add-test.xml << EOF
  <disk type='file' device='disk'>
        <driver name='qemu' type='qcow2'/>
        <source file='/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2' 
index='1'/>
        <backingStore type='file' index='2'>
          <format type='qcow2'/>
          <source file='/var/lib/libvirt/images/testdisk-snap-base.qcow2'/>
          <backingStore/>
        </backingStore>
        <target dev='vdc' bus='virtio'/>
  </disk>
  EOF
  
  # prep a guest to attach the disk to
  uvt-simplestreams-libvirt --verbose sync --source 
http://cloud-images.ubuntu.com/daily arch=amd64 label=daily release=focal
  uvt-kvm create --host-passthrough --password=ubuntu f-test2 release=focal 
arch=amd64 label=daily
  
- We have checked the "effective" profile that is loaded. JJohanssen didn't see 
an obvious flaw in it. But for reference here:
+ 
+ We have together checked the "effective" profile that is loaded. JJohanssen 
didn't see an obvious flaw in it. But for reference here:
  root@h-libvirt-orig:~# apparmor_parser -p 
/etc/apparmor.d/libvirt/libvirt-e033b910-be06-4975-826f-b6fba368d928 | 
pastebinit
  https://paste.ubuntu.com/p/yxdmMT6638/
  
- # attach the disk
+ 
+ # attach the disk to trigger the issue
  virsh attach-device f-test2 hot-add-test.xml
  
- Libvirt (as usual) now adds rules for the image file.
+ Libvirt (as usual) now adds rules for the image file (seen in dmesg).
  With the new code from the PPA it does so twice - once for the snapshot and 
once the base file.
  But in any case we allow that access through our override.
- You can see that on a little gimmick, since the flattened profile always has 
those rules - and when libvirt is adding them we will sometimes now see 
info="same as current profile, skipping".
+ You can see that at a little gimmick, since the flattened profile always has 
those rules - and when libvirt is adding them we will sometimes now see 
info="same as current profile, skipping".
  If the local override isn't in place that does not show up.
  So the rules libvirt adds are identical to what we have put into the local 
override and that should allow the access.
  
  But we get:
  
  [699826.573035] audit: type=1400 audit(1611044559.393:2047):
  apparmor="DENIED" operation="open" namespace="root//lxd-h-libvirt-new2_
  <var-snap-lxd-common-lxd>" profile="libvirt-85410987-d91b-487a-8f2c-
  911f0136c877" name="/var/lib/libvirt/images/testdisk-snap-base.qcow2"
  pid=111701 comm="qemu-system-x86" requested_mask="r" denied_mask="r"
  fsuid=64055 ouid=64055
  
- 
  ## 3. tracking apparmor_parser
  
- I was using (for debug) a wrapper for apparmro_parser but all really
+ I was using (for debug) a wrapper for apparmor_parser but all really
  looks as expected
  
  $ cat /usr/sbin/apparmor_parser.wrap
  #!/bin/bash
  echo "ARGS $@" | /usr/bin/systemd-cat
  echo "Content of ${2}:" | /usr/bin/systemd-cat
  /usr/bin/ls -laF "${2}" | /usr/bin/systemd-cat
  /usr/bin/cat "${2}" | /usr/bin/systemd-cat
  echo "Content of ${2}.files:" | /usr/bin/systemd-cat
  /usr/bin/ls -laF "${2}.files" | /usr/bin/systemd-cat
  /usr/bin/cat "${2}.files" | /usr/bin/systemd-cat
  /usr/sbin/apparmor_parser.orig "$@" | /usr/bin/systemd-cat
  
- It is called with "-r filepath" and the content of that filepath contain
- the aforementioned rules. So what else might be wrong - why else could
- the access be denied despite the rule that should allow it.
+ It is called with "-r filepath" and the content of that filepath contain the 
aforementioned rules. So what else might be wrong? Why else could the access be 
denied despite the rule that should allow it being present?
+ Also do mind that it blocks only the secondary access - which means it has 
read the snapshot file and determined that it also needs to open the base file
  
  
- ## "special combination" 
+ ## "special combination"
  
- As explained in later comments it works on bare metal, it works with the old 
libvirt code, it works if attaching one disk (no two calls to apparmor_parser 
from libvirt), it works if I do the same manually.
+ As explained in later comments it works on bare metal, it works with the old 
libvirt code, it works if attaching just one disk (no two calls to 
apparmor_parser from libvirt), it works if I do the same manually.
  But still the bad case fails - puzzling - please help.

-- 
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to apparmor in Ubuntu.
https://bugs.launchpad.net/bugs/1912214

Title:
  qemu can't access files - even with manual overrides (only an issue in
  Containers)

Status in apparmor package in Ubuntu:
  New

Bug description:
  Hi,
  to be clear I consider it quite likely that the error is on my side, but I'd 
appreciate guidance how to continue resolve. I have rubber ducked my setup with 
a coworker and talked to jjohansen if the profile would contain an obvious 
fault (it did not).

  So hereby I'm filing a bug hoping to get some help on this case.

  ## 1. What happened

  I was trying to add a feature to libvirt to support chained qcow files.
  That means instead of adding just one path libvirt has to process the chain 
of backing files and add all of those. That works already on guest start, but 
not on hot-add of devices.

  But while I see my code appending the rules I'd expect and issuing the
  apparmor_parser calls I'd expect it still fails.

  I'd love to get the profile at runtime to ensure things really are loaded as 
expected.
  But as discussed on IRC that won't work. So I tested and simplified and 
wonder for the test below why things fail.

  ## 2. how to reproduce

  I was debugging various cases (see below updates) and found that just one
  combination fails (also see comment #5).

  I'm rather sure now that libvirt adds the path correctly, but even
  adding the denied path that is needed to the local overrides does not
  help.

  Use a Hirsute Container and enable this PPA:
  https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/4393

  
  # First get a KVM capable container (the issue only triggers in containers)
  $ lxc profile create kvm
  $ cat > kvm_profile.yaml << EOF
  config:
    boot.autostart: "true"
    linux.kernel_modules: openvswitch,nbd,ip_tables,ip6_tables,kvm
    security.nesting: "true"
    security.privileged: "true"
  description: ""
  devices:
    eth0:
      mtu: "9000"
      name: eth0
      nictype: bridged
      parent: lxdbr0
      type: nic
    kvm:
      path: /dev/kvm
      type: unix-char
    mem:
      path: /dev/mem
      type: unix-char
    tun:
      path: /dev/net/tun
      type: unix-char
  name: kvm
  EOF
  $ lxc profile edit kvm < kvm_profile.yaml
  $ lxc launch ubuntu-daily:h h-libvirt-new --profile default --profile kvm

  
  All following steps are in the container
  $ sudo add-apt-repository ppa:ci-train-ppa-service/4393
  $ apt update
  $ apt upgrade -y
  $ apt install uvtool-libvirt

  
  Then - to underline that the rules should be ok - also add it manually as 
local override (therefore we don't need to theorize or check if the rule was 
indeed added by libvirt - but since it works on bare metal we know it was 
added).
  # Allow the access that libvirt does not yet allow
  $ echo '"/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2" rwk,' > 
/etc/apparmor.d/local/abstractions/libvirt-qemu
  $ echo '"/var/lib/libvirt/images/testdisk-snap-base.qcow2" rwk,' >> 
/etc/apparmor.d/local/abstractions/libvirt-qemu

  
  # Check the profile can be loaded and contains these
  $ apparmor_parser -r 
/etc/apparmor.d/libvirt/libvirt-85410987-d91b-487a-8f2c-911f0136c877
  $ apparmor_parser -p 
/etc/apparmor.d/libvirt/libvirt-85410987-d91b-487a-8f2c-911f0136c877 | grep snap
  "/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2" rwk,
  "/var/lib/libvirt/images/testdisk-snap-base.qcow2" rwk,

  
  # prep the disk chain to be hot-added
  qemu-img create -f qcow2 /var/lib/libvirt/images/testdisk-snap-base.qcow2 100M
  qemu-img create -f qcow2 -b /var/lib/libvirt/images/testdisk-snap-base.qcow2 
/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2
  cat > hot-add-test.xml << EOF
  <disk type='file' device='disk'>
        <driver name='qemu' type='qcow2'/>
        <source file='/var/lib/libvirt/images/testdisk-snap-snapshot.qcow2' 
index='1'/>
        <backingStore type='file' index='2'>
          <format type='qcow2'/>
          <source file='/var/lib/libvirt/images/testdisk-snap-base.qcow2'/>
          <backingStore/>
        </backingStore>
        <target dev='vdc' bus='virtio'/>
  </disk>
  EOF

  # prep a guest to attach the disk to
  uvt-simplestreams-libvirt --verbose sync --source 
http://cloud-images.ubuntu.com/daily arch=amd64 label=daily release=focal
  uvt-kvm create --host-passthrough --password=ubuntu f-test2 release=focal 
arch=amd64 label=daily

  
  We have together checked the "effective" profile that is loaded. JJohanssen 
didn't see an obvious flaw in it. But for reference here:
  root@h-libvirt-orig:~# apparmor_parser -p 
/etc/apparmor.d/libvirt/libvirt-e033b910-be06-4975-826f-b6fba368d928 | 
pastebinit
  https://paste.ubuntu.com/p/yxdmMT6638/

  
  # attach the disk to trigger the issue
  virsh attach-device f-test2 hot-add-test.xml

  Libvirt (as usual) now adds rules for the image file (seen in dmesg).
  With the new code from the PPA it does so twice - once for the snapshot and 
once the base file.
  But in any case we allow that access through our override.
  You can see that at a little gimmick, since the flattened profile always has 
those rules - and when libvirt is adding them we will sometimes now see 
info="same as current profile, skipping".
  If the local override isn't in place that does not show up.
  So the rules libvirt adds are identical to what we have put into the local 
override and that should allow the access.

  But we get:

  [699826.573035] audit: type=1400 audit(1611044559.393:2047):
  apparmor="DENIED" operation="open" namespace="root//lxd-h-libvirt-
  new2_<var-snap-lxd-common-lxd>" profile="libvirt-85410987-d91b-487a-
  8f2c-911f0136c877" name="/var/lib/libvirt/images/testdisk-snap-
  base.qcow2" pid=111701 comm="qemu-system-x86" requested_mask="r"
  denied_mask="r" fsuid=64055 ouid=64055

  ## 3. tracking apparmor_parser

  I was using (for debug) a wrapper for apparmor_parser but all really
  looks as expected

  $ cat /usr/sbin/apparmor_parser.wrap
  #!/bin/bash
  echo "ARGS $@" | /usr/bin/systemd-cat
  echo "Content of ${2}:" | /usr/bin/systemd-cat
  /usr/bin/ls -laF "${2}" | /usr/bin/systemd-cat
  /usr/bin/cat "${2}" | /usr/bin/systemd-cat
  echo "Content of ${2}.files:" | /usr/bin/systemd-cat
  /usr/bin/ls -laF "${2}.files" | /usr/bin/systemd-cat
  /usr/bin/cat "${2}.files" | /usr/bin/systemd-cat
  /usr/sbin/apparmor_parser.orig "$@" | /usr/bin/systemd-cat

  It is called with "-r filepath" and the content of that filepath contain the 
aforementioned rules. So what else might be wrong? Why else could the access be 
denied despite the rule that should allow it being present?
  Also do mind that it blocks only the secondary access - which means it has 
read the snapshot file and determined that it also needs to open the base file

  
  ## "special combination"

  As explained in later comments it works on bare metal, it works with the old 
libvirt code, it works if attaching just one disk (no two calls to 
apparmor_parser from libvirt), it works if I do the same manually.
  But still the bad case fails - puzzling - please help.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/apparmor/+bug/1912214/+subscriptions

-- 
Mailing list: https://launchpad.net/~touch-packages
Post to     : touch-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~touch-packages
More help   : https://help.launchpad.net/ListHelp

Reply via email to