[Bug 1525310] Re: virsh with apparmor misconfigures libvirt-UUID files during snapshot
Hi, I was asked about a very similar case and needed to debug it. So I thought I give the issue reported here a try how it looks like today. virt install creates a guest with a command like: -drive file=/dev/LVMpool/test-snapshot-virtinst,format=raw,if=none,id=drive-virtio-disk0,cache=none,aio=native ## Solved - confusion of pre-existing other apparmor rules as pools are unsupported by libvirt/apparmor ## In this case virt-inst pre-creates a VG of a given size and passes the guest just that. This is the different to using the actual pool feature. With that I'm "ok" that it doesn't need a special apparmor rule already. >From the guest/apparmor point of view when the guest starts the path is known >and added to the guests profile. (With a pool ref in the guest that would not have worked) ## experiments - setup ## Lets define a guest which has a qcow and a lvm disk that we can snapshot for experiments. We will use the disk created in the test above, but in a uvtool guest to get rid of all virt-install special quirks. The other disk is just a qcow file. $ sudo qemu-img create -f qcow2 /var/lib/uvtool/libvirt/images/eoan-snapshot-test.qcow 1G Formatting '/var/lib/uvtool/libvirt/images/eoan-snapshot-test.qcow', fmt=qcow2 size=1073741824 cluster_size=65536 lazy_refcounts=off refcount_bits=16 The config for those looks like: qcow: CMD: -drive file=/var/lib/uvtool/libvirt/images/eoan-snapshot-test.qcow,format=qcow2,if=none,id=drive-virtio-disk2 apparmor: "/var/lib/uvtool/libvirt/images/eoan-snapshot-test.qcow" rwk, disk: CMD: -drive file=/dev/LVMpool/test-snapshot-virtinst,format=raw,if=none,id=drive-virtio-disk3 apparmor: "/dev/dm-11" rwk, which is a match as $ ll /dev/LVMpool/test-snapshot-virtinst lrwxrwxrwx 1 root root 8 Sep 11 05:14 /dev/LVMpool/test-snapshot-virtinst -> ../dm-11 ## experiments - snapshotting ## Details of the spec see: https://libvirt.org/formatdomain.html Snapshot of just the qcow file: $ virsh snapshot-create-as --print-xml --domain eoan-snapshot --disk-only --atomic --diskspec vda,snapshot=no --diskspec vdb,snapshot=no --diskspec vdc,file=/var/lib/libvirt/images/eoan-snapshot-test.thesnapshot.qcow,snapshot=external --diskspec vdd,snapshot=no $ virsh snapshot-list eoan-snapshot Name Creation Time State 1568196836 2019-09-11 06:13:56 -0400 disk-snapshot The snapshot got added to the apparmor profile: "/var/lib/libvirt/images/eoan-snapshot-test.thesnapshot.qcow" rwk, The position shows that this was done with the "append" feature of virt-aa-helper. So it did not re-parse the guest but just add one more entry (as it would do on hotplug). I'm not trying to LVM-snapshot as that seems not what was asked for. And further LVM would have own capabilties to do so. ## check status after snapshot ## The guest now has the new snapshot as main file and the old one as backing file (COW-chain) Please do mind that this is the "runtime view", once shut down you'll only see the new snapshot. This is confirmed by the metadata in the qcow file. $ sudo qemu-img info /var/lib/libvirt/images/eoan-snapshot-test.thesnapshot.qcow image: /var/lib/libvirt/images/eoan-snapshot-test.thesnapshot.qcow file format: qcow2 virtual size: 1.0G (1073741824 bytes) disk size: 196K cluster_size: 65536 backing file: /var/lib/uvtool/libvirt/images/eoan-snapshot-test.qcow backing file format: qcow2 Format specific information: compat: 1.1 lazy refcounts: false refcount bits: 16 corrupt: false ## restart guest ## XML of inactive guest as expected: The guest starts just fine (with still all 4 disk entries): ../dm-11 lrwxrwxrwx 1 root root8 Sep 11 06:44 test-snapshot-virtinst-snap -> ../dm-14 Still dm-11 that matches apparmor "/dev/dm-11" rwk, Checking if it is an issue to restart the guest with the LVM snapshot attached. No, shutdown and start work fine still. With that said I think we can close this old bug nowadays. P.S. There is a case a friend of mine reported with qcow snapshots on LVM which sounds odd and is broken. This I'll track down in another place as it has nothing to do with the issue that was reported here. ** Changed in: apparmor (Ubuntu) Status: New => Fix Released -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1525310 Title:
[Bug 1525310] Re: virsh with apparmor misconfigures libvirt-UUID files during snapshot
** Tags added: trusty ** Tags added: wily ** Changed in: apparmor (Ubuntu) Importance: Undecided => Medium -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1525310 Title: virsh with apparmor misconfigures libvirt-UUID files during snapshot To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/apparmor/+bug/1525310/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1525310] Re: virsh with apparmor misconfigures libvirt-UUID files during snapshot
This bug also appears in Ubuntu 15.10 $ apt-cache policy apparmor apparmor: Installed: 2.10-0ubuntu6 Candidate: 2.10-0ubuntu6 Version table: *** 2.10-0ubuntu6 0 500 http://us.archive.ubuntu.com/ubuntu/ wily/main amd64 Packages 100 /var/lib/dpkg/status $ apt-cache policy libvirt-bin libvirt-bin: Installed: 1.2.16-2ubuntu11.15.10.1 Candidate: 1.2.16-2ubuntu11.15.10.1 Version table: *** 1.2.16-2ubuntu11.15.10.1 0 500 http://us.archive.ubuntu.com/ubuntu/ wily-updates/main amd64 Packages 100 /var/lib/dpkg/status 1.2.16-2ubuntu11 0 500 http://us.archive.ubuntu.com/ubuntu/ wily/main amd64 Packages -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1525310 Title: virsh with apparmor misconfigures libvirt-UUID files during snapshot To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/apparmor/+bug/1525310/+subscriptions -- ubuntu-bugs mailing list ubuntu-bugs@lists.ubuntu.com https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
[Bug 1525310] Re: virsh with apparmor misconfigures libvirt-UUID files during snapshot
** Description changed:
Reproducible: Yes, every time.
Background:
When you create a virtual machine (VM) under KVM/Qemu in Ubuntu,
apparmor files are created as:
/etc/apparmor.d/libvirt/libvirt-
- and
+ and
/etc/apparmor.d/libvirt/libvirt-.files
And in the file /etc/apparmor.d/libvirt/libvirt-.files there is
the line
- "PATH_to_BLOCK_DEVICE" rw,
+ "PATH_to_BLOCK_DEVICE" rw,
- where PATH_to_BLOCK_DEVICE is the full path name of the image. ( E.g.
something like /var/lib/libvirtd/images/asdf.qcow2)
- and is the UUID of the VM container.
+ where PATH_to_BLOCK_DEVICE is the full path name of the image. ( E.g.
something like /var/lib/libvirtd/images/asdf.qcow2)
+ and is the UUID of the VM container.
The problem:
When creating a shapshot of a running VM under KVM/Qemu you run the
command
$ sudo virsh snapshot-create-as DOMAIN_NAME DESCRIPTION --no-
metadata --disk-only --atomic
which creates a new file and stops writing to the old VM block device.
However: the old PATH_to_BLOCK_DEVICE in /etc/apparmor.d/libvirt
/libvirt-UUID.files is deleted and replaced with the new block device
info BEFORE virsh is done creating the snapshot. So you get the error
error: internal error: unable to execute QEMU command 'transaction':
Could not open 'PATH_to_BLOCK_DEVICE': Could not open
'PATH_to_BLOCK_DEVICE': Permission denied: Permission denied
and in /var/log/syslog you get the error:
type=1400 audit(1449752104.054:539): apparmor="DENIED" operation="open"
profile="libvirt-" name="PATH_to_BLOCK_DEVICE" pid=8710 comm
="qemu-system-x86" requested_mask="r" denied_mask="r" fsuid=106 ouid=106
+ When you look now at /etc/apparmor.d/libvirt/libvirt-.files you
+ find that the line that was there
- When you look now at /etc/apparmor.d/libvirt/libvirt-.files you find
that the line that was there
+ "PATH_to_BLOCK_DEVICE" rw,
- "PATH_to_BLOCK_DEVICE" rw,
+ has been replaced with
+ "PATH_to_BLOCK_DEVICE.DESCRIPTION" rw,
- has been replaced with
-
- "PATH_to_BLOCK_DEVICE.DESCRIPTION" rw,
-
-
- but you need BOTH LINES. in order for the command "virsh snapshot-create-as"
to work. (or at least have the old file have read permissions)
+ but you need BOTH LINES. in order for the command "virsh snapshot-
+ create-as" to work. (or at least have the old file have read
+ permissions)
-
Workarounds:
1. Disable apparmor for libvirtd
- or
+ or
2. Change /etc/apparmor.d/libvirt/libvirt- to look like this
+
--
#
# This profile is for the domain whose UUID matches this file.
- #
-
+ #
+
#include
-
+
profile libvirt-UUID {
- #include
- #include
-
- "PATH_to_BLOCK_DEVICE*" rw,
- }
+ #include
+ #include
+
+ "PATH_to_BLOCK_DEVICE*" rw,
+ }
---
+ (
+ So if the old line was
+ "/var/lib/libvirtd/images/asdf.qcow2" rw,
+ , the line you can add would read something like this
+
+ "/var/lib/libvirtd/images/asdf*" rw,
+
+ )
+
Details on server:
# lsb_release -rd
Description:Ubuntu 14.04.3 LTS
Release:14.04
-
# apt-cache policy apparmor
apparmor:
- Installed: 2.8.95~2430-0ubuntu5.3
- Candidate: 2.8.95~2430-0ubuntu5.3
- Version table:
- *** 2.8.95~2430-0ubuntu5.3 0
- 500 http://us.archive.ubuntu.com/ubuntu/ trusty-updates/main amd64
Packages
- 100 /var/lib/dpkg/status
- 2.8.95~2430-0ubuntu5.1 0
- 500 http://security.ubuntu.com/ubuntu/ trusty-security/main amd64
Packages
- 2.8.95~2430-0ubuntu5 0
- 500 http://us.archive.ubuntu.com/ubuntu/ trusty/main amd64 Packages
+ Installed: 2.8.95~2430-0ubuntu5.3
+ Candidate: 2.8.95~2430-0ubuntu5.3
+ Version table:
+ *** 2.8.95~2430-0ubuntu5.3 0
+ 500 http://us.archive.ubuntu.com/ubuntu/ trusty-updates/main amd64
Packages
+ 100 /var/lib/dpkg/status
+ 2.8.95~2430-0ubuntu5.1 0
+ 500 http://security.ubuntu.com/ubuntu/ trusty-security/main amd64
Packages
+ 2.8.95~2430-0ubuntu5 0
+ 500 http://us.archive.ubuntu.com/ubuntu/ trusty/main amd64 Packages
# apt-cache policy libvirt-bin
libvirt-bin:
- Installed: 1.2.2-0ubuntu13.1.14
- Candidate: 1.2.2-0ubuntu13.1.14
- Version table:
- *** 1.2.2-0ubuntu13.1.14 0
- 500 http://us.archive.ubuntu.com/ubuntu/ trusty-updates/main amd64
Packages
- 100 /var/lib/dpkg/status
- 1.2.2-0ubuntu13.1.7 0
- 500 http://security.u buntu.com/ubuntu/ trusty-security/main amd64
Packages
- 1.2.2-0ubuntu13 0
- 500 http://us.archive.ubuntu.com/ubuntu/ trusty/main amd64 Packages
-
+ Installed: 1.2.2-0ubuntu13.1.14
+ Candidate: 1.2.2-0ubuntu13.1.14
+ Version table:
+ *** 1.2.2-0ubuntu13.1.14 0
+ 500 http://us.archive.ubuntu.com/ubuntu/ trusty-updates/main amd64
Packages
+ 100 /var/lib/dpkg/status
+ 1.2.2-0ubuntu13.1.7 0
+ 500
