As you explained I acknowledge that we are changing behaviour in Focal
here, but I agree with you that this is the least worst option.

** Description changed:

  [Impact]
  
-  * The defaults of virt-manager for disk allocation always worked fine
-    when qcow2 had nothing but sparse support. So sparse=True vs
-    sparse=False made no difference. So it always set sparse=False.
-    Then qcow2 grows non-sparse support, and virt-manager is suddenly
-    defaulting to it, which is not the intention.
+  * The defaults of virt-manager for disk allocation always worked fine
+    when qcow2 had nothing but sparse support. So sparse=True vs
+    sparse=False made no difference. So it always set sparse=False.
+    Then qcow2 grows non-sparse support, and virt-manager is suddenly
+    defaulting to it, which is not the intention.
  
+  * For upgraders from pre-Focal this was a regression in two ways:
+     a) allocation of qcow changed to non-sparse (fallocate based)
+        potentially consuming more space
+     b) depending on the underlying FS this made the allocation much
+        slower
  
-  * For upgraders from pre-Focal this was a regression in two ways:
-     a) allocation of qcow changed to non-sparse (fallocate based) 
-        potentially consuming more space
-     b) depending on the underlying FS this made the allocation much 
-        slower
- 
-  * Fix by applying the upstream change that Defaults to sparse when 
-    requested format isn't raw (for raw the assumption ill stay that 
-    users do that for runtime performance, so non-sparse on those stays 
-    as-is)
+  * Fix by applying the upstream change that Defaults to sparse when
+    requested format isn't raw (for raw the assumption ill stay that
+    users do that for runtime performance, so non-sparse on those stays
+    as-is)
  
  [Test Case]
  
-  * open virt-manager and create a new guest which includes creating a 
-    new image for it
-    * when clicking "finish" the image will be created (which e.g. on ZFS 
-      will take quite some time)
-    * In the background grep ps output if the qemu-img call for qcow (the 
-      default) is using "preallocation=fallocate" (wrong) or 
-      "preallocation=metadata" (correct)
+  * open virt-manager and create a new guest which includes creating a
+    new image for it
+    * when clicking "finish" the image will be created (which e.g. on ZFS
+      will take quite some time)
+    * In the background grep ps output if the qemu-img call for qcow (the
+      default) is using "preallocation=fallocate" (wrong) or
+      "preallocation=metadata" (correct)
  
  [Regression Potential]
  
-  * For upgraders from pre-focal it actually ensures behavior stays as is 
-    (no change/regression there)
-  * For users of ZFS where the allocation was slow it fixes this 
-    slowness.
-  * But for people using Focal all the time and on non-ZFS there will be 
-    a behavior change in no more doing falloc pre-allocating the qemu 
-    image. To be cleear there are four modes:
-    1. no allocation
-    2. metadata allocation - is initially larger but can improve 
-       performance when the image needs to grow
-    3. falloc - preallocates space for image by calling posix_fallocate
-    4. full - writes data to the underlying storage
-    The fix changes the default from 3->2 which it always was and should 
-    be. As the patch says: "If users want to override it, they can
-    do it via custom created storage."
-    (To be clear, upstream changed this as we fixed it here, so on 
-     further upgrades what we apply here will be the behavior anyway, it 
-     seems wrong to keep Focal the only one in between kept different)
+  * For upgraders from pre-focal it actually ensures behavior stays as is
+    (no change/regression there)
+  * For users of ZFS where the allocation was slow it fixes this
+    slowness.
+  * But for people using Focal all the time and on non-ZFS there will be
+    a behavior change in no more doing falloc pre-allocating the qemu
+    image. To be cleear there are four modes:
+    1. no allocation
+    2. metadata allocation - is initially larger but can improve
+       performance when the image needs to grow
+    3. falloc - preallocates space for image by calling posix_fallocate
+    4. full - writes data to the underlying storage
+    The fix changes the default from 3->2 which it always was and should
+    be. As the patch says: "If users want to override it, they can
+    do it via custom created storage."
+    (To be clear, upstream changed this as we fixed it here, so on
+     further upgrades what we apply here will be the behavior anyway, it
+     seems wrong to keep Focal the only one in between kept different)
+  * [racb] The code path being added special cases the 'raw' format and in 
handling of the default sparse setting. Areas of potential regression are 
therefore in the handling of the default and override of the default in the 
cases of both 'raw' and not 'raw'.
  
  [Other Info]
-  
-  * The slowness effect might be further mitigated by ZFS implementing 
-    more falloc options (thanks Richard for that hint) but that won't be 
-    in Focal.
+ 
+  * The slowness effect might be further mitigated by ZFS implementing
+    more falloc options (thanks Richard for that hint) but that won't be
+    in Focal.
  
  ----
  
- 
- This is a regression in eoan for me. I use virt-manager to create vms, and I 
noticed that creating one now takes more than a minute.
+ This is a regression in eoan for me. I use virt-manager to create vms,
+ and I noticed that creating one now takes more than a minute.
  
  Looking at the process listing while the backing disk is being created, I see 
this qemu-img command line:
  15658 ?        Ssl    0:00 /usr/sbin/libvirtd
  23726 ?        Sl     0:04  \_ /usr/bin/qemu-img create -f qcow2 -o 
preallocation=falloc,compat=1.1,lazy_refcounts 
/var/lib/libvirt/images/live-server.qcow2 41943040K
  
  If I run qemu-img with that preallocation parameter set, even on bionic,
  then it also takes a very long time.
  
  On eoan, for comparison:
  andreas@nsn7:~$ time qemu-img create -f qcow2 no-prealloc-image.qcow2 40G
  Formatting 'no-prealloc-image.qcow2', fmt=qcow2 size=42949672960 
cluster_size=65536 lazy_refcounts=off refcount_bits=16
  
  real  0m0,016s
  user  0m0,010s
  sys   0m0,006s
  andreas@nsn7:~$ qemu-img info no-prealloc-image.qcow2
  image: no-prealloc-image.qcow2
  file format: qcow2
  virtual size: 40G (42949672960 bytes)
  disk size: 17K
  cluster_size: 65536
  Format specific information:
      compat: 1.1
      lazy refcounts: false
      refcount bits: 16
      corrupt: false
  andreas@nsn7:~$ du -hs no-prealloc-image.qcow2
  17K   no-prealloc-image.qcow2
  andreas@nsn7:~$
  
  and now with preallocation=falloc:
  andreas@nsn7:~$ time qemu-img create -f qcow2 -o preallocation=falloc 
with-prealloc-image.qcow2 40G
  Formatting 'with-prealloc-image.qcow2', fmt=qcow2 size=42949672960 
cluster_size=65536 preallocation=falloc lazy_refcounts=off refcount_bits=16
  
  real  1m43,196s
  user  0m3,564s
  sys   1m26,720s
  andreas@nsn7:~$ qemu-img info with-prealloc-image.qcow2
  image: with-prealloc-image.qcow2
  file format: qcow2
  virtual size: 40G (42949672960 bytes)
  disk size: 2.7M
  cluster_size: 65536
  Format specific information:
      compat: 1.1
      lazy refcounts: false
      refcount bits: 16
      corrupt: false
  andreas@nsn7:~$ du -hs with-prealloc-image.qcow2
  2,8M  with-prealloc-image.qcow2
  andreas@nsn7:~$
  
  ProblemType: Bug
  DistroRelease: Ubuntu 19.10
  Package: libvirt-daemon 5.4.0-0ubuntu5
  ProcVersionSignature: Ubuntu 5.3.0-13.14-generic 5.3.0
  Uname: Linux 5.3.0-13-generic x86_64
  NonfreeKernelModules: zfs zunicode zavl icp zcommon znvpair
  ApportVersion: 2.20.11-0ubuntu7
  Architecture: amd64
  Date: Mon Oct  7 11:36:03 2019
  InstallationDate: Installed on 2019-10-07 (0 days ago)
  InstallationMedia: Ubuntu 19.10 "Eoan Ermine" - Beta amd64 (20191006)
  SourcePackage: libvirt
  UpgradeStatus: No upgrade log present (probably fresh install)
  modified.conffile..etc.libvirt.nwfilter.allow-arp.xml: [inaccessible: [Errno 
13] Permission denied: '/etc/libvirt/nwfilter/allow-arp.xml']
  modified.conffile..etc.libvirt.nwfilter.allow-dhcp-server.xml: [inaccessible: 
[Errno 13] Permission denied: '/etc/libvirt/nwfilter/allow-dhcp-server.xml']
  modified.conffile..etc.libvirt.nwfilter.allow-dhcp.xml: [inaccessible: [Errno 
13] Permission denied: '/etc/libvirt/nwfilter/allow-dhcp.xml']
  modified.conffile..etc.libvirt.nwfilter.allow-incoming-ipv4.xml: 
[inaccessible: [Errno 13] Permission denied: 
'/etc/libvirt/nwfilter/allow-incoming-ipv4.xml']
  modified.conffile..etc.libvirt.nwfilter.allow-ipv4.xml: [inaccessible: [Errno 
13] Permission denied: '/etc/libvirt/nwfilter/allow-ipv4.xml']
  modified.conffile..etc.libvirt.nwfilter.clean-traffic-gateway.xml: 
[inaccessible: [Errno 13] Permission denied: 
'/etc/libvirt/nwfilter/clean-traffic-gateway.xml']
  modified.conffile..etc.libvirt.nwfilter.clean-traffic.xml: [inaccessible: 
[Errno 13] Permission denied: '/etc/libvirt/nwfilter/clean-traffic.xml']
  modified.conffile..etc.libvirt.nwfilter.no-arp-ip-spoofing.xml: 
[inaccessible: [Errno 13] Permission denied: 
'/etc/libvirt/nwfilter/no-arp-ip-spoofing.xml']
  modified.conffile..etc.libvirt.nwfilter.no-arp-mac-spoofing.xml: 
[inaccessible: [Errno 13] Permission denied: 
'/etc/libvirt/nwfilter/no-arp-mac-spoofing.xml']
  modified.conffile..etc.libvirt.nwfilter.no-arp-spoofing.xml: [inaccessible: 
[Errno 13] Permission denied: '/etc/libvirt/nwfilter/no-arp-spoofing.xml']
  modified.conffile..etc.libvirt.nwfilter.no-ip-multicast.xml: [inaccessible: 
[Errno 13] Permission denied: '/etc/libvirt/nwfilter/no-ip-multicast.xml']
  modified.conffile..etc.libvirt.nwfilter.no-ip-spoofing.xml: [inaccessible: 
[Errno 13] Permission denied: '/etc/libvirt/nwfilter/no-ip-spoofing.xml']
  modified.conffile..etc.libvirt.nwfilter.no-mac-broadcast.xml: [inaccessible: 
[Errno 13] Permission denied: '/etc/libvirt/nwfilter/no-mac-broadcast.xml']
  modified.conffile..etc.libvirt.nwfilter.no-mac-spoofing.xml: [inaccessible: 
[Errno 13] Permission denied: '/etc/libvirt/nwfilter/no-mac-spoofing.xml']
  modified.conffile..etc.libvirt.nwfilter.no-other-l2-traffic.xml: 
[inaccessible: [Errno 13] Permission denied: 
'/etc/libvirt/nwfilter/no-other-l2-traffic.xml']
  modified.conffile..etc.libvirt.nwfilter.no-other-rarp-traffic.xml: 
[inaccessible: [Errno 13] Permission denied: 
'/etc/libvirt/nwfilter/no-other-rarp-traffic.xml']
  modified.conffile..etc.libvirt.nwfilter.qemu-announce-self-rarp.xml: 
[inaccessible: [Errno 13] Permission denied: 
'/etc/libvirt/nwfilter/qemu-announce-self-rarp.xml']
  modified.conffile..etc.libvirt.nwfilter.qemu-announce-self.xml: 
[inaccessible: [Errno 13] Permission denied: 
'/etc/libvirt/nwfilter/qemu-announce-self.xml']
  modified.conffile..etc.libvirt.qemu.conf: [inaccessible: [Errno 13] 
Permission denied: '/etc/libvirt/qemu.conf']
  modified.conffile..etc.libvirt.qemu.networks.default.xml: [inaccessible: 
[Errno 13] Permission denied: '/etc/libvirt/qemu/networks/default.xml']

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to zfs-linux in Ubuntu.
https://bugs.launchpad.net/bugs/1847105

Title:
  very slow disk creation, snapshotting

Status in virt-manager:
  Fix Released
Status in Native ZFS for Linux:
  New
Status in libvirt package in Ubuntu:
  Triaged
Status in virt-manager package in Ubuntu:
  Fix Released
Status in zfs-linux package in Ubuntu:
  New
Status in libvirt source package in Bionic:
  Invalid
Status in virt-manager source package in Bionic:
  Invalid
Status in zfs-linux source package in Bionic:
  Won't Fix
Status in libvirt source package in Disco:
  Won't Fix
Status in virt-manager source package in Disco:
  Won't Fix
Status in zfs-linux source package in Disco:
  Won't Fix
Status in libvirt source package in Focal:
  Triaged
Status in libvirt source package in Groovy:
  Triaged

Bug description:
  [Impact]

   * The defaults of virt-manager for disk allocation always worked fine
     when qcow2 had nothing but sparse support. So sparse=True vs
     sparse=False made no difference. So it always set sparse=False.
     Then qcow2 grows non-sparse support, and virt-manager is suddenly
     defaulting to it, which is not the intention.

   * For upgraders from pre-Focal this was a regression in two ways:
      a) allocation of qcow changed to non-sparse (fallocate based)
         potentially consuming more space
      b) depending on the underlying FS this made the allocation much
         slower

   * Fix by applying the upstream change that Defaults to sparse when
     requested format isn't raw (for raw the assumption ill stay that
     users do that for runtime performance, so non-sparse on those stays
     as-is)

  [Test Case]

   * open virt-manager and create a new guest which includes creating a
     new image for it
     * when clicking "finish" the image will be created (which e.g. on ZFS
       will take quite some time)
     * In the background grep ps output if the qemu-img call for qcow (the
       default) is using "preallocation=fallocate" (wrong) or
       "preallocation=metadata" (correct)

  [Regression Potential]

   * For upgraders from pre-focal it actually ensures behavior stays as is
     (no change/regression there)
   * For users of ZFS where the allocation was slow it fixes this
     slowness.
   * But for people using Focal all the time and on non-ZFS there will be
     a behavior change in no more doing falloc pre-allocating the qemu
     image. To be cleear there are four modes:
     1. no allocation
     2. metadata allocation - is initially larger but can improve
        performance when the image needs to grow
     3. falloc - preallocates space for image by calling posix_fallocate
     4. full - writes data to the underlying storage
     The fix changes the default from 3->2 which it always was and should
     be. As the patch says: "If users want to override it, they can
     do it via custom created storage."
     (To be clear, upstream changed this as we fixed it here, so on
      further upgrades what we apply here will be the behavior anyway, it
      seems wrong to keep Focal the only one in between kept different)
   * [racb] The code path being added special cases the 'raw' format and in 
handling of the default sparse setting. Areas of potential regression are 
therefore in the handling of the default and override of the default in the 
cases of both 'raw' and not 'raw'.

  [Other Info]

   * The slowness effect might be further mitigated by ZFS implementing
     more falloc options (thanks Richard for that hint) but that won't be
     in Focal.

  ----

  This is a regression in eoan for me. I use virt-manager to create vms,
  and I noticed that creating one now takes more than a minute.

  Looking at the process listing while the backing disk is being created, I see 
this qemu-img command line:
  15658 ?        Ssl    0:00 /usr/sbin/libvirtd
  23726 ?        Sl     0:04  \_ /usr/bin/qemu-img create -f qcow2 -o 
preallocation=falloc,compat=1.1,lazy_refcounts 
/var/lib/libvirt/images/live-server.qcow2 41943040K

  If I run qemu-img with that preallocation parameter set, even on
  bionic, then it also takes a very long time.

  On eoan, for comparison:
  andreas@nsn7:~$ time qemu-img create -f qcow2 no-prealloc-image.qcow2 40G
  Formatting 'no-prealloc-image.qcow2', fmt=qcow2 size=42949672960 
cluster_size=65536 lazy_refcounts=off refcount_bits=16

  real  0m0,016s
  user  0m0,010s
  sys   0m0,006s
  andreas@nsn7:~$ qemu-img info no-prealloc-image.qcow2
  image: no-prealloc-image.qcow2
  file format: qcow2
  virtual size: 40G (42949672960 bytes)
  disk size: 17K
  cluster_size: 65536
  Format specific information:
      compat: 1.1
      lazy refcounts: false
      refcount bits: 16
      corrupt: false
  andreas@nsn7:~$ du -hs no-prealloc-image.qcow2
  17K   no-prealloc-image.qcow2
  andreas@nsn7:~$

  and now with preallocation=falloc:
  andreas@nsn7:~$ time qemu-img create -f qcow2 -o preallocation=falloc 
with-prealloc-image.qcow2 40G
  Formatting 'with-prealloc-image.qcow2', fmt=qcow2 size=42949672960 
cluster_size=65536 preallocation=falloc lazy_refcounts=off refcount_bits=16

  real  1m43,196s
  user  0m3,564s
  sys   1m26,720s
  andreas@nsn7:~$ qemu-img info with-prealloc-image.qcow2
  image: with-prealloc-image.qcow2
  file format: qcow2
  virtual size: 40G (42949672960 bytes)
  disk size: 2.7M
  cluster_size: 65536
  Format specific information:
      compat: 1.1
      lazy refcounts: false
      refcount bits: 16
      corrupt: false
  andreas@nsn7:~$ du -hs with-prealloc-image.qcow2
  2,8M  with-prealloc-image.qcow2
  andreas@nsn7:~$

  ProblemType: Bug
  DistroRelease: Ubuntu 19.10
  Package: libvirt-daemon 5.4.0-0ubuntu5
  ProcVersionSignature: Ubuntu 5.3.0-13.14-generic 5.3.0
  Uname: Linux 5.3.0-13-generic x86_64
  NonfreeKernelModules: zfs zunicode zavl icp zcommon znvpair
  ApportVersion: 2.20.11-0ubuntu7
  Architecture: amd64
  Date: Mon Oct  7 11:36:03 2019
  InstallationDate: Installed on 2019-10-07 (0 days ago)
  InstallationMedia: Ubuntu 19.10 "Eoan Ermine" - Beta amd64 (20191006)
  SourcePackage: libvirt
  UpgradeStatus: No upgrade log present (probably fresh install)
  modified.conffile..etc.libvirt.nwfilter.allow-arp.xml: [inaccessible: [Errno 
13] Permission denied: '/etc/libvirt/nwfilter/allow-arp.xml']
  modified.conffile..etc.libvirt.nwfilter.allow-dhcp-server.xml: [inaccessible: 
[Errno 13] Permission denied: '/etc/libvirt/nwfilter/allow-dhcp-server.xml']
  modified.conffile..etc.libvirt.nwfilter.allow-dhcp.xml: [inaccessible: [Errno 
13] Permission denied: '/etc/libvirt/nwfilter/allow-dhcp.xml']
  modified.conffile..etc.libvirt.nwfilter.allow-incoming-ipv4.xml: 
[inaccessible: [Errno 13] Permission denied: 
'/etc/libvirt/nwfilter/allow-incoming-ipv4.xml']
  modified.conffile..etc.libvirt.nwfilter.allow-ipv4.xml: [inaccessible: [Errno 
13] Permission denied: '/etc/libvirt/nwfilter/allow-ipv4.xml']
  modified.conffile..etc.libvirt.nwfilter.clean-traffic-gateway.xml: 
[inaccessible: [Errno 13] Permission denied: 
'/etc/libvirt/nwfilter/clean-traffic-gateway.xml']
  modified.conffile..etc.libvirt.nwfilter.clean-traffic.xml: [inaccessible: 
[Errno 13] Permission denied: '/etc/libvirt/nwfilter/clean-traffic.xml']
  modified.conffile..etc.libvirt.nwfilter.no-arp-ip-spoofing.xml: 
[inaccessible: [Errno 13] Permission denied: 
'/etc/libvirt/nwfilter/no-arp-ip-spoofing.xml']
  modified.conffile..etc.libvirt.nwfilter.no-arp-mac-spoofing.xml: 
[inaccessible: [Errno 13] Permission denied: 
'/etc/libvirt/nwfilter/no-arp-mac-spoofing.xml']
  modified.conffile..etc.libvirt.nwfilter.no-arp-spoofing.xml: [inaccessible: 
[Errno 13] Permission denied: '/etc/libvirt/nwfilter/no-arp-spoofing.xml']
  modified.conffile..etc.libvirt.nwfilter.no-ip-multicast.xml: [inaccessible: 
[Errno 13] Permission denied: '/etc/libvirt/nwfilter/no-ip-multicast.xml']
  modified.conffile..etc.libvirt.nwfilter.no-ip-spoofing.xml: [inaccessible: 
[Errno 13] Permission denied: '/etc/libvirt/nwfilter/no-ip-spoofing.xml']
  modified.conffile..etc.libvirt.nwfilter.no-mac-broadcast.xml: [inaccessible: 
[Errno 13] Permission denied: '/etc/libvirt/nwfilter/no-mac-broadcast.xml']
  modified.conffile..etc.libvirt.nwfilter.no-mac-spoofing.xml: [inaccessible: 
[Errno 13] Permission denied: '/etc/libvirt/nwfilter/no-mac-spoofing.xml']
  modified.conffile..etc.libvirt.nwfilter.no-other-l2-traffic.xml: 
[inaccessible: [Errno 13] Permission denied: 
'/etc/libvirt/nwfilter/no-other-l2-traffic.xml']
  modified.conffile..etc.libvirt.nwfilter.no-other-rarp-traffic.xml: 
[inaccessible: [Errno 13] Permission denied: 
'/etc/libvirt/nwfilter/no-other-rarp-traffic.xml']
  modified.conffile..etc.libvirt.nwfilter.qemu-announce-self-rarp.xml: 
[inaccessible: [Errno 13] Permission denied: 
'/etc/libvirt/nwfilter/qemu-announce-self-rarp.xml']
  modified.conffile..etc.libvirt.nwfilter.qemu-announce-self.xml: 
[inaccessible: [Errno 13] Permission denied: 
'/etc/libvirt/nwfilter/qemu-announce-self.xml']
  modified.conffile..etc.libvirt.qemu.conf: [inaccessible: [Errno 13] 
Permission denied: '/etc/libvirt/qemu.conf']
  modified.conffile..etc.libvirt.qemu.networks.default.xml: [inaccessible: 
[Errno 13] Permission denied: '/etc/libvirt/qemu/networks/default.xml']

To manage notifications about this bug go to:
https://bugs.launchpad.net/virt-manager/+bug/1847105/+subscriptions

-- 
Mailing list: https://launchpad.net/~kernel-packages
Post to     : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help   : https://help.launchpad.net/ListHelp

Reply via email to