Hi folks,
Summary/Questions:
1. Is the extremely large minimum-IO value of 256KB for the dom0 block devices
representing Q4 VM volume in the thin pool ... intentional?
2. And if so, to what purpose (e.g. performance, etc.)?
3. And if so, has the impact of this value on depending on discards for
returning unused disk space to the pool been factored in?
---discussion and supporting cmd output follows---
As you can see below, the MIN-IO (minimum IO size for R/W/D) and DISC-GRAN
(minimum size allowed for discard/trim commands) on most of the thin pool
volumes are both set to 256KB. Shown below, you can see this is the case for
the debian 9 VM and the dom0 root volume. Same for all the VM volumes that I
cut out of the output below for brevity/privacy.
Everything else in the stack, the drives, partitions, luks/crypt container and
even some of the non-VM filesystem pool volumes and/or metadata have much more
reasonable MIN-IO and DISC-GRAN values of 512 bytes or 4K...including dom0 swap!
The result is that turning on automatic trimming of the filesystems within VMs
requires large holes to be created on the virtual disk before triggering
discards that can be transmitted down the stack during deletions. To rephrase:
in the default configuration, for data to be recovered from VM volumes back
into the pool after deletions, the deletions must include files with large
contiguous sections. Also, this negatively impacts physical disk trimming, if
the user has configured it.
The 256K value may explain why folks have only found that manually invoking
'sudo fstrim /av' is the only guaranteed way to trigger full release of storage
back into the pool from VMs, leaving users who do not regularly trim from
inside their VMs at risk of the pool running out of room.
--command-to-create-output--
% lsblk -o
TYPE,ROTA,FSTYPE,PHY-SEC,LOG-SEC,MIN-IO,DISC-GRAN,DISC-ZERO,VENDOR,MODEL,REV,NAME
-x NAME -x TYPE|sort|uniq
--trimmed-output--
TYPE ROTA FSTYPE PHY-SEC LOG-SEC MIN-IO DISC-GRAN DISC-ZERO VENDOR
MODEL REV NAME
crypt 0 LVM2_member 512 512 512 512B 0
luks-xxxxxxxx-xxxx-xxxx-xxxx
-xxxxxxxxxxxx
disk 0 4096 512 4096 4K 0 ATA
xxxxxxxxxxxxxxxx xxxx sdc
disk 0 4096 512 4096 4K 0 ATA
xxxxxxxxxxxxxxxx xxxx sdb
disk 0 512 512 512 512B 0 ATA
xxxxxxxxxxxxxxxx xxxx sda
loop 1 ext3 512 512 512 4K 0
loop0
...
lvm 0 512 512 262144 256K 0
qubes_dom0-vm--debian--9--private
lvm 0 512 512 262144 256K 0
qubes_dom0-vm--debian--9--root
...
lvm 0 512 512 262144 512B 0
qubes_dom0-pool00
lvm 0 512 512 262144 512B 0
qubes_dom0-pool00-tpool
lvm 0 512 512 512 512B 0
qubes_dom0-pool00_tdata
lvm 0 512 512 512 512B 0
qubes_dom0-pool00_tmeta
lvm 0 ext4 512 512 262144 256K 0
qubes_dom0-root
lvm 0 swap 512 512 512 512B 0
qubes_dom0-swap
...
part 0 crypto_LUKS 512 512 512 512B 0
sda2
part 0 ext4 512 512 512 512B 0
sda1
For fun, here's a one-liner I put together to keep an eye on discards on
/dev/sd* or /dev/xvd* volumes in dom0 or a VM as I was exploring this issue.
watch -n 1 -d \
"if [ -d /sys/block/sda ] ; then pat=sd ; else pat=xvd ; fi ; sync;echo
--DISCARD TOTALS--;cat /sys/block/\$pat*/stat|awk 'BEGIN {print \"DRIVE IOS
QMERGES SECTORS MSWAITS\"} {printf \"%5i %-8s %s %15s
%11s\\n\",NR,\$12,\$13,\$14,\$15}'"
Brendan
--
You received this message because you are subscribed to the Google Groups
"qubes-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/qubes-users/44f1ae64-2da1-480f-aa30-98c5f22653ba%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
