Russell Coker wrote: > On Friday, 4 June 2021 6:49:17 PM AEST Trent W. Buck via luv-main wrote: > > > > https://virtio-fs.gitlab.io/ > > > > > > I tested it and found it to be about half as fast as NFS. > > > > Interesting. I am using it in the lab, but haven't gotten as far as fio > > benching. > > > > Were you using NFSv4 sec=sys, or something else? > > Fairly default NFSv4, not using NFS for root though. > > > Were guest and host both Debian 11 / Linux 5.10, or something else? > > Both Debian/Testing. > > > Was libvirt-daemon 7.x? > > > > Was qemu-system-x86 5.2 or 6.x? > > Whatever the Debian/Testing versions were at the start of this year. > > > It is currently on Debian 11, because you have to patch domain.xml by hand. > > That should get better by Debian 12, though. > > I run kvm with a shell script that passes all relevant parameters on the > command line.
Right-o.
FTR, for booting off virtiofs,
attached is the CLI options libvirtd used, the script I used to
make the chroot (VM), and the domain.xml after GUI fiddle-farting
around to make virtd pass the right options to qemu and virtiofsd.
debian 11 / linux 5.10 / libvirtd 7 / qemu 5.2.
The chroot is 468MiB (271MiB with compression=zstd).
Obviously that can be tuned better.
To boot Debian 9 off NFSv4 instead of NFSv3, you need
apt install nfs-common # not needed to boot off NFSv3!
>/usr/share/initramfs-tools/hooks/zz-nfs4 cat <<-'EOF' && chmod +x
/usr/share/initramfs-tools/hooks/zz-nfs4
# Use nfs-utils mount.nfs (not klibc-utils nfsmount) so rootfs mounts as
NFSv4 (not NFSv3) [#32658]
[ "$1" = prereqs ]||(. /usr/share/initramfs-tools/hook-functions;copy_exec
/sbin/mount.nfs /bin/nfsmount)
EOF
# install kernel, or run "update-initramfs -u -k all"
I'm not sure if this is still needed for Debian 10 and Debian 11.
I *think* it is.
That's for sec=sys i.e. "classic" insecure NFS.
If you want kerberization, that's more work.
No fio benches, sorry, I can't be arsed tonight setting up an NFS VM
side-by-side for a fair comparison.
tmp.sh
Description: Bourne shell script
2021-06-05 20:15:55.522+0000: Starting external device: virtiofsd /usr/lib/qemu/virtiofsd --fd=34 -o source=/var/tmp/my-cool-vm 2021-06-05 20:15:55.698+0000: starting up libvirt version: 7.0.0, package: 3 (Andrea Bolognani <[email protected]> Fri, 26 Feb 2021 16:46:34 +0100), qemu version: 5.2.0Debian 1:5.2+dfsg-10, kernel: 5.10.0-6-amd64, hostname: hera.lan LC_ALL=C \ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \ HOME=/var/lib/libvirt/qemu/domain-3-debiantesting \ XDG_DATA_HOME=/var/lib/libvirt/qemu/domain-3-debiantesting/.local/share \ XDG_CACHE_HOME=/var/lib/libvirt/qemu/domain-3-debiantesting/.cache \ XDG_CONFIG_HOME=/var/lib/libvirt/qemu/domain-3-debiantesting/.config \ QEMU_AUDIO_DRV=spice \ /usr/bin/qemu-system-x86_64 \ -name guest=debiantesting,debug-threads=on \ -S \ -object secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-3-debiantesting/master-key.aes \ -machine pc-q35-5.2,accel=kvm,usb=off,vmport=off,dump-guest-core=off,memory-backend=pc.ram \ -cpu Skylake-Client-IBRS,ss=on,vmx=on,pdcm=on,hypervisor=on,tsc-adjust=on,clflushopt=on,umip=on,md-clear=on,stibp=on,arch-capabilities=on,ssbd=on,xsaves=on,pdpe1gb=on,ibpb=on,amd-stibp=on,amd-ssbd=on,rdctl-no=on,skip-l1dfl-vmentry=on,pschange-mc-no=on,hle=off,rtm=off \ -m 1024 \ -object memory-backend-file,id=pc.ram,mem-path=/dev/shm/libvirt/qemu/3-debiantesting/pc.ram,share=yes,size=1073741824 \ -overcommit mem-lock=off \ -smp 2,sockets=2,cores=1,threads=1 \ -uuid 725e8c6f-4627-49bb-a780-7c06df32288c \ -no-user-config \ -nodefaults \ -chardev socket,id=charmonitor,fd=34,server,nowait \ -mon chardev=charmonitor,id=monitor,mode=control \ -rtc base=utc,driftfix=slew \ -global kvm-pit.lost_tick_policy=delay \ -no-hpet \ -no-shutdown \ -global ICH9-LPC.disable_s3=1 \ -global ICH9-LPC.disable_s4=1 \ -boot strict=on \ -kernel /var/tmp/my-cool-vm/vmlinuz \ -initrd /var/tmp/my-cool-vm//initrd.img \ -append 'rootfstype=virtiofs root=my-cool-vm-rootfs' \ -device pcie-root-port,port=0x10,chassis=1,id=pci.1,bus=pcie.0,multifunction=on,addr=0x2 \ -device pcie-root-port,port=0x11,chassis=2,id=pci.2,bus=pcie.0,addr=0x2.0x1 \ -device pcie-root-port,port=0x12,chassis=3,id=pci.3,bus=pcie.0,addr=0x2.0x2 \ -device pcie-root-port,port=0x13,chassis=4,id=pci.4,bus=pcie.0,addr=0x2.0x3 \ -device pcie-root-port,port=0x14,chassis=5,id=pci.5,bus=pcie.0,addr=0x2.0x4 \ -device pcie-root-port,port=0x15,chassis=6,id=pci.6,bus=pcie.0,addr=0x2.0x5 \ -device qemu-xhci,p2=15,p3=15,id=usb,bus=pci.2,addr=0x0 \ -device virtio-serial-pci,id=virtio-serial0,bus=pci.3,addr=0x0 \ -chardev socket,id=chr-vu-fs0,path=/var/lib/libvirt/qemu/domain-3-debiantesting/fs0-fs.sock \ -device vhost-user-fs-pci,chardev=chr-vu-fs0,tag=my-cool-vm-rootfs,bus=pci.6,addr=0x0 \ -netdev tap,fd=36,id=hostnet0,vhost=on,vhostfd=37 \ -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:05:d9:e4,bus=pci.1,addr=0x0 \ -chardev pty,id=charserial0 \ -device isa-serial,chardev=charserial0,id=serial0 \ -chardev socket,id=charchannel0,fd=38,server,nowait \ -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 \ -chardev spicevmc,id=charchannel1,name=vdagent \ -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=com.redhat.spice.0 \ -device usb-tablet,id=input0,bus=usb.0,port=1 \ -spice port=5900,addr=127.0.0.1,disable-ticketing,image-compression=off,seamless-migration=on \ -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,max_outputs=1,bus=pcie.0,addr=0x1 \ -device ich9-intel-hda,id=sound0,bus=pcie.0,addr=0x1b \ -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 \ -chardev spicevmc,id=charredir0,name=usbredir \ -device usb-redir,chardev=charredir0,id=redir0,bus=usb.0,port=2 \ -chardev spicevmc,id=charredir1,name=usbredir \ -device usb-redir,chardev=charredir1,id=redir1,bus=usb.0,port=3 \ -device virtio-balloon-pci,id=balloon0,bus=pci.4,addr=0x0 \ -object rng-random,id=objrng0,filename=/dev/urandom \ -device virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.5,addr=0x0 \ -sandbox on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny \ -msg timestamp=on char device redirected to /dev/pts/4 (label charserial0)
debiantesting.xml
Description: XML document
_______________________________________________ luv-main mailing list -- [email protected] To unsubscribe send an email to [email protected]
