On 12/02/2017 00:23, Dustin Wenz wrote: > I have noticed significant storage amplification for my zvols; that could > very well be the reason. I would like to know more about why it happens. > > Since the volblocksize is 512 bytes, I certainly expect extra cpu overhead > (and maybe an extra 1k or so worth of checksums for each 128k block in the > vm), but how do you get a 10X expansion in stored data? > > What is the recommended zvol block size for a FreeBSD/ZFS guest? Perhaps 4k, > to match the most common mass storage sector size? > > - .Dustin > >> On Dec 1, 2017, at 9:18 PM, K. Macy <km...@freebsd.org> wrote: >> >> One thing to watch out for with chyves if your virtual disk is more >> than 20G is the fact that it uses 512 byte blocks for the zvols it >> creates. I ended up using up 1.4TB only half filling up a 250G zvol. >> Chyves is quick and easy, but it's not exactly production ready. >> >> -M >> >> >> >>> On Thu, Nov 30, 2017 at 3:15 PM, Dustin Wenz <dustinw...@ebureau.com> wrote: >>> I'm using chyves on FreeBSD 11.1 RELEASE to manage a few VMs (guest OS is >>> also FreeBSD 11.1). Their sole purpose is to house some medium-sized >>> Postgres databases (100-200GB). The host system has 64GB of real memory and >>> 112GB of swap. I have configured each guest to only use 16GB of memory, yet >>> while doing my initial database imports in the VMs, bhyve will quickly grow >>> to use all available system memory and then be killed by the kernel: >>> >>> kernel: swap_pager: I/O error - pageout failed; blkno 1735,size >>> 4096, error 12 >>> kernel: swap_pager: I/O error - pageout failed; blkno 1610,size >>> 4096, error 12 >>> kernel: swap_pager: I/O error - pageout failed; blkno 1763,size >>> 4096, error 12 >>> kernel: pid 41123 (bhyve), uid 0, was killed: out of swap space >>> >>> The OOM condition seems related to doing moderate IO within the VM, though >>> nothing within the VM itself shows high memory usage. This is the chyves >>> config for one of them: >>> >>> bargs -A -H -P -S >>> bhyve_disk_type virtio-blk >>> bhyve_net_type virtio-net >>> bhyveload_flags >>> chyves_guest_version 0300 >>> cpu 4 >>> creation Created on Mon Oct 23 16:17:04 CDT 2017 >>> by chyves v0.2.0 2016/09/11 using __create() >>> loader bhyveload >>> net_ifaces tap51 >>> os default >>> ram 16G >>> rcboot 0 >>> revert_to_snapshot >>> revert_to_snapshot_method off >>> serial nmdm51 >>> template no >>> uuid 8495a130-b837-11e7-b092-0025909a8b56 >>> >>> >>> I've also tried using different bhyve_disk_types, with no improvement. How >>> is it that bhyve can use far more memory that I'm specifying? >>> >>> - .Dustin > _______________________________________________ > freebsd-virtualization@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization > To unsubscribe, send any mail to > "freebsd-virtualization-unsubscr...@freebsd.org" >
Storage amplification usually has to do with ZFS RAID-Z padding. If your ZVOL block size does not make sense with your disk sector size, and RAID-Z level, you can get pretty silly numbers. -- Allan Jude _______________________________________________ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
