On 2017-11-30 22:10, Dustin Wenz wrote:
> I am using a zvol as the storage for the VM, and I do not have any ARC
> limits set. However, the bhyve process itself ends up grabbing the vast
> majority of memory. 
> 
> I’ll run a test tomorrow to get the exact output from top.
> 
>    - .Dustin
> 
> On Nov 30, 2017, at 5:28 PM, Allan Jude <allanj...@freebsd.org
> <mailto:allanj...@freebsd.org>> wrote:
> 
>> On 11/30/2017 18:15, Dustin Wenz wrote:
>>> I'm using chyves on FreeBSD 11.1 RELEASE to manage a few VMs (guest
>>> OS is also FreeBSD 11.1). Their sole purpose is to house some
>>> medium-sized Postgres databases (100-200GB). The host system has 64GB
>>> of real memory and 112GB of swap. I have configured each guest to
>>> only use 16GB of memory, yet while doing my initial database imports
>>> in the VMs, bhyve will quickly grow to use all available system
>>> memory and then be killed by the kernel:
>>>
>>>    kernel: swap_pager: I/O error - pageout failed; blkno 1735,size
>>> 4096, error 12
>>>    kernel: swap_pager: I/O error - pageout failed; blkno 1610,size
>>> 4096, error 12
>>>    kernel: swap_pager: I/O error - pageout failed; blkno 1763,size
>>> 4096, error 12
>>>    kernel: pid 41123 (bhyve), uid 0, was killed: out of swap space
>>>
>>> The OOM condition seems related to doing moderate IO within the VM,
>>> though nothing within the VM itself shows high memory usage. This is
>>> the chyves config for one of them:
>>>
>>>    bargs                      -A -H -P -S
>>>    bhyve_disk_type            virtio-blk
>>>    bhyve_net_type             virtio-net
>>>    bhyveload_flags
>>>    chyves_guest_version       0300
>>>    cpu                        4
>>>    creation                   Created on Mon Oct 23 16:17:04 CDT 2017
>>> by chyves v0.2.0 2016/09/11 using __create()
>>>    loader                     bhyveload
>>>    net_ifaces                 tap51
>>>    os                         default
>>>    ram                        16G
>>>    rcboot                     0
>>>    revert_to_snapshot
>>>    revert_to_snapshot_method  off
>>>    serial                     nmdm51
>>>    template                   no
>>>    uuid                       8495a130-b837-11e7-b092-0025909a8b56
>>>
>>>
>>> I've also tried using different bhyve_disk_types, with no
>>> improvement. How is it that bhyve can use far more memory that I'm
>>> specifying?
>>>
>>>    - .Dustin
>>>
>>
>> Can you show 'top' output. What makes you think bhyve is using the
>> memory? Are you using ZFS? Have you limited the vfs.zfs.arc_max to leave
>> some free RAM for the bhyve instances?
>>
>> -- 
>> Allan Jude
>> _______________________________________________
>> freebsd-virtualization@freebsd.org
>> <mailto:freebsd-virtualization@freebsd.org> mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
>> To unsubscribe, send any mail to
>> "freebsd-virtualization-unsubscr...@freebsd.org
>> <mailto:freebsd-virtualization-unsubscr...@freebsd.org>"

The default limit for the ARC is 99% of your ram, so you are definitely
going to want to reduce that to something like: 90% of your 64GB of ram,
less the total amount of RAM you have given to all VMs.

-- 
Allan Jude
_______________________________________________
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Reply via email to