Re: Run VM with 16G or more?

[Mike Larkin]

On Tue, Jan 02, 2024 at 08:29:03PM +0100, Kirill A. Korinsky wrote:
> And one more noticed bug in vmd regarding memory.
>
> If I changed memory in /etc/vm.conf for running machine, run rcctl reload vmd,
> and restart VM... It has no effect.
>
> The VM should be shutdown before reload.
>
> --
> wbr, Kirill
>

yes, vmctl reload does not reload specifications for currently running VMs.

Re: Run VM with 16G or more?

[Kirill A. Korinsky]

And one more noticed bug in vmd regarding memory.

If I changed memory in /etc/vm.conf for running machine, run rcctl reload vmd,
and restart VM... It has no effect.

The VM should be shutdown before reload.

--
wbr, Kirill

Re: Run VM with 16G or more?

[Kirill A. Korinsky]

> On 2. Jan 2024, at 19:58, Kirill A. Korinsky  wrote:
> 
> Anyway, right now it fails as:
> 
>> vmctl: start vm command failed: Invalid argument
> 
> and if I revert may changes (to 10G for example) at cat /etc/login.conf.d/vmd 
> from:
>> vmd:\
>>  :datasize=100G:\
>>  :tc=daemon:
> 
> 
> it's failed as (which is expected):
>> vmctl: start vm command failed: Cannot allocate memory
> 
> 

Forgotten log for the error "Invalid argument" that says nothing I assume

> Jan  2 20:14:07 island vmd: vmd: config_setvm: vm 3 restarted after 11.346817 
> seconds, limit 0/3
> Jan  2 20:14:07 island vmd: vmd: vm_opentty: vm podman tty /dev/ttyp2 uid 
> 1000 gid 4 mode 620
> Jan  2 20:14:07 island vmd: vmm: vm_register: registering vm 3
> Jan  2 20:14:07 island vmd: vmm: vm_remove: vmm vmm_start_vm removing vm 3 
> from running config
> Jan  2 20:14:07 island vmd: vmm: vm_stop: vmm vmm_start_vm stopping vm 3
> Jan  2 20:14:07 island vmd: vmd: podman: failed to start vm
> Jan  2 20:14:07 island vmd: vmd: vm_stop: vmd vmd_dispatch_vmm stopping vm 3
> Jan  2 20:14:07 island vmd: vmm: vmm_sighdlr: handling signal 20


--
wbr, Kirill

Re: Run VM with 16G or more?

[Kirill A. Korinsky]

> On 2. Jan 2024, at 20:13, Mischa  wrote:
> 
> On 2024-01-02 19:58, Kirill A. Korinsky wrote:
>>> On 2. Jan 2024, at 19:17, Dave Voutila  wrote:
 vmd: failed to start vm podman
 vmd: vm_stop: vmd config_setvm stopping vm 3
 This machine runs 4 more VM and this one (huge) should be 5th.
>>> Try this:
>>> # cd /dev && sh MAKEDEV tap4
>>> By default I believe on amd64 we create tap[0-3]. You might need to
>>> define additional special files to represent 4+ taps.
>> I really think that this should be documented at 
>> https://www.openbsd.org/faq/faq16.html 
>> 
> 
> It's in man vm.conf, to an extend:
> 
> CAVEATS
> Each guest requires one tap(4) device per assigned interface and one
> pty(4) device.  Administrators may need to create additional devices
> using MAKEDEV(8).

ok, I agree that this is documented.  But not original issue with memory.

--
wbr, Kirill

Re: Run VM with 16G or more?

[Mischa]

On 2024-01-02 19:58, Kirill A. Korinsky wrote:

On 2. Jan 2024, at 19:17, Dave Voutila  wrote:


vmd: failed to start vm podman
vmd: vm_stop: vmd config_setvm stopping vm 3

This machine runs 4 more VM and this one (huge) should be 5th.


Try this:

# cd /dev && sh MAKEDEV tap4

By default I believe on amd64 we create tap[0-3]. You might need to
define additional special files to represent 4+ taps.


I really think that this should be documented at 
https://www.openbsd.org/faq/faq16.html 



It's in man vm.conf, to an extend:

CAVEATS
 Each guest requires one tap(4) device per assigned interface and 
one
 pty(4) device.  Administrators may need to create additional 
devices

 using MAKEDEV(8).


Anyway, right now it fails as:


vmctl: start vm command failed: Invalid argument


and if I revert may changes (to 10G for example) at cat 
/etc/login.conf.d/vmd from:

vmd:\
:datasize=100G:\
:tc=daemon:



it's failed as (which is expected):

vmctl: start vm command failed: Cannot allocate memory



--
wbr, Kirill

Re: Run VM with 16G or more?

[Kirill A. Korinsky]

> On 2. Jan 2024, at 19:17, Dave Voutila  wrote:
> 
>> vmd: failed to start vm podman
>> vmd: vm_stop: vmd config_setvm stopping vm 3
>> 
>> This machine runs 4 more VM and this one (huge) should be 5th.
> 
> Try this:
> 
> # cd /dev && sh MAKEDEV tap4
> 
> By default I believe on amd64 we create tap[0-3]. You might need to
> define additional special files to represent 4+ taps.

I really think that this should be documented at 
https://www.openbsd.org/faq/faq16.html 

Anyway, right now it fails as:

> vmctl: start vm command failed: Invalid argument

and if I revert may changes (to 10G for example) at cat /etc/login.conf.d/vmd 
from:
> vmd:\
>   :datasize=100G:\
>   :tc=daemon:


it's failed as (which is expected):
> vmctl: start vm command failed: Cannot allocate memory


--
wbr, Kirill

Re: Run VM with 16G or more?

[Dave Voutila]

"Kirill A. Korinsky"  writes:

> [[PGP Signed Part:Undecided]]
>> On 2. Jan 2024, at 18:41, Dave Voutila  wrote:
>> "Kirill A. Korinsky"  writes:
>
>>> vmctl -v start... doesn't help a bit
>>
>> How much physicaly memory does the host machine have? We currently don't
>> allow oversubscribing memory with vmm/vmd. If the host only has 16GB
>> that could be the cause.
>
> hw.physmem=137257779200
> hw.usermem=133537726464
>
> and machine is used only for run VMs.
>
>> If that's not the case, can you run vmd in debug mode and get the log
>> output?
>
> Sure, I run /usr/sbin/vmd -vvv -d and the error is:
>
> vmd: config_setvm: vm 3 restarted after 9.757221 seconds, limit 0/3
> vmd: config_setvm: can't open tap tap

Ah, that.

> vmd: failed to start vm podman
> vmd: vm_stop: vmd config_setvm stopping vm 3
>
> This machine runs 4 more VM and this one (huge) should be 5th.

Try this:

# cd /dev && sh MAKEDEV tap4

By default I believe on amd64 we create tap[0-3]. You might need to
define additional special files to represent 4+ taps.

-dv

Re: Run VM with 16G or more?

[Mischa]

On 2024-01-02 19:16, Kirill A. Korinsky wrote:

On 2. Jan 2024, at 18:41, Dave Voutila  wrote:
"Kirill A. Korinsky"  writes:



vmctl -v start... doesn't help a bit


How much physicaly memory does the host machine have? We currently 
don't

allow oversubscribing memory with vmm/vmd. If the host only has 16GB
that could be the cause.


hw.physmem=137257779200
hw.usermem=133537726464

and machine is used only for run VMs.


If that's not the case, can you run vmd in debug mode and get the log
output?


Sure, I run /usr/sbin/vmd -vvv -d and the error is:

vmd: config_setvm: vm 3 restarted after 9.757221 seconds, limit 0/3
vmd: config_setvm: can't open tap tap
vmd: failed to start vm podman
vmd: vm_stop: vmd config_setvm stopping vm 3

This machine runs 4 more VM and this one (huge) should be 5th.


You need to add more tap interfaces:

# cd /dev
# for i in $(jot XX 4); do sh MAKEDEV tap$i; done

Where XX is the number of taps you want to add.

Mischa

Re: Run VM with 16G or more?

[Kirill A. Korinsky]

> On 2. Jan 2024, at 18:41, Dave Voutila  wrote:
> "Kirill A. Korinsky"  writes:

>> vmctl -v start... doesn't help a bit
> 
> How much physicaly memory does the host machine have? We currently don't
> allow oversubscribing memory with vmm/vmd. If the host only has 16GB
> that could be the cause.

hw.physmem=137257779200
hw.usermem=133537726464

and machine is used only for run VMs.

> If that's not the case, can you run vmd in debug mode and get the log
> output?

Sure, I run /usr/sbin/vmd -vvv -d and the error is:

vmd: config_setvm: vm 3 restarted after 9.757221 seconds, limit 0/3
vmd: config_setvm: can't open tap tap
vmd: failed to start vm podman
vmd: vm_stop: vmd config_setvm stopping vm 3

This machine runs 4 more VM and this one (huge) should be 5th.

--
wbr, Kirill

Re: Run VM with 16G or more?

[Dave Voutila]

"Kirill A. Korinsky"  writes:

>> On 2. Jan 2024, at 12:07, Kirill A. Korinsky  wrote:
>>
>> Confirmed that it is:
>>
>> island$ grep '^vmd:' -A 2 /etc/login.conf
>> vmd:\
>>  :datasize=16384M:\
>>  :tc=daemon:
>> island$
>
>
> Wel.. after that changes error has been changed to:
>
>> vmctl: start vm command failed: Unknown error: -1

yeah that error description sucks...should be changed

>
>
> vmctl -v start... doesn't help a bit

How much physicaly memory does the host machine have? We currently don't
allow oversubscribing memory with vmm/vmd. If the host only has 16GB
that could be the cause.

If that's not the case, can you run vmd in debug mode and get the log
output?

Re: Run VM with 16G or more?

[Kirill A. Korinsky]

> On 2. Jan 2024, at 12:07, Kirill A. Korinsky  wrote:
> 
> Confirmed that it is:
> 
> island$ grep '^vmd:' -A 2 /etc/login.conf
> vmd:\
>   :datasize=16384M:\
>   :tc=daemon:
> island$


Wel.. after that changes error has been changed to:

> vmctl: start vm command failed: Unknown error: -1


vmctl -v start... doesn't help a bit

--
wbr, Kirill

Re: Run VM with 16G or more?

[Kirill A. Korinsky]

> On 2. Jan 2024, at 08:58, Janne Johansson  wrote:
> 
> Den mån 1 jan. 2024 kl 21:44 skrev Kirill A. Korinsky :
>> 
>> How can I run a VM with more than 16G of memory?
>> A naive approach fails with error:
>>> vmctl: start vm command failed: Cannot allocate memory
>> 
>> Yes, the host machine has that memory and much more.
> 
> Check datasize in ulimits as set by the shell and login.conf for the
> user that the VM runs as.
> 

Confirmed that it is:

island$ grep '^vmd:' -A 2 /etc/login.conf
vmd:\
:datasize=16384M:\
:tc=daemon:
island$

Thanks!

--
wbr, Kirill

Re: Run VM with 16G or more?

[Janne Johansson]

Den mån 1 jan. 2024 kl 21:44 skrev Kirill A. Korinsky :
>
> Greetings,
> How can I run a VM with more than 16G of memory?
> A naive approach fails with error:
> > vmctl: start vm command failed: Cannot allocate memory
>
> Yes, the host machine has that memory and much more.

Check datasize in ulimits as set by the shell and login.conf for the
user that the VM runs as.

-- 
May the most significant bit of your life be positive.