Re: Checking bhyve supported features (sysctls)

[Marcelo Araujo]

2018-08-17 0:07 GMT+08:00 Fabian Freyer :

> On 16 Aug 2018, at 12:16, Matt Churchyard wrote:
>
>> I'm looking for better ways to check for bhyve support / available
>> features without trying to scan through dmesg output.
>>
>
> There’s a few patches floating around for checking features [1,2] -
> however none of these are yet committed.
> For [2], there’s also an RFC mailing list thread [3].
>
> CC’ing novel@, as he wrote those patches.
>
> The hackish way libvirt checks for features right now is to parse error
> messages from invalid invocations of bhyve(8).
>

Hi,

I have committed r338210 the solution [1] that seemed to be the simplest
one comparing with [2].
Thank you and novel@ to keep working on bhyve libvirt support, appreciate
it.

Best,


>
> [1] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=210111
> [2] https://reviews.freebsd.org/D15992
> [3] https://lists.freebsd.org/pipermail/freebsd-virtualization/
> 2018-June/006536.html
>
> ___
> freebsd-virtualization@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> To unsubscribe, send any mail to "freebsd-virtualization-unsubs
> cr...@freebsd.org"
>



-- 

-- 
Marcelo Araujo(__)ara...@freebsd.org
\\\'',)http://www.FreeBSD.org    \/  \ ^
Power To Server. .\. /_)
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: Checking bhyve supported features (sysctls)

[Rodney W. Grimes]

> From: owner-freebsd-virtualizat...@freebsd.org 
>  On Behalf Of Rodney W. Grimes
> Sent: 16 August 2018 18:31
> To: Allan Jude 
> Cc: Matt Churchyard ; 
> freebsd-virtualization@freebsd.org
> Subject: Re: Checking bhyve supported features (sysctls)
> 
> > On August 16, 2018 5:28:05 PM GMT+01:00, "Rodney W. Grimes" 
> >  wrote:
> > >> 
> > >> Text manually wrapped to 80, any broken quoting is my fault - rwg
> > >> 
> > >> > > Hello,
> > >> > > 
> > >> > > I'm looking for better ways to check for  bhyve support /
> > >available
> > >> > > features without trying to scan through dmesg output.
> > >> > 
> > >> > >Yes, it would be very good to remove that, as it usually tries 
> > >> > >to grep a non-existent file /var/run/dmesg.boot that is not 
> > >> > >created until after vm_bhyve has been called from
> > >/usr/local/etc/rc.d
> > >> > >when you have things set to autostartup >in /etc/rc.conf
> > >> > 
> > >> > 
> > >> > > 
> > >> > > I notice that the following 2 sysctl's appear to be set to 1 as
> > >soon
> > >> > > as the vmm module is loaded
> > >> > > 
> > >> > > hw.vmm.vmx.initialized: 1
> > >> > > hw.vmm.vmx.cap.unrestricted_guest: 1
> > >> > > 
> > >> > > Will these be available on both Intel & AMD processors as a way 
> > >> > > to determine if the module has loaded successfully and can run
> > >guests?
> > >> > > 
> > >> > > I also see the below sysctl related to iommu.
> > >> > > 
> > >> > > hw.vmm.iommu.initialized
> > >> > > 
> > >> > > Again, will this be set to 1 as soon as the module is loaded if 
> > >> > > iommu is supported, or only when it is used?
> > >> > > There also seems to be a vmm.amdvi.enable sysctl.
> > >> > > Would both these need checking or is vmm.iommu enough to 
> > >> > > determine support on any processor.
> > >> > 
> > >> > >Probalby the safest way for a shell script to decide if bhyve is 
> > >> > >up and running is to stat /dev/vmm, if that exists then the
> > >modules
> > >> > >have loaded and initialized and bhyve should be ready to process
> > >guests.
> > >> > 
> > >> > Hmm, I don't get /dev/vmm unless I actually have running guests.
> > >> 
> > >> I'll investigate that, I was pretty sure that you should get this 
> > >> as soon as the vmm.ko module is finished initialzing, but you might 
> > >> be right in that it takes a first vm to cause its creation.
> > >> Confirmed, /dev/vmm does not exist until the first vm is created.
> > >> 
> > >> > 
> > >> > >sysctl's mentiond above would be a poor way to make this
> > >determination.
> > >> > 
> > >> > It would be nice if sysctls were better documented.
> > >> 
> > >> Agreed.
> > >> 
> > >> > If vmx.initialized is set once vmm has successfully loaded, I 
> > >> > can't
> > >see a better way of checking for bhyve support (assuming it's not 
> > >Intel specific). This entry definitely exists and is set to 0 if you 
> > >load the module on a non-supported system, and set to 1 as soon as 
> > >vmm loads on my Intel test system.
> > >> 
> > >> Given its undocumented status you would be relying on an 
> > >> undocumented feature that could change in either name or behavior, 
> > >> and that is not desirable.
> > >> 
> > >> Let me see if I can come up with something else.
> > >
> > >I looked at the code for bhyvectl, bhyveload and byhve.  They do not 
> > >actually try to decide if vmm is supported or not, they simply 
> > >process the error from a vm_create() or vm_open() call and exit with 
> > >an error code if they can not handle it (some of the code can handle 
> > >a vm_create failure if infact we are trying to create a vm that 
> > >already exists).
> > >
> > >If you want to maintain full compatibility a similiar stratergy may 
> > >be in order.
> > >
> > >Why is it that vm-bhyve specifically needs to know if the kernel has 
> > >v

RE: Checking bhyve supported features (sysctls)

[Matt Churchyard]

-Original Message-
From: owner-freebsd-virtualizat...@freebsd.org 
 On Behalf Of Marcelo Araujo
Sent: 17 August 2018 10:14
To: Matt Churchyard 
Cc: freebsd-virtualization@freebsd.org
Subject: Re: Checking bhyve supported features (sysctls)

2018-08-17 16:54 GMT+08:00 Matt Churchyard :

>
>
>
>
> 2018-08-17 16:25 GMT+08:00 Matt Churchyard :
>
> -Original Message-
> From: owner-freebsd-virtualizat...@freebsd.org < 
> owner-freebsd-virtualizat...@freebsd.org> On Behalf Of Rodney W. 
> Grimes
> Sent: 16 August 2018 18:31
> To: Allan Jude 
> Cc: Matt Churchyard ; 
> freebsd-virtualization@ freebsd.org
> Subject: Re: Checking bhyve supported features (sysctls)
>
> > On August 16, 2018 5:28:05 PM GMT+01:00, "Rodney W. Grimes" <
> freebsd-...@pdx.rh.cn85.dnsmgr.net> wrote:
> > >>
> > >> Text manually wrapped to 80, any broken quoting is my fault - rwg
> > >>
> > >> > > Hello,
> > >> > >
> > >> > > I'm looking for better ways to check for  bhyve support /
> > >available
> > >> > > features without trying to scan through dmesg output.
> > >> >
> > >> > >Yes, it would be very good to remove that, as it usually tries 
> > >> > >to grep a non-existent file /var/run/dmesg.boot that is not 
> > >> > >created until after vm_bhyve has been called from
> > >/usr/local/etc/rc.d
> > >> > >when you have things set to autostartup >in /etc/rc.conf
> > >> >
> > >> >
> > >> > >
> > >> > > I notice that the following 2 sysctl's appear to be set to 1 
> > >> > > as
> > >soon
> > >> > > as the vmm module is loaded
> > >> > >
> > >> > > hw.vmm.vmx.initialized: 1
> > >> > > hw.vmm.vmx.cap.unrestricted_guest: 1
> > >> > >
> > >> > > Will these be available on both Intel & AMD processors as a 
> > >> > > way to determine if the module has loaded successfully and 
> > >> > > can run
> > >guests?
> > >> > >
> > >> > > I also see the below sysctl related to iommu.
> > >> > >
> > >> > > hw.vmm.iommu.initialized
> > >> > >
> > >> > > Again, will this be set to 1 as soon as the module is loaded 
> > >> > > if iommu is supported, or only when it is used?
> > >> > > There also seems to be a vmm.amdvi.enable sysctl.
> > >> > > Would both these need checking or is vmm.iommu enough to 
> > >> > > determine support on any processor.
> > >> >
> > >> > >Probalby the safest way for a shell script to decide if bhyve 
> > >> > >is up and running is to stat /dev/vmm, if that exists then the
> > >modules
> > >> > >have loaded and initialized and bhyve should be ready to 
> > >> > >process
> > >guests.
> > >> >
> > >> > Hmm, I don't get /dev/vmm unless I actually have running guests.
> > >>
> > >> I'll investigate that, I was pretty sure that you should get this 
> > >> as soon as the vmm.ko module is finished initialzing, but you 
> > >> might be right in that it takes a first vm to cause its creation.
> > >> Confirmed, /dev/vmm does not exist until the first vm is created.
> > >>
> > >> >
> > >> > >sysctl's mentiond above would be a poor way to make this
> > >determination.
> > >> >
> > >> > It would be nice if sysctls were better documented.
> > >>
> > >> Agreed.
> > >>
> > >> > If vmx.initialized is set once vmm has successfully loaded, I 
> > >> > can't
> > >see a better way of checking for bhyve support (assuming it's not 
> > >Intel specific). This entry definitely exists and is set to 0 if 
> > >you load the module on a non-supported system, and set to 1 as soon 
> > >as vmm loads on my Intel test system.
> > >>
> > >> Given its undocumented status you would be relying on an 
> > >> undocumented feature that could change in either name or 
> > >> behavior, and that is not desirable.
> > >>
> > >> Let me see if I can come up with something else.
> > >
> > >I looked at the code for bhyvectl, bhyveload and byhve.  They do 
> > >not actually try t

Re: Checking bhyve supported features (sysctls)

[Marcelo Araujo]

2018-08-17 16:54 GMT+08:00 Matt Churchyard :

>
>
>
>
> 2018-08-17 16:25 GMT+08:00 Matt Churchyard :
>
> -Original Message-
> From: owner-freebsd-virtualizat...@freebsd.org <
> owner-freebsd-virtualizat...@freebsd.org> On Behalf Of Rodney W. Grimes
> Sent: 16 August 2018 18:31
> To: Allan Jude 
> Cc: Matt Churchyard ; freebsd-virtualization@
> freebsd.org
> Subject: Re: Checking bhyve supported features (sysctls)
>
> > On August 16, 2018 5:28:05 PM GMT+01:00, "Rodney W. Grimes" <
> freebsd-...@pdx.rh.cn85.dnsmgr.net> wrote:
> > >>
> > >> Text manually wrapped to 80, any broken quoting is my fault - rwg
> > >>
> > >> > > Hello,
> > >> > >
> > >> > > I'm looking for better ways to check for  bhyve support /
> > >available
> > >> > > features without trying to scan through dmesg output.
> > >> >
> > >> > >Yes, it would be very good to remove that, as it usually tries
> > >> > >to grep a non-existent file /var/run/dmesg.boot that is not
> > >> > >created until after vm_bhyve has been called from
> > >/usr/local/etc/rc.d
> > >> > >when you have things set to autostartup >in /etc/rc.conf
> > >> >
> > >> >
> > >> > >
> > >> > > I notice that the following 2 sysctl's appear to be set to 1 as
> > >soon
> > >> > > as the vmm module is loaded
> > >> > >
> > >> > > hw.vmm.vmx.initialized: 1
> > >> > > hw.vmm.vmx.cap.unrestricted_guest: 1
> > >> > >
> > >> > > Will these be available on both Intel & AMD processors as a way
> > >> > > to determine if the module has loaded successfully and can run
> > >guests?
> > >> > >
> > >> > > I also see the below sysctl related to iommu.
> > >> > >
> > >> > > hw.vmm.iommu.initialized
> > >> > >
> > >> > > Again, will this be set to 1 as soon as the module is loaded if
> > >> > > iommu is supported, or only when it is used?
> > >> > > There also seems to be a vmm.amdvi.enable sysctl.
> > >> > > Would both these need checking or is vmm.iommu enough to
> > >> > > determine support on any processor.
> > >> >
> > >> > >Probalby the safest way for a shell script to decide if bhyve is
> > >> > >up and running is to stat /dev/vmm, if that exists then the
> > >modules
> > >> > >have loaded and initialized and bhyve should be ready to process
> > >guests.
> > >> >
> > >> > Hmm, I don't get /dev/vmm unless I actually have running guests.
> > >>
> > >> I'll investigate that, I was pretty sure that you should get this
> > >> as soon as the vmm.ko module is finished initialzing, but you might
> > >> be right in that it takes a first vm to cause its creation.
> > >> Confirmed, /dev/vmm does not exist until the first vm is created.
> > >>
> > >> >
> > >> > >sysctl's mentiond above would be a poor way to make this
> > >determination.
> > >> >
> > >> > It would be nice if sysctls were better documented.
> > >>
> > >> Agreed.
> > >>
> > >> > If vmx.initialized is set once vmm has successfully loaded, I
> > >> > can't
> > >see a better way of checking for bhyve support (assuming it's not
> > >Intel specific). This entry definitely exists and is set to 0 if you
> > >load the module on a non-supported system, and set to 1 as soon as
> > >vmm loads on my Intel test system.
> > >>
> > >> Given its undocumented status you would be relying on an
> > >> undocumented feature that could change in either name or behavior,
> > >> and that is not desirable.
> > >>
> > >> Let me see if I can come up with something else.
> > >
> > >I looked at the code for bhyvectl, bhyveload and byhve.  They do not
> > >actually try to decide if vmm is supported or not, they simply
> > >process the error from a vm_create() or vm_open() call and exit with
> > >an error code if they can not handle it (some of the code can handle
> > >a vm_create failure if infact we are trying to create a vm that
> > >already exists).
> > >
> > >If you 

RE: Checking bhyve supported features (sysctls)

[Matt Churchyard]

2018-08-17 16:25 GMT+08:00 Matt Churchyard 
mailto:matt.churchy...@userve.net>>:
-Original Message-
From: 
owner-freebsd-virtualizat...@freebsd.org<mailto:owner-freebsd-virtualizat...@freebsd.org>
 
mailto:owner-freebsd-virtualizat...@freebsd.org>>
 On Behalf Of Rodney W. Grimes
Sent: 16 August 2018 18:31
To: Allan Jude mailto:allanj...@freebsd.org>>
Cc: Matt Churchyard 
mailto:matt.churchy...@userve.net>>; 
freebsd-virtualization@freebsd.org<mailto:freebsd-virtualization@freebsd.org>
Subject: Re: Checking bhyve supported features (sysctls)

> On August 16, 2018 5:28:05 PM GMT+01:00, "Rodney W. Grimes" 
> mailto:freebsd-...@pdx.rh.cn85.dnsmgr.net>>
>  wrote:
> >>
> >> Text manually wrapped to 80, any broken quoting is my fault - rwg
> >>
> >> > > Hello,
> >> > >
> >> > > I'm looking for better ways to check for  bhyve support /
> >available
> >> > > features without trying to scan through dmesg output.
> >> >
> >> > >Yes, it would be very good to remove that, as it usually tries
> >> > >to grep a non-existent file /var/run/dmesg.boot that is not
> >> > >created until after vm_bhyve has been called from
> >/usr/local/etc/rc.d
> >> > >when you have things set to autostartup >in /etc/rc.conf
> >> >
> >> >
> >> > >
> >> > > I notice that the following 2 sysctl's appear to be set to 1 as
> >soon
> >> > > as the vmm module is loaded
> >> > >
> >> > > hw.vmm.vmx.initialized: 1
> >> > > hw.vmm.vmx.cap.unrestricted_guest: 1
> >> > >
> >> > > Will these be available on both Intel & AMD processors as a way
> >> > > to determine if the module has loaded successfully and can run
> >guests?
> >> > >
> >> > > I also see the below sysctl related to iommu.
> >> > >
> >> > > hw.vmm.iommu.initialized
> >> > >
> >> > > Again, will this be set to 1 as soon as the module is loaded if
> >> > > iommu is supported, or only when it is used?
> >> > > There also seems to be a vmm.amdvi.enable sysctl.
> >> > > Would both these need checking or is vmm.iommu enough to
> >> > > determine support on any processor.
> >> >
> >> > >Probalby the safest way for a shell script to decide if bhyve is
> >> > >up and running is to stat /dev/vmm, if that exists then the
> >modules
> >> > >have loaded and initialized and bhyve should be ready to process
> >guests.
> >> >
> >> > Hmm, I don't get /dev/vmm unless I actually have running guests.
> >>
> >> I'll investigate that, I was pretty sure that you should get this
> >> as soon as the vmm.ko module is finished initialzing, but you might
> >> be right in that it takes a first vm to cause its creation.
> >> Confirmed, /dev/vmm does not exist until the first vm is created.
> >>
> >> >
> >> > >sysctl's mentiond above would be a poor way to make this
> >determination.
> >> >
> >> > It would be nice if sysctls were better documented.
> >>
> >> Agreed.
> >>
> >> > If vmx.initialized is set once vmm has successfully loaded, I
> >> > can't
> >see a better way of checking for bhyve support (assuming it's not
> >Intel specific). This entry definitely exists and is set to 0 if you
> >load the module on a non-supported system, and set to 1 as soon as
> >vmm loads on my Intel test system.
> >>
> >> Given its undocumented status you would be relying on an
> >> undocumented feature that could change in either name or behavior,
> >> and that is not desirable.
> >>
> >> Let me see if I can come up with something else.
> >
> >I looked at the code for bhyvectl, bhyveload and byhve.  They do not
> >actually try to decide if vmm is supported or not, they simply
> >process the error from a vm_create() or vm_open() call and exit with
> >an error code if they can not handle it (some of the code can handle
> >a vm_create failure if infact we are trying to create a vm that
> >already exists).
> >
> >If you want to maintain full compatibility a similiar stratergy may
> >be in order.
> >
> >Why is it that vm-bhyve specifically needs to know if the kernel has
> >vmm support or not?
> >Cant it just be written to handle the errors returned if the
> >suppo

Re: Checking bhyve supported features (sysctls)

[Marcelo Araujo]

2018-08-17 16:25 GMT+08:00 Matt Churchyard :

> -Original Message-
> From: owner-freebsd-virtualizat...@freebsd.org <
> owner-freebsd-virtualizat...@freebsd.org> On Behalf Of Rodney W. Grimes
> Sent: 16 August 2018 18:31
> To: Allan Jude 
> Cc: Matt Churchyard ; freebsd-virtualization@
> freebsd.org
> Subject: Re: Checking bhyve supported features (sysctls)
>
> > On August 16, 2018 5:28:05 PM GMT+01:00, "Rodney W. Grimes" <
> freebsd-...@pdx.rh.cn85.dnsmgr.net> wrote:
> > >>
> > >> Text manually wrapped to 80, any broken quoting is my fault - rwg
> > >>
> > >> > > Hello,
> > >> > >
> > >> > > I'm looking for better ways to check for  bhyve support /
> > >available
> > >> > > features without trying to scan through dmesg output.
> > >> >
> > >> > >Yes, it would be very good to remove that, as it usually tries
> > >> > >to grep a non-existent file /var/run/dmesg.boot that is not
> > >> > >created until after vm_bhyve has been called from
> > >/usr/local/etc/rc.d
> > >> > >when you have things set to autostartup >in /etc/rc.conf
> > >> >
> > >> >
> > >> > >
> > >> > > I notice that the following 2 sysctl's appear to be set to 1 as
> > >soon
> > >> > > as the vmm module is loaded
> > >> > >
> > >> > > hw.vmm.vmx.initialized: 1
> > >> > > hw.vmm.vmx.cap.unrestricted_guest: 1
> > >> > >
> > >> > > Will these be available on both Intel & AMD processors as a way
> > >> > > to determine if the module has loaded successfully and can run
> > >guests?
> > >> > >
> > >> > > I also see the below sysctl related to iommu.
> > >> > >
> > >> > > hw.vmm.iommu.initialized
> > >> > >
> > >> > > Again, will this be set to 1 as soon as the module is loaded if
> > >> > > iommu is supported, or only when it is used?
> > >> > > There also seems to be a vmm.amdvi.enable sysctl.
> > >> > > Would both these need checking or is vmm.iommu enough to
> > >> > > determine support on any processor.
> > >> >
> > >> > >Probalby the safest way for a shell script to decide if bhyve is
> > >> > >up and running is to stat /dev/vmm, if that exists then the
> > >modules
> > >> > >have loaded and initialized and bhyve should be ready to process
> > >guests.
> > >> >
> > >> > Hmm, I don't get /dev/vmm unless I actually have running guests.
> > >>
> > >> I'll investigate that, I was pretty sure that you should get this
> > >> as soon as the vmm.ko module is finished initialzing, but you might
> > >> be right in that it takes a first vm to cause its creation.
> > >> Confirmed, /dev/vmm does not exist until the first vm is created.
> > >>
> > >> >
> > >> > >sysctl's mentiond above would be a poor way to make this
> > >determination.
> > >> >
> > >> > It would be nice if sysctls were better documented.
> > >>
> > >> Agreed.
> > >>
> > >> > If vmx.initialized is set once vmm has successfully loaded, I
> > >> > can't
> > >see a better way of checking for bhyve support (assuming it's not
> > >Intel specific). This entry definitely exists and is set to 0 if you
> > >load the module on a non-supported system, and set to 1 as soon as
> > >vmm loads on my Intel test system.
> > >>
> > >> Given its undocumented status you would be relying on an
> > >> undocumented feature that could change in either name or behavior,
> > >> and that is not desirable.
> > >>
> > >> Let me see if I can come up with something else.
> > >
> > >I looked at the code for bhyvectl, bhyveload and byhve.  They do not
> > >actually try to decide if vmm is supported or not, they simply
> > >process the error from a vm_create() or vm_open() call and exit with
> > >an error code if they can not handle it (some of the code can handle
> > >a vm_create failure if infact we are trying to create a vm that
> > >already exists).
> > >
> > >If you want to maintain full compatibility a similiar stratergy may
> > >be in ord

RE: Checking bhyve supported features (sysctls)

[Matt Churchyard]

-Original Message-
From: owner-freebsd-virtualizat...@freebsd.org 
 On Behalf Of Rodney W. Grimes
Sent: 16 August 2018 18:31
To: Allan Jude 
Cc: Matt Churchyard ; 
freebsd-virtualization@freebsd.org
Subject: Re: Checking bhyve supported features (sysctls)

> On August 16, 2018 5:28:05 PM GMT+01:00, "Rodney W. Grimes" 
>  wrote:
> >> 
> >> Text manually wrapped to 80, any broken quoting is my fault - rwg
> >> 
> >> > > Hello,
> >> > > 
> >> > > I'm looking for better ways to check for  bhyve support /
> >available
> >> > > features without trying to scan through dmesg output.
> >> > 
> >> > >Yes, it would be very good to remove that, as it usually tries 
> >> > >to grep a non-existent file /var/run/dmesg.boot that is not 
> >> > >created until after vm_bhyve has been called from
> >/usr/local/etc/rc.d
> >> > >when you have things set to autostartup >in /etc/rc.conf
> >> > 
> >> > 
> >> > > 
> >> > > I notice that the following 2 sysctl's appear to be set to 1 as
> >soon
> >> > > as the vmm module is loaded
> >> > > 
> >> > > hw.vmm.vmx.initialized: 1
> >> > > hw.vmm.vmx.cap.unrestricted_guest: 1
> >> > > 
> >> > > Will these be available on both Intel & AMD processors as a way 
> >> > > to determine if the module has loaded successfully and can run
> >guests?
> >> > > 
> >> > > I also see the below sysctl related to iommu.
> >> > > 
> >> > > hw.vmm.iommu.initialized
> >> > > 
> >> > > Again, will this be set to 1 as soon as the module is loaded if 
> >> > > iommu is supported, or only when it is used?
> >> > > There also seems to be a vmm.amdvi.enable sysctl.
> >> > > Would both these need checking or is vmm.iommu enough to 
> >> > > determine support on any processor.
> >> > 
> >> > >Probalby the safest way for a shell script to decide if bhyve is 
> >> > >up and running is to stat /dev/vmm, if that exists then the
> >modules
> >> > >have loaded and initialized and bhyve should be ready to process
> >guests.
> >> > 
> >> > Hmm, I don't get /dev/vmm unless I actually have running guests.
> >> 
> >> I'll investigate that, I was pretty sure that you should get this 
> >> as soon as the vmm.ko module is finished initialzing, but you might 
> >> be right in that it takes a first vm to cause its creation.
> >> Confirmed, /dev/vmm does not exist until the first vm is created.
> >> 
> >> > 
> >> > >sysctl's mentiond above would be a poor way to make this
> >determination.
> >> > 
> >> > It would be nice if sysctls were better documented.
> >> 
> >> Agreed.
> >> 
> >> > If vmx.initialized is set once vmm has successfully loaded, I 
> >> > can't
> >see a better way of checking for bhyve support (assuming it's not 
> >Intel specific). This entry definitely exists and is set to 0 if you 
> >load the module on a non-supported system, and set to 1 as soon as 
> >vmm loads on my Intel test system.
> >> 
> >> Given its undocumented status you would be relying on an 
> >> undocumented feature that could change in either name or behavior, 
> >> and that is not desirable.
> >> 
> >> Let me see if I can come up with something else.
> >
> >I looked at the code for bhyvectl, bhyveload and byhve.  They do not 
> >actually try to decide if vmm is supported or not, they simply 
> >process the error from a vm_create() or vm_open() call and exit with 
> >an error code if they can not handle it (some of the code can handle 
> >a vm_create failure if infact we are trying to create a vm that 
> >already exists).
> >
> >If you want to maintain full compatibility a similiar stratergy may 
> >be in order.
> >
> >Why is it that vm-bhyve specifically needs to know if the kernel has 
> >vmm support or not?
> >Cant it just be written to handle the errors returned if the 
> >supported functions do not exist?
> 
> I think the question vm-bhyve wants to answer is: does the CPU have 
> the required features to run a multicore VM.

>Why does it need to know that?  If it tries to start a multicore/thread VM and 
>the system can not support it an error is returned and the bhyve command fails.

>Ac

Re: Checking bhyve supported features (sysctls)

[Matt Churchyard]

On 16 Aug 2018, at 19:55, Marcelo Araujo 
mailto:araujobsdp...@gmail.com>> wrote:



2018-08-17 0:53 GMT+08:00 Allan Jude 
mailto:allanj...@freebsd.org>>:
On August 16, 2018 5:28:05 PM GMT+01:00, "Rodney W. Grimes" 
mailto:freebsd-...@pdx.rh.cn85.dnsmgr.net>> 
wrote:
>>
>> Text manually wrapped to 80, any broken quoting is my fault - rwg
>>
>> > > Hello,
>> > >
>> > > I'm looking for better ways to check for  bhyve support /
>available
>> > > features without trying to scan through dmesg output.
>> >
>> > >Yes, it would be very good to remove that, as it usually tries
>> > >to grep a non-existent file /var/run/dmesg.boot that is not
>> > >created until after vm_bhyve has been called from
>/usr/local/etc/rc.d
>> > >when you have things set to autostartup >in /etc/rc.conf
>> >
>> >
>> > >
>> > > I notice that the following 2 sysctl's appear to be set to 1 as
>soon
>> > > as the vmm module is loaded
>> > >
>> > > hw.vmm.vmx.initialized: 1
>> > > hw.vmm.vmx.cap.unrestricted_guest: 1
>> > >
>> > > Will these be available on both Intel & AMD processors as a way
>> > > to determine if the module has loaded successfully and can run
>guests?
>> > >
>> > > I also see the below sysctl related to iommu.
>> > >
>> > > hw.vmm.iommu.initialized
>> > >
>> > > Again, will this be set to 1 as soon as the module is loaded if
>> > > iommu is supported, or only when it is used?
>> > > There also seems to be a vmm.amdvi.enable sysctl.
>> > > Would both these need checking or is vmm.iommu enough to
>> > > determine support on any processor.
>> >
>> > >Probalby the safest way for a shell script to decide if bhyve is
>> > >up and running is to stat /dev/vmm, if that exists then the
>modules
>> > >have loaded and initialized and bhyve should be ready to process
>guests.
>> >
>> > Hmm, I don't get /dev/vmm unless I actually have running guests.
>>
>> I'll investigate that, I was pretty sure that you should get this
>> as soon as the vmm.ko module is finished initialzing, but you might
>> be right in that it takes a first vm to cause its creation.
>> Confirmed, /dev/vmm does not exist until the first vm
>> is created.
>>
>> >
>> > >sysctl's mentiond above would be a poor way to make this
>determination.
>> >
>> > It would be nice if sysctls were better documented.
>>
>> Agreed.
>>
>> > If vmx.initialized is set once vmm has successfully loaded, I can't
>see a better way of checking for bhyve support (assuming it's not Intel
>specific). This entry definitely exists and is set to 0 if you load the
>module on a non-supported system, and set to 1 as soon as vmm loads on
>my Intel test system.
>>
>> Given its undocumented status you would be relying on an
>> undocumented feature that could change in either name or
>> behavior, and that is not desirable.
>>
>> Let me see if I can come up with something else.
>
>I looked at the code for bhyvectl, bhyveload and
>byhve.  They do not actually try to decide if vmm
>is supported or not, they simply process the error
>from a vm_create() or vm_open() call and exit
>with an error code if they can not handle it
>(some of the code can handle a vm_create failure
>if infact we are trying to create a vm that
>already exists).
>
>If you want to maintain full compatibility a similiar
>stratergy may be in order.
>
>Why is it that vm-bhyve specifically needs to know
>if the kernel has vmm support or not?
>Cant it just be written to handle the errors returned
>if the supported functions do not exist?

I think the question vm-bhyve wants to answer is: does the CPU have the 
required features to run a multicore VM.

These or similar sysctls do seem to be the correct way to communicate that 
support.


You are correct!

The question in case as I understood was about CPU feature supported, actually 
vmm(8) knows all this information! Some examples such like CPU with VMX 
unrestricted mode support (UG) that is necessary for guest VMs running with 
multiple vCPU or like VT-d necessary for PCI device passthrough.

I have a patch that exposes a sysctl saying what bhyve(8) is capable to run, 
however it needs to be polished a bit more to be more informative.
I think for third part software like vm-bhyve these information are crucial as 
these software can get advantage of these information prior to run a certain 
set that will end up in a fail because of a partial CPU support.

Best,


As mentioned in my first email, it does seem like some of these exist already 
in the way of
vmm.vmx.cap.* sysctls.

We could look at bhyve output and try to process that, but that seems more 
messy if there
are sysctls that expose support, especially as the vmm module does seem to know 
what
features the cpu/hardware supports. vm-bhyve has already forked into the 
background
by the time bhyve runs so can't easily provide feedback to the caller other 
than through
the log file.

We do also try and take action in some cases, such as reducing cpu count to 1 
if UG support
isn't found, rather than just having bhyve fail. (Of course you 

Re: Checking bhyve supported features (sysctls)

[Marcelo Araujo]

2018-08-17 0:53 GMT+08:00 Allan Jude :

> On August 16, 2018 5:28:05 PM GMT+01:00, "Rodney W. Grimes" <
> freebsd-...@pdx.rh.cn85.dnsmgr.net> wrote:
> >>
> >> Text manually wrapped to 80, any broken quoting is my fault - rwg
> >>
> >> > > Hello,
> >> > >
> >> > > I'm looking for better ways to check for  bhyve support /
> >available
> >> > > features without trying to scan through dmesg output.
> >> >
> >> > >Yes, it would be very good to remove that, as it usually tries
> >> > >to grep a non-existent file /var/run/dmesg.boot that is not
> >> > >created until after vm_bhyve has been called from
> >/usr/local/etc/rc.d
> >> > >when you have things set to autostartup >in /etc/rc.conf
> >> >
> >> >
> >> > >
> >> > > I notice that the following 2 sysctl's appear to be set to 1 as
> >soon
> >> > > as the vmm module is loaded
> >> > >
> >> > > hw.vmm.vmx.initialized: 1
> >> > > hw.vmm.vmx.cap.unrestricted_guest: 1
> >> > >
> >> > > Will these be available on both Intel & AMD processors as a way
> >> > > to determine if the module has loaded successfully and can run
> >guests?
> >> > >
> >> > > I also see the below sysctl related to iommu.
> >> > >
> >> > > hw.vmm.iommu.initialized
> >> > >
> >> > > Again, will this be set to 1 as soon as the module is loaded if
> >> > > iommu is supported, or only when it is used?
> >> > > There also seems to be a vmm.amdvi.enable sysctl.
> >> > > Would both these need checking or is vmm.iommu enough to
> >> > > determine support on any processor.
> >> >
> >> > >Probalby the safest way for a shell script to decide if bhyve is
> >> > >up and running is to stat /dev/vmm, if that exists then the
> >modules
> >> > >have loaded and initialized and bhyve should be ready to process
> >guests.
> >> >
> >> > Hmm, I don't get /dev/vmm unless I actually have running guests.
> >>
> >> I'll investigate that, I was pretty sure that you should get this
> >> as soon as the vmm.ko module is finished initialzing, but you might
> >> be right in that it takes a first vm to cause its creation.
> >> Confirmed, /dev/vmm does not exist until the first vm
> >> is created.
> >>
> >> >
> >> > >sysctl's mentiond above would be a poor way to make this
> >determination.
> >> >
> >> > It would be nice if sysctls were better documented.
> >>
> >> Agreed.
> >>
> >> > If vmx.initialized is set once vmm has successfully loaded, I can't
> >see a better way of checking for bhyve support (assuming it's not Intel
> >specific). This entry definitely exists and is set to 0 if you load the
> >module on a non-supported system, and set to 1 as soon as vmm loads on
> >my Intel test system.
> >>
> >> Given its undocumented status you would be relying on an
> >> undocumented feature that could change in either name or
> >> behavior, and that is not desirable.
> >>
> >> Let me see if I can come up with something else.
> >
> >I looked at the code for bhyvectl, bhyveload and
> >byhve.  They do not actually try to decide if vmm
> >is supported or not, they simply process the error
> >from a vm_create() or vm_open() call and exit
> >with an error code if they can not handle it
> >(some of the code can handle a vm_create failure
> >if infact we are trying to create a vm that
> >already exists).
> >
> >If you want to maintain full compatibility a similiar
> >stratergy may be in order.
> >
> >Why is it that vm-bhyve specifically needs to know
> >if the kernel has vmm support or not?
> >Cant it just be written to handle the errors returned
> >if the supported functions do not exist?
>
> I think the question vm-bhyve wants to answer is: does the CPU have the
> required features to run a multicore VM.
>
> These or similar sysctls do seem to be the correct way to communicate that
> support.
>


You are correct!

The question in case as I understood was about CPU feature supported,
actually vmm(8) knows all this information! Some examples such like CPU
with VMX unrestricted mode support (UG) that is necessary for guest VMs
running with multiple vCPU or like VT-d necessary for PCI device
passthrough.

I have a patch that exposes a sysctl saying what bhyve(8) is capable to
run, however it needs to be polished a bit more to be more informative.
I think for third part software like vm-bhyve these information are crucial
as these software can get advantage of these information prior to run a
certain set that will end up in a fail because of a partial CPU support.

Best,



> --
> Allan Jude
> ___
> freebsd-virtualization@freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> To unsubscribe, send any mail to "freebsd-virtualization-
> unsubscr...@freebsd.org"
>



-- 

-- 
Marcelo Araujo(__)ara...@freebsd.org
\\\'',)http://www.FreeBSD.org    \/  \ ^
Power To Server. .\. /_)
___
freebsd-virtualization@freebsd.org mailing list

Re: Checking bhyve supported features (sysctls)

[Rodney W. Grimes]

> On August 16, 2018 5:28:05 PM GMT+01:00, "Rodney W. Grimes" 
>  wrote:
> >> 
> >> Text manually wrapped to 80, any broken quoting is my fault - rwg
> >> 
> >> > > Hello,
> >> > > 
> >> > > I'm looking for better ways to check for  bhyve support /
> >available
> >> > > features without trying to scan through dmesg output.
> >> > 
> >> > >Yes, it would be very good to remove that, as it usually tries
> >> > >to grep a non-existent file /var/run/dmesg.boot that is not
> >> > >created until after vm_bhyve has been called from
> >/usr/local/etc/rc.d
> >> > >when you have things set to autostartup >in /etc/rc.conf
> >> > 
> >> > 
> >> > > 
> >> > > I notice that the following 2 sysctl's appear to be set to 1 as
> >soon 
> >> > > as the vmm module is loaded
> >> > > 
> >> > > hw.vmm.vmx.initialized: 1
> >> > > hw.vmm.vmx.cap.unrestricted_guest: 1
> >> > > 
> >> > > Will these be available on both Intel & AMD processors as a way
> >> > > to determine if the module has loaded successfully and can run
> >guests?
> >> > > 
> >> > > I also see the below sysctl related to iommu.
> >> > > 
> >> > > hw.vmm.iommu.initialized
> >> > > 
> >> > > Again, will this be set to 1 as soon as the module is loaded if
> >> > > iommu is supported, or only when it is used?
> >> > > There also seems to be a vmm.amdvi.enable sysctl.
> >> > > Would both these need checking or is vmm.iommu enough to
> >> > > determine support on any processor.
> >> > 
> >> > >Probalby the safest way for a shell script to decide if bhyve is
> >> > >up and running is to stat /dev/vmm, if that exists then the
> >modules
> >> > >have loaded and initialized and bhyve should be ready to process
> >guests.
> >> > 
> >> > Hmm, I don't get /dev/vmm unless I actually have running guests.
> >> 
> >> I'll investigate that, I was pretty sure that you should get this
> >> as soon as the vmm.ko module is finished initialzing, but you might
> >> be right in that it takes a first vm to cause its creation.
> >> Confirmed, /dev/vmm does not exist until the first vm
> >> is created.
> >> 
> >> > 
> >> > >sysctl's mentiond above would be a poor way to make this
> >determination.
> >> > 
> >> > It would be nice if sysctls were better documented.
> >> 
> >> Agreed.
> >> 
> >> > If vmx.initialized is set once vmm has successfully loaded, I can't
> >see a better way of checking for bhyve support (assuming it's not Intel
> >specific). This entry definitely exists and is set to 0 if you load the
> >module on a non-supported system, and set to 1 as soon as vmm loads on
> >my Intel test system.
> >> 
> >> Given its undocumented status you would be relying on an
> >> undocumented feature that could change in either name or
> >> behavior, and that is not desirable.
> >> 
> >> Let me see if I can come up with something else.
> >
> >I looked at the code for bhyvectl, bhyveload and
> >byhve.  They do not actually try to decide if vmm
> >is supported or not, they simply process the error
> >from a vm_create() or vm_open() call and exit
> >with an error code if they can not handle it
> >(some of the code can handle a vm_create failure
> >if infact we are trying to create a vm that
> >already exists).
> >
> >If you want to maintain full compatibility a similiar
> >stratergy may be in order.
> >
> >Why is it that vm-bhyve specifically needs to know
> >if the kernel has vmm support or not?
> >Cant it just be written to handle the errors returned
> >if the supported functions do not exist?
> 
> I think the question vm-bhyve wants to answer is: does
> the CPU have the required features to run a multicore VM.

Why does it need to know that?  If it tries to start a multicore/thread
VM and the system can not support it an error is returned and the bhyve
command fails.

Actually determining that specific issue is non-trivial iirc as
some vmm supported CPU's can only run a single VM with a single
thread in that VM (early core cpu's).

> 
> These or similar sysctls do seem to be the correct way
> to communicate that support.

I do not believe any of those sysctl's communicate that
your on a broken cpu or to what extent you can run vm's
with multiple threads.


I went and looked at why vm-bhyve is groveling around
in /var/run/dmesg.boot and found that it is simply
trying to determine if the host CPU is vmm capable,
specifically:
util::check_bhyve_support(){
...
# get features2 line which should include popcnt
_mesg=$(grep -E '^[ ]+Features2' /var/run/dmesg.boot | tail -n 1)

# only check if we found it
if [ -n "${_mesg}" ]; then

# look for pop cnt
_result=$(echo "${_mesg}" |grep "POPCNT")
[ -z "${_result}" ] && util::err "it doesn't look like your cpu 
supports bhyve (missing POPCNT)"
fi

# check ept for intel
_mesg=$(grep -E '^[ ]+VT-x' /var/run/dmesg.boot | tail -n 1)

if [ -n "${_mesg}" ]; then

# look for ept
_result=$(echo "${_mesg}" |grep "EPT")
[ -z "${_result}" ] && util::err "it doesn't look like your cpu 

Re: Checking bhyve supported features (sysctls)

[Allan Jude]

On August 16, 2018 5:28:05 PM GMT+01:00, "Rodney W. Grimes" 
 wrote:
>> 
>> Text manually wrapped to 80, any broken quoting is my fault - rwg
>> 
>> > > Hello,
>> > > 
>> > > I'm looking for better ways to check for  bhyve support /
>available
>> > > features without trying to scan through dmesg output.
>> > 
>> > >Yes, it would be very good to remove that, as it usually tries
>> > >to grep a non-existent file /var/run/dmesg.boot that is not
>> > >created until after vm_bhyve has been called from
>/usr/local/etc/rc.d
>> > >when you have things set to autostartup >in /etc/rc.conf
>> > 
>> > 
>> > > 
>> > > I notice that the following 2 sysctl's appear to be set to 1 as
>soon 
>> > > as the vmm module is loaded
>> > > 
>> > > hw.vmm.vmx.initialized: 1
>> > > hw.vmm.vmx.cap.unrestricted_guest: 1
>> > > 
>> > > Will these be available on both Intel & AMD processors as a way
>> > > to determine if the module has loaded successfully and can run
>guests?
>> > > 
>> > > I also see the below sysctl related to iommu.
>> > > 
>> > > hw.vmm.iommu.initialized
>> > > 
>> > > Again, will this be set to 1 as soon as the module is loaded if
>> > > iommu is supported, or only when it is used?
>> > > There also seems to be a vmm.amdvi.enable sysctl.
>> > > Would both these need checking or is vmm.iommu enough to
>> > > determine support on any processor.
>> > 
>> > >Probalby the safest way for a shell script to decide if bhyve is
>> > >up and running is to stat /dev/vmm, if that exists then the
>modules
>> > >have loaded and initialized and bhyve should be ready to process
>guests.
>> > 
>> > Hmm, I don't get /dev/vmm unless I actually have running guests.
>> 
>> I'll investigate that, I was pretty sure that you should get this
>> as soon as the vmm.ko module is finished initialzing, but you might
>> be right in that it takes a first vm to cause its creation.
>> Confirmed, /dev/vmm does not exist until the first vm
>> is created.
>> 
>> > 
>> > >sysctl's mentiond above would be a poor way to make this
>determination.
>> > 
>> > It would be nice if sysctls were better documented.
>> 
>> Agreed.
>> 
>> > If vmx.initialized is set once vmm has successfully loaded, I can't
>see a better way of checking for bhyve support (assuming it's not Intel
>specific). This entry definitely exists and is set to 0 if you load the
>module on a non-supported system, and set to 1 as soon as vmm loads on
>my Intel test system.
>> 
>> Given its undocumented status you would be relying on an
>> undocumented feature that could change in either name or
>> behavior, and that is not desirable.
>> 
>> Let me see if I can come up with something else.
>
>I looked at the code for bhyvectl, bhyveload and
>byhve.  They do not actually try to decide if vmm
>is supported or not, they simply process the error
>from a vm_create() or vm_open() call and exit
>with an error code if they can not handle it
>(some of the code can handle a vm_create failure
>if infact we are trying to create a vm that
>already exists).
>
>If you want to maintain full compatibility a similiar
>stratergy may be in order.
>
>Why is it that vm-bhyve specifically needs to know
>if the kernel has vmm support or not?
>Cant it just be written to handle the errors returned
>if the supported functions do not exist?

I think the question vm-bhyve wants to answer is: does the CPU have the 
required features to run a multicore VM.

These or similar sysctls do seem to be the correct way to communicate that 
support.
-- 
Allan Jude
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: Checking bhyve supported features (sysctls)

[Rodney W. Grimes]

> 
> Text manually wrapped to 80, any broken quoting is my fault - rwg
> 
> > > Hello,
> > > 
> > > I'm looking for better ways to check for  bhyve support / available
> > > features without trying to scan through dmesg output.
> > 
> > >Yes, it would be very good to remove that, as it usually tries
> > >to grep a non-existent file /var/run/dmesg.boot that is not
> > >created until after vm_bhyve has been called from /usr/local/etc/rc.d
> > >when you have things set to autostartup >in /etc/rc.conf
> > 
> > 
> > > 
> > > I notice that the following 2 sysctl's appear to be set to 1 as soon 
> > > as the vmm module is loaded
> > > 
> > > hw.vmm.vmx.initialized: 1
> > > hw.vmm.vmx.cap.unrestricted_guest: 1
> > > 
> > > Will these be available on both Intel & AMD processors as a way
> > > to determine if the module has loaded successfully and can run guests?
> > > 
> > > I also see the below sysctl related to iommu.
> > > 
> > > hw.vmm.iommu.initialized
> > > 
> > > Again, will this be set to 1 as soon as the module is loaded if
> > > iommu is supported, or only when it is used?
> > > There also seems to be a vmm.amdvi.enable sysctl.
> > > Would both these need checking or is vmm.iommu enough to
> > > determine support on any processor.
> > 
> > >Probalby the safest way for a shell script to decide if bhyve is
> > >up and running is to stat /dev/vmm, if that exists then the modules
> > >have loaded and initialized and bhyve should be ready to process guests.
> > 
> > Hmm, I don't get /dev/vmm unless I actually have running guests.
> 
> I'll investigate that, I was pretty sure that you should get this
> as soon as the vmm.ko module is finished initialzing, but you might
> be right in that it takes a first vm to cause its creation.
> Confirmed, /dev/vmm does not exist until the first vm
> is created.
> 
> > 
> > >sysctl's mentiond above would be a poor way to make this determination.
> > 
> > It would be nice if sysctls were better documented.
> 
> Agreed.
> 
> > If vmx.initialized is set once vmm has successfully loaded, I can't see a 
> > better way of checking for bhyve support (assuming it's not Intel 
> > specific). This entry definitely exists and is set to 0 if you load the 
> > module on a non-supported system, and set to 1 as soon as vmm loads on my 
> > Intel test system.
> 
> Given its undocumented status you would be relying on an
> undocumented feature that could change in either name or
> behavior, and that is not desirable.
> 
> Let me see if I can come up with something else.

I looked at the code for bhyvectl, bhyveload and
byhve.  They do not actually try to decide if vmm
is supported or not, they simply process the error
from a vm_create() or vm_open() call and exit
with an error code if they can not handle it
(some of the code can handle a vm_create failure
if infact we are trying to create a vm that
already exists).

If you want to maintain full compatibility a similiar
stratergy may be in order.

Why is it that vm-bhyve specifically needs to know
if the kernel has vmm support or not?
Cant it just be written to handle the errors returned
if the supported functions do not exist?


-- 
Rod Grimes rgri...@freebsd.org
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: Checking bhyve supported features (sysctls)

[Rodney W. Grimes]

Text manually wrapped to 80, any broken quoting is my fault - rwg

> > Hello,
> > 
> > I'm looking for better ways to check for  bhyve support / available
> > features without trying to scan through dmesg output.
> 
> >Yes, it would be very good to remove that, as it usually tries
> >to grep a non-existent file /var/run/dmesg.boot that is not
> >created until after vm_bhyve has been called from /usr/local/etc/rc.d
> >when you have things set to autostartup >in /etc/rc.conf
> 
> 
> > 
> > I notice that the following 2 sysctl's appear to be set to 1 as soon 
> > as the vmm module is loaded
> > 
> > hw.vmm.vmx.initialized: 1
> > hw.vmm.vmx.cap.unrestricted_guest: 1
> > 
> > Will these be available on both Intel & AMD processors as a way
> > to determine if the module has loaded successfully and can run guests?
> > 
> > I also see the below sysctl related to iommu.
> > 
> > hw.vmm.iommu.initialized
> > 
> > Again, will this be set to 1 as soon as the module is loaded if
> > iommu is supported, or only when it is used?
> > There also seems to be a vmm.amdvi.enable sysctl.
> > Would both these need checking or is vmm.iommu enough to
> > determine support on any processor.
> 
> >Probalby the safest way for a shell script to decide if bhyve is
> >up and running is to stat /dev/vmm, if that exists then the modules
> >have loaded and initialized and bhyve should be ready to process guests.
> 
> Hmm, I don't get /dev/vmm unless I actually have running guests.

I'll investigate that, I was pretty sure that you should get this
as soon as the vmm.ko module is finished initialzing, but you might
be right in that it takes a first vm to cause its creation.
Confirmed, /dev/vmm does not exist until the first vm
is created.

> 
> >sysctl's mentiond above would be a poor way to make this determination.
> 
> It would be nice if sysctls were better documented.

Agreed.

> If vmx.initialized is set once vmm has successfully loaded, I can't see a 
> better way of checking for bhyve support (assuming it's not Intel specific). 
> This entry definitely exists and is set to 0 if you load the module on a 
> non-supported system, and set to 1 as soon as vmm loads on my Intel test 
> system.

Given its undocumented status you would be relying on an
undocumented feature that could change in either name or
behavior, and that is not desirable.

Let me see if I can come up with something else.


-- 
Rod Grimes rgri...@freebsd.org
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: Checking bhyve supported features (sysctls)

[Fabian Freyer]

On 16 Aug 2018, at 12:16, Matt Churchyard wrote:
I'm looking for better ways to check for bhyve support / available 
features without trying to scan through dmesg output.


There’s a few patches floating around for checking features [1,2] - 
however none of these are yet committed.

For [2], there’s also an RFC mailing list thread [3].

CC’ing novel@, as he wrote those patches.

The hackish way libvirt checks for features right now is to parse error 
messages from invalid invocations of bhyve(8).


[1] https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=210111
[2] https://reviews.freebsd.org/D15992
[3] 
https://lists.freebsd.org/pipermail/freebsd-virtualization/2018-June/006536.html

___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

RE: Checking bhyve supported features (sysctls)

[Matt Churchyard]

> Hello,
> 
> I'm looking for better ways to check for bhyve support / available features 
> without trying to scan through dmesg output.

>Yes, it would be very good to remove that, as it usually tries to grep a 
>non-existent file /var/run/dmesg.boot that is not created until after vm_bhyve 
>has been called from /usr/local/etc/rc.d when you have things set to 
>autostartup >in /etc/rc.conf


> 
> I notice that the following 2 sysctl's appear to be set to 1 as soon 
> as the vmm module is loaded
> 
> hw.vmm.vmx.initialized: 1
> hw.vmm.vmx.cap.unrestricted_guest: 1
> 
> Will these be available on both Intel & AMD processors as a way to determine 
> if the module has loaded successfully and can run guests?
> 
> I also see the below sysctl related to iommu.
> 
> hw.vmm.iommu.initialized
> 
> Again, will this be set to 1 as soon as the module is loaded if iommu is 
> supported, or only when it is used?
> There also seems to be a vmm.amdvi.enable sysctl. Would both these need 
> checking or is vmm.iommu enough to determine support on any processor.

>Probalby the safest way for a shell script to decide if bhyve is up and 
>running is to stat /dev/vmm, if that exists then the modules have loaded and 
>initialized and bhyve should be ready to process guests.

Hmm, I don't get /dev/vmm unless I actually have running guests.

>sysctl's mentiond above would be a poor way to make this determination.

It would be nice if sysctls were better documented. If vmx.initialized is set 
once vmm has successfully loaded, I can't see a better way of checking for 
bhyve support (assuming it's not Intel specific). This entry definitely exists 
and is set to 0 if you load the module on a non-supported system, and set to 1 
as soon as vmm loads on my Intel test system.

Matt

-- 
Rod Grimes rgri...@freebsd.org
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: Checking bhyve supported features (sysctls)

[Rodney W. Grimes]

> Hello,
> 
> I'm looking for better ways to check for bhyve support / available features 
> without trying to scan through dmesg output.

Yes, it would be very good to remove that,
as it usually tries to grep a non-existent file /var/run/dmesg.boot
that is not created until after vm_bhyve has been
called from /usr/local/etc/rc.d when you have things
set to autostartup in /etc/rc.conf


> 
> I notice that the following 2 sysctl's appear to be set to 1 as soon as the 
> vmm module is loaded
> 
> hw.vmm.vmx.initialized: 1
> hw.vmm.vmx.cap.unrestricted_guest: 1
> 
> Will these be available on both Intel & AMD processors as a way to determine 
> if the module has loaded successfully and can run guests?
> 
> I also see the below sysctl related to iommu.
> 
> hw.vmm.iommu.initialized
> 
> Again, will this be set to 1 as soon as the module is loaded if iommu is 
> supported, or only when it is used?
> There also seems to be a vmm.amdvi.enable sysctl. Would both these need 
> checking or is vmm.iommu enough to determine support on any processor.

Probalby the safest way for a shell script to decide if
bhyve is up and running is to stat /dev/vmm, if that
exists then the modules have loaded and initialized
and bhyve should be ready to process guests.

sysctl's mentiond above would be a poor way to make
this determination.


-- 
Rod Grimes rgri...@freebsd.org
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Checking bhyve supported features (sysctls)

[Matt Churchyard]

Hello,

I'm looking for better ways to check for bhyve support / available features 
without trying to scan through dmesg output.

I notice that the following 2 sysctl's appear to be set to 1 as soon as the vmm 
module is loaded

hw.vmm.vmx.initialized: 1
hw.vmm.vmx.cap.unrestricted_guest: 1

Will these be available on both Intel & AMD processors as a way to determine if 
the module has loaded successfully and can run guests?

I also see the below sysctl related to iommu.

hw.vmm.iommu.initialized

Again, will this be set to 1 as soon as the module is loaded if iommu is 
supported, or only when it is used?
There also seems to be a vmm.amdvi.enable sysctl. Would both these need 
checking or is vmm.iommu enough to determine support on any processor.

Matt
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"