2018-08-17 16:25 GMT+08:00 Matt Churchyard <[email protected]<mailto:[email protected]>>: -----Original Message----- From: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> On Behalf Of Rodney W. Grimes Sent: 16 August 2018 18:31 To: Allan Jude <[email protected]<mailto:[email protected]>> Cc: Matt Churchyard <[email protected]<mailto:[email protected]>>; [email protected]<mailto:[email protected]> Subject: Re: Checking bhyve supported features (sysctls)
> On August 16, 2018 5:28:05 PM GMT+01:00, "Rodney W. Grimes" > <[email protected]<mailto:[email protected]>> > wrote: > >> > >> Text manually wrapped to 80, any broken quoting is my fault - rwg > >> > >> > > Hello, > >> > > > >> > > I'm looking for better ways to check for bhyve support / > >available > >> > > features without trying to scan through dmesg output. > >> > > >> > >Yes, it would be very good to remove that, as it usually tries > >> > >to grep a non-existent file /var/run/dmesg.boot that is not > >> > >created until after vm_bhyve has been called from > >/usr/local/etc/rc.d > >> > >when you have things set to autostartup >in /etc/rc.conf > >> > > >> > > >> > > > >> > > I notice that the following 2 sysctl's appear to be set to 1 as > >soon > >> > > as the vmm module is loaded > >> > > > >> > > hw.vmm.vmx.initialized: 1 > >> > > hw.vmm.vmx.cap.unrestricted_guest: 1 > >> > > > >> > > Will these be available on both Intel & AMD processors as a way > >> > > to determine if the module has loaded successfully and can run > >guests? > >> > > > >> > > I also see the below sysctl related to iommu. > >> > > > >> > > hw.vmm.iommu.initialized > >> > > > >> > > Again, will this be set to 1 as soon as the module is loaded if > >> > > iommu is supported, or only when it is used? > >> > > There also seems to be a vmm.amdvi.enable sysctl. > >> > > Would both these need checking or is vmm.iommu enough to > >> > > determine support on any processor. > >> > > >> > >Probalby the safest way for a shell script to decide if bhyve is > >> > >up and running is to stat /dev/vmm, if that exists then the > >modules > >> > >have loaded and initialized and bhyve should be ready to process > >guests. > >> > > >> > Hmm, I don't get /dev/vmm unless I actually have running guests. > >> > >> I'll investigate that, I was pretty sure that you should get this > >> as soon as the vmm.ko module is finished initialzing, but you might > >> be right in that it takes a first vm to cause its creation. > >> Confirmed, /dev/vmm does not exist until the first vm is created. > >> > >> > > >> > >sysctl's mentiond above would be a poor way to make this > >determination. > >> > > >> > It would be nice if sysctls were better documented. > >> > >> Agreed. > >> > >> > If vmx.initialized is set once vmm has successfully loaded, I > >> > can't > >see a better way of checking for bhyve support (assuming it's not > >Intel specific). This entry definitely exists and is set to 0 if you > >load the module on a non-supported system, and set to 1 as soon as > >vmm loads on my Intel test system. > >> > >> Given its undocumented status you would be relying on an > >> undocumented feature that could change in either name or behavior, > >> and that is not desirable. > >> > >> Let me see if I can come up with something else. > > > >I looked at the code for bhyvectl, bhyveload and byhve. They do not > >actually try to decide if vmm is supported or not, they simply > >process the error from a vm_create() or vm_open() call and exit with > >an error code if they can not handle it (some of the code can handle > >a vm_create failure if infact we are trying to create a vm that > >already exists). > > > >If you want to maintain full compatibility a similiar stratergy may > >be in order. > > > >Why is it that vm-bhyve specifically needs to know if the kernel has > >vmm support or not? > >Cant it just be written to handle the errors returned if the > >supported functions do not exist? > > I think the question vm-bhyve wants to answer is: does the CPU have > the required features to run a multicore VM. >Why does it need to know that? If it tries to start a multicore/thread VM and >the system can not support it an error is returned and the bhyve command fails. >Actually determining that specific issue is non-trivial iirc as some vmm >supported CPU's can only run a single VM with a single thread in that VM >(early core cpu's). > > These or similar sysctls do seem to be the correct way to communicate > that support. >I do not believe any of those sysctl's communicate that your on a broken cpu >or to what extent you can run vm's with multiple threads. So cap.unrestricted_guest from the vmm "capabilities" set of sysctls is not a valid way to determine if the host has unrestricted guest support (required for non-freebsd or multicore freebsd guests, and as you say missing from some early VT-x capable processors)? >I went and looked at why vm-bhyve is groveling around in /var/run/dmesg.boot >and found that it is simply trying to determine if the host CPU is vmm capable, >specifically: >util::check_bhyve_support(){ >... >These checks are already built into the kernel. >This can all go in the bit bucket, if you try to start a VM on an unsupported >system an error is returned, recoding this in shell is just setting yourself >up for "future" bugs. The kernel knows what features are supported but does not expose these, so all I can do is similar to libvirt and run bhyve with different options to see what errors pop up. I think I'll just remove all checking for now and let users discover the issue for themselves if bhyve won't run. Hopefully the vmx.initialized / cap.* sysctls will at some point become a defined way of seeing if vmm is ready / testing for vmm features, as apparently these serve no purpose at the moment. Matt _______________________________________________ [email protected]<mailto:[email protected]> mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "[email protected]<mailto:[email protected]>" >This is something that is in my long to-do list, I will try to get back to >this in couple weeks. >I think the way how you are dealing with it nowadays is the best way to try to >discover the CPU capabilities, better in this way than let the users blind. >Best, Thanks Marcelo, It’s not exactly critical. I just saw another project using vmx.initialized and it seemed a much cleaner way to determine support by seeing if vmm loads rather than probing log files. I asked here expecting someone to just say “yes if that’s 1 then the host will run guests”, or “no you can’t tell if guests will run just by that sysctl”. Matt _______________________________________________ [email protected] mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "[email protected]"
