Re: Proposal for better support of hypervisors and their synthetic drivers at boot-time

2013-05-29 Thread Larry Melia
Hi Alexander,

Thanks for your help. I've been working on getting the dummy driver to load
at boot time, but having some issues. After rereading your comments,
however, I'm wondering if I'm overriding the correct driver. I've been
trying to override the pci-ata driver, but should I be PCI bus driver one
level up (pci.c)? I suspect I'm trying to override the wrong driver.



On Tue, Apr 23, 2013 at 10:48 AM, Alexander Motin m...@freebsd.org wrote:

 On 23.04.2013 17:07, Larry Melia wrote:

 I’m on a team of developers working on synthetic device drivers for
 FreeBSD on Hyper-V. In late March, I briefly mentioned (on this list)
 that we're trying to get our drivers included with the FreeBSD
 distribution. Also noted were some necessary patches to the ATA driver.
 The changes are necessary to achieve significant performance gains by
 replacing the native ATA driver with our synthetic storage driver
 when when Hyper-V is detected. Alexander Motin, the maintainer of the
 ATA code-base, however, expressed some concerns about making these
 modifications that are so specific to Hyper-V and the AMD64 build. We
 understand his concerns and have subsequently removed these patches from
 our vendor branch. Our drivers are now completely independent and
 require no changes to the kernel or other drivers. The only touch-point
 is the common makefile for modules/drivers.

 Removing our ATA patches, on the other hand, results in a huge
 performance loss. This is because the root file system is managed by the
 ATA driver, which is emulated under Hyper-V. Furthermore, there may be
 other native drivers that we wish to replace with synthetic ones at boot
 time (e.g., mouse driver, video driver), but, there appears to be no
 easy way to do this.Therefore, we would like to work with the developer
 community to come up with better solution that would improve support for
 synthetic drivers at boot-time.

 One possible solution:

 (1) Move the call to init_param1() (in sys/kern/subr_parm.c), which is
 used for hypervisor detection, to an earlier point in the boot process.
 Presently, it appears to be called after the ATA driver is selected,
 which is too late in the boot process. (This was discovered after some
 testing with the ATA driver.) Therefore, before the bus drivers and
 native controllers are detected and selected, discovery of a host
 hypervisor should be done first.

 (2) Extend the boot process to dynamically insert and prioritize
 synthetic drivers over native ones. Presently, FreeBSD supports the
 selection of generic drivers for a given device, but these can be
 overridden when a more specific driver is available. This priority
 scheme could be extended by first detecting the presence of a
 hypervisor, then allowing this priority mechanism to override the native
 drivers with their synthetic cousins (i.e., we only want to override a
 native driver when a specific hypervisor is detected and synthetic
 drivers for it have been configured). This could be arranged in a
 separate, table-driven, input file or perhaps, by extending the existing
 driver/module configuration file with additional mappings of native to
 synthetic drivers.

 (3) Upgrade the init_param1() function (in sys/kern/subr_parm.c) to use
 the more recent approach to hypervisor detection. This approach uses the
 CPU-identify functions to retrieve a unique signature consisting of a
 fixed string of ASCII characters. This was done on Linux about five
 years. For backward compatibility, however, the existing logic would be
 retained, but augmented with this new approach. It would also be
 conditionally added only for x86/AMD64 builds.

 When reviewing the changes we're not looking at a lot of lines of
 code.The difficultly lies, however, in where the changes are made.
 Obviously, we need to work closely with the committers responsible for
 the kernel modules we're looking to touch. Retrospectively, these
 upgrades only bring FreeBSD up to the same level of hypervisor support
 already present in Linux.There are other issues that we would also like
 to discuss, but the improvements described here are on the top of the
 list.

 We look forward to working more closely with the FreeBSD community and
 welcome your remarks and feedback on this proposal.


 I am sorry, I don't really understand what is your proposition, so I just
 describe my vision of the solution here, hoping it will be useful.

 You are right that FreeBSD has priority mechanism to select best driver
 for each device. That mechanism should be sufficient to prevent default OS
 driver from attaching to emulated ATA controller when Hyper-V environment
 is detected and paravirtual drivers are installed. To do that you should
 just create a very small dummy driver for PCI bus, implementing single
 probe() method. That probe() method should in your preferable way check
 that Hyper-V environment is active and match PCI ID of the device against
 the list of used by Hyper-V for emulated ATA controller. 

Re: [Xen-devel] FreeBSD PVHVM call for testing

2013-05-29 Thread Matt Wilson
On Wed, May 29, 2013 at 07:03:40PM +0200, Roger Pau Monné wrote:
 
 Hello,
 
 Thanks Matt and Colin for the testing and help! I've pushed yet another
 version, now it's branch pvhvm_v12, which I *think* should solve the
 issues with cpuid != acpi_id:
 
 http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=shortlog;h=refs/heads/pvhvm_v12
 
 Since I'm not able to reproduce the cpuid != acpi_id case, could you
 give it a try and report the results?

Colin, can you build an AMI with this new kernel?
 
[...]

 On 28/05/13 23:33, Colin Percival wrote:
  On a cc2.8xlarge EC2 instance, the lines which come after this are
  GEOM: new disk xbd1
  GEOM: new disk xbd2
  GEOM: new disk xbd3
  GEOM: new disk xbd4
  Trying to mount root from ufs:/dev/ad0a [rw]...
  start_init: trying /sbin/init
  and then the userland boot process; have you made any bug fixes after
  your pvhvm_v7 which would explain why tasting disks was hanging?
 
 I'm not sure I follow, did you found a regression from previous
 branches? i.e. it used to work with branch pvhvm_v6 and not pvhvm_v7?

Colin was saying that his local change only moved the boot process a
bit farther for cr1.8xlarge. Perhaps some of the other changes you
made in the latest pvhvm_v12 branch will get the VM all the way up.

--msw
___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org


Re: [Xen-devel] FreeBSD PVHVM call for testing

2013-05-29 Thread Colin Percival
On 05/29/13 10:45, Roger Pau Monné wrote:
 Oh, sure, more changes where needed in order to get it to work, like
 using acpi_id to map the vcpu_info and perform the cpu bindings.

Ah, that explains it.  I looked at timer.c for other places where that
change was needed but it didn't occur to me that the same problem would
exist in other parts of the tree.

 On 29/05/13 19:22, Matt Wilson wrote:
 On Wed, May 29, 2013 at 07:03:40PM +0200, Roger Pau Monné wrote:
 http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=shortlog;h=refs/heads/pvhvm_v12

 Colin, can you build an AMI with this new kernel?

Done, ami-95177dfc in us-east-1.

This is now booting successfully on cr1.8xlarge; are there any other instance
types I should test?  I don't know which Xen versions you have deployed across
the entire fleet.

-- 
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid

___
freebsd-virtualization@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
freebsd-virtualization-unsubscr...@freebsd.org