Directly reserve an interrupt IDT entry for Hyper-V
Hello, Sending to Xen, drivers and virtualization mailing lists since this might be of interest to the folks on these aliases. I am working for Microsoft to improve the performance of FreeBSD running on Hyper-V. Right now I am adding a feature in the vmbus driver which could handle the host-guest channel communications on all vCPUs simultaneously. In order to achieve this, the hypervisor will send same interrupt concurrently on all the vCPUs. The traditional way on FreeBSD to set up interrupt handling for devicse, such as calling bus_alloc_resource() to reserve an IRQ line, and then calling bus_setup_intr() to create a vector, doesn't seem to work in this case. It seems if the interrupt is routed via legacy IRQ, it can only be active on one vCPU at a time. In order to allow the same interrupt to be handled on all vCPUs concurrently, all I need is an IDT entry, not an IRQ line. I checked current FreeBSD code. It looks to me Xen directly uses the vector number IDT_EVTCHN (0x93) to achieve the same purpose. I am proposing both Xen and Hyper-V share this same vector. Following is a little bit detail of my proposal for the changes in the current kernel. 1. In machdep.c: #ifdef XENHVM setidt(IDT_EVTCHN, IDTVEC(xen_intr_upcall), SDT_SYSIGT, SEL_UPL, 0); #else setidt(IDT_EVTCHN, IDTVEC(hv_vmbus_intr), SDT_SYSIGT, SEL_UPL, 0); #endif 2. Apic_vector.S Add IDTVEC(hv_vmbus_intr) to call Hyper-V vmbus interrupt service routine. Any thoughts, objections and feedbacks are all welcome. Thanks, Wei Hu Open Source Technology Center Microsoft China ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to freebsd-virtualization-unsubscr...@freebsd.org
Re: Directly reserve an interrupt IDT entry for Hyper-V
On 20/08/14 11:19, Wei Hu wrote: Hello, Sending to Xen, drivers and virtualization mailing lists since this might be of interest to the folks on these aliases. I am working for Microsoft to improve the performance of FreeBSD running on Hyper-V. Right now I am adding a feature in the vmbus driver which could handle the host-guest channel communications on all vCPUs simultaneously. In order to achieve this, the hypervisor will send same interrupt concurrently on all the vCPUs. The traditional way on FreeBSD to set up interrupt handling for devicse, such as calling bus_alloc_resource() to reserve an IRQ line, and then calling bus_setup_intr() to create a vector, doesn't seem to work in this case. It seems if the interrupt is routed via legacy IRQ, it can only be active on one vCPU at a time. In order to allow the same interrupt to be handled on all vCPUs concurrently, all I need is an IDT entry, not an IRQ line. I checked current FreeBSD code. It looks to me Xen directly uses the vector number IDT_EVTCHN (0x93) to achieve the same purpose. I am proposing both Xen and Hyper-V share this same vector. Following is a little bit detail of my proposal for the changes in the current kernel. 1. In machdep.c: #ifdef XENHVM setidt(IDT_EVTCHN, IDTVEC(xen_intr_upcall), SDT_SYSIGT, SEL_UPL, 0); #else setidt(IDT_EVTCHN, IDTVEC(hv_vmbus_intr), SDT_SYSIGT, SEL_UPL, 0); #endif 2. Apic_vector.S Add IDTVEC(hv_vmbus_intr) to call Hyper-V vmbus interrupt service routine. Any thoughts, objections and feedbacks are all welcome. Hello, I don't think using the same IDT vector is the right approach, I would just pick a different IDT vector and use that for Hyper-V. Using the same IDT vector (like your suggestion above) would prevent shipping a kernel with with both Hyper-V and Xen support (like it's done now in GENERIC). Roger. ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to freebsd-virtualization-unsubscr...@freebsd.org
Re: Directly reserve an interrupt IDT entry for Hyper-V
On Wednesday, August 20, 2014 9:31:54 am Roger Pau Monné wrote: On 20/08/14 11:19, Wei Hu wrote: Hello, Sending to Xen, drivers and virtualization mailing lists since this might be of interest to the folks on these aliases. I am working for Microsoft to improve the performance of FreeBSD running on Hyper-V. Right now I am adding a feature in the vmbus driver which could handle the host-guest channel communications on all vCPUs simultaneously. In order to achieve this, the hypervisor will send same interrupt concurrently on all the vCPUs. The traditional way on FreeBSD to set up interrupt handling for devicse, such as calling bus_alloc_resource() to reserve an IRQ line, and then calling bus_setup_intr() to create a vector, doesn't seem to work in this case. It seems if the interrupt is routed via legacy IRQ, it can only be active on one vCPU at a time. In order to allow the same interrupt to be handled on all vCPUs concurrently, all I need is an IDT entry, not an IRQ line. I checked current FreeBSD code. It looks to me Xen directly uses the vector number IDT_EVTCHN (0x93) to achieve the same purpose. I am proposing both Xen and Hyper-V share this same vector. Following is a little bit detail of my proposal for the changes in the current kernel. 1. In machdep.c: #ifdef XENHVM setidt(IDT_EVTCHN, IDTVEC(xen_intr_upcall), SDT_SYSIGT, SEL_UPL, 0); #else setidt(IDT_EVTCHN, IDTVEC(hv_vmbus_intr), SDT_SYSIGT, SEL_UPL, 0); #endif 2. Apic_vector.S Add IDTVEC(hv_vmbus_intr) to call Hyper-V vmbus interrupt service routine. Any thoughts, objections and feedbacks are all welcome. Hello, I don't think using the same IDT vector is the right approach, I would just pick a different IDT vector and use that for Hyper-V. Using the same IDT vector (like your suggestion above) would prevent shipping a kernel with with both Hyper-V and Xen support (like it's done now in GENERIC). Roger. Hmm, can't you make this a runtime check to only call setidt() if you detect you are under the appropriate hypervisor? Also, bhyve currently has a hackish way of requesting a free IDT slot. Perhaps it would be best if I added little API to reserve an IDT slot assuming that callers could accept a dynamic IDT vector rather than a static one. -- John Baldwin ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to freebsd-virtualization-unsubscr...@freebsd.org
Re: Directly reserve an interrupt IDT entry for Hyper-V
On 20/08/14 17:31, John Baldwin wrote: On Wednesday, August 20, 2014 9:31:54 am Roger Pau Monné wrote: Hello, I don't think using the same IDT vector is the right approach, I would just pick a different IDT vector and use that for Hyper-V. Using the same IDT vector (like your suggestion above) would prevent shipping a kernel with with both Hyper-V and Xen support (like it's done now in GENERIC). Roger. Hmm, can't you make this a runtime check to only call setidt() if you detect you are under the appropriate hypervisor? Also, bhyve currently has a hackish way of requesting a free IDT slot. Perhaps it would be best if I added little API to reserve an IDT slot assuming that callers could accept a dynamic IDT vector rather than a static one. That would work for Xen. The IDT vector doesn't need to be fixed since it's registered with Xen when the system boots. Roger. ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to freebsd-virtualization-unsubscr...@freebsd.org
Re: Directly reserve an interrupt IDT entry for Hyper-V
Hi John, On Wed, Aug 20, 2014 at 8:31 AM, John Baldwin j...@freebsd.org wrote: On Wednesday, August 20, 2014 9:31:54 am Roger Pau Monné wrote: On 20/08/14 11:19, Wei Hu wrote: Hello, Sending to Xen, drivers and virtualization mailing lists since this might be of interest to the folks on these aliases. I am working for Microsoft to improve the performance of FreeBSD running on Hyper-V. Right now I am adding a feature in the vmbus driver which could handle the host-guest channel communications on all vCPUs simultaneously. In order to achieve this, the hypervisor will send same interrupt concurrently on all the vCPUs. The traditional way on FreeBSD to set up interrupt handling for devicse, such as calling bus_alloc_resource() to reserve an IRQ line, and then calling bus_setup_intr() to create a vector, doesn't seem to work in this case. It seems if the interrupt is routed via legacy IRQ, it can only be active on one vCPU at a time. In order to allow the same interrupt to be handled on all vCPUs concurrently, all I need is an IDT entry, not an IRQ line. I checked current FreeBSD code. It looks to me Xen directly uses the vector number IDT_EVTCHN (0x93) to achieve the same purpose. I am proposing both Xen and Hyper-V share this same vector. Following is a little bit detail of my proposal for the changes in the current kernel. 1. In machdep.c: #ifdef XENHVM setidt(IDT_EVTCHN, IDTVEC(xen_intr_upcall), SDT_SYSIGT, SEL_UPL, 0); #else setidt(IDT_EVTCHN, IDTVEC(hv_vmbus_intr), SDT_SYSIGT, SEL_UPL, 0); #endif 2. Apic_vector.S Add IDTVEC(hv_vmbus_intr) to call Hyper-V vmbus interrupt service routine. Any thoughts, objections and feedbacks are all welcome. Hello, I don't think using the same IDT vector is the right approach, I would just pick a different IDT vector and use that for Hyper-V. Using the same IDT vector (like your suggestion above) would prevent shipping a kernel with with both Hyper-V and Xen support (like it's done now in GENERIC). Roger. Hmm, can't you make this a runtime check to only call setidt() if you detect you are under the appropriate hypervisor? Also, bhyve currently has a hackish way of requesting a free IDT slot. Perhaps it would be best if I added little API to reserve an IDT slot assuming that callers could accept a dynamic IDT vector rather than a static one. Yup, it'll be good to get rid of vmm_ipi.c. best Neel -- John Baldwin ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to freebsd-virtualization-unsubscr...@freebsd.org ___ freebsd-virtualization@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to freebsd-virtualization-unsubscr...@freebsd.org
