Re: [kvm-devel] [PATCH] Make virtio devices multi-function
Ian Kirk wrote: Avi Kivity wrote: For mass storage, we should follow the SCSI model with a single device serving multiple disks, similar to what you suggest. Not sure if the device should have a single queue or one queue per disk. Don't you just end up re-implementing SCSI then, at which point you might as well stick with a 'fake' SCSI device in the guest? A virtio-scsi controller is indeed useful as it can control tapes, media changers, and other fancy stuff in addition to ordinary disks. For disks, I'd like to avoid the overhead of scsi command generation and parsing. -- Any sufficiently difficult bug is indistinguishable from a feature. - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] Make virtio devices multi-function
Anthony Liguori wrote: BTW, I've never been that convinced that hotplugging devices is as useful as people make it out to be. I also think that's particularly true when it comes to hot adding/removing very large numbers of disks. On the contrary, the more disks you have, the more likely one is to fail, so you'd need to hotreplace it (think a setup with redundancy like zfs). -- Any sufficiently difficult bug is indistinguishable from a feature. - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] Make virtio devices multi-function
* Anthony Liguori ([EMAIL PROTECTED]) wrote: Logically speaking, virtio is a bus. virtio supports all of the features of a bus (discover, hot add, hot remove). Right now, we map virtio devices directly onto the PCI bus. The problem we're trying to address is limitations of the PCI bus. We have a couple options: First question is if we have a real limitation with multiple busses? 1) add a virtio device that supports multiple disks. we need to reinvent hotplug within this device. 2) add a new PCI virtio transport that supports multiple virtio-blk devices within a single PCI slot 3) add a generic PCI virtio transport that supports multiple virtio devices within a single PCI slot compare and contrast above with HBA and disks (makes most sense from my point of view). for 2 and 3, only difference is whether you want to be able to support nics, balloons, and block devices on same pci slot (at which point it's a bridge, how is it different from 4?) 4) add a generic virtio bridge that supports multiple virtio devices within a single virtio device. #4 may seem strange, but it's no different from a PCI-to-PCI bridge. I like #4 the most, but #2 is probably the most practical. Also, your current patch does not work for hotplug disk. thanks, -chris - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] Make virtio devices multi-function
It sounds reasonable to expect so. ACPI has objects for devices, not functions (IIRC). So what I dislike about multifunction devices is the fact that a single slot shares an IRQ, and that special code is required in the QEMU drivers (virtio guest capability might not always be present). Actually this is not true. It is possible to have separate IRQ's for different devices residing in the same slot, changes are required to the ACPI tables though. I don't see any need for using them if we can extend PCI slots... Maybe require explicit device/function assignment on the command line? It will be managed anyway. ACPI does support hotplugging of individual functions inside slots, not sure how well does Linux (and other OSes) support that.. should be transparent though. - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
[kvm-devel] [PATCH] Make virtio devices multi-function
This patch changes virtio devices to be multi-function devices whenever
possible. This increases the number of virtio devices we can support now by
a factor of 8.
With this patch, I've been able to launch a guest with either 220 disks or 220
network adapters.
I haven't tested the Windows virtio drivers.
Signed-off-by: Anthony Liguori [EMAIL PROTECTED]
diff --git a/qemu/hw/pci.h b/qemu/hw/pci.h
index 60e4094..df3a878 100644
--- a/qemu/hw/pci.h
+++ b/qemu/hw/pci.h
@@ -33,7 +33,7 @@ typedef struct PCIIORegion {
#define PCI_ROM_SLOT 6
#define PCI_NUM_REGIONS 7
-#define PCI_DEVICES_MAX 64
+#define PCI_DEVICES_MAX 256
#define PCI_VENDOR_ID 0x00/* 16 bits */
#define PCI_DEVICE_ID 0x02/* 16 bits */
diff --git a/qemu/hw/virtio.c b/qemu/hw/virtio.c
index 9100bb1..9ea14d3 100644
--- a/qemu/hw/virtio.c
+++ b/qemu/hw/virtio.c
@@ -405,9 +405,18 @@ VirtIODevice *virtio_init_pci(PCIBus *bus, const char
*name,
PCIDevice *pci_dev;
uint8_t *config;
uint32_t size;
+static int devfn = 7;
+
+if ((devfn % 8) == 7)
+ devfn = -1;
+else
+ devfn++;
pci_dev = pci_register_device(bus, name, struct_size,
- -1, NULL, NULL);
+ devfn, NULL, NULL);
+
+devfn = pci_dev-devfn;
+
vdev = to_virtio_device(pci_dev);
vdev-status = 0;
@@ -435,6 +444,10 @@ VirtIODevice *virtio_init_pci(PCIBus *bus, const char
*name,
config[0x3d] = 1;
+/* Mark device as multi-function */
+if ((devfn % 8) == 0)
+ config[0x0e] |= 0x80;
+
vdev-name = name;
vdev-config_len = config_size;
if (vdev-config_len)
diff --git a/qemu/net.h b/qemu/net.h
index 13daa27..3bada75 100644
--- a/qemu/net.h
+++ b/qemu/net.h
@@ -42,7 +42,7 @@ void net_client_uninit(NICInfo *nd);
/* NIC info */
-#define MAX_NICS 8
+#define MAX_NICS 256
struct NICInfo {
uint8_t macaddr[6];
diff --git a/qemu/sysemu.h b/qemu/sysemu.h
index b645fb7..7992a77 100644
--- a/qemu/sysemu.h
+++ b/qemu/sysemu.h
@@ -151,7 +151,7 @@ typedef struct DriveInfo {
#define MAX_IDE_DEVS 2
#define MAX_SCSI_DEVS 7
-#define MAX_DRIVES 32
+#define MAX_DRIVES 256
int nb_drives;
DriveInfo drives_table[MAX_DRIVES+1];
diff --git a/qemu/vl.c b/qemu/vl.c
index 7dd0094..e203a4d 100644
--- a/qemu/vl.c
+++ b/qemu/vl.c
@@ -8754,7 +8754,7 @@ static BOOL WINAPI qemu_ctrl_handler(DWORD type)
}
#endif
-#define MAX_NET_CLIENTS 32
+#define MAX_NET_CLIENTS 512
static int saved_argc;
static char **saved_argv;
-
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
___
kvm-devel mailing list
kvm-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] Make virtio devices multi-function
Anthony Liguori wrote: This patch changes virtio devices to be multi-function devices whenever possible. This increases the number of virtio devices we can support now by a factor of 8. With this patch, I've been able to launch a guest with either 220 disks or 220 network adapters. Does this play well with hotplug? Perhaps we need to allocate a new device on hotplug. (certainly if we have a device with one function, which then gets converted to a multifunction device) -- error compiling committee.c: too many arguments to function - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] Make virtio devices multi-function
Avi Kivity wrote: Anthony Liguori wrote: This patch changes virtio devices to be multi-function devices whenever possible. This increases the number of virtio devices we can support now by a factor of 8. With this patch, I've been able to launch a guest with either 220 disks or 220 network adapters. Does this play well with hotplug? Perhaps we need to allocate a new device on hotplug. Probably not. I imagine you can only hotplug devices, not individual functions? Regards, Anthony Liguori (certainly if we have a device with one function, which then gets converted to a multifunction device) - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] Make virtio devices multi-function
Anthony Liguori wrote: Avi Kivity wrote: Anthony Liguori wrote: This patch changes virtio devices to be multi-function devices whenever possible. This increases the number of virtio devices we can support now by a factor of 8. With this patch, I've been able to launch a guest with either 220 disks or 220 network adapters. Does this play well with hotplug? Perhaps we need to allocate a new device on hotplug. Probably not. I imagine you can only hotplug devices, not individual functions? It sounds reasonable to expect so. ACPI has objects for devices, not functions (IIRC). Maybe require explicit device/function assignment on the command line? It will be managed anyway. -- error compiling committee.c: too many arguments to function - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] Make virtio devices multi-function
On Tue, Apr 22, 2008 at 4:15 PM, Anthony Liguori [EMAIL PROTECTED] wrote:
This patch changes virtio devices to be multi-function devices whenever
possible. This increases the number of virtio devices we can support now by
a factor of 8.
[...]
diff --git a/qemu/hw/virtio.c b/qemu/hw/virtio.c
index 9100bb1..9ea14d3 100644
--- a/qemu/hw/virtio.c
+++ b/qemu/hw/virtio.c
@@ -405,9 +405,18 @@ VirtIODevice *virtio_init_pci(PCIBus *bus, const char
*name,
PCIDevice *pci_dev;
uint8_t *config;
uint32_t size;
+static int devfn = 7;
+
+if ((devfn % 8) == 7)
+ devfn = -1;
+else
+ devfn++;
This code look strange... devfn should be passed to virtio_init_pci by
virtio-{net,blk} init functions, no?
Luca
-
This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don't miss this year's exciting event. There's still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone
___
kvm-devel mailing list
kvm-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] Make virtio devices multi-function
* Anthony Liguori [EMAIL PROTECTED] [2008-04-22 09:16]: This patch changes virtio devices to be multi-function devices whenever possible. This increases the number of virtio devices we can support now by a factor of 8. With this patch, I've been able to launch a guest with either 220 disks or 220 network adapters. Have you confirmed that the network devices show up? I was playing around with some of the limits last night and while it is easy to get QEMU to create the adapters, so far I've only had a guest see 29 pci nics (e1000). -- Ryan Harper Software Engineer; Linux Technology Center IBM Corp., Austin, Tx (512) 838-9253 T/L: 678-9253 [EMAIL PROTECTED] - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] Make virtio devices multi-function
Ryan Harper wrote: * Anthony Liguori [EMAIL PROTECTED] [2008-04-22 09:16]: This patch changes virtio devices to be multi-function devices whenever possible. This increases the number of virtio devices we can support now by a factor of 8. With this patch, I've been able to launch a guest with either 220 disks or 220 network adapters. Have you confirmed that the network devices show up? I was playing around with some of the limits last night and while it is easy to get QEMU to create the adapters, so far I've only had a guest see 29 pci nics (e1000). Yup, I had an eth219 Regards, Anthony Liguori - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] Make virtio devices multi-function
On Tue, Apr 22, 2008 at 05:32:45PM +0300, Avi Kivity wrote: Anthony Liguori wrote: This patch changes virtio devices to be multi-function devices whenever possible. This increases the number of virtio devices we can support now by a factor of 8. With this patch, I've been able to launch a guest with either 220 disks or 220 network adapters. Does this play well with hotplug? Perhaps we need to allocate a new device on hotplug. (certainly if we have a device with one function, which then gets converted to a multifunction device) Would have to change the hotplug code to handle functions... It sounds less hacky to just extend the PCI slots instead of (ab)using multiple functions per-slot. - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] Make virtio devices multi-function
Marcelo Tosatti wrote: Maybe require explicit device/function assignment on the command line? It will be managed anyway. ACPI does support hotplugging of individual functions inside slots, not sure how well does Linux (and other OSes) support that.. should be transparent though. I think we need to decide what we want to target in terms of upper limits. With a bridge or two, we can probably easily do 128. If we really want to push things, I think we should do a PCI based virtio controller. I doubt a large number of PCI devices is ever going to perform very well b/c of interrupt sharing and some of the assumptions in virtio_pci. If we implement a controller, we can use a single interrupt, but multiplex multiple notifications on that single interrupt. We can also be more aggressive about using shared memory instead of PCI config space which would reduce the overall number of exits. We could easily support a very large number of devices this way. But again, what do we want to target for now? Regards, Anthony Liguori - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] Make virtio devices multi-function
Anthony Liguori wrote: I think we need to decide what we want to target in terms of upper limits. With a bridge or two, we can probably easily do 128. If we really want to push things, I think we should do a PCI based virtio controller. I doubt a large number of PCI devices is ever going to perform very well b/c of interrupt sharing and some of the assumptions in virtio_pci. If we implement a controller, we can use a single interrupt, but multiplex multiple notifications on that single interrupt. We can also be more aggressive about using shared memory instead of PCI config space which would reduce the overall number of exits. We could easily support a very large number of devices this way. But again, what do we want to target for now? I think that for networking we should keep things as is. I don't see anybody using 100 virtual NICs. For mass storage, we should follow the SCSI model with a single device serving multiple disks, similar to what you suggest. Not sure if the device should have a single queue or one queue per disk. -- error compiling committee.c: too many arguments to function - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] Make virtio devices multi-function
Avi Kivity wrote: Anthony Liguori wrote: I think we need to decide what we want to target in terms of upper limits. With a bridge or two, we can probably easily do 128. If we really want to push things, I think we should do a PCI based virtio controller. I doubt a large number of PCI devices is ever going to perform very well b/c of interrupt sharing and some of the assumptions in virtio_pci. If we implement a controller, we can use a single interrupt, but multiplex multiple notifications on that single interrupt. We can also be more aggressive about using shared memory instead of PCI config space which would reduce the overall number of exits. We could easily support a very large number of devices this way. But again, what do we want to target for now? I think that for networking we should keep things as is. I don't see anybody using 100 virtual NICs. For mass storage, we should follow the SCSI model with a single device serving multiple disks, similar to what you suggest. Not sure if the device should have a single queue or one queue per disk. My latest thought it to do a virtio-based virtio controller. We could avoid creating one in QEMU unless we detect an abnormally large number of disks or something. Regards, Anthony Liguori - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] Make virtio devices multi-function
On Tue, Apr 22, 2008 at 11:31:11AM -0500, Anthony Liguori wrote: Avi Kivity wrote: Anthony Liguori wrote: I think we need to decide what we want to target in terms of upper limits. With a bridge or two, we can probably easily do 128. If we really want to push things, I think we should do a PCI based virtio controller. I doubt a large number of PCI devices is ever going to perform very well b/c of interrupt sharing and some of the assumptions in virtio_pci. If we implement a controller, we can use a single interrupt, but multiplex multiple notifications on that single interrupt. We can also be more aggressive about using shared memory instead of PCI config space which would reduce the overall number of exits. We should increase the number of interrupt lines, perhaps to 16. Using shared memory to avoid exits sounds very good idea. We could easily support a very large number of devices this way. But again, what do we want to target for now? I think that for networking we should keep things as is. I don't see anybody using 100 virtual NICs. The target was along the lines of 20 nics + 80 disks. Dan? For mass storage, we should follow the SCSI model with a single device serving multiple disks, similar to what you suggest. Not sure if the device should have a single queue or one queue per disk. My latest thought it to do a virtio-based virtio controller. Why do you dislike multiple disks per virtio-blk controller? As mentioned this seems a natural way forward. - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] Make virtio devices multi-function
For mass storage, we should follow the SCSI model with a single device serving multiple disks, similar to what you suggest. Not sure if the device should have a single queue or one queue per disk. My latest thought it to do a virtio-based virtio controller. Why do you dislike multiple disks per virtio-blk controller? As mentioned this seems a natural way forward. Logically speaking, virtio is a bus. virtio supports all of the features of a bus (discover, hot add, hot remove). Right now, we map virtio devices directly onto the PCI bus. The problem we're trying to address is limitations of the PCI bus. We have a couple options: 1) add a virtio device that supports multiple disks. we need to reinvent hotplug within this device. 2) add a new PCI virtio transport that supports multiple virtio-blk devices within a single PCI slot 3) add a generic PCI virtio transport that supports multiple virtio devices within a single PCI slot 4) add a generic virtio bridge that supports multiple virtio devices within a single virtio device. #4 may seem strange, but it's no different from a PCI-to-PCI bridge. I like #4 the most, but #2 is probably the most practical. Regards, Anthony Liguori - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] Make virtio devices multi-function
Avi Kivity wrote: For mass storage, we should follow the SCSI model with a single device serving multiple disks, similar to what you suggest. Not sure if the device should have a single queue or one queue per disk. Don't you just end up re-implementing SCSI then, at which point you might as well stick with a 'fake' SCSI device in the guest? - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] Make virtio devices multi-function
Marcelo Tosatti wrote: On Tue, Apr 22, 2008 at 05:32:45PM +0300, Avi Kivity wrote: Anthony Liguori wrote: This patch changes virtio devices to be multi-function devices whenever possible. This increases the number of virtio devices we can support now by a factor of 8. With this patch, I've been able to launch a guest with either 220 disks or 220 network adapters. Does this play well with hotplug? Perhaps we need to allocate a new device on hotplug. (certainly if we have a device with one function, which then gets converted to a multifunction device) Would have to change the hotplug code to handle functions... BTW, I've never been that convinced that hotplugging devices is as useful as people make it out to be. I also think that's particularly true when it comes to hot adding/removing very large numbers of disks. I think if you created all virtio devices as multifunction devices, but didn't add additional functions until you ran out of PCI slots, it would be a pretty acceptable solution. Hotplug works just as it does today until you get much higher than 32 devices. Even then, hotplug still works with most of your devices (until you hit the absolute maximum number of devices of course). Regards, Anthony Liguori It sounds less hacky to just extend the PCI slots instead of (ab)using multiple functions per-slot. - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
Re: [kvm-devel] [PATCH] Make virtio devices multi-function
On Tue, Apr 22, 2008 at 02:26:45PM -0300, Marcelo Tosatti wrote: On Tue, Apr 22, 2008 at 11:31:11AM -0500, Anthony Liguori wrote: Avi Kivity wrote: Anthony Liguori wrote: I think we need to decide what we want to target in terms of upper limits. With a bridge or two, we can probably easily do 128. If we really want to push things, I think we should do a PCI based virtio controller. I doubt a large number of PCI devices is ever going to perform very well b/c of interrupt sharing and some of the assumptions in virtio_pci. If we implement a controller, we can use a single interrupt, but multiplex multiple notifications on that single interrupt. We can also be more aggressive about using shared memory instead of PCI config space which would reduce the overall number of exits. We should increase the number of interrupt lines, perhaps to 16. Using shared memory to avoid exits sounds very good idea. We could easily support a very large number of devices this way. But again, what do we want to target for now? I think that for networking we should keep things as is. I don't see anybody using 100 virtual NICs. The target was along the lines of 20 nics + 80 disks. Dan? I've already had people ask for ability to as many as 64 disks and 32 nics with Xen, so to my mind, the more we support the better. 100's if possible. Dan. -- |: Red Hat, Engineering, Boston -o- http://people.redhat.com/berrange/ :| |: http://libvirt.org -o- http://virt-manager.org -o- http://ovirt.org :| |: http://autobuild.org -o- http://search.cpan.org/~danberr/ :| |: GnuPG: 7D3B9505 -o- F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :| - This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone ___ kvm-devel mailing list kvm-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/kvm-devel
[kvm-devel] [PATCH] Make virtio devices multi-function (v2)
This patch changes virtio devices to be multi-function devices whenever
possible. This increases the number of virtio devices we can support now by
a factor of 8.
With this patch, I've been able to launch a guest with either 220 disks or 220
network adapters.
Since v1, I've changed the way virtio devices are allocated to be as follows:
1) Always use a slot as long as they are available. We can extend this to
use a PCI when we get that working more reliably.
2) When PCI slots are exhausted, fall back add device as an additional
function on an existing slot
This way, hotplug continues to work just as well as it does now. Once you
exceed the number of PCI slots, you need an OS that can do hotplug of
individual PCI functions if you care about doing hotplug. I think this is a
pretty reasonable trade-off.
Signed-off-by: Anthony Liguori [EMAIL PROTECTED]
diff --git a/qemu/hw/pci.c b/qemu/hw/pci.c
index a23a466..5d5d1a5 100644
--- a/qemu/hw/pci.c
+++ b/qemu/hw/pci.c
@@ -146,6 +146,41 @@ int pci_device_load(PCIDevice *s, QEMUFile *f)
return 0;
}
+/* Search the bus for a multifunction device with a free function that
+ * matches vendor_id_filter and device_id_filter. -1 can be passed as
+ * a filter value to accept any id.
+ */
+int pci_bus_find_device_function(PCIBus *bus, int vendor_id_filter,
+int device_id_filter)
+{
+int devfn;
+
+for (devfn = bus-devfn_min; devfn 256; devfn += 8) {
+ int vendor_id, device_id;
+ PCIDevice *pci_dev;
+
+ if (!bus-devices[devfn])
+ continue;
+
+ pci_dev = bus-devices[devfn];
+ vendor_id = pci_dev-config[0x01] 8 | pci_dev-config[0x00];
+ device_id = pci_dev-config[0x03] 8 | pci_dev-config[0x02];
+
+ if ((vendor_id_filter == -1 || vendor_id_filter == vendor_id)
+ (device_id_filter == -1 || device_id_filter == device_id)
+ ((pci_dev-config[0x0e] 0x80) == 0x80)) {
+ int i;
+
+ for (i = 1; i 8; i++) {
+ if (!bus-devices[devfn + i])
+ return devfn + i;
+ }
+ }
+}
+
+return -1;
+}
+
/* -1 for devfn means auto assign */
PCIDevice *pci_register_device(PCIBus *bus, const char *name,
int instance_size, int devfn,
diff --git a/qemu/hw/pci.h b/qemu/hw/pci.h
index 60e4094..84d6a29 100644
--- a/qemu/hw/pci.h
+++ b/qemu/hw/pci.h
@@ -33,7 +33,7 @@ typedef struct PCIIORegion {
#define PCI_ROM_SLOT 6
#define PCI_NUM_REGIONS 7
-#define PCI_DEVICES_MAX 64
+#define PCI_DEVICES_MAX 256
#define PCI_VENDOR_ID 0x00/* 16 bits */
#define PCI_DEVICE_ID 0x02/* 16 bits */
@@ -105,6 +105,9 @@ void pci_info(void);
PCIBus *pci_bridge_init(PCIBus *bus, int devfn, uint32_t id,
pci_map_irq_fn map_irq, const char *name);
+int pci_bus_find_device_function(PCIBus *bus, int vendor_id_filter,
+int device_id_filter);
+
/* lsi53c895a.c */
#define LSI_MAX_DEVS 7
void lsi_scsi_attach(void *opaque, BlockDriverState *bd, int id);
diff --git a/qemu/hw/virtio.c b/qemu/hw/virtio.c
index 6a50001..361455d 100644
--- a/qemu/hw/virtio.c
+++ b/qemu/hw/virtio.c
@@ -405,12 +405,22 @@ VirtIODevice *virtio_init_pci(PCIBus *bus, const char
*name,
PCIDevice *pci_dev;
uint8_t *config;
uint32_t size;
+int devfn = -1;
-pci_dev = pci_register_device(bus, name, struct_size,
- -1, NULL, NULL);
-if (!pci_dev)
+pci_dev = pci_register_device(bus, name, struct_size, -1, NULL, NULL);
+
+if (pci_dev == NULL) {
+ devfn = pci_bus_find_device_function(bus, vendor, -1);
+ if (devfn != -1)
+ pci_dev = pci_register_device(bus, name, struct_size,
+ devfn, NULL, NULL);
+}
+
+if (pci_dev == NULL)
return NULL;
+devfn = pci_dev-devfn;
+
vdev = to_virtio_device(pci_dev);
vdev-status = 0;
@@ -438,6 +448,10 @@ VirtIODevice *virtio_init_pci(PCIBus *bus, const char
*name,
config[0x3d] = 1;
+/* Mark device as multi-function */
+if ((devfn % 8) == 0)
+ config[0x0e] |= 0x80;
+
vdev-name = name;
vdev-config_len = config_size;
if (vdev-config_len)
diff --git a/qemu/net.h b/qemu/net.h
index 13daa27..3bada75 100644
--- a/qemu/net.h
+++ b/qemu/net.h
@@ -42,7 +42,7 @@ void net_client_uninit(NICInfo *nd);
/* NIC info */
-#define MAX_NICS 8
+#define MAX_NICS 256
struct NICInfo {
uint8_t macaddr[6];
diff --git a/qemu/sysemu.h b/qemu/sysemu.h
index c60072d..4385802 100644
--- a/qemu/sysemu.h
+++ b/qemu/sysemu.h
@@ -149,7 +149,7 @@ typedef struct DriveInfo {
#define MAX_IDE_DEVS 2
#define MAX_SCSI_DEVS 7
-#define MAX_DRIVES 32
+#define MAX_DRIVES 256
int nb_drives;
DriveInfo drives_table[MAX_DRIVES+1];
diff --git a/qemu/vl.c b/qemu/vl.c
index 74be059..824e331 100644
--- a/qemu/vl.c
+++ b/qemu/vl.c
@@ -8717,7
