Re: [Qemu-devel] Re: Hitting 29 NIC limit
Hi, What is missing for multiple pci buses is - a simple P2P bridge emulator for pc architecture - seabios support - DSDT for a new machine which has multi pci buses(IRQ routing) Is this really needed? IIRC there are (used to be?) multiport nic pci cards which are simply multiple ethernet card pci chips behind a pci-to-pci bridge. Such a pci-to-pci bridge should be able to operate without bios support, no? So one cold just do ... -device pci-bridge,id=br -device e1000,bus=br.0 -device e1000,bus=br.0 to create a bridge with two nics behind it? cheers, Gerd
Re: [Qemu-devel] Re: Hitting 29 NIC limit
On Thu, Nov 18, 2010 at 12:54:29PM +0100, Gerd Hoffmann wrote: Hi, What is missing for multiple pci buses is - a simple P2P bridge emulator for pc architecture - seabios support - DSDT for a new machine which has multi pci buses(IRQ routing) Is this really needed? Q35 IRQ routing is different from piix4's. So new one is necessary. Possibly it won't depend on the number of pci buses, I haven't confirmed it. -- yamahata
[Qemu-devel] Re: Hitting 29 NIC limit
Right now, qemu is internally ready to handle pci-to-pci bridge emulators and it's quite easy to add a new simple p2p bridge emulator. Just a single function to initialize vid/did and a single PCIDevinceInfo. But it's difficult to use multi pci buses at the moment because other part isn't ready. It's only for developers who are interested in this area. BTW, how many pci buses/slots/functions do you want at most? What we have in the upstream repository is - various fixes for p2p bridge support - the library for p2p bridge support - pcie root/upstream/downstream port emulators which is p2p bridges ioh3420/xio3130_upstream/xio3130_downstream What is missing for multiple pci buses is - a new PC machine description which has multiple pci buses My Q35 chipset patch will provide this. - a simple P2P bridge emulator for pc architecture - seabios support - DSDT for a new machine which has multi pci buses(IRQ routing) - pv pci bus numbering this isn't a must, I suppose I have those patches locally. thanks, On Wed, Nov 17, 2010 at 12:58:43PM +0200, Michael S. Tsirkin wrote: On Tue, Nov 16, 2010 at 11:20:52AM -0800, Anjali Kulkarni wrote: Hi, Can you elaborate what you mean by bridge support? I would really like to go to 256 interfaces. Anjali Isaku Yamahata wrote most of it, I'll let him comment. -- MST -- yamahata
Re: [Qemu-devel] Re: Hitting 29 NIC limit
Isaku, Could you explain more on irq routing in dsdt and related code in qemu? I encountered a problem: emulated user networking (-net nic -net user) cannot work in my qemu-kvm+q35+pcie code. Your original q35 (pure qemu) works fine. Thanks! Wei On 11/17/10 7:09 PM, Isaku Yamahata yamah...@valinux.co.jp wrote: Right now, qemu is internally ready to handle pci-to-pci bridge emulators and it's quite easy to add a new simple p2p bridge emulator. Just a single function to initialize vid/did and a single PCIDevinceInfo. But it's difficult to use multi pci buses at the moment because other part isn't ready. It's only for developers who are interested in this area. BTW, how many pci buses/slots/functions do you want at most? What we have in the upstream repository is - various fixes for p2p bridge support - the library for p2p bridge support - pcie root/upstream/downstream port emulators which is p2p bridges ioh3420/xio3130_upstream/xio3130_downstream What is missing for multiple pci buses is - a new PC machine description which has multiple pci buses My Q35 chipset patch will provide this. - a simple P2P bridge emulator for pc architecture - seabios support - DSDT for a new machine which has multi pci buses(IRQ routing) - pv pci bus numbering this isn't a must, I suppose I have those patches locally. thanks, On Wed, Nov 17, 2010 at 12:58:43PM +0200, Michael S. Tsirkin wrote: On Tue, Nov 16, 2010 at 11:20:52AM -0800, Anjali Kulkarni wrote: Hi, Can you elaborate what you mean by bridge support? I would really like to go to 256 interfaces. Anjali Isaku Yamahata wrote most of it, I'll let him comment. -- MST
Re: [Qemu-devel] Re: Hitting 29 NIC limit
On Wed, Nov 17, 2010 at 09:41:03PM -0800, Wei Xu wrote: Could you explain more on irq routing in dsdt and related code in qemu? I encountered a problem: emulated user networking (-net nic -net user) cannot work in my qemu-kvm+q35+pcie code. Your original q35 (pure qemu) works fine. The related functions are ich9_lpc_set_irq_pic() and ich9_lpc_set_irq() in q35.c. I'm aware that there is something wrong. I'll fix it when I rebase the patches, but I haven't found time yet. -- yamahata
[Qemu-devel] Re: Hitting 29 NIC limit
On Thu, Nov 18, 2010 at 12:09:18PM +0900, Isaku Yamahata wrote: Right now, qemu is internally ready to handle pci-to-pci bridge emulators and it's quite easy to add a new simple p2p bridge emulator. Just a single function to initialize vid/did and a single PCIDevinceInfo. But it's difficult to use multi pci buses at the moment because other part isn't ready. It's only for developers who are interested in this area. BTW, how many pci buses/slots/functions do you want at most? What we have in the upstream repository is - various fixes for p2p bridge support - the library for p2p bridge support - pcie root/upstream/downstream port emulators which is p2p bridges ioh3420/xio3130_upstream/xio3130_downstream What is missing for multiple pci buses is - a new PC machine description which has multiple pci buses My Q35 chipset patch will provide this. IMHO, what we really need is a way to specify bridges through qdev. Not a machine with a hard-coded buses. The firmware paths used by Gleb's code might supply a good syntax to specify topology. - a simple P2P bridge emulator for pc architecture - seabios support - DSDT for a new machine which has multi pci buses(IRQ routing) - pv pci bus numbering Interesting. What's the motivation for this? this isn't a must, I suppose I have those patches locally. thanks, On Wed, Nov 17, 2010 at 12:58:43PM +0200, Michael S. Tsirkin wrote: On Tue, Nov 16, 2010 at 11:20:52AM -0800, Anjali Kulkarni wrote: Hi, Can you elaborate what you mean by bridge support? I would really like to go to 256 interfaces. Anjali Isaku Yamahata wrote most of it, I'll let him comment. -- MST -- yamahata
[Qemu-devel] Re: Hitting 29 NIC limit
On Thu, Nov 18, 2010 at 08:46:43AM +0200, Michael S. Tsirkin wrote: What is missing for multiple pci buses is - a new PC machine description which has multiple pci buses My Q35 chipset patch will provide this. IMHO, what we really need is a way to specify bridges through qdev. Not a machine with a hard-coded buses. Yes, it's long-term goal. The firmware paths used by Gleb's code might supply a good syntax to specify topology. - a simple P2P bridge emulator for pc architecture - seabios support - DSDT for a new machine which has multi pci buses(IRQ routing) - pv pci bus numbering Interesting. What's the motivation for this? For hot-plugging pci-to-pci bridge. It requires sparse pci bus numbering. In fact, it's not so strong requirement. Possibly I would have to dig into dsdt. -- yamahata
[Qemu-devel] Re: Hitting 29 NIC limit
On Thu, Nov 18, 2010 at 04:09:34PM +0900, Isaku Yamahata wrote: On Thu, Nov 18, 2010 at 08:46:43AM +0200, Michael S. Tsirkin wrote: What is missing for multiple pci buses is - a new PC machine description which has multiple pci buses My Q35 chipset patch will provide this. IMHO, what we really need is a way to specify bridges through qdev. Not a machine with a hard-coded buses. Yes, it's long-term goal. The firmware paths used by Gleb's code might supply a good syntax to specify topology. - a simple P2P bridge emulator for pc architecture - seabios support - DSDT for a new machine which has multi pci buses(IRQ routing) - pv pci bus numbering Interesting. What's the motivation for this? For hot-plugging pci-to-pci bridge. It requires sparse pci bus numbering. Not sure I understand. pci bus numbering should be completely up to the guest. In fact, it's not so strong requirement. Possibly I would have to dig into dsdt. Yes, some guest-side technology like acpi might be needed. -- yamahata
[Qemu-devel] Re: Hitting 29 NIC limit
On Thu, Nov 18, 2010 at 09:15:19AM +0200, Michael S. Tsirkin wrote: On Thu, Nov 18, 2010 at 04:09:34PM +0900, Isaku Yamahata wrote: On Thu, Nov 18, 2010 at 08:46:43AM +0200, Michael S. Tsirkin wrote: What is missing for multiple pci buses is - a new PC machine description which has multiple pci buses My Q35 chipset patch will provide this. IMHO, what we really need is a way to specify bridges through qdev. Not a machine with a hard-coded buses. Yes, it's long-term goal. The firmware paths used by Gleb's code might supply a good syntax to specify topology. - a simple P2P bridge emulator for pc architecture - seabios support - DSDT for a new machine which has multi pci buses(IRQ routing) - pv pci bus numbering Interesting. What's the motivation for this? For hot-plugging pci-to-pci bridge. It requires sparse pci bus numbering. Not sure I understand. pci bus numbering should be completely up to the guest. Yes. In fact, it's not so strong requirement. Possibly I would have to dig into dsdt. Yes, some guest-side technology like acpi might be needed. OSes like Linux tries to keep bios settings as much as possible. It means seabios needs to understand dsdt and initialize pci buses properly according to it. I think pv numbering is much easier than teaching seabios dsdt parsing. -- yamahata
[Qemu-devel] Re: Hitting 29 NIC limit
On Thu, Oct 14, 2010 at 05:17:36PM -0500, Anthony Liguori wrote: On 10/14/2010 05:12 PM, Anjali Kulkarni wrote: Thanks. Does this work for e1000 as well? Haven't tried. I don't know how various e1000 drivers would react. Also, does it support pci hotplug? No, but that's fixable down the road. Regards, Anthony Liguori Probably not.
[Qemu-devel] Re: Hitting 29 NIC limit
On Thu, Oct 14, 2010 at 02:07:17PM +0200, Avi Kivity wrote:
On 10/14/2010 12:54 AM, Anthony Liguori wrote:
On 10/13/2010 05:32 PM, Anjali Kulkarni wrote:
Hi,
Using the legacy way of starting up NICs, I am hitting a
limitation after 29
NICs ie no more than 29 are detected (that's because of the 32 PCI slot
limit on a single bus- 3 are already taken up)
I had initially increased the MAX_NICS to 48, just on my tree, to get to
more, but ofcource that wont work.
Is there any way to go beyond 29 NICs the legacy way? What is
the maximum
that can be supported by the qdev mothod?
I got up to 104 without trying very hard using the following script:
args=
for slot in 5 6 7 8 9 10 11 12 13 14 15 16 17; do
for fn in 0 1 2 3 4 5 6 7; do
args=$args -netdev user,id=eth${slot}_${fn}
args=$args -device
virtio-net-pci,addr=${slot}.${fn},netdev=eth${slot}_${fn},multifunction=on,romfile=
done
done
x86_64-softmmu/qemu-system-x86_64 -hda ~/images/linux.img ${args}
-enable-kvm
The key is to make the virtio-net devices multifunction and to
fill out all 8 functions for each slot.
This is unlikely to work right wrt pci hotplug. If we want to
support a large number of interfaces, we need true multiport cards.
Bridge support seems to be working mostly fine here.
That will let you go up to 256 devices without multifunction.
--
MST
[Qemu-devel] Re: Hitting 29 NIC limit
On 11/16/2010 11:23 AM, Michael S. Tsirkin wrote: On Thu, Oct 14, 2010 at 05:17:36PM -0500, Anthony Liguori wrote: On 10/14/2010 05:12 PM, Anjali Kulkarni wrote: Thanks. Does this work for e1000 as well? Haven't tried. I don't know how various e1000 drivers would react. Also, does it support pci hotplug? No, but that's fixable down the road. Regards, Anthony Liguori Probably not. FWIW, by fixable, I meant, we could add the ability to hot plug a multi-port NIC device. For instance, we could have -device virtio-net-pci-x8 which would be an 8-port virtio-net card. Regards, Anthony Liguori
