Fischer, Anna wrote:
> A current trend in virtualization seems to be to move more functionality into
> hardware and also to remove the hypervisor (like e.g. Xen) from the main I/O
> data path. This means that VMs can directly transmit on the physical hardware
> (that might be shared with between multiple VMs, e.g. through PCI-SIG IOV
> capabilities) which has a big positive performance impact. Do you support
> this kind of direct I/O with Crossbow? I can imagine that it's difficult to
> do rate (bandwidth) controlling with such a design, and this seems to be one
> of the most important features of Crossbow.
>
There are two kinds of rate limiting.
The first is inbound (RX) traffic. The second is outbound (TX) traffic.
For inbound, you have no control over the bits on the wire. So you can
only ensure that one domain (or ring) cannot fill up and thereby cause
another one to fill up. PCI-IOV is complementary to this. (A VM may
have its rings fill up because it can't process them quickly enough...
perhaps due to CPU resource considerations ... but it can't create a DoS
for other VMs with different rings.)
For outbound, we have a different problem. It sort of depends on the
hardware. One can imagine that you can assign rings to VMs, but the
question is whether they have a priority associated with them or not.
If the rings are all fair-share, then rate limiting is not necessary.
(One assumes that then you want to allow a VM to use all resource
available, but not more than its fair share if other VMs are
competing.) But if you assign a high-priority (high QoS) ring to a VM,
then that VM could starve other VMs. So I guess it depends on the hardware.
As far as limiting the bandwidth used by a particular ethernet port
overall, if you want that level of control, then you probably need to
use Crossbow soft rings instead of hardware rings. I don't see any
other solution. But most people only want to use resource control to
divvy up the bandwidth between different VMs or processes, and would
like to use the maximum bandwidth to the box if possible. So PCI IOV is
useful there.
Btw, there are other solutions besides PCI IOV. Any NIC which has
multiple rings and separate rx/tx interrupts can have the same feature
with Crossbow. The nxge hardware is an example of this. I believe the
Intel Zoar NIC is also multiple-ring capable, though I believe that the
driver that just got integrated does not have support it yet.
> Also, can Crossbow make use of new Intel VT-d features? If not, is this
> something you look at for future work?
>
I believe we (Sun) are planning on enhancements to support VT-d. Much
of what we want to do with VT-d is not just virtualization. VT-d can
also be used to provide fault isolation and reduce or eliminate the need
for DMA bounce buffers.
-- Garrett
> Thanks,
> Anna
>
>
>> -----Original Message-----
>> From: Roamer.Lu at sun.com [mailto:Roamer.Lu at sun.com] On Behalf
>> Of Yunsong (Roamer) Lu
>> Sent: 09 January 2008 15:22
>> To: Fischer, Anna
>> Cc: crossbow-discuss at opensolaris.org
>> Subject: Re: [crossbow-discuss] Crossbow and HW-assisted
>> virtualization
>>
>> Hi Anna,
>> Current Crossbow architecture doesn't make use of PCI-SIG IOV
>> functionalities. In fact, Crossbow can virtualize any NIC
>> even the hardware doesn't support any PCI-SIG IOV technologies.
>>
>> Crossbow framework manage all hardware resources exposed by
>> driver and hardware so that IOV-capable NICs can be handled
>> well if the driver can configure the device properly.
>>
>> We are considering to have explicit interfaces for IOV
>> functionalities in the future.
>>
>> Thanks,
>>
>> Roamer
>>
>> Fischer, Anna wrote:
>>
>>> In what way does Crossbow make use of functionalities of
>>>
>> PCI-SIG IOV-capable NICs that export multiple virtual NICs to
>> the OS? Is this already supported in the current Crossbow
>> implementation and if yes, how much performance impact does
>> it have to use IOV-capable NICs with Crossbow?
>>
>>> Thanks,
>>> Anna
>>> _______________________________________________
>>> crossbow-discuss mailing list
>>> crossbow-discuss at opensolaris.org
>>> http://mail.opensolaris.org/mailman/listinfo/crossbow-discuss
>>>
>> --
>>
>> # telnet (650)-786-6759 (x86759)
>> Connected to Solaris.Sun.COM.
>> login: Lu, Yunsong
>> Last login: January 2, 2007 from beyond.sfbay
>> Yunsong.Lu at Sun.COM v1.03 Since Mon Dec. 22, 2003
>> [Roamer at Solaris Networking]# cd ..
>>
>>
> _______________________________________________
> crossbow-discuss mailing list
> crossbow-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/crossbow-discuss
>
