Re: [vfio-users] Is there some method to merge 2 iommu group if I disable ACS

[Alex Williamson]

On Tue, 8 Aug 2017 06:58:40 +
"Wuzongyong (Euler Dept)"  wrote:

> > On Mon, 7 Aug 2017 06:43:30 -0600
> > Alex Williamson  wrote:
> >   
> > > On Aug 7, 2017 5:39 AM, "Wuzongyong (Euler Dept)"
> > > 
> > > wrote:
> > >
> > > Hi,
> > >
> > >
> > >
> > > Assume that an endpoint device(called ep1) belongs to iommu group 1,
> > > and another endpoint device(called ep2) belongs to iommu group 2.
> > >
> > > Moreover, these two devices locate in different downstream ports of
> > > the same switch respectively. If I disable the ACS of these downstream
> > >
> > > ports,  we know ep1 and ep2 should locate in the same iommu group. So
> > > the question is if I can regenerate a iommu group to let ep1 and ep2
> > >
> > > locate in the same iommu group and let ep1 and ep2  can’t be assigned
> > > to two different VMs without  host rebooting?  
> > 
> > 
> > Wow, complete formatting fail from gmail on my phone.  Sorry for that.
> > Question is:
> >   
> > > Why would you want to be able to do this?  
> > 
> > Thanks,
> > 
> > Alex  
> 
> I want to do peer-to-peer communication between the two pci-e eps.

A significant problem here is the I/O virtual address (IOVA) space in
which the device operates when used with vfio.  Peer-to-peer will
operate that the physical address level while userspace drivers like
QEMU will operate with IOVAs.  The proper why to get both is for the
endpoint to support ATS and ACS on the downstream switch port to
support Direct Translation.  If these two things were both enabled,
then a transaction from the endpoint with the address translation field
set would be able to do direct p2p with another device while
non-translated transactions would go through the IOMMU.

Therefore, I don't think what you're about to try is going to work and
you're on your own for support.  That said, you can manipulate the ACS
control bits on the downstream switch ports with setpci to disable
ACS.  You might then be able to use the "remove" sysfs entry for the
PCI endpoint and the /sys/bus/pci/rescan entry to re-add the device,
which would hopefully detect the isolation issue and group the
endpoints together.  You might still run into problems that the
downstream ports are already in separate groups and this won't work
without also removing and rescanning the downstream port itself, but
this risks that ACS will be re-enabled on that port.  IOW, there are
some hacks you can try, but this is not an encouraged or supported
procedure and don't expect support for it.  Thanks,

Alex

___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] FLR supporting serial chips?

[Laszlo Ersek]

On 08/08/17 00:12, Alex Williamson wrote:
> On Mon, 7 Aug 2017 17:52:04 -0400
> "taii...@gmx.com"  wrote:
> 
>> I want to add physical serial ports to a VM but I can't find any FLR 
>> supporting serial cards (I have looked at spec sheets for the chips no luck)
>>
>> Ideally there would be a multi port card (say 4 or so) where every port 
>> can be separately assigned.
>>
>> Anyone have any info? - Thanks
> 
> I think you're headed in the wrong direction, serial ports and PCI
> device assignment can be done, but it's not a great match.  For
> starters, serial ports don't do DMA, so the whole IOMMU thing is a bit
> overkill.  You're also not likely to find a serial ASIC designed within
> the last 20yrs, so things like ACS and FLR are mostly out of the
> question, besides the device state of a serial port is pretty minimal
> and a device specific reset could probably be made to handle it, but
> I'm not sure it's even necessary.  Performance is also likely to be not
> much, if any, better than just serial passthrough because the register
> set of a serial device doesn't fill a 4K page or may be implemented as
> I/O port rather than MMIO, so you'll probably trap to QEMU for every
> access anyway unless you can get 4k alignment on the MMIO BARs.  USB
> serial passthrough might be a better option, but FLR seems
> exceptionally unlikely. Thanks,

To quote more specifics about "serial passthrough", from the QEMU manual:

   -serial dev
   Redirect the virtual serial port to host character device
   dev. The default device is "vc" in graphical mode and
   "stdio" in non graphical mode.

   This option can be used several times to simulate up to 4
   serial ports.
[...]
   Available character devices are:
[...]
   /dev/XXX
   [Linux only] Use host tty, e.g. /dev/ttyS0. The host
   serial port parameters are set according to the
   emulated ones.

Libvirt: .

Thanks
Laszlo

___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

[vfio-users] Is there some method to merge 2 iommu group if I disable ACS ?

[Wuzongyong (Euler Dept)]

Hi,

Assume that an endpoint device(called ep1) belongs to iommu group 1, and 
another endpoint device(called ep2) belongs to iommu group 2.
Moreover, these two devices locate in different downstream ports of the same 
switch respectively. If I disable the ACS of these downstream
ports,  we know ep1 and ep2 should locate in the same iommu group. So the 
question is if I can regenerate a iommu group to let ep1 and ep2
locate in the same iommu group and let ep1 and ep2  can't be assigned to two 
different VMs without  host rebooting?


Thanks,
Cordius Wu

___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users