Re: [vfio-users] Z170X IOMMU Groups

[Wei Xu]

On 2016年09月23日 12:39, Nick Sarnie wrote:

Wei,

I submitted a ticket here:
http://www.gigabyte.us/support-downloads/support-downloads.aspx


Thanks a lot. :)

Wei


Thanks,
Nick

On Fri, Sep 23, 2016 at 12:06 AM, Brett Peckinpaugh > wrote:

Same here I have a ud5 that I would like a bios that does not need
the ACS patch.

On September 22, 2016 8:59:57 PM PDT, Wei Xu > wrote:



On 2016年09月23日 02:47, Nick Sarnie wrote:

Hi again,

Very much to my surprise, Gigabyte replied and sent me a
fixed BIOS. The
new IOMMU groups (with ACS override patch kernel commandline
removed for
this boot), as well as my lspci information are below. I see
four
messages the following messages in dmesg now:

[ 0.523892] pci :00:1c.0: Intel SPT PCH root port ACS
workaround
enabled
[ 0.524031] pci :00:1c.4: Intel SPT PCH root port ACS
workaround
enabled
[ 0.524159] pci :00:1c.5: Intel SPT PCH root port ACS
workaround
enabled
[ 0.524292] pci :00:1d.0: Intel SPT PCH root port ACS
workaround
enabled


IOMMU Groups: http://pastebin.com/raw/0dcHk8Xk

lspci: http://pastebin.com/raw/1zAZuPBM



That's cool, how did you report your issue to Gigabyte? I'd like
to have
a try as well.

Wei


Alex, please let me know if they missed anything else, so I
can report
it to them.

Thanks,
Nick

On Sun, Sep 18, 2016 at 4:03 PM, Nick Sarnie

>> wrote:

Hi again,

Thanks a lot for investigating. I've reported the issue to the
manufacturer.


Thanks,
sarnex

On Sat, Sep 17, 2016 at 5:35 PM, Alex Williamson

>>
wrote:

On Sat, Sep 17, 2016 at 12:29 PM, Nick Sarnie

>> wrote:

Hi Alex,

The output is here: http://pastebin.com/raw/qjnpuaVr

>


Ok, you need to go complain to your motherboard manufacturer,
they're the ones hiding the ACS capability. PCIe capabilities
always start at 0x100, the dword there is:

100: 01 00 01 22 = 0x22010001

Breaking that down, the capability at 0x100 is ID 0x0001 (AER),
version 0x1, and the next capability is at 0x220. So we do the
same there:

220: 19 00 01 00 = 0x00010019

Capability ID 0x0019 (Secondary PCIe), version 0x1, next
capability 0x0, terminating the capability list.

Per Intel documentation for the chipset

(http://www.intel.com/content/www/us/en/chipsets/100-series-chipset-datasheet-vol-2.html



>),
the ACS capability and control registers live at 0x144 and 0x148
respectively and we can see that you do have data here matching
the default value of the capability register:

140: 00 00 00 00 0f 00 00 00 00 00 00 00 00 00 00 00

ie. default value of 0x144 is 0xf. It appears that this BIOS
vendor didn't connect the capability into the chain or fill in
the capability header. The registers to do this are RW/O, ie.
Read-Write-Once. IOW, the registers can only be written once,
which is intended to be used by the BIOS. The capability bits
themselves are RW/O, allowing vendors to expose different sets
of ACS capabilities. Given that this vendor has not exposed the
capability, we have no basis to believe that the default value
of the register represents the real capabilities of the system
and 

Re: [vfio-users] Z170X IOMMU Groups

[Nick Sarnie]

Wei,

I submitted a ticket here:
http://www.gigabyte.us/support-downloads/support-downloads.aspx

Thanks,
Nick

On Fri, Sep 23, 2016 at 12:06 AM, Brett Peckinpaugh 
wrote:

> Same here I have a ud5 that I would like a bios that does not need the ACS
> patch.
>
> On September 22, 2016 8:59:57 PM PDT, Wei Xu  wrote:
>
>>
>>
>> On 2016年09月23日 02:47, Nick Sarnie wrote:
>>
>>>  Hi again,
>>>
>>>  Very much to my surprise, Gigabyte replied and sent me a fixed BIOS. The
>>>  new IOMMU groups (with ACS override patch kernel commandline removed for
>>>  this boot), as well as my lspci information are below. I see four
>>>  messages the following messages in dmesg now:
>>>
>>>  [0.523892] pci :00:1c.0: Intel SPT PCH root port ACS workaround
>>>  enabled
>>>  [0.524031] pci :00:1c.4: Intel SPT PCH root port ACS workaround
>>>  enabled
>>>  [0.524159] pci :00:1c.5: Intel SPT PCH root port ACS workaround
>>>  enabled
>>>  [0.524292] pci :00:1d.0: Intel SPT PCH root port ACS workaround
>>>  enabled
>>>
>>>
>>>  IOMMU Groups: http://pastebin.com/raw/0dcHk8Xk
>>>
>>> lspci: http://pastebin.com/raw/1zAZuPBM
>>>
>>
>> That's cool, how did you report your issue to Gigabyte? I'd like to have
>> a try as well.
>>
>> Wei
>>
>>
>>>  Alex, please let me know if they missed anything else, so I can report
>>>  it to them.
>>>
>>>  Thanks,
>>>  Nick
>>>
>>>  On Sun, Sep 18, 2016 at 4:03 PM, Nick Sarnie >>  > wrote:
>>>
>>>  Hi again,
>>>
>>>  Thanks a lot for investigating. I've reported the issue to the
>>>  manufacturer.
>>>
>>>
>>>  Thanks,
>>>  sarnex
>>>
>>>  On Sat, Sep 17, 2016 at 5:35 PM, Alex Williamson
>>>  >
>>>  wrote:
>>>
>>>  On Sat, Sep
>>> 17, 2016 at 12:29 PM, Nick Sarnie
>>>  > wrote:
>>>
>>>  Hi Alex,
>>>
>>>  The output is here: http://pastebin.com/raw/qjnpuaVr
>>>  
>>>
>>>
>>>  Ok, you need to go complain to your motherboard manufacturer,
>>>  they're the ones hiding the ACS capability.  PCIe capabilities
>>>  always start at 0x100, the dword there is:
>>>
>>>  100: 01 00 01 22 = 0x22010001
>>>
>>>  Breaking that down, the capability at 0x100 is ID 0x0001 (AER),
>>>  version 0x1, and the next capability is at 0x220.  So we do the
>>>  same there:
>>>
>>>  220: 19 00 01 00 = 0x00010019
>>>
>>>  Capability ID 0x0019 (Secondary PCIe), version 0x1, next
>>>
>>> capability 0x0, terminating the capability list.
>>>
>>>  Per Intel documentation for the chipset
>>>  
>>> (http://www.intel.com/content/www/us/en/chipsets/100-series-chipset-datasheet-vol-2.html
>>>  
>>> ),
>>>  the ACS capability and control registers live at 0x144 and 0x148
>>>  respectively and we can see that you do have data here matching
>>>  the default value of the capability register:
>>>
>>>  140: 00 00 00 00 0f 00 00 00 00 00 00 00 00 00 00 00
>>>
>>>  ie. default value of 0x144 is 0xf.  It appears that this BIOS
>>>  vendor didn't connect the capability into the chain or fill in
>>>  the
>>> capability header.  The registers to do this are RW/O, ie.
>>>  Read-Write-Once.  IOW, the registers can only be written once,
>>>  which is intended to be used by the BIOS.  The capability bits
>>>  themselves are RW/O, allowing vendors to expose different sets
>>>  of ACS capabilities.  Given that this vendor has not exposed the
>>>  capability, we have no basis to believe that the default value
>>>  of the register represents the real capabilities of the system
>>>  and therefore we cannot assume we're able to control ACS.  File
>>>  a bug with the vendor or look for a BIOS update where they may
>>>  have already fixed this.
>>>
>>>  Also, is there any way we could move the USB controller into
>>>  its own group, or remove the Ethernet and SATA controller
>>>  into a seperate group? Ideally, I could pass the USB
>>>  Controller in group 7 without
>>> the ACS patch.
>>>
>>>
>>>  That's not how IOMMU groups works.  See
>>>  http://vfio.blogspot.com/2014/08/iommu-groups-inside-and-out.html 
>>> 
>>>We aren't creating these groups arbitrarily, we base them on
>>>  the information provided to use by the IOMMU driver and PCI
>>>  topology features, including ACS.  If we cannot determine that
>>>  there is isolation between 

Re: [vfio-users] Z170X IOMMU Groups

[Brett Peckinpaugh]

Same here I have a ud5 that I would like a bios that does not need the ACS 
patch. 

On September 22, 2016 8:59:57 PM PDT, Wei Xu  wrote:
>
>
>On 2016年09月23日 02:47, Nick Sarnie wrote:
>> Hi again,
>>
>> Very much to my surprise, Gigabyte replied and sent me a fixed BIOS.
>The
>> new IOMMU groups (with ACS override patch kernel commandline removed
>for
>> this boot), as well as my lspci information are below. I see four
>> messages the following messages in dmesg now:
>>
>> [0.523892] pci :00:1c.0: Intel SPT PCH root port ACS
>workaround
>> enabled
>> [0.524031] pci :00:1c.4: Intel SPT PCH root port ACS
>workaround
>> enabled
>> [0.524159] pci :00:1c.5: Intel SPT PCH root port ACS
>workaround
>> enabled
>> [0.524292] pci :00:1d.0: Intel SPT PCH root port ACS
>workaround
>> enabled
>>
>>
>> IOMMU Groups: http://pastebin.com/raw/0dcHk8Xk
>> lspci: http://pastebin.com/raw/1zAZuPBM
>
>That's cool, how did you report your issue to Gigabyte? I'd like to
>have 
>a try as well.
>
>Wei
>
>>
>> Alex, please let me know if they missed anything else, so I can
>report
>> it to them.
>>
>> Thanks,
>> Nick
>>
>> On Sun, Sep 18, 2016 at 4:03 PM, Nick Sarnie > > wrote:
>>
>> Hi again,
>>
>> Thanks a lot for investigating. I've reported the issue to the
>> manufacturer.
>>
>>
>> Thanks,
>> sarnex
>>
>> On Sat, Sep 17, 2016 at 5:35 PM, Alex Williamson
>> >
>> wrote:
>>
>> On Sat, Sep 17, 2016 at 12:29 PM, Nick Sarnie
>> >
>wrote:
>>
>> Hi Alex,
>>
>> The output is here: http://pastebin.com/raw/qjnpuaVr
>> 
>>
>>
>> Ok, you need to go complain to your motherboard manufacturer,
>> they're the ones hiding the ACS capability.  PCIe
>capabilities
>> always start at 0x100, the dword there is:
>>
>> 100: 01 00 01 22 = 0x22010001
>>
>> Breaking that down, the capability at 0x100 is ID 0x0001
>(AER),
>> version 0x1, and the next capability is at 0x220.  So we do
>the
>> same there:
>>
>> 220: 19 00 01 00 = 0x00010019
>>
>> Capability ID 0x0019 (Secondary PCIe), version 0x1, next
>> capability 0x0, terminating the capability list.
>>
>> Per Intel documentation for the chipset
>>
>(http://www.intel.com/content/www/us/en/chipsets/100-series-chipset-datasheet-vol-2.html
>>
>),
>> the ACS capability and control registers live at 0x144 and
>0x148
>> respectively and we can see that you do have data here
>matching
>> the default value of the capability register:
>>
>> 140: 00 00 00 00 0f 00 00 00 00 00 00 00 00 00 00 00
>>
>> ie. default value of 0x144 is 0xf.  It appears that this BIOS
>> vendor didn't connect the capability into the chain or fill
>in
>> the capability header.  The registers to do this are RW/O,
>ie.
>> Read-Write-Once.  IOW, the registers can only be written
>once,
>> which is intended to be used by the BIOS.  The capability
>bits
>> themselves are RW/O, allowing vendors to expose different
>sets
>> of ACS capabilities.  Given that this vendor has not exposed
>the
>> capability, we have no basis to believe that the default
>value
>> of the register represents the real capabilities of the
>system
>> and therefore we cannot assume we're able to control ACS. 
>File
>> a bug with the vendor or look for a BIOS update where they
>may
>> have already fixed this.
>>
>> Also, is there any way we could move the USB controller
>into
>> its own group, or remove the Ethernet and SATA controller
>> into a seperate group? Ideally, I could pass the USB
>> Controller in group 7 without the ACS patch.
>>
>>
>> That's not how IOMMU groups works.  See
>>
>http://vfio.blogspot.com/2014/08/iommu-groups-inside-and-out.html
>
>>   We aren't creating these groups arbitrarily, we base them
>on
>> the information provided to use by the IOMMU driver and PCI
>> topology features, including ACS.  If we cannot determine
>that
>> there is isolation between components, we must assume that
>they
>> are not isolated.  Your choices are to run an unsupported
>(and
>> unsupportable) configuration using the ACS override patch,
>get
>> your hardware vendor to fix their platform, or upgrade to
>better
>> hardware with better isolation characteristics.
>>
>>
>>
>>
>>
>> 

Re: [vfio-users] Z170X IOMMU Groups

[Wei Xu]

On 2016年09月23日 02:47, Nick Sarnie wrote:

Hi again,

Very much to my surprise, Gigabyte replied and sent me a fixed BIOS. The
new IOMMU groups (with ACS override patch kernel commandline removed for
this boot), as well as my lspci information are below. I see four
messages the following messages in dmesg now:

[0.523892] pci :00:1c.0: Intel SPT PCH root port ACS workaround
enabled
[0.524031] pci :00:1c.4: Intel SPT PCH root port ACS workaround
enabled
[0.524159] pci :00:1c.5: Intel SPT PCH root port ACS workaround
enabled
[0.524292] pci :00:1d.0: Intel SPT PCH root port ACS workaround
enabled


IOMMU Groups: http://pastebin.com/raw/0dcHk8Xk
lspci: http://pastebin.com/raw/1zAZuPBM


That's cool, how did you report your issue to Gigabyte? I'd like to have 
a try as well.


Wei



Alex, please let me know if they missed anything else, so I can report
it to them.

Thanks,
Nick

On Sun, Sep 18, 2016 at 4:03 PM, Nick Sarnie > wrote:

Hi again,

Thanks a lot for investigating. I've reported the issue to the
manufacturer.


Thanks,
sarnex

On Sat, Sep 17, 2016 at 5:35 PM, Alex Williamson
>
wrote:

On Sat, Sep 17, 2016 at 12:29 PM, Nick Sarnie
> wrote:

Hi Alex,

The output is here: http://pastebin.com/raw/qjnpuaVr



Ok, you need to go complain to your motherboard manufacturer,
they're the ones hiding the ACS capability.  PCIe capabilities
always start at 0x100, the dword there is:

100: 01 00 01 22 = 0x22010001

Breaking that down, the capability at 0x100 is ID 0x0001 (AER),
version 0x1, and the next capability is at 0x220.  So we do the
same there:

220: 19 00 01 00 = 0x00010019

Capability ID 0x0019 (Secondary PCIe), version 0x1, next
capability 0x0, terminating the capability list.

Per Intel documentation for the chipset

(http://www.intel.com/content/www/us/en/chipsets/100-series-chipset-datasheet-vol-2.html

),
the ACS capability and control registers live at 0x144 and 0x148
respectively and we can see that you do have data here matching
the default value of the capability register:

140: 00 00 00 00 0f 00 00 00 00 00 00 00 00 00 00 00

ie. default value of 0x144 is 0xf.  It appears that this BIOS
vendor didn't connect the capability into the chain or fill in
the capability header.  The registers to do this are RW/O, ie.
Read-Write-Once.  IOW, the registers can only be written once,
which is intended to be used by the BIOS.  The capability bits
themselves are RW/O, allowing vendors to expose different sets
of ACS capabilities.  Given that this vendor has not exposed the
capability, we have no basis to believe that the default value
of the register represents the real capabilities of the system
and therefore we cannot assume we're able to control ACS.  File
a bug with the vendor or look for a BIOS update where they may
have already fixed this.

Also, is there any way we could move the USB controller into
its own group, or remove the Ethernet and SATA controller
into a seperate group? Ideally, I could pass the USB
Controller in group 7 without the ACS patch.


That's not how IOMMU groups works.  See
http://vfio.blogspot.com/2014/08/iommu-groups-inside-and-out.html 

  We aren't creating these groups arbitrarily, we base them on
the information provided to use by the IOMMU driver and PCI
topology features, including ACS.  If we cannot determine that
there is isolation between components, we must assume that they
are not isolated.  Your choices are to run an unsupported (and
unsupportable) configuration using the ACS override patch, get
your hardware vendor to fix their platform, or upgrade to better
hardware with better isolation characteristics.





___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users



___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Z170X IOMMU Groups

[Rich Mingin (vfio-users)]

Well, I'd been considering getting a Z170 setup, if I do, I now know what
vendor. Kudos indeed.

On Thu, Sep 22, 2016 at 2:55 PM, Alex Williamson <
alex.l.william...@gmail.com> wrote:

> On Thu, Sep 22, 2016 at 12:47 PM, Nick Sarnie 
> wrote:
>
>> Hi again,
>>
>> Very much to my surprise, Gigabyte replied and sent me a fixed BIOS. The
>> new IOMMU groups (with ACS override patch kernel commandline removed for
>> this boot), as well as my lspci information are below. I see four messages
>> the following messages in dmesg now:
>>
>> [0.523892] pci :00:1c.0: Intel SPT PCH root port ACS workaround
>> enabled
>> [0.524031] pci :00:1c.4: Intel SPT PCH root port ACS workaround
>> enabled
>> [0.524159] pci :00:1c.5: Intel SPT PCH root port ACS workaround
>> enabled
>> [0.524292] pci :00:1d.0: Intel SPT PCH root port ACS workaround
>> enabled
>>
>>
>> IOMMU Groups: http://pastebin.com/raw/0dcHk8Xk
>> lspci: http://pastebin.com/raw/1zAZuPBM
>>
>> Alex, please let me know if they missed anything else, so I can report it
>> to them.
>>
>
> :-O
>
> /me picks jaw off the ground
>
> Kudos to Gigabyte!
>
> So we now have:
>
>  100: 01 00 01 14
>
> 0x14010001 = ID 0x0001 (AER), version 0x1, next 0x140
>
> 140: 0d 00 01 22
>
> 0x2201000d = ID 0x000d (ACS), version 0x1, next 0x220
>
> 220: 19 00 01 00
>
> 0x00010019 = ID 0x0019 (Secondary PCIe), version 0x1, next 0x0
>
> So they've just added ACS into the chain, perfect.  Thanks,
>
> Alex
>
> ___
> vfio-users mailing list
> vfio-users@redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users
>
>
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Z170X IOMMU Groups

[Alex Williamson]

On Thu, Sep 22, 2016 at 12:47 PM, Nick Sarnie 
wrote:

> Hi again,
>
> Very much to my surprise, Gigabyte replied and sent me a fixed BIOS. The
> new IOMMU groups (with ACS override patch kernel commandline removed for
> this boot), as well as my lspci information are below. I see four messages
> the following messages in dmesg now:
>
> [0.523892] pci :00:1c.0: Intel SPT PCH root port ACS workaround
> enabled
> [0.524031] pci :00:1c.4: Intel SPT PCH root port ACS workaround
> enabled
> [0.524159] pci :00:1c.5: Intel SPT PCH root port ACS workaround
> enabled
> [0.524292] pci :00:1d.0: Intel SPT PCH root port ACS workaround
> enabled
>
>
> IOMMU Groups: http://pastebin.com/raw/0dcHk8Xk
> lspci: http://pastebin.com/raw/1zAZuPBM
>
> Alex, please let me know if they missed anything else, so I can report it
> to them.
>

:-O

/me picks jaw off the ground

Kudos to Gigabyte!

So we now have:

 100: 01 00 01 14

0x14010001 = ID 0x0001 (AER), version 0x1, next 0x140

140: 0d 00 01 22

0x2201000d = ID 0x000d (ACS), version 0x1, next 0x220

220: 19 00 01 00

0x00010019 = ID 0x0019 (Secondary PCIe), version 0x1, next 0x0

So they've just added ACS into the chain, perfect.  Thanks,

Alex
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Z170X IOMMU Groups

[Nick Sarnie]

Hi again,

Thanks a lot for investigating. I've reported the issue to the manufacturer.


Thanks,
sarnex

On Sat, Sep 17, 2016 at 5:35 PM, Alex Williamson <
alex.l.william...@gmail.com> wrote:

> On Sat, Sep 17, 2016 at 12:29 PM, Nick Sarnie 
> wrote:
>
>> Hi Alex,
>>
>> The output is here: http://pastebin.com/raw/qjnpuaVr
>>
>
> Ok, you need to go complain to your motherboard manufacturer, they're the
> ones hiding the ACS capability.  PCIe capabilities always start at 0x100,
> the dword there is:
>
> 100: 01 00 01 22 = 0x22010001
>
> Breaking that down, the capability at 0x100 is ID 0x0001 (AER), version
> 0x1, and the next capability is at 0x220.  So we do the same there:
>
> 220: 19 00 01 00 = 0x00010019
>
> Capability ID 0x0019 (Secondary PCIe), version 0x1, next capability 0x0,
> terminating the capability list.
>
> Per Intel documentation for the chipset (http://www.intel.com/content/
> www/us/en/chipsets/100-series-chipset-datasheet-vol-2.html), the ACS
> capability and control registers live at 0x144 and 0x148 respectively and
> we can see that you do have data here matching the default value of the
> capability register:
>
> 140: 00 00 00 00 0f 00 00 00 00 00 00 00 00 00 00 00
>
> ie. default value of 0x144 is 0xf.  It appears that this BIOS vendor
> didn't connect the capability into the chain or fill in the capability
> header.  The registers to do this are RW/O, ie. Read-Write-Once.  IOW, the
> registers can only be written once, which is intended to be used by the
> BIOS.  The capability bits themselves are RW/O, allowing vendors to expose
> different sets of ACS capabilities.  Given that this vendor has not exposed
> the capability, we have no basis to believe that the default value of the
> register represents the real capabilities of the system and therefore we
> cannot assume we're able to control ACS.  File a bug with the vendor or
> look for a BIOS update where they may have already fixed this.
>
>
>> Also, is there any way we could move the USB controller into its own
>> group, or remove the Ethernet and SATA controller into a seperate group?
>> Ideally, I could pass the USB Controller in group 7 without the ACS patch.
>>
>
> That's not how IOMMU groups works.  See  http://vfio.blogspot.com/
> 2014/08/iommu-groups-inside-and-out.html  We aren't creating these groups
> arbitrarily, we base them on the information provided to use by the IOMMU
> driver and PCI topology features, including ACS.  If we cannot determine
> that there is isolation between components, we must assume that they are
> not isolated.  Your choices are to run an unsupported (and unsupportable)
> configuration using the ACS override patch, get your hardware vendor to fix
> their platform, or upgrade to better hardware with better isolation
> characteristics.
>
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Z170X IOMMU Groups

[Alex Williamson]

On Sat, Sep 17, 2016 at 9:12 AM, Alex Williamson <
alex.l.william...@gmail.com> wrote:

> On Sat, Sep 17, 2016 at 9:00 AM, Nick Sarnie 
> wrote:
>
>> Hi Alex,
>>
>> I'm on 4.7.4 which includes this patch, and there are the IOMMU groups.
>> Is there some extra info I can provide?
>>
>
> Hmm, based on the info you sent me previously, your PCH root ports don't
> even attempt to include the broken ACS capability, therefore the quirk
> doesn't get enabled on your system.  Perhaps the Z170X is an especially
> broken version of Z170 :-\
>

Could you pastebin a dump of PCI config space for the PCH root ports?  ie.
"sudo lspci -s 1c."
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Z170X IOMMU Groups

[Alex Williamson]

On Sat, Sep 17, 2016 at 9:00 AM, Nick Sarnie 
wrote:

> Hi Alex,
>
> I'm on 4.7.4 which includes this patch, and there are the IOMMU groups. Is
> there some extra info I can provide?
>

Hmm, based on the info you sent me previously, your PCH root ports don't
even attempt to include the broken ACS capability, therefore the quirk
doesn't get enabled on your system.  Perhaps the Z170X is an especially
broken version of Z170 :-\
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Z170X IOMMU Groups

[Nick Sarnie]

Hi Alex,

I'm on 4.7.4 which includes this patch, and there are the IOMMU groups. Is
there some extra info I can provide?

Thanks,
Sarnex

On Sat, Sep 17, 2016 at 4:40 AM, Philip Abernethy 
wrote:

> You definitely are. Running Skylake on Arch myself. Had the mainline
> package before, now using stock 4.7. I decided to buy a USB card to pass
> through.
>
> On Sat, 17 Sep 2016, 03:58 Brett Peckinpaugh,  wrote:
>
>> So I might be able to drop the ACS patch now that 4.7 is out on Arch.
>>
>> On September 16, 2016 5:39:43 PM PDT, Alex Williamson <
>> alex.l.william...@gmail.com> wrote:
>>>
>>>
>>> On Fri, Sep 16, 2016 at 6:30 PM, Nick Sarnie 
>>> wrote:
>>>
 Hi,

 I was wondering if we could split group 7 any more. CPU is the 6700k.
 I'd like to be able to pass through the USB controller without the ACS
 patch.

>>>
>>> Run a newer kernel, Intel botched ACS in Skylake:
>>>
>>> http://git.kernel.org/cgit/linux/kernel/git/torvalds/
>>> linux.git/commit/drivers/pci/quirks.c?id=1bf2bf229b64540f91ac6fa3af37c8
>>> 1249037a0b
>>>
>>> PCH PCIe root ports have isolation starting in v4.7.
>>>
>> vfio-users mailing list
>> vfio-users@redhat.com
>> https://www.redhat.com/mailman/listinfo/vfio-users
>>
>> ___
>> vfio-users mailing list
>> vfio-users@redhat.com
>> https://www.redhat.com/mailman/listinfo/vfio-users
>>
>
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Z170X IOMMU Groups

[Philip Abernethy]

You definitely are. Running Skylake on Arch myself. Had the mainline
package before, now using stock 4.7. I decided to buy a USB card to pass
through.

On Sat, 17 Sep 2016, 03:58 Brett Peckinpaugh,  wrote:

> So I might be able to drop the ACS patch now that 4.7 is out on Arch.
>
> On September 16, 2016 5:39:43 PM PDT, Alex Williamson <
> alex.l.william...@gmail.com> wrote:
>>
>>
>> On Fri, Sep 16, 2016 at 6:30 PM, Nick Sarnie 
>> wrote:
>>
>>> Hi,
>>>
>>> I was wondering if we could split group 7 any more. CPU is the 6700k.
>>> I'd like to be able to pass through the USB controller without the ACS
>>> patch.
>>>
>>
>> Run a newer kernel, Intel botched ACS in Skylake:
>>
>>
>> http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/drivers/pci/quirks.c?id=1bf2bf229b64540f91ac6fa3af37c81249037a0b
>>
>> PCH PCIe root ports have isolation starting in v4.7.
>>
> vfio-users mailing list
> vfio-users@redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users
>
> ___
> vfio-users mailing list
> vfio-users@redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users
>
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Z170X IOMMU Groups

[Alex Williamson]

On Fri, Sep 16, 2016 at 6:30 PM, Nick Sarnie 
wrote:

> Hi,
>
> I was wondering if we could split group 7 any more. CPU is the 6700k. I'd
> like to be able to pass through the USB controller without the ACS patch.
>

Run a newer kernel, Intel botched ACS in Skylake:

http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/drivers/pci/quirks.c?id=1bf2bf229b64540f91ac6fa3af37c81249037a0b

PCH PCIe root ports have isolation starting in v4.7.
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Z170X IOMMU Groups

[Brett Peckinpaugh]

Running skylake myself and the IOMMU groups suck. Only options are the 
unsupported ACS patch or to passthrough the USB devices and leave the 
controller with the host. 

On September 16, 2016 5:30:38 PM PDT, Nick Sarnie  
wrote:
>Hi,
>
>I was wondering if we could split group 7 any more. CPU is the 6700k.
>I'd
>like to be able to pass through the USB controller without the ACS
>patch.
>
>Thanks,
>Sarnex
>
>IOMMU group 0
>00:00.0 Host bridge [0600]: Intel Corporation Skylake Host
>Bridge/DRAM Registers [8086:191f] (rev 07)
>IOMMU group 1
>00:01.0 PCI bridge [0604]: Intel Corporation Skylake PCIe
>Controller (x16) [8086:1901] (rev 07)
>  01:00.0 VGA compatible controller [0300]: Advanced Micro Devices,
>Inc. [AMD/ATI] Ellesmere [Radeon RX 480] [1002:67df] (rev c7)
>01:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI]
>Device [1002:aaf0]
>IOMMU group 2
>   00:02.0 Display controller [0380]: Intel Corporation HD Graphics
>530 [8086:1912] (rev 06)
>IOMMU group 3
>   00:14.0 USB controller [0c03]: Intel Corporation Sunrise Point-H
>USB 3.0 xHCI Controller [8086:a12f] (rev 31)
>00:14.2 Signal processing controller [1180]: Intel Corporation
>Sunrise Point-H Thermal subsystem [8086:a131] (rev 31)
>IOMMU group 4
> 00:16.0 Communication controller [0780]: Intel Corporation Sunrise
>Point-H CSME HECI #1 [8086:a13a] (rev 31)
>IOMMU group 5
>  00:17.0 SATA controller [0106]: Intel Corporation Sunrise Point-H
>SATA controller [AHCI mode] [8086:a102] (rev 31)
>IOMMU group 6
>   00:1b.0 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI
>Root Port #17 [8086:a167] (rev f1)
>IOMMU group 7
>   00:1c.0 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI
>Express Root Port #1 [8086:a110] (rev f1)
>   00:1c.4 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI
>Express Root Port #5 [8086:a114] (rev f1)
>   00:1c.5 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI
>Express Root Port #6 [8086:a115] (rev f1)
> 03:00.0 PCI bridge [0604]: Intel Corporation DSL6540 Thunderbolt 3
>Bridge [Alpine Ridge 4C 2015] [8086:1578]
> 04:00.0 PCI bridge [0604]: Intel Corporation DSL6540 Thunderbolt 3
>Bridge [Alpine Ridge 4C 2015] [8086:1578]
> 04:01.0 PCI bridge [0604]: Intel Corporation DSL6540 Thunderbolt 3
>Bridge [Alpine Ridge 4C 2015] [8086:1578]
> 04:02.0 PCI bridge [0604]: Intel Corporation DSL6540 Thunderbolt 3
>Bridge [Alpine Ridge 4C 2015] [8086:1578]
> 04:04.0 PCI bridge [0604]: Intel Corporation DSL6540 Thunderbolt 3
>Bridge [Alpine Ridge 4C 2015] [8086:1578]
>   07:00.0 USB controller [0c03]: Intel Corporation DSL6540 USB 3.1
>Controller [Alpine Ridge] [8086:15b6]
> 09:00.0 Ethernet controller [0200]: Intel Corporation I211 Gigabit
>Network Connection [8086:1539] (rev 03)
>0a:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062
>Serial ATA Controller [1b21:0612] (rev 02)
>IOMMU group 8
>   00:1d.0 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI
>Express Root Port #9 [8086:a118] (rev f1)
>   00:1d.4 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI
>Express Root Port #13 [8086:a11c] (rev f1)
>0b:00.0 Network controller [0280]: Qualcomm Atheros AR93xx Wireless
>Network Adapter [168c:0030] (rev 01)
>IOMMU group 9
>   00:1f.0 ISA bridge [0601]: Intel Corporation Sunrise Point-H LPC
>Controller [8086:a145] (rev 31)
>00:1f.2 Memory controller [0580]: Intel Corporation Sunrise Point-H
>PMC [8086:a121] (rev 31)
>  00:1f.3 Audio device [0403]: Intel Corporation Sunrise Point-H HD
>Audio [8086:a170] (rev 31)
>00:1f.4 SMBus [0c05]: Intel Corporation Sunrise Point-H SMBus
>[8086:a123] (rev 31)
>IOMMU group 10
>00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet
>Connection (2) I219-V [8086:15b8] (rev 31)
>
>
>
>
>___
>vfio-users mailing list
>vfio-users@redhat.com
>https://www.redhat.com/mailman/listinfo/vfio-users
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

[vfio-users] Z170X IOMMU Groups

[Nick Sarnie]

Hi,

I was wondering if we could split group 7 any more. CPU is the 6700k. I'd
like to be able to pass through the USB controller without the ACS patch.

Thanks,
Sarnex

IOMMU group 0
00:00.0 Host bridge [0600]: Intel Corporation Skylake Host
Bridge/DRAM Registers [8086:191f] (rev 07)
IOMMU group 1
00:01.0 PCI bridge [0604]: Intel Corporation Skylake PCIe
Controller (x16) [8086:1901] (rev 07)
01:00.0 VGA compatible controller [0300]: Advanced Micro Devices,
Inc. [AMD/ATI] Ellesmere [Radeon RX 480] [1002:67df] (rev c7)
01:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI]
Device [1002:aaf0]
IOMMU group 2
00:02.0 Display controller [0380]: Intel Corporation HD Graphics
530 [8086:1912] (rev 06)
IOMMU group 3
00:14.0 USB controller [0c03]: Intel Corporation Sunrise Point-H
USB 3.0 xHCI Controller [8086:a12f] (rev 31)
00:14.2 Signal processing controller [1180]: Intel Corporation
Sunrise Point-H Thermal subsystem [8086:a131] (rev 31)
IOMMU group 4
00:16.0 Communication controller [0780]: Intel Corporation Sunrise
Point-H CSME HECI #1 [8086:a13a] (rev 31)
IOMMU group 5
00:17.0 SATA controller [0106]: Intel Corporation Sunrise Point-H
SATA controller [AHCI mode] [8086:a102] (rev 31)
IOMMU group 6
00:1b.0 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI
Root Port #17 [8086:a167] (rev f1)
IOMMU group 7
00:1c.0 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI
Express Root Port #1 [8086:a110] (rev f1)
00:1c.4 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI
Express Root Port #5 [8086:a114] (rev f1)
00:1c.5 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI
Express Root Port #6 [8086:a115] (rev f1)
03:00.0 PCI bridge [0604]: Intel Corporation DSL6540 Thunderbolt 3
Bridge [Alpine Ridge 4C 2015] [8086:1578]
04:00.0 PCI bridge [0604]: Intel Corporation DSL6540 Thunderbolt 3
Bridge [Alpine Ridge 4C 2015] [8086:1578]
04:01.0 PCI bridge [0604]: Intel Corporation DSL6540 Thunderbolt 3
Bridge [Alpine Ridge 4C 2015] [8086:1578]
04:02.0 PCI bridge [0604]: Intel Corporation DSL6540 Thunderbolt 3
Bridge [Alpine Ridge 4C 2015] [8086:1578]
04:04.0 PCI bridge [0604]: Intel Corporation DSL6540 Thunderbolt 3
Bridge [Alpine Ridge 4C 2015] [8086:1578]
07:00.0 USB controller [0c03]: Intel Corporation DSL6540 USB 3.1
Controller [Alpine Ridge] [8086:15b6]
09:00.0 Ethernet controller [0200]: Intel Corporation I211 Gigabit
Network Connection [8086:1539] (rev 03)
0a:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062
Serial ATA Controller [1b21:0612] (rev 02)
IOMMU group 8
00:1d.0 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI
Express Root Port #9 [8086:a118] (rev f1)
00:1d.4 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI
Express Root Port #13 [8086:a11c] (rev f1)
0b:00.0 Network controller [0280]: Qualcomm Atheros AR93xx Wireless
Network Adapter [168c:0030] (rev 01)
IOMMU group 9
00:1f.0 ISA bridge [0601]: Intel Corporation Sunrise Point-H LPC
Controller [8086:a145] (rev 31)
00:1f.2 Memory controller [0580]: Intel Corporation Sunrise Point-H
PMC [8086:a121] (rev 31)
00:1f.3 Audio device [0403]: Intel Corporation Sunrise Point-H HD
Audio [8086:a170] (rev 31)
00:1f.4 SMBus [0c05]: Intel Corporation Sunrise Point-H SMBus
[8086:a123] (rev 31)
IOMMU group 10
00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet
Connection (2) I219-V [8086:15b8] (rev 31)
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users