Re: [Xen-devel] pre Sandy bridge IOMMU support (gm45)

[Thierry Laurion]

Sorry for the precedent post that was written a bit too fast. Libreboot was
flashed when I wrote it, which is the equivalent of a having vt-d
deactivated (iommu=0). Thanks to a user that read this post and wrote to me
personally so I could do my mea culpa. Sorry for the precedent misleading
post.

Xen on a GM45 chipset and with IGD i915 driver is still getting the system
hanged when vt-d is activated. I'm willing to borrow a machine to the Xen
developer that could fix the iommu=no-igfx code for gm45 chipset to
actually work.

A ticket is opened here with current states of thing:
https://github.com/QubesOS/qubes-issues/issues/1594#issuecomment-209213917

Sorry about that.
Thierry

Le dim. 28 févr. 2016 à 14:08, Thierry Laurion 
a écrit :

> The problem wasn't with xen iommu support but kms/drm and i915 driver.
>
> Passing to the kernel i915.preliminary_hw_support=1 fixes it all :)
>
> Thanks
>
> Le sam. 20 févr. 2016 à 22:44, Thierry Laurion 
> a écrit :
>
>> Le mar. 26 janv. 2016 à 05:52, Jan Beulich  a écrit :
>>
>>> >>> On 25.01.16 at 22:49,  wrote:
>>> > The case is 1) disabling iommu for IGD, unilaterally since i915 + gm45
>>> > doesn't play well together. Iommu is still desired to isolate usb and
>>> > network devices, so we don't want to disable iommu completely. The side
>>> > effect of this would be to have IGD only for dom0, which would also
>>> > completely make sense in this use case.
>>> >
>>> > The point is the iommu=no-igfx doesn't fix the issue, since remapping
>>> seems
>>> > to still happen for IGD. Does that make sense ?
>>>
>>> It certainly may make sense, just that in what you have written so
>>> far I don't think I've been able to spot any evidence thereof. Since,
>>> as you say, nothing interesting gets logged by Xen, you must be
>>> drawing this conclusion from something (or else you wouldn't say
>>> "doesn't fix the issue").
>>>
>>> Jan
>>>
>>>
>> Here is some interesting lines showing Xen failing without iommu=no-igfx:
>>
>> --- /home/john/Downloads/amtterm/x200_xen_debug-normal-no_ts.txt
>> +++ /home/john/Downloads/amtterm/x200_xen_debug-iommu-no_igfx-no_ts.txt
>> @@ -339,23 +339,10 @@
>> (XEN) [VT-D]iommu.c:1465: d0:PCI: map :00:1f.3
>> (XEN) [VT-D]iommu.c:1453: d0:PCIe: map :03:00.0
>> (XEN) [VT-D]iommu.c:729: iommu_enable_translation: iommu->reg =
>> 82c000205000
>> -(XEN) [VT-D]iommu.c:729: iommu_enable_translation: iommu->reg =
>> 82c000203000
>> +(XEN) [VT-D]iommu.c:719: BIOS did not enable IGD for VT properly.
>> Disabling IGD VT-d engine.
>> (XEN) [VT-D]iommu.c:729: iommu_enable_translation: iommu->reg =
>> 82c000201000
>> (XEN) [VT-D]iommu.c:729: iommu_enable_translation: iommu->reg =
>> 82c000207000
>> (XEN) Scrubbing Free RAM on 1 nodes using 2 CPUs
>> -(XEN) [VT-D]iommu.c:873: iommu_fault_status: Fault Overflow
>> -(XEN) [VT-D]iommu.c:875: iommu_fault_status: Primary Pending Fault
>> -(XEN) [VT-D]DMAR:[DMA Write] Request device [:00:02.0] fault addr
>> ff000, iommu reg = 82c000203000
>> -(XEN) [VT-D]DMAR: reason 05 - PTE Write access is not set
>> -(XEN) print_vtd_entries: iommu 8301363fa7d0 dev :00:02.0 gmfn
>> ff
>> -(XEN) root_entry = 8301363f4000
>> -(XEN) root_entry[0] = 80fa001
>> -(XEN) context = 8300080fa000
>> -(XEN) context[10] = 1_8ae0001
>> -(XEN) l3 = 830008ae
>> -(XEN) l3_index = 3f
>> -(XEN) l3[3f] = 0
>> -(XEN) l3[3f] not present
>> (XEN) done.
>> (XEN) Initial low memory virq threshold set at 0x4000 pages.
>> (XEN) Std. Loglevel: All
>>
>> I restate my comprehension.
>> iommu=no-igfx needs to be passed to hypervisor for it to boot.
>> iommu=dom0-passthrough would also be needed for dom0 tobe able to unset
>> iommu usage for itself?
>>
>> For Dom0 to have access to device, I also understand that
>> intel_iommu=igfx_off kernel option would need to be passed to i915 driver
>> of dom0?
>>
>> Doing so still fails without error. Any hint?
>> Doing so by providing dom0-pass
>>
>>
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] pre Sandy bridge IOMMU support (gm45)

[Thierry Laurion]

The problem wasn't with xen iommu support but kms/drm and i915 driver.

Passing to the kernel i915.preliminary_hw_support=1 fixes it all :)

Thanks

Le sam. 20 févr. 2016 à 22:44, Thierry Laurion 
a écrit :

> Le mar. 26 janv. 2016 à 05:52, Jan Beulich  a écrit :
>
>> >>> On 25.01.16 at 22:49,  wrote:
>> > The case is 1) disabling iommu for IGD, unilaterally since i915 + gm45
>> > doesn't play well together. Iommu is still desired to isolate usb and
>> > network devices, so we don't want to disable iommu completely. The side
>> > effect of this would be to have IGD only for dom0, which would also
>> > completely make sense in this use case.
>> >
>> > The point is the iommu=no-igfx doesn't fix the issue, since remapping
>> seems
>> > to still happen for IGD. Does that make sense ?
>>
>> It certainly may make sense, just that in what you have written so
>> far I don't think I've been able to spot any evidence thereof. Since,
>> as you say, nothing interesting gets logged by Xen, you must be
>> drawing this conclusion from something (or else you wouldn't say
>> "doesn't fix the issue").
>>
>> Jan
>>
>>
> Here is some interesting lines showing Xen failing without iommu=no-igfx:
>
> --- /home/john/Downloads/amtterm/x200_xen_debug-normal-no_ts.txt
> +++ /home/john/Downloads/amtterm/x200_xen_debug-iommu-no_igfx-no_ts.txt
> @@ -339,23 +339,10 @@
> (XEN) [VT-D]iommu.c:1465: d0:PCI: map :00:1f.3
> (XEN) [VT-D]iommu.c:1453: d0:PCIe: map :03:00.0
> (XEN) [VT-D]iommu.c:729: iommu_enable_translation: iommu->reg =
> 82c000205000
> -(XEN) [VT-D]iommu.c:729: iommu_enable_translation: iommu->reg =
> 82c000203000
> +(XEN) [VT-D]iommu.c:719: BIOS did not enable IGD for VT properly.
> Disabling IGD VT-d engine.
> (XEN) [VT-D]iommu.c:729: iommu_enable_translation: iommu->reg =
> 82c000201000
> (XEN) [VT-D]iommu.c:729: iommu_enable_translation: iommu->reg =
> 82c000207000
> (XEN) Scrubbing Free RAM on 1 nodes using 2 CPUs
> -(XEN) [VT-D]iommu.c:873: iommu_fault_status: Fault Overflow
> -(XEN) [VT-D]iommu.c:875: iommu_fault_status: Primary Pending Fault
> -(XEN) [VT-D]DMAR:[DMA Write] Request device [:00:02.0] fault addr
> ff000, iommu reg = 82c000203000
> -(XEN) [VT-D]DMAR: reason 05 - PTE Write access is not set
> -(XEN) print_vtd_entries: iommu 8301363fa7d0 dev :00:02.0 gmfn
> ff
> -(XEN) root_entry = 8301363f4000
> -(XEN) root_entry[0] = 80fa001
> -(XEN) context = 8300080fa000
> -(XEN) context[10] = 1_8ae0001
> -(XEN) l3 = 830008ae
> -(XEN) l3_index = 3f
> -(XEN) l3[3f] = 0
> -(XEN) l3[3f] not present
> (XEN) done.
> (XEN) Initial low memory virq threshold set at 0x4000 pages.
> (XEN) Std. Loglevel: All
>
> I restate my comprehension.
> iommu=no-igfx needs to be passed to hypervisor for it to boot.
> iommu=dom0-passthrough would also be needed for dom0 tobe able to unset
> iommu usage for itself?
>
> For Dom0 to have access to device, I also understand that
> intel_iommu=igfx_off kernel option would need to be passed to i915 driver
> of dom0?
>
> Doing so still fails without error. Any hint?
> Doing so by providing dom0-pass
>
>
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] pre Sandy bridge IOMMU support (gm45)

[Thierry Laurion]

Le mar. 26 janv. 2016 à 05:52, Jan Beulich  a écrit :

> >>> On 25.01.16 at 22:49,  wrote:
> > The case is 1) disabling iommu for IGD, unilaterally since i915 + gm45
> > doesn't play well together. Iommu is still desired to isolate usb and
> > network devices, so we don't want to disable iommu completely. The side
> > effect of this would be to have IGD only for dom0, which would also
> > completely make sense in this use case.
> >
> > The point is the iommu=no-igfx doesn't fix the issue, since remapping
> seems
> > to still happen for IGD. Does that make sense ?
>
> It certainly may make sense, just that in what you have written so
> far I don't think I've been able to spot any evidence thereof. Since,
> as you say, nothing interesting gets logged by Xen, you must be
> drawing this conclusion from something (or else you wouldn't say
> "doesn't fix the issue").
>
> Jan
>
>
Here is some interesting lines showing Xen failing without iommu=no-igfx:

--- /home/john/Downloads/amtterm/x200_xen_debug-normal-no_ts.txt
+++ /home/john/Downloads/amtterm/x200_xen_debug-iommu-no_igfx-no_ts.txt
@@ -339,23 +339,10 @@
(XEN) [VT-D]iommu.c:1465: d0:PCI: map :00:1f.3
(XEN) [VT-D]iommu.c:1453: d0:PCIe: map :03:00.0
(XEN) [VT-D]iommu.c:729: iommu_enable_translation: iommu->reg =
82c000205000
-(XEN) [VT-D]iommu.c:729: iommu_enable_translation: iommu->reg =
82c000203000
+(XEN) [VT-D]iommu.c:719: BIOS did not enable IGD for VT properly.
Disabling IGD VT-d engine.
(XEN) [VT-D]iommu.c:729: iommu_enable_translation: iommu->reg =
82c000201000
(XEN) [VT-D]iommu.c:729: iommu_enable_translation: iommu->reg =
82c000207000
(XEN) Scrubbing Free RAM on 1 nodes using 2 CPUs
-(XEN) [VT-D]iommu.c:873: iommu_fault_status: Fault Overflow
-(XEN) [VT-D]iommu.c:875: iommu_fault_status: Primary Pending Fault
-(XEN) [VT-D]DMAR:[DMA Write] Request device [:00:02.0] fault addr
ff000, iommu reg = 82c000203000
-(XEN) [VT-D]DMAR: reason 05 - PTE Write access is not set
-(XEN) print_vtd_entries: iommu 8301363fa7d0 dev :00:02.0 gmfn
ff
-(XEN) root_entry = 8301363f4000
-(XEN) root_entry[0] = 80fa001
-(XEN) context = 8300080fa000
-(XEN) context[10] = 1_8ae0001
-(XEN) l3 = 830008ae
-(XEN) l3_index = 3f
-(XEN) l3[3f] = 0
-(XEN) l3[3f] not present
(XEN) done.
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Std. Loglevel: All

I restate my comprehension.
iommu=no-igfx needs to be passed to hypervisor for it to boot.
iommu=dom0-passthrough would also be needed for dom0 tobe able to unset
iommu usage for itself?

For Dom0 to have access to device, I also understand that
intel_iommu=igfx_off kernel option would need to be passed to i915 driver
of dom0?

Doing so still fails without error. Any hint?
Doing so by providing dom0-pass
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] pre Sandy bridge IOMMU support (gm45)

[Thierry Laurion]

Le lun. 1 févr. 2016 à 07:35, Jan Beulich  a écrit :

> >>> On 01.02.16 at 13:28,  wrote:
> > On Mon, Feb 01, 2016 at 12:59:00AM -0700, Jan Beulich wrote:
> >> >>> On 30.01.16 at 02:47,  wrote:
> >> > On Tue, Jan 26, 2016 at 04:37:05AM -0700, Jan Beulich wrote:
> >> >> (re-adding xen-devel)
> >> >>
> >> >> >>> On 26.01.16 at 12:28,  wrote:
> >> >> > Iommu=0 let the whole Qubes system work, without enforcing hardware
> >> >> > compartimentalisation (iommu is enforced in software mode)
> >> >> >
> >> >> > When iommu=no-igfx is enforced, shell console boot up works
> flawlessly.
> > All
> >> >> > domu machines get booted up. A system hang will happen at the
> moment a
> > domu
> >> >> > machine does graphic rendering,
> >> >>
> >> >> And this is (other than I originally implied) without passing through
> >> >> the IGD to the DomU? If so, I can't see the difference between a
> >> >> guest rendering to its display (and vncviewer or whatever frontend
> >> >> you use converting this to rendering on the host) and rendering
> >> >> which originates in the host.
> >> >
> >> > Not sure if relevant, but window content is mapped from PV domU
> directly
> >> > into X server (in dom0) address space, using xc_map_foreign_pages. It
> is
> >> > done by hacking XShmAttach function. Not sure what graphics driver do
> >> > with it next. Theoretically it could be possible that driver will
> direct IGD
> >> > to do DMA directly from that place, but I guess it does not.
> >>
> >> Interesting. This then really needs to be investigated from the
> >> Qubes end rather than here. Possible resulting patches, if
> >> relevant outside of that unusual setup, would then of course be
> >> appreciated to be sent here.
> >
>
Interesting fact: kde corrupts video buffer and make system hang faster
then it's xfce counterparts.

> > Note that Thierry said "The point is the iommu=no-igfx doesn't fix the
> > issue", so either it is totally unrelated issue, or iommu=no-igfx is
> > broken. Does iommu=no-igfx have any meaning when IDG is _not_ passed
> > through to some domU?
>
> Yes: Iirc that option turns off the IOMMU responsible for IGD.
>
>  I suppose passing iommu=dom0-passthrough to hypervisor and passing
*intel_iommu*=*igfx_off to dom0 *may be what is desired here, but
hypervisor refuses to boot without iommu=no-igfx flag.

> And generally - is IOMMU used in any way for dom0
> > devices?
>
> Of course; by how much depends on what options you use (see
> the command line doc).
>
Qubes does't use any specifics.

>
> > Is Linux kernel able to utilize it for its own purposes (my
> > guess: no)?
>
> Under Xen? If so - indeed, no.
>
Wouldn't it be more natural if i915 iommu would be deactivated in kernel
with *intel_iommu*=*igfx_off option being passed to dom0?*

>
> > Somehow unrelated: I've tried to get p2m iommu mapping using `xl
> > debug-key o`, but it's too long (and xenconsoled isn't fast enough to
> > catch it into hypervisor.log). Is is possible to enlarge console ring
> > buffer at runtime?
>
> No.
>
> > If not, is `conring_size` the right option?
>
> Yes.
>
> > How large
> > it should be (aprox) for `xl debug-key o` output?
>
> Depends on the amount of memory page tables need to be created
> for as well as how fragmented memory was at the time memory
> allocation for domains happens. I'm afraid you can only try it out.
>
> Jan
>
> Would it be possible to lend a x200 laptop to the most
interested/motivated developer in making i915/iommu work correctly for this
laptop? I don't see how I can troubleshoot this deeper by myself alone.

Thierry
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] pre Sandy bridge IOMMU support (gm45)

[Jan Beulich]

>>> On 30.01.16 at 02:47,  wrote:
> On Tue, Jan 26, 2016 at 04:37:05AM -0700, Jan Beulich wrote:
>> (re-adding xen-devel)
>> 
>> >>> On 26.01.16 at 12:28,  wrote:
>> > Iommu=0 let the whole Qubes system work, without enforcing hardware
>> > compartimentalisation (iommu is enforced in software mode)
>> > 
>> > When iommu=no-igfx is enforced, shell console boot up works flawlessly. All
>> > domu machines get booted up. A system hang will happen at the moment a domu
>> > machine does graphic rendering,
>> 
>> And this is (other than I originally implied) without passing through
>> the IGD to the DomU? If so, I can't see the difference between a
>> guest rendering to its display (and vncviewer or whatever frontend
>> you use converting this to rendering on the host) and rendering
>> which originates in the host.
> 
> Not sure if relevant, but window content is mapped from PV domU directly
> into X server (in dom0) address space, using xc_map_foreign_pages. It is
> done by hacking XShmAttach function. Not sure what graphics driver do
> with it next. Theoretically it could be possible that driver will direct IGD
> to do DMA directly from that place, but I guess it does not.

Interesting. This then really needs to be investigated from the
Qubes end rather than here. Possible resulting patches, if
relevant outside of that unusual setup, would then of course be
appreciated to be sent here.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] pre Sandy bridge IOMMU support (gm45)

[Marek Marczykowski-Górecki]

On Mon, Feb 01, 2016 at 12:59:00AM -0700, Jan Beulich wrote:
> >>> On 30.01.16 at 02:47,  wrote:
> > On Tue, Jan 26, 2016 at 04:37:05AM -0700, Jan Beulich wrote:
> >> (re-adding xen-devel)
> >> 
> >> >>> On 26.01.16 at 12:28,  wrote:
> >> > Iommu=0 let the whole Qubes system work, without enforcing hardware
> >> > compartimentalisation (iommu is enforced in software mode)
> >> > 
> >> > When iommu=no-igfx is enforced, shell console boot up works flawlessly. 
> >> > All
> >> > domu machines get booted up. A system hang will happen at the moment a 
> >> > domu
> >> > machine does graphic rendering,
> >> 
> >> And this is (other than I originally implied) without passing through
> >> the IGD to the DomU? If so, I can't see the difference between a
> >> guest rendering to its display (and vncviewer or whatever frontend
> >> you use converting this to rendering on the host) and rendering
> >> which originates in the host.
> > 
> > Not sure if relevant, but window content is mapped from PV domU directly
> > into X server (in dom0) address space, using xc_map_foreign_pages. It is
> > done by hacking XShmAttach function. Not sure what graphics driver do
> > with it next. Theoretically it could be possible that driver will direct IGD
> > to do DMA directly from that place, but I guess it does not.
> 
> Interesting. This then really needs to be investigated from the
> Qubes end rather than here. Possible resulting patches, if
> relevant outside of that unusual setup, would then of course be
> appreciated to be sent here.

Note that Thierry said "The point is the iommu=no-igfx doesn't fix the
issue", so either it is totally unrelated issue, or iommu=no-igfx is
broken. Does iommu=no-igfx have any meaning when IDG is _not_ passed
through to some domU? And generally - is IOMMU used in any way for dom0
devices? Is Linux kernel able to utilize it for its own purposes (my
guess: no)?

Somehow unrelated: I've tried to get p2m iommu mapping using `xl
debug-key o`, but it's too long (and xenconsoled isn't fast enough to
catch it into hypervisor.log). Is is possible to enlarge console ring
buffer at runtime? If not, is `conring_size` the right option? How large
it should be (aprox) for `xl debug-key o` output? 

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?


signature.asc
Description: PGP signature
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] pre Sandy bridge IOMMU support (gm45)

[Jan Beulich]

>>> On 01.02.16 at 13:28,  wrote:
> On Mon, Feb 01, 2016 at 12:59:00AM -0700, Jan Beulich wrote:
>> >>> On 30.01.16 at 02:47,  wrote:
>> > On Tue, Jan 26, 2016 at 04:37:05AM -0700, Jan Beulich wrote:
>> >> (re-adding xen-devel)
>> >> 
>> >> >>> On 26.01.16 at 12:28,  wrote:
>> >> > Iommu=0 let the whole Qubes system work, without enforcing hardware
>> >> > compartimentalisation (iommu is enforced in software mode)
>> >> > 
>> >> > When iommu=no-igfx is enforced, shell console boot up works flawlessly. 
> All
>> >> > domu machines get booted up. A system hang will happen at the moment a 
> domu
>> >> > machine does graphic rendering,
>> >> 
>> >> And this is (other than I originally implied) without passing through
>> >> the IGD to the DomU? If so, I can't see the difference between a
>> >> guest rendering to its display (and vncviewer or whatever frontend
>> >> you use converting this to rendering on the host) and rendering
>> >> which originates in the host.
>> > 
>> > Not sure if relevant, but window content is mapped from PV domU directly
>> > into X server (in dom0) address space, using xc_map_foreign_pages. It is
>> > done by hacking XShmAttach function. Not sure what graphics driver do
>> > with it next. Theoretically it could be possible that driver will direct 
>> > IGD
>> > to do DMA directly from that place, but I guess it does not.
>> 
>> Interesting. This then really needs to be investigated from the
>> Qubes end rather than here. Possible resulting patches, if
>> relevant outside of that unusual setup, would then of course be
>> appreciated to be sent here.
> 
> Note that Thierry said "The point is the iommu=no-igfx doesn't fix the
> issue", so either it is totally unrelated issue, or iommu=no-igfx is
> broken. Does iommu=no-igfx have any meaning when IDG is _not_ passed
> through to some domU?

Yes: Iirc that option turns off the IOMMU responsible for IGD.

> And generally - is IOMMU used in any way for dom0
> devices?

Of course; by how much depends on what options you use (see
the command line doc).

> Is Linux kernel able to utilize it for its own purposes (my
> guess: no)?

Under Xen? If so - indeed, no.

> Somehow unrelated: I've tried to get p2m iommu mapping using `xl
> debug-key o`, but it's too long (and xenconsoled isn't fast enough to
> catch it into hypervisor.log). Is is possible to enlarge console ring
> buffer at runtime?

No.

> If not, is `conring_size` the right option?

Yes.

> How large
> it should be (aprox) for `xl debug-key o` output? 

Depends on the amount of memory page tables need to be created
for as well as how fragmented memory was at the time memory
allocation for domains happens. I'm afraid you can only try it out.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] pre Sandy bridge IOMMU support (gm45)

[Marek Marczykowski-Górecki]

On Tue, Jan 26, 2016 at 04:37:05AM -0700, Jan Beulich wrote:
> (re-adding xen-devel)
> 
> >>> On 26.01.16 at 12:28,  wrote:
> > Iommu=0 let the whole Qubes system work, without enforcing hardware
> > compartimentalisation (iommu is enforced in software mode)
> > 
> > When iommu=no-igfx is enforced, shell console boot up works flawlessly. All
> > domu machines get booted up. A system hang will happen at the moment a domu
> > machine does graphic rendering,
> 
> And this is (other than I originally implied) without passing through
> the IGD to the DomU? If so, I can't see the difference between a
> guest rendering to its display (and vncviewer or whatever frontend
> you use converting this to rendering on the host) and rendering
> which originates in the host.

Not sure if relevant, but window content is mapped from PV domU directly
into X server (in dom0) address space, using xc_map_foreign_pages. It is
done by hacking XShmAttach function. Not sure what graphics driver do
with it next. Theoretically it could be possible that driver will direct IGD
to do DMA directly from that place, but I guess it does not.

> 
> Jan
> 
> > which often results in tray icon being
> > fuzzy just before the system gets unresponsive(netvm showing it get
> > connected through nm - applet rendering) , or a notification starting to
> > show up while the system hangs before it disappears with some minor/major
> > visual glitch being visible (usb-vm showing device attribution to another
> > vm).
> > 
> > Again, if iommu=0 is passed to xen, there is no system hang while not
> > having any added isolation security from usb devices being in a domu and
> > network devices being in another one, while applications sit in seperate
> > ones. This is why Qubes strongly suggest but doesn't require iommu;stronger
> > isolation.
> > IGD has a bad history of iommu support. A quick list :
> > 
> > -http://lists.freedesktop.org/archives/dri-devel/2013-January/033662.html 
> > -https://lists.ubuntu.com/archives/kernel-team/2013-February/024796.html 
> > 
> > Isolation of netvm and usb is a required use case in Qubes. IGD passthrough
> > would be nice to have, but isn't required. I don't really see why someone
> > would want to passthrouh IGD to a Windows domu, gm45 based laptops are
> > definitely not gaming laptops. Since i915 and gm45 have a bad iommu
> > history, just being able to completely disable iommu for IGD would suffice.
> > 
> > 
> > Thierry
> > 
> > Le mar. 26 janv. 2016 05:52, Jan Beulich  a écrit :
> > 
> >> >>> On 25.01.16 at 22:49,  wrote:
> >> > The case is 1) disabling iommu for IGD, unilaterally since i915 + gm45
> >> > doesn't play well together. Iommu is still desired to isolate usb and
> >> > network devices, so we don't want to disable iommu completely. The side
> >> > effect of this would be to have IGD only for dom0, which would also
> >> > completely make sense in this use case.
> >> >
> >> > The point is the iommu=no-igfx doesn't fix the issue, since remapping
> >> seems
> >> > to still happen for IGD. Does that make sense ?
> >>
> >> It certainly may make sense, just that in what you have written so
> >> far I don't think I've been able to spot any evidence thereof. Since,
> >> as you say, nothing interesting gets logged by Xen, you must be
> >> drawing this conclusion from something (or else you wouldn't say
> >> "doesn't fix the issue").
> >>
> >> Jan
> >>
> >>
> 
> 
> 
> ___
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?


signature.asc
Description: PGP signature
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] pre Sandy bridge IOMMU support (gm45)

[Jan Beulich]

>>> On 25.01.16 at 22:49,  wrote:
> The case is 1) disabling iommu for IGD, unilaterally since i915 + gm45
> doesn't play well together. Iommu is still desired to isolate usb and
> network devices, so we don't want to disable iommu completely. The side
> effect of this would be to have IGD only for dom0, which would also
> completely make sense in this use case.
> 
> The point is the iommu=no-igfx doesn't fix the issue, since remapping seems
> to still happen for IGD. Does that make sense ?

It certainly may make sense, just that in what you have written so
far I don't think I've been able to spot any evidence thereof. Since,
as you say, nothing interesting gets logged by Xen, you must be
drawing this conclusion from something (or else you wouldn't say
"doesn't fix the issue").

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] pre Sandy bridge IOMMU support (gm45)

[Jan Beulich]

>>> On 26.01.16 at 12:57,  wrote:
> Only dom0 talks directly to the i915 driver, other appvm being pv, which is
> why I put in question the complete deactivation of IGD by iommu=no-igfx.
> 
> Is there anything I can provide to troubleshoot?

Hard to tell. The VT-d maintainers may be able to give you better
guidance.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] pre Sandy bridge IOMMU support (gm45)

[Jan Beulich]

(re-adding xen-devel)

>>> On 26.01.16 at 12:28,  wrote:
> Iommu=0 let the whole Qubes system work, without enforcing hardware
> compartimentalisation (iommu is enforced in software mode)
> 
> When iommu=no-igfx is enforced, shell console boot up works flawlessly. All
> domu machines get booted up. A system hang will happen at the moment a domu
> machine does graphic rendering,

And this is (other than I originally implied) without passing through
the IGD to the DomU? If so, I can't see the difference between a
guest rendering to its display (and vncviewer or whatever frontend
you use converting this to rendering on the host) and rendering
which originates in the host.

Jan

> which often results in tray icon being
> fuzzy just before the system gets unresponsive(netvm showing it get
> connected through nm - applet rendering) , or a notification starting to
> show up while the system hangs before it disappears with some minor/major
> visual glitch being visible (usb-vm showing device attribution to another
> vm).
> 
> Again, if iommu=0 is passed to xen, there is no system hang while not
> having any added isolation security from usb devices being in a domu and
> network devices being in another one, while applications sit in seperate
> ones. This is why Qubes strongly suggest but doesn't require iommu;stronger
> isolation.
> IGD has a bad history of iommu support. A quick list :
> 
> -http://lists.freedesktop.org/archives/dri-devel/2013-January/033662.html 
> -https://lists.ubuntu.com/archives/kernel-team/2013-February/024796.html 
> 
> Isolation of netvm and usb is a required use case in Qubes. IGD passthrough
> would be nice to have, but isn't required. I don't really see why someone
> would want to passthrouh IGD to a Windows domu, gm45 based laptops are
> definitely not gaming laptops. Since i915 and gm45 have a bad iommu
> history, just being able to completely disable iommu for IGD would suffice.
> 
> 
> Thierry
> 
> Le mar. 26 janv. 2016 05:52, Jan Beulich  a écrit :
> 
>> >>> On 25.01.16 at 22:49,  wrote:
>> > The case is 1) disabling iommu for IGD, unilaterally since i915 + gm45
>> > doesn't play well together. Iommu is still desired to isolate usb and
>> > network devices, so we don't want to disable iommu completely. The side
>> > effect of this would be to have IGD only for dom0, which would also
>> > completely make sense in this use case.
>> >
>> > The point is the iommu=no-igfx doesn't fix the issue, since remapping
>> seems
>> > to still happen for IGD. Does that make sense ?
>>
>> It certainly may make sense, just that in what you have written so
>> far I don't think I've been able to spot any evidence thereof. Since,
>> as you say, nothing interesting gets logged by Xen, you must be
>> drawing this conclusion from something (or else you wouldn't say
>> "doesn't fix the issue").
>>
>> Jan
>>
>>



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] pre Sandy bridge IOMMU support (gm45)

[Tian, Kevin]

> From: Jan Beulich
> Sent: Tuesday, January 26, 2016 8:28 PM
> 
> >>> On 26.01.16 at 12:57,  wrote:
> > Only dom0 talks directly to the i915 driver, other appvm being pv, which is
> > why I put in question the complete deactivation of IGD by iommu=no-igfx.
> >
> > Is there anything I can provide to troubleshoot?
> 
> Hard to tell. The VT-d maintainers may be able to give you better
> guidance.
> 
> Jan
> 

I'm a bit confused what's being exactly asked here. Did you mean
that IOMMU is not actually disabled when you set iommu=no-gfx?

Thanks
Kevin

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] pre Sandy bridge IOMMU support (gm45)

[Thierry Laurion]

It seems that IGD iommu is not completely deactivated, yes, since memory
corruption happens (graphic glitches and then system hang)

General iommu is still reported as being active by xen, as desired.

Thierry

Le mar. 26 janv. 2016 17:21, Tian, Kevin  a écrit :

> > From: Jan Beulich
> > Sent: Tuesday, January 26, 2016 8:28 PM
> >
> > >>> On 26.01.16 at 12:57,  wrote:
> > > Only dom0 talks directly to the i915 driver, other appvm being pv,
> which is
> > > why I put in question the complete deactivation of IGD by
> iommu=no-igfx.
> > >
> > > Is there anything I can provide to troubleshoot?
> >
> > Hard to tell. The VT-d maintainers may be able to give you better
> > guidance.
> >
> > Jan
> >
>
> I'm a bit confused what's being exactly asked here. Did you mean
> that IOMMU is not actually disabled when you set iommu=no-gfx?
>
> Thanks
> Kevin
>
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] pre Sandy bridge IOMMU support (gm45)

[Thierry Laurion]

Only dom0 talks directly to the i915 driver, other appvm being pv, which is
why I put in question the complete deactivation of IGD by iommu=no-igfx.

Is there anything I can provide to troubleshoot?

Le mar. 26 janv. 2016 06:37, Jan Beulich  a écrit :

> (re-adding xen-devel)
>
> >>> On 26.01.16 at 12:28,  wrote:
> > Iommu=0 let the whole Qubes system work, without enforcing hardware
> > compartimentalisation (iommu is enforced in software mode)
> >
> > When iommu=no-igfx is enforced, shell console boot up works flawlessly.
> All
> > domu machines get booted up. A system hang will happen at the moment a
> domu
> > machine does graphic rendering,
>
> And this is (other than I originally implied) without passing through
> the IGD to the DomU? If so, I can't see the difference between a
> guest rendering to its display (and vncviewer or whatever frontend
> you use converting this to rendering on the host) and rendering
> which originates in the host.
>
> Jan
>
> > which often results in tray icon being
> > fuzzy just before the system gets unresponsive(netvm showing it get
> > connected through nm - applet rendering) , or a notification starting to
> > show up while the system hangs before it disappears with some minor/major
> > visual glitch being visible (usb-vm showing device attribution to another
> > vm).
> >
> > Again, if iommu=0 is passed to xen, there is no system hang while not
> > having any added isolation security from usb devices being in a domu and
> > network devices being in another one, while applications sit in seperate
> > ones. This is why Qubes strongly suggest but doesn't require
> iommu;stronger
> > isolation.
> > IGD has a bad history of iommu support. A quick list :
> >
> > -
> http://lists.freedesktop.org/archives/dri-devel/2013-January/033662.html
> > -https://lists.ubuntu.com/archives/kernel-team/2013-February/024796.html
> >
> > Isolation of netvm and usb is a required use case in Qubes. IGD
> passthrough
> > would be nice to have, but isn't required. I don't really see why someone
> > would want to passthrouh IGD to a Windows domu, gm45 based laptops are
> > definitely not gaming laptops. Since i915 and gm45 have a bad iommu
> > history, just being able to completely disable iommu for IGD would
> suffice.
> >
> >
> > Thierry
> >
> > Le mar. 26 janv. 2016 05:52, Jan Beulich  a écrit :
> >
> >> >>> On 25.01.16 at 22:49,  wrote:
> >> > The case is 1) disabling iommu for IGD, unilaterally since i915 + gm45
> >> > doesn't play well together. Iommu is still desired to isolate usb and
> >> > network devices, so we don't want to disable iommu completely. The
> side
> >> > effect of this would be to have IGD only for dom0, which would also
> >> > completely make sense in this use case.
> >> >
> >> > The point is the iommu=no-igfx doesn't fix the issue, since remapping
> >> seems
> >> > to still happen for IGD. Does that make sense ?
> >>
> >> It certainly may make sense, just that in what you have written so
> >> far I don't think I've been able to spot any evidence thereof. Since,
> >> as you say, nothing interesting gets logged by Xen, you must be
> >> drawing this conclusion from something (or else you wouldn't say
> >> "doesn't fix the issue").
> >>
> >> Jan
> >>
> >>
>
>
>
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] pre Sandy bridge IOMMU support (gm45)

[Jan Beulich]

>>> On 23.01.16 at 08:38,  wrote:
> Thinkpad x200 p8600 laptops have vt-d, vt-x and tpm. They also have intel
> integrated graphics 4 Series (gm45 chipset), supported through i915 driver.
> 
> In December, a fix got introduced to Xen 4.6 through iommu=no-igfx switch.
> Before that fix, it was impossible to boot xen without passing iommu=0.
> 
> With iommu=no-igfx passed on, Qubes boots xen, kernel, dom0 and domu until
> some graphic rendering is done from a domu to dom0 xserver.
> 
> I'm trying to push forward IOMMU support of gm45 chipset here. The problem
> is between i915 and xen iommu support for sure, but there is no crash or
> interesting debugging information given on a serial console.

To be honest, even after having read you second mail I'm not quite
clear what it you want and/or what doesn't work. Adding some
guessing, I could interpret the above as you wanting to pass through
the graphics device. If that's the case, then this is quite clearly a
pretty bad idea with the IOMMU behind it disabled.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] pre Sandy bridge IOMMU support (gm45)

[Thierry Laurion]

The case is 1) disabling iommu for IGD, unilaterally since i915 + gm45
doesn't play well together. Iommu is still desired to isolate usb and
network devices, so we don't want to disable iommu completely. The side
effect of this would be to have IGD only for dom0, which would also
completely make sense in this use case.

The point is the iommu=no-igfx doesn't fix the issue, since remapping seems
to still happen for IGD. Does that make sense ?

Le lun. 25 janv. 2016 09:30, Jan Beulich  a écrit :

> >>> On 23.01.16 at 08:38,  wrote:
> > Thinkpad x200 p8600 laptops have vt-d, vt-x and tpm. They also have intel
> > integrated graphics 4 Series (gm45 chipset), supported through i915
> driver.
> >
> > In December, a fix got introduced to Xen 4.6 through iommu=no-igfx
> switch.
> > Before that fix, it was impossible to boot xen without passing iommu=0.
> >
> > With iommu=no-igfx passed on, Qubes boots xen, kernel, dom0 and domu
> until
> > some graphic rendering is done from a domu to dom0 xserver.
> >
> > I'm trying to push forward IOMMU support of gm45 chipset here. The
> problem
> > is between i915 and xen iommu support for sure, but there is no crash or
> > interesting debugging information given on a serial console.
>
> To be honest, even after having read you second mail I'm not quite
> clear what it you want and/or what doesn't work. Adding some
> guessing, I could interpret the above as you wanting to pass through
> the graphics device. If that's the case, then this is quite clearly a
> pretty bad idea with the IOMMU behind it disabled.
>
> Jan
>
>
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] pre Sandy bridge IOMMU support (gm45)

[Andrew Cooper]

On 24/01/16 18:21, Thierry Laurion wrote:
> Hi devs!
>
> XEN devs:
> As per short discussion with ktemkin earlier in January in #xen:
>
> "ktemkin Jan 10, 2016 16:21:50
> This test patch did appear to make the system work, though:
> https://gist.github.com/ktemkin/0e81b93654ae800a5609
>
> ktemkin Jan 10, 2016 16:24:55
> Only real difference I see between that and the upstream behavior
> (besides limiting things to dom0 so things weren't accidentally passed
> through) is the call to disable_pmr on line 117 before aborting."
>
>
>
> Makes total sense to my early understanding, since it seems that it is
> said that vt-d engine gets disabled, but disable_pmr(iommu) function
> is not called to enforce.
>
> What do you think?

There is some confusion here.

"Unfortunately, quirks specific to the Clarkdale/Nehalem integrated
graphics device (IGD) do not function correctly with Xen's VT-d
implementation"

Either
1) There is a chipset errata which prevents it from functioning
correctly, or
2) Xen's VT-d code doesn't program the chip correctly.

Which is it?

If it is 1), then there is a very good case for quirking the affected
chipsets and unilaterally disabling them.  If it is 2), then the VT-d
code should be corrected.

~Andrew
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] pre Sandy bridge IOMMU support (gm45)

[Thierry Laurion]

Hi devs!

XEN devs:
As per short discussion with ktemkin earlier in January in #xen:

"ktemkin Jan 10, 2016 16:21:50
This test patch did appear to make the system work, though:
https://gist.github.com/ktemkin/0e81b93654ae800a5609

ktemkin Jan 10, 2016 16:24:55
Only real difference I see between that and the upstream behavior (besides
limiting things to dom0 so things weren't accidentally passed through) is
the call to disable_pmr on line 117 before aborting."



Makes total sense to my early understanding, since it seems that it is said
that vt-d engine gets disabled, but disable_pmr(iommu) function is not
called to enforce.

What do you think?

QUBES devs:
I'm still trying to understand how to apply this patch to qubes_builder to
actually build a test iso or xen.gz image and report. All Qubes patches
seem to be applied from git to local directory structure. Looking inside
the code to understand how to generate the provided patch to git can apply
it to local chrooted environment when building. Any documentation you could
point me to would be greatly appreciated, as any feedback to actually fix
the issue stopping this laptop from being a nearly perfect candidate for
Qubes.


Thierry

Le sam. 23 janv. 2016 à 02:37, Thierry Laurion 
a écrit :

> Hey devs,
>
> Thinkpad x200 p8600 laptops have vt-d, vt-x and tpm. They also have intel
> integrated graphics 4 Series (gm45 chipset), supported through i915 driver.
>
> In December, a fix got introduced to Xen 4.6 through iommu=no-igfx switch.
> Before that fix, it was impossible to boot xen without passing iommu=0.
>
> With iommu=no-igfx passed on, Qubes boots xen, kernel, dom0 and domu until
> some graphic rendering is done from a domu to dom0 xserver.
>
> I'm trying to push forward IOMMU support of gm45 chipset here. The problem
> is between i915 and xen iommu support for sure, but there is no crash or
> interesting debugging information given on a serial console.
>
> Any dev help is welcome since that beast and t400 would be excellent Qubes
> candidates once that problem is fixed. I posted in December on the list
> just before Christmas but I guess the timing wasn't right;)
>
> Thanks for your help.
> Thierry
>
--- ./drivers/passthrough/vtd/iommu.c.bak	2016-01-24 12:55:15.020267553 -0500
+++ ./drivers/passthrough/vtd/iommu.c	2016-01-24 12:57:14.754138262 -0500
@@ -717,6 +717,7 @@
 {
 dprintk(XENLOG_WARNING VTDPREFIX,
 "BIOS did not enable IGD for VT properly.  Disabling IGD VT-d engine.\n");
+disable_pmr(iommu);
 return;
 }
 }
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[Xen-devel] pre Sandy bridge IOMMU support (gm45)

[Thierry Laurion]

Hey devs,

Thinkpad x200 p8600 laptops have vt-d, vt-x and tpm. They also have intel
integrated graphics 4 Series (gm45 chipset), supported through i915 driver.

In December, a fix got introduced to Xen 4.6 through iommu=no-igfx switch.
Before that fix, it was impossible to boot xen without passing iommu=0.

With iommu=no-igfx passed on, Qubes boots xen, kernel, dom0 and domu until
some graphic rendering is done from a domu to dom0 xserver.

I'm trying to push forward IOMMU support of gm45 chipset here. The problem
is between i915 and xen iommu support for sure, but there is no crash or
interesting debugging information given on a serial console.

Any dev help is welcome since that beast and t400 would be excellent Qubes
candidates once that problem is fixed. I posted in December on the list
just before Christmas but I guess the timing wasn't right;)

Thanks for your help.
Thierry
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel