Re: [qubes-devel] Re: GVT-g Discussion

[thorsten...@gmail.com]

Hello everyone,

it has been a while since the last post here, is there any update on this 
topic?
Any gpu virtualization on the horizon in the (near) future?

Thorsten Schierer 

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-devel/8ab4ca70-eca5-453f-9b89-a5cb5e3c58a4n%40googlegroups.com.

Re: [qubes-devel] Re: GVT-g Discussion

[Ivan Kardykov]

On 2020-02-11 17:51, Marek Marczykowski-Górecki wrote:
> On Thu, Feb 06, 2020 at 02:53:09PM +0300, Ivan Kardykov wrote:
>> Greetings,
> 
>> there is an update of our experimental GVT-g integration with Qubes.
> 
>> Code was slightly improved to get rid of qemu. Vgpu init routine was
>> moved to libxl and it works with stubdomain.
> 
> 
>> https://github.com/tabit-pro/tabit-qubes-repo
> 
> This is really interesting development!
> Can you say something more about it? Specifically:
> > 1. What a VM have access to really? Full GPU, some context? If a
> context, how is it enforced?
> 
> 2. What components are involved in GPU commands processing and how are
> they isolated?
> 

In theory [1] I see that each VM has access to frame and command buffers
in system memory. The command buffer reused across VMs with smart
shadowing and GVT framework validates graphics memory addresses before
they used by the GPU.
Access to the privileged resources (GPU Page Table Entries and I/O
registers) is handled by xengt module, based on trap-and-emulation.
Render context switching is used per vGPU.

In practice, security audit is the matter of our further experiments.

[1]
https://www.usenix.org/system/files/conference/atc14/atc14-paper-tian.pdf

> 3. Is it possible to enable it only for some VMs - in a way outside of VM
> control?
> 

We made several changes to the existing implementation to use it with
qubes via qvm-features (video-model):
- Patch to libxl creates vGPU instance via sysfs, during hvm domain
execution.
- Stubdomain part extends video model to prevent unnecessary
initialization of emulated VGA.
- Libvirt configuration adapted to parse xengt parameters.


---
Regads,
IK
tabit.pro

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-devel/a2c9b683-171f-6186-9a4c-4e0eca739802%40tabit.pro.


signature.asc
Description: OpenPGP digital signature

Re: [qubes-devel] Re: GVT-g Discussion

[Marek Marczykowski-Górecki]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

On Thu, Feb 06, 2020 at 02:53:09PM +0300, Ivan Kardykov wrote:
> Greetings,
> 
> there is an update of our experimental GVT-g integration with Qubes.
> 
> Code was slightly improved to get rid of qemu. Vgpu init routine was
> moved to libxl and it works with stubdomain.
> 
> 
> https://github.com/tabit-pro/tabit-qubes-repo

This is really interesting development!
Can you say something more about it? Specifically:

1. What a VM have access to really? Full GPU, some context? If a
context, how is it enforced?

2. What components are involved in GPU commands processing and how are
they isolated?

3. Is it possible to enable it only for some VMs - in a way outside of VM
control?

- -- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
-BEGIN PGP SIGNATURE-

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAl5Cv4gACgkQ24/THMrX
1ywSQAf/bZhoYV4kd++KmF/o38mZkBUblyZZFfRlXRYuvDYqAFWdmCMVmLyLzLBn
fRDTjph/4XuY3UC0hADBXpVYDSAoSVlCw4E2dza5g+gFipvXEC8l0dCAP3hS3nob
N03t4UGmqBS+Vq6dekVpvgCyLnUGVc1SUDtgM04LNdtJbt+NfeRdsL9QFa5tBaCI
MFmg1HTvdz5/Qg4gmxvLIuwMRwXi6tSPWAcd84a3+3c+kGR8paZsJhztdNbyhrY+
QPPWqNxwNbL5aUEv/HqljM9Qcxt/iXRrVkv0y3ZXHnZMN3ff2ZVru6r3+Ie+tCWc
ZjgSKxHf6YnEsEkxyPd8DOrkdwacZA==
=TCpb
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-devel/20200211145152.GO15453%40mail-itl.

Re: [qubes-devel] Re: GVT-g Discussion

[Ivan Kardykov]

Greetings,

there is an update of our experimental GVT-g integration with Qubes.

Code was slightly improved to get rid of qemu. Vgpu init routine was
moved to libxl and it works with stubdomain.


https://github.com/tabit-pro/tabit-qubes-repo

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-devel/3277ad93-e5c6-ccba-9007-639b1195775d%40tabit.pro.


signature.asc
Description: OpenPGP digital signature

Re: [qubes-devel] Re: GVT-g Discussion

[Ivan Kardykov]

On 2020-01-02 18:34, Demi M. Obenour wrote:
> On 2020-01-02 15:33, Kardykov Ivan wrote:
>> Sorry for that bumping, but looks like my posts from work domain (tabit.pro) 
>> stucked at moderate or
>> spam checks.. So my message was about GVT-g.
>>
>> We have some success with adaptation Intel gtv-g technology to Qubes OS.
>> At now it is rather proof of concept state, but it probably could help
>> to test guivm features and discuss of applicability in practice.
>>
>> In this [1] repository we placed specs and patches to build core Qubes
>> components with ability to run and work with GPU accelerated VM. We
>> oriented to Fedora 29 dom0 and Xen 4.12 versions with plans of 4.1
>> compatibility. Now it is up to Fedora 31 and Xen 4.13 and we will get to
>> this versions in some time.
>>
>> We made and adapted patches to xen, kernel, libvirt, qemu to dom0 and
>> xorg dummy driver to guest environment. Our work mostly based on Intel
>> [2][3] and XenServer [4] public repositories.
>>
>> The most controversial question is probably running qemu without any
>> restrictions, but there are several ways to improve it.
> 
> As Marek has stated before, this is a non-starter.  QEMU is not
> considered trusted.  Xen’s attack surface is already too high.
> 

As I see it, useful results could be with testing it to get more about
advantages and finding ways to minimize risks. May be smth with this [1]
movements are compatible with gvt-g.

[1] https://wiki.xenproject.org/wiki/Linux_stub_domains

> The approach I would like to see is a paravirtualized approach
> similar to Genode’s: a small (<10K SLOC), auditable (ideally
> formally verified) driver running in dom0 and/or its own domain that
> communicates with a paravirtualized driver in the VMs.  Any needed
> emulation is done in the stubdomain.  This is likely to be difficult
> for many reasons, but is the only way that I see can hardware
> accelerated graphics being secure enough for Qubes with commodity
> hardware.  If you are willing to pay literally thousands of dollars
> for server-grade GPUs, then SR-IOV and friends become options, which
> make secure hardware acceleration much easier.  However, those are
> far too expensive for virtually all Qubes users.  In particular,
> they are far more expensive than just buying another computer for
> graphics-intensive activities.
> 
> The main activities I know of that need hardware-accelerated graphics,
> and which a Qubes user is likely to want to partake in, are gaming
> and GPGPU.  Video games, in particular, should be considered completely
> untrusted, and so need to be walled off as tightly as possible.
> 
> Sincerely,
> 
> Demi
> 


-- 
You received this message because you are subscribed to the Google Groups 
"qubes-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-devel/8dce113c-6afa-3181-9f1c-7eb140b0b725%40tabit.pro.


signature.asc
Description: OpenPGP digital signature

Re: [qubes-devel] Re: GVT-g Discussion

[Ivan Kardykov]

On 2020-01-02 15:26, Kardykov Ivan wrote:
> bump thread
> 
> On Friday, November 22, 2019 at 5:02:14 AM UTC-5, Dylanger Daly wrote:
> 
> Will the Qubes team consider enabling the use of GVT-g so we can
> enjoy hardware accelerated graphics?
> 
> After upgrading to DDR4 Memory I've noticed a significant increase
> in performance, I assume this is because of the many, many memory
> copies taking place, GVT-g should lighten the load.
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "qubes-devel" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to qubes-devel+unsubscr...@googlegroups.com
> .
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/qubes-devel/f608a18f-15e2-4ad1-94b3-da7ae8b1c553%40googlegroups.com
> .

Sorry for that bumping, but looks like my posts stuck at moderate or
spam checks.. So my message was about GVT-g.

We have some success with adaptation Intel gtv-g technology to Qubes OS.
At now it is rather proof of concept state, but it probably could help
to test guivm features and discuss of applicability in practice.

In this [1] repository we placed specs and patches to build core Qubes
components with ability to run and work with GPU accelerated VM. We
oriented to Fedora 29 dom0 and Xen 4.12 versions with plans of 4.1
compatibility. Now it is up to Fedora 31 and Xen 4.13 and we will get to
this versions in some time.

We made and adapted patches to xen, kernel, libvirt, qemu to dom0 and
xorg dummy driver to guest environment. Our work mostly based on Intel
[2][3] and XenServer [4] public repositories.

The most controversial question is probably running qemu without any
restrictions, but there are several ways to improve it.

At now it works with some issues and limitations, most of them described
in repository readme. We have tested it on Lenovo X1 gen5/gen6 and T480
laptops with KabyLake/KabyLake R chipsets.

[1] https://github.com/tabit-pro/tabit-qubes-repo
[2] https://github.com/intel/gvt-linux
[3] https://github.com/intel/igvtg-qemu
[4] https://github.com/xenserver/xen.pg.git


Regards,
Ivan Kardykov

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-devel/9ffc85a2-9b21-b126-bfbb-831ebad2d7ed%40tabit.pro.


signature.asc
Description: OpenPGP digital signature

Re: [qubes-devel] Re: GVT-g Discussion

['Dylanger Daly' via qubes-devel]

 My bad, I accidentally replied directly to Demi

Hey Ivan,

This is fantastic work! I'll test this out today, I think this will help 
with Firefox's WebRender engine, meaning much less CPU spikes, and reduced 
wattage when I have like 2 Firefox Tabs open.
Not to mention watching YouTube.

Demi you make some valid Security points, however one major GPU activity is 
WebRender (Web Content being rendered within the iGPU)
It's also my understanding Marek mentioned R4.1 would support GVT-g.

I would love to see an approach similar to Genode as well, I just don't 
think it will happen, I'm almost certain GVT-g will be used.

Cheers!

*Demi Replied with*

> Hey Ivan,
> 
> This is fantastic work! I'll test this out today, I think this will help
> with Firefox's WebRender engine, meaning much less CPU spikes, and reduced
> wattage when I have like 2 Firefox Tabs open.
> Not to mention watching YouTube.
> 
> Demi you make some valid Security points, however one major GPU activity 
is
> WebRender (Web Content being rendered within the iGPU)
> It's also my understanding Marek mentioned R4.1 would support GVT-g.
> 
> I would love to see an approach similar to Genode as well, I just don't
> think it will happen, I'm almost certain GVT-g will be used.
> 
> Cheers!

Regarding WebRender: My personal approach (and recommendation) is to
just accept the higher power consumption.  If you can’t, a separate
computer is probably your best option.  The trend in QubesOS is to
emulate more devices (such as audio hardware), and rely on QEMU being
sandboxed for security.

GVT-g may be supported in QubesOS at some point, but not until it
doesn’t require unsandboxed QEMU.  Honestly, I consider QEMU
being in stubdoms to be one of QubesOS’s main selling points.
Especially since dom0 is so out of date that there is almost certain
to be a trivially-exploitable vulnerability there.

Sincerely,

Demi

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-devel/4e5e8079-8aba-4c9f-95c1-7d119db418eb%40googlegroups.com.

Re: [qubes-devel] Re: GVT-g Discussion

[Demi M. Obenour]

On 2020-01-02 15:33, Kardykov Ivan wrote:
> Sorry for that bumping, but looks like my posts from work domain (tabit.pro) 
> stucked at moderate or
> spam checks.. So my message was about GVT-g.
> 
> We have some success with adaptation Intel gtv-g technology to Qubes OS.
> At now it is rather proof of concept state, but it probably could help
> to test guivm features and discuss of applicability in practice.
> 
> In this [1] repository we placed specs and patches to build core Qubes
> components with ability to run and work with GPU accelerated VM. We
> oriented to Fedora 29 dom0 and Xen 4.12 versions with plans of 4.1
> compatibility. Now it is up to Fedora 31 and Xen 4.13 and we will get to
> this versions in some time.
> 
> We made and adapted patches to xen, kernel, libvirt, qemu to dom0 and
> xorg dummy driver to guest environment. Our work mostly based on Intel
> [2][3] and XenServer [4] public repositories.
> 
> The most controversial question is probably running qemu without any
> restrictions, but there are several ways to improve it.

As Marek has stated before, this is a non-starter.  QEMU is not
considered trusted.  Xen’s attack surface is already too high.

The approach I would like to see is a paravirtualized approach
similar to Genode’s: a small (<10K SLOC), auditable (ideally
formally verified) driver running in dom0 and/or its own domain that
communicates with a paravirtualized driver in the VMs.  Any needed
emulation is done in the stubdomain.  This is likely to be difficult
for many reasons, but is the only way that I see can hardware
accelerated graphics being secure enough for Qubes with commodity
hardware.  If you are willing to pay literally thousands of dollars
for server-grade GPUs, then SR-IOV and friends become options, which
make secure hardware acceleration much easier.  However, those are
far too expensive for virtually all Qubes users.  In particular,
they are far more expensive than just buying another computer for
graphics-intensive activities.

The main activities I know of that need hardware-accelerated graphics,
and which a Qubes user is likely to want to partake in, are gaming
and GPGPU.  Video games, in particular, should be considered completely
untrusted, and so need to be walled off as tightly as possible.

Sincerely,

Demi

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-devel/430e381e-7118-5660-38b5-9db65399b02d%40gmail.com.


signature.asc
Description: OpenPGP digital signature

[qubes-devel] Re: GVT-g Discussion

[Kardykov Ivan]

On Thursday, January 2, 2020 at 3:26:54 PM UTC-5, Kardykov Ivan wrote:
>
> bump thread
>
> On Friday, November 22, 2019 at 5:02:14 AM UTC-5, Dylanger Daly wrote:
>>
>> Will the Qubes team consider enabling the use of GVT-g so we can enjoy 
>> hardware accelerated graphics?
>>
>> After upgrading to DDR4 Memory I've noticed a significant increase in 
>> performance, I assume this is because of the many, many memory copies 
>> taking place, GVT-g should lighten the load.
>>
>
Sorry for that bumping, but looks like my posts from work domain (tabit.pro) 
stucked at moderate or
spam checks.. So my message was about GVT-g.

We have some success with adaptation Intel gtv-g technology to Qubes OS.
At now it is rather proof of concept state, but it probably could help
to test guivm features and discuss of applicability in practice.

In this [1] repository we placed specs and patches to build core Qubes
components with ability to run and work with GPU accelerated VM. We
oriented to Fedora 29 dom0 and Xen 4.12 versions with plans of 4.1
compatibility. Now it is up to Fedora 31 and Xen 4.13 and we will get to
this versions in some time.

We made and adapted patches to xen, kernel, libvirt, qemu to dom0 and
xorg dummy driver to guest environment. Our work mostly based on Intel
[2][3] and XenServer [4] public repositories.

The most controversial question is probably running qemu without any
restrictions, but there are several ways to improve it.

At now it works with some issues and limitations, most of them described
in repository readme. We have tested it on Lenovo X1 gen5/gen6 and T480
laptops with KabyLake/KabyLake R chipsets.

[1] https://github.com/tabit-pro/tabit-qubes-repo
[2] https://github.com/intel/gvt-linux
[3] https://github.com/intel/igvtg-qemu
[4] https://github.com/xenserver/xen.pg.git


Regards,
Ivan Kardykov
tabit.pro

 

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-devel/d36c7519-82b8-40ca-b2dd-b24c023bc7b7%40googlegroups.com.

[qubes-devel] Re: GVT-g Discussion

[Kardykov Ivan]

bump thread

On Friday, November 22, 2019 at 5:02:14 AM UTC-5, Dylanger Daly wrote:
>
> Will the Qubes team consider enabling the use of GVT-g so we can enjoy 
> hardware accelerated graphics?
>
> After upgrading to DDR4 Memory I've noticed a significant increase in 
> performance, I assume this is because of the many, many memory copies 
> taking place, GVT-g should lighten the load.
>

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-devel/f608a18f-15e2-4ad1-94b3-da7ae8b1c553%40googlegroups.com.

[qubes-devel] Re: GVT-g Discussion

[Qubes123]

As far as I know in Qubes R4.1 will have split admin and gui domains, so 
neither of these functions will run in dom0.
For that I assume there's already intensive work and testing are being done 
to test GPU passthrough to guivm. Considering the significantly larger 
Intel install base - I guess the integrated Intel GPU passthrough this will 
come first. Then probably the AMD integrated GPU passthrough support, then 
the rest of the other PCI cards. However, full HW acceleration (especially 
3D) might not be trivial as the Qubes gui agent implementation is code 
developed by the Qubes dev team, so not all features and possibilities of 
Xen- HW gui acceleration capabilities is implemented within. But I could be 
mistaken here, and it could be, that as soon as the guivm separation is 
done and tested (eg. also security wise), then it'll be much more easy to 
have HW acceleration (2D->3D) in an AppVM. 

On Friday, November 22, 2019 at 10:02:14 AM UTC, Dylanger Daly wrote:
>
> Will the Qubes team consider enabling the use of GVT-g so we can enjoy 
> hardware accelerated graphics?
>
> After upgrading to DDR4 Memory I've noticed a significant increase in 
> performance, I assume this is because of the many, many memory copies 
> taking place, GVT-g should lighten the load.
>

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-devel/b0e6a890-4305-4de1-a52c-e19b64668702%40googlegroups.com.

[qubes-devel] Re: GVT-g Discussion

[David Schissler]

I've been lurking on Qubes channels for a long time.  For one particular 
minor version some years ago 25% of the TODOs were small quality of life 
improvements that I had suggested.  My story is that I've tried to use 
Qubes but have ended up not due to many quality of life issues since I 
don't really need that level of protection.  I also live out of a bag so 
carrying two laptops isn't very practical.  Also I've found more benefit in 
upgrading notebooks than staying back to use Qubes.  That said, if I could 
get GPU acceleration for just my green zones for local development work 
then it would be amazing and probably worth the jump for me.  In my case 
I'd likely put some proprietary games in their own environment and browse 
the web unaccelerated.  I think that if this feature were available even if 
it needed to be turned on manually from the CLI (to make sure people are 
aware of the ramifications) then it would bring a lot more mind share to 
Qubes.  The less than ideal degraded GPU security would be better than what 
I'm running now.  I think that other than the labor the question is if 
Qubes wants to offer the most protection to many different use cases and 
threat levels or if its only interested in being the most ideal security 
environment period.




On Saturday, November 30, 2019 at 7:53:14 PM UTC-7, pixel fairy wrote:
>
> This comes up often, and i suspect qubes-os is the inspiration for the xen 
> version of it, but it also opens up a big can of attack surface. maybe 
> after splitting the guivm off from dom0?
>
> On Friday, November 22, 2019 at 2:02:14 AM UTC-8, Dylanger Daly wrote:
>>
>> Will the Qubes team consider enabling the use of GVT-g so we can enjoy 
>> hardware accelerated graphics?
>>
>> After upgrading to DDR4 Memory I've noticed a significant increase in 
>> performance, I assume this is because of the many, many memory copies 
>> taking place, GVT-g should lighten the load.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-devel/451ef0be-3127-4ad2-8956-423d33a7d204%40googlegroups.com.

[qubes-devel] Re: GVT-g Discussion

[pixel fairy]

This comes up often, and i suspect qubes-os is the inspiration for the xen 
version of it, but it also opens up a big can of attack surface. maybe 
after splitting the guivm off from dom0?

On Friday, November 22, 2019 at 2:02:14 AM UTC-8, Dylanger Daly wrote:
>
> Will the Qubes team consider enabling the use of GVT-g so we can enjoy 
> hardware accelerated graphics?
>
> After upgrading to DDR4 Memory I've noticed a significant increase in 
> performance, I assume this is because of the many, many memory copies 
> taking place, GVT-g should lighten the load.
>

-- 
You received this message because you are subscribed to the Google Groups 
"qubes-devel" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to qubes-devel+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/qubes-devel/49dcc8d8-18ec-4b47-8329-f0fcb84581b6%40googlegroups.com.