Re: [EXTERNAL] Re: [RFC PATCH 0/4] DirectX on Linux
Hi! > Thanks for the discussion. I may not be able to immediately answer all of > your questions, but I'll do my best . > Could you do something with your email settings? Because this is not how you should use email on lkml. "[EXTERNAL]" in the subject, top-posting, unwrapped lines... Thank you, Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
RE: [EXTERNAL] Re: [RFC PATCH 0/4] DirectX on Linux
[resending as plain text, sorry about that] Thanks Daniel, more below. From: Daniel Vetter <mailto:dan...@ffwll.ch> Sent: Wednesday, May 20, 2020 12:41 AM To: Steve Pronovost <mailto:spron...@microsoft.com> Cc: Dave Airlie <mailto:airl...@gmail.com>; Sasha Levin <mailto:sas...@kernel.org>; mailto:linux-hyp...@vger.kernel.org; Stephen Hemminger <mailto:sthem...@microsoft.com>; Ursulin, Tvrtko <mailto:tvrtko.ursu...@intel.com>; Greg Kroah-Hartman <mailto:gre...@linuxfoundation.org>; Haiyang Zhang <mailto:haiya...@microsoft.com>; LKML <mailto:linux-kernel@vger.kernel.org>; dri-devel <mailto:dri-de...@lists.freedesktop.org>; Chris Wilson <mailto:ch...@chris-wilson.co.uk>; Linux Fbdev development list <mailto:linux-fb...@vger.kernel.org>; Iouri Tarassov <mailto:iou...@microsoft.com>; Deucher, Alexander <mailto:alexander.deuc...@amd.com>; KY Srinivasan <mailto:k...@microsoft.com>; Wei Liu <mailto:wei....@kernel.org>; Hawking Zhang <mailto:hawking.zh...@amd.com> Subject: Re: [EXTERNAL] Re: [RFC PATCH 0/4] DirectX on Linux Hi Steve, Sounds all good, some more comments and details below. On Wed, May 20, 2020 at 5:47 AM Steve Pronovost <mailto:spron...@microsoft.com> wrote: Hey guys, Thanks for the discussion. I may not be able to immediately answer all of your questions, but I'll do my best 😊. drivers/hyperv sounds like it could be a better location. We weren't too sure where to put this, we though /drivers/gpu would be appropriate given this deal with GPUs, but I get your point... this is a vGPU driver that really only works when being run under Hyper-V, so drivers/hyperv is likely more appropriate. I think "it's a virtual gpu" is the wrong sales pitch, as is "only runs on $platform". We have lots of drm drivers in drivers/gpu that fit that bill. The better pitch I think is "it's a not a gpu, it's a dx12 protocol pipe" and "we actually do not want to integrate with the linux gpu ecosystem and primitives, we want to integrate with dx12 ecosystem and primitives to make the seamless rdp/rail/vail stuff work nicely". Below some more thoughts on the technical said. [spronovo] Agreed. As I mentioned in another reply, that protocol isn’t tied to DX… but the point you are making is still valid. This is really a projection of the Windows native abstraction of a GPU that windows user mode driver (dx, gl, cl, vulkan, cuda, etc…) are familiar with and use to communicate with the GPU… This effectively enable porting of these user mode driver to Linux inside of WSL and allow them to share the GPU with the host. Our goal is to offer CL/EGL/GLX/CUDA/… API support for applications running inside of WSL and integrate their output on the Windows desktop through the Wayland compositor we are building. The fact that we are using layer to implement some of these APIs (to reduce our partners work among other thing) is just an implementation details that most application shouldn’t have to worry about… “it just works” 😊. From that perspective we’re fine moving the driver under a different node than /driver/gpu 😊. In term of presentation, I need to clarify a few things. We announced today that we're also adding support for Linux GUI applications. The way this will work is roughly as follow. We're writing a Wayland compositor that will essentially bridge over RDP-RAIL (RAIL=Remote Application Integrated Locally). We're starting from a Weston base. Weston already has an RDP Backend, but that's for a full desktop remoting scheme. Weston draws a desktop and remote it over RDP... and then you can peek at that desktop using an rdp client on the Windows side. RAIL works differently. In that case our wayland compositor no longer paint a desktop... instead it simply forward individual visual / wl_surface over the RDP RAIL channel such that these visual can be displayed on the Windows desktop. The RDP client create proxy window for each of these top level visual and their content is filled with the data coming over the RDP channel. All pixels are owned by the RDP server/WSL... so these windows looks different than native window are they are painted and themed by WSL. The proxy window on the host gather input and inject back over RDP... This is essentially how application remoting works on windows and this is all publicly documented as part of the various RDP protocol specification. As a matter of fact, for the RDP server on the Weston side we are looking at continue to leverage FreeRDP (and provide fixes/enhancement as needed to the public project). Further, we're looking at further improvement down this path to avoid having to copy the content over the RAIL channel and instead just share/swap buffer between the guest and the host. We have extension to the RDP protocol, called VAIL
Re: [EXTERNAL] Re: [RFC PATCH 0/4] DirectX on Linux
Hi Steve, thank you for the fast reply. Am 20.05.20 um 09:42 schrieb Steve Pronovost: >> Echoing what others said, you're not making a DRM driver. The driver should >> live outside of the DRM code. > > Agreed, please see my earlier reply. We'll be moving the driver to > drivers/hyperv node or something similar. Apology for the confusion here. > >> I have one question about the driver API: on Windows, DirectX versions are >> loosly tied to Windows releases. So I guess you can change the kernel >> interface among DirectX versions? >> If so, how would this work on Linux in the long term? If there ever is a >> DirectX 13 or 14 with incompatible kernel interfaces, how would you plan to >> update the Linux driver? > > You should think of the communication over the VM Bus for the vGPU projection > as a strongly versioned interface. We will be keeping compatibility with > older version of that interface as it evolves over time so we can continue to > run older guest (we already do). This protocol isn't actually tied to the DX > API. It is a generic abstraction for the GPU that can be used for any APIs > (for example the NVIDIA CUDA driver that we announced is going over the same > protocol to access the GPU). > > New version of user mode DX can either take advantage or sometime require new > services from this kernel abstraction. This mean that pulling a new version > of user mode DX can mean having to also pull a new version of this vGPU > kernel driver. For WSL, these essentially ships together. The kernel driver > ships as part of our WSL2 Linux Kernel integration. User mode DX bits ships > with Windows. Just a friendly advise: maintaining a proprietary component within a Linux environment is tough. You will need a good plan for long-term interface stability and compatibility with the other components. Best regards Thomas > > -Original Message- > From: Thomas Zimmermann > Sent: Wednesday, May 20, 2020 12:11 AM > To: Sasha Levin ; alexander.deuc...@amd.com; > ch...@chris-wilson.co.uk; ville.syrj...@linux.intel.com; > hawking.zh...@amd.com; tvrtko.ursu...@intel.com > Cc: linux-kernel@vger.kernel.org; linux-hyp...@vger.kernel.org; KY Srinivasan > ; Haiyang Zhang ; Stephen > Hemminger ; wei@kernel.org; Steve Pronovost > ; Iouri Tarassov ; > dri-de...@lists.freedesktop.org; linux-fb...@vger.kernel.org; > gre...@linuxfoundation.org > Subject: [EXTERNAL] Re: [RFC PATCH 0/4] DirectX on Linux > > Hi > > Am 19.05.20 um 18:32 schrieb Sasha Levin: >> There is a blog post that goes into more detail about the bigger >> picture, and walks through all the required pieces to make this work. >> It is available here: >> https://devblogs.microsoft.com/directx/directx-heart-linux . The rest >> of this cover letter will focus on the Linux Kernel bits. > > That's quite a surprise. Thanks for your efforts to contribute. > >> >> Overview >> >> >> This is the first draft of the Microsoft Virtual GPU (vGPU) driver. >> The driver exposes a paravirtualized GPU to user mode applications >> running in a virtual machine on a Windows host. This enables hardware >> acceleration in environment such as WSL (Windows Subsystem for Linux) >> where the Linux virtual machine is able to share the GPU with the >> Windows host. >> >> The projection is accomplished by exposing the WDDM (Windows Display >> Driver Model) interface as a set of IOCTL. This allows APIs and user >> mode driver written against the WDDM GPU abstraction on Windows to be >> ported to run within a Linux environment. This enables the port of the >> D3D12 and DirectML APIs as well as their associated user mode driver >> to Linux. This also enables third party APIs, such as the popular >> NVIDIA Cuda compute API, to be hardware accelerated within a WSL environment. >> >> Only the rendering/compute aspect of the GPU are projected to the >> virtual machine, no display functionality is exposed. Further, at this >> time there are no presentation integration. So although the D3D12 API >> can be use to render graphics offscreen, there is no path (yet) for >> pixel to flow from the Linux environment back onto the Windows host >> desktop. This GPU stack is effectively side-by-side with the native >> Linux graphics stack. >> >> The driver creates the /dev/dxg device, which can be opened by user >> mode application and handles their ioctls. The IOCTL interface to the >> driver is defined in dxgkmthk.h (Dxgkrnl Graphics Port Driver ioctl >> definitions). The interface matches the D3DKMT interface on Windows. >> Ioctls are implemented in ioctl.c. > > Echoing what others said, you're not making a DRM driver. The driver should > live outside of the DRM code. > > I have one question about the driver API: on Windows, DirectX versions are > loosly tied to Windows releases. So I guess you can change the kernel > interface among DirectX versions? > > If so, how would this work on Linux in the long term? If there ev
RE: [EXTERNAL] Re: [RFC PATCH 0/4] DirectX on Linux
>Echoing what others said, you're not making a DRM driver. The driver should >live outside of the DRM code. Agreed, please see my earlier reply. We'll be moving the driver to drivers/hyperv node or something similar. Apology for the confusion here. > I have one question about the driver API: on Windows, DirectX versions are > loosly tied to Windows releases. So I guess you can change the kernel > interface among DirectX versions? > If so, how would this work on Linux in the long term? If there ever is a > DirectX 13 or 14 with incompatible kernel interfaces, how would you plan to > update the Linux driver? You should think of the communication over the VM Bus for the vGPU projection as a strongly versioned interface. We will be keeping compatibility with older version of that interface as it evolves over time so we can continue to run older guest (we already do). This protocol isn't actually tied to the DX API. It is a generic abstraction for the GPU that can be used for any APIs (for example the NVIDIA CUDA driver that we announced is going over the same protocol to access the GPU). New version of user mode DX can either take advantage or sometime require new services from this kernel abstraction. This mean that pulling a new version of user mode DX can mean having to also pull a new version of this vGPU kernel driver. For WSL, these essentially ships together. The kernel driver ships as part of our WSL2 Linux Kernel integration. User mode DX bits ships with Windows. -Original Message- From: Thomas Zimmermann Sent: Wednesday, May 20, 2020 12:11 AM To: Sasha Levin ; alexander.deuc...@amd.com; ch...@chris-wilson.co.uk; ville.syrj...@linux.intel.com; hawking.zh...@amd.com; tvrtko.ursu...@intel.com Cc: linux-kernel@vger.kernel.org; linux-hyp...@vger.kernel.org; KY Srinivasan ; Haiyang Zhang ; Stephen Hemminger ; wei@kernel.org; Steve Pronovost ; Iouri Tarassov ; dri-de...@lists.freedesktop.org; linux-fb...@vger.kernel.org; gre...@linuxfoundation.org Subject: [EXTERNAL] Re: [RFC PATCH 0/4] DirectX on Linux Hi Am 19.05.20 um 18:32 schrieb Sasha Levin: > There is a blog post that goes into more detail about the bigger > picture, and walks through all the required pieces to make this work. > It is available here: > https://devblogs.microsoft.com/directx/directx-heart-linux . The rest > of this cover letter will focus on the Linux Kernel bits. That's quite a surprise. Thanks for your efforts to contribute. > > Overview > > > This is the first draft of the Microsoft Virtual GPU (vGPU) driver. > The driver exposes a paravirtualized GPU to user mode applications > running in a virtual machine on a Windows host. This enables hardware > acceleration in environment such as WSL (Windows Subsystem for Linux) > where the Linux virtual machine is able to share the GPU with the > Windows host. > > The projection is accomplished by exposing the WDDM (Windows Display > Driver Model) interface as a set of IOCTL. This allows APIs and user > mode driver written against the WDDM GPU abstraction on Windows to be > ported to run within a Linux environment. This enables the port of the > D3D12 and DirectML APIs as well as their associated user mode driver > to Linux. This also enables third party APIs, such as the popular > NVIDIA Cuda compute API, to be hardware accelerated within a WSL environment. > > Only the rendering/compute aspect of the GPU are projected to the > virtual machine, no display functionality is exposed. Further, at this > time there are no presentation integration. So although the D3D12 API > can be use to render graphics offscreen, there is no path (yet) for > pixel to flow from the Linux environment back onto the Windows host > desktop. This GPU stack is effectively side-by-side with the native > Linux graphics stack. > > The driver creates the /dev/dxg device, which can be opened by user > mode application and handles their ioctls. The IOCTL interface to the > driver is defined in dxgkmthk.h (Dxgkrnl Graphics Port Driver ioctl > definitions). The interface matches the D3DKMT interface on Windows. > Ioctls are implemented in ioctl.c. Echoing what others said, you're not making a DRM driver. The driver should live outside of the DRM code. I have one question about the driver API: on Windows, DirectX versions are loosly tied to Windows releases. So I guess you can change the kernel interface among DirectX versions? If so, how would this work on Linux in the long term? If there ever is a DirectX 13 or 14 with incompatible kernel interfaces, how would you plan to update the Linux driver? Best regards Thomas > > When a VM starts, hyper-v on the host adds virtual GPU devices to the > VM via the hyper-v driver. The host offers several VM bus channels to > the > VM: the global channel and one channel per virtual GPU, assigned to > the VM. > > The driver registers with the hyper-v driver (hv_driver) for the > arrival of
RE: [EXTERNAL] Re: [RFC PATCH 0/4] DirectX on Linux
Hey guys, Thanks for the discussion. I may not be able to immediately answer all of your questions, but I'll do my best 😊. drivers/hyperv sounds like it could be a better location. We weren't too sure where to put this, we though /drivers/gpu would be appropriate given this deal with GPUs, but I get your point... this is a vGPU driver that really only works when being run under Hyper-V, so drivers/hyperv is likely more appropriate. In term of presentation, I need to clarify a few things. We announced today that we're also adding support for Linux GUI applications. The way this will work is roughly as follow. We're writing a Wayland compositor that will essentially bridge over RDP-RAIL (RAIL=Remote Application Integrated Locally). We're starting from a Weston base. Weston already has an RDP Backend, but that's for a full desktop remoting scheme. Weston draws a desktop and remote it over RDP... and then you can peek at that desktop using an rdp client on the Windows side. RAIL works differently. In that case our wayland compositor no longer paint a desktop... instead it simply forward individual visual / wl_surface over the RDP RAIL channel such that these visual can be displayed on the Windows desktop. The RDP client create proxy window for each of these top level visual and their content is filled with the data coming over the RDP channel. All pixels are owned by the RDP server/WSL... so these windows looks different than native window are they are painted and themed by WSL. The proxy window on the host gather input and inject back over RDP... This is essentially how application remoting works on windows and this is all publicly documented as part of the various RDP protocol specification. As a matter of fact, for the RDP server on the Weston side we are looking at continue to leverage FreeRDP (and provide fixes/enhancement as needed to the public project). Further, we're looking at further improvement down this path to avoid having to copy the content over the RAIL channel and instead just share/swap buffer between the guest and the host. We have extension to the RDP protocol, called VAIL (Virtualized Application Integrated Locally) which does that today. Today this is only use in Windows on Windows for very specific scenario. We're looking at extending the public RDP protocol with these VAIL extension to make this an official Microsoft supported protocol which would allow us to target this in WSL. We have finished designing this part in details. Our goal would be to leverage something along the line of wl_drm, dma-buf, dma-fence, etc... This compositor and all our contribution to FreeRDP will be fully open source, including our design doc. We're not quite sure yet whether this will be offered as a separate project entirely distinct from it's Weston root... or if we'll propose an extension to Weston to operate in this mode. We would like to build it such that in theory any Wayland compositor could add support for this mode of operation if they want to remote application to a Windows host (over the network, or on the same box). We see /dev/dxg really as a projection of the GPU when running in WSL such that the GPU can be shared between WSL and the host... not something that would coexist "at the same time" with a real DRM GPU. We have consider the possibility of bringing DX to Linux with no Windows cord attached. I'm not ready to discuss this at this time 😊... but in the hypothetical that we were do this, DX would be running on top of DRI/DRM on native Linux. We likely would be contributing some changes to DRM to address area of divergence and get better mapping for our user mode driver, but we wouldn't try to shoehorn /dev/dxg into the picture. In that hypothetical world, we would essentially have DX target DRM on native Linux and DX continue to target DXG in WSL to share the GPU with the host. I think this further reinforce the point you guys were making that the right place for our current dxgkrnl driver to live in would be /drivers/hyperv/dxgkrnl. In insight, I totally agree 😊. I think this cover all questions, let me know if I missed anything. Thanks, Steve -Original Message- From: Daniel Vetter Sent: Tuesday, May 19, 2020 4:01 PM To: Dave Airlie Cc: Sasha Levin ; linux-hyp...@vger.kernel.org; Stephen Hemminger ; Ursulin, Tvrtko ; Greg Kroah-Hartman ; Haiyang Zhang ; LKML ; dri-devel ; Chris Wilson ; Steve Pronovost ; Linux Fbdev development list ; Iouri Tarassov ; Deucher, Alexander ; KY Srinivasan ; Wei Liu ; Hawking Zhang Subject: [EXTERNAL] Re: [RFC PATCH 0/4] DirectX on Linux On Wed, May 20, 2020 at 12:42 AM Dave Airlie wrote: > > On Wed, 20 May 2020 at 02:33, Sasha Levin wrote: > > > > There is a blog post that goes into more detail about the bigger > > picture, and walks through all the required pieces to make this > > work. It is available here: > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F