Re: Moving amber into separate repo?

[Jason Ekstrand]

On September 24, 2022 04:04:48 "Filip Gawin"  wrote:




2. repo is growing large. Amber kinda requires long history, modern
mesa not. This may be good spot to split if cleanup is required.


mesa absolutely uses long history. there is nothing to clean up. those
bytes of disk space are well worth it.

(Neutral on the other points, I don't work on stuff suported in Amber)
I thought here about traces in issues. (Sometimes traces are uploaded onto 
gitlab by users.) I'm guessing it should be possible to reedit really old 
closed issues and remove attachments.


If the fd.o admins are complaining about disk usage, we can take steps to 
reduce that but it's entirely irrelevant to the discussion of whether amber 
should get it's own repo. Pulling amber into a separate repo won't reduce 
disk usage. If anything, it'll slightly increase it but not by an amount 
that's likely to matter.


Let's stay focused on actual reasons that might matter, like user confusion 
(if that's actually happening).


--Jason

Re: Xbox Series S/X UWP

[Jason Ekstrand]

On June 6, 2022 21:07:32 Marek Olšák  wrote:

I'd love to see Mesa used to bring Vulkan to consoles!


Ever heard of Steam Deck? ;)


I have! I've even heard a rumour that a line it two of my code is running 
in it. :-P But why not take over ALL the consoles?


--Jason



Marek



On Mon, Jun 6, 2022 at 12:59 PM Jason Ekstrand  wrote:

On Mon, Jun 6, 2022 at 11:38 AM Jesse Natalie  wrote:


(Hopefully this goes through and not to spam like last time I tried to 
respond…)




No, neither of these would currently work with UWP.



The primary reason is that neither Khronos API has extensions to initialize 
the winsys on top of the UWP core window infrastructure. In theory, you 
could initialize Dozen for offscreen rendering and then explicitly marshal 
the contents out – that would probably work actually. There’s 2 more 
gotchas there though:


The ICD loaders (OpenGL32.dll, Vulkan-1.dll) are not available in the UWP 
environment. You could explicitly use the non-ICD version of GL (i.e. 
Mesa’s OpenGL32.dll from the libgl-gdi target), include the open-source 
Vulkan ICD loader, or use the ICD version of either 
(OpenGLOn12.dll/libgallium_wgl.dll for GL – I plan to delete the former at 
some point and just use the latter at some point; vulkan_dzn.dll for VK).
If the objective here is to write a Vulkan app and ship it in UWP, I don't 
see any reason why it couldn't be used for that eventually.  Without the 
loader, you can still load the driver DLL directly.  You just have to use 
vkGetInstance/DeviceProcAddr for everything because drivers don't expose 
the Vulkan 1.0 API symbols (the loader does).  We'd also have to come up 
with a story for window-system integration, as Jesse alluded.  I'd love to 
see Mesa used to bring Vulkan to consoles!  Unfortunately, no one's working 
on that currently or has any plans, as far as I know.




--Jason



There’s not currently extensions for D3D12 interop either spec’d or 
implemented.




There’s one more problem for GL that I don’t think is problematic for VK, 
which is that it uses APIs that are banned from the UWP environment, 
specifically around inserting window hooks for Win32 framebuffer lifetime 
management. So you’d probably have to build a custom version that has all 
of that stuff stripped out to get it to be shippable in a UWP.




We (Microsoft) don’t really have plans to add this kind of stuff, at least 
not in the near future, but I’d be open to accepting contributions that 
enable this.




-Jesse



From: mesa-dev  On Behalf Of Daniel 
Price

Sent: Monday, June 6, 2022 5:41 AM
To: mesa-dev@lists.freedesktop.org
Subject: [EXTERNAL] Xbox Series S/X UWP





You don't often get email from riverpr...@hotmail.com. Learn why this is 
important




Hi, I was wandering if these two layers would work with UWP on Xbox Series 
Console or if not will there be plans to add support?




https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14766



https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14881



Many Thanks

Dan

Re: Xbox Series S/X UWP

[Jason Ekstrand]

On Mon, Jun 6, 2022 at 11:38 AM Jesse Natalie 
wrote:

> (Hopefully this goes through and not to spam like last time I tried to
> respond…)
>
>
>
> No, neither of these would currently work with UWP.
>
>
>
> The primary reason is that neither Khronos API has extensions to
> initialize the winsys on top of the UWP core window infrastructure. In
> theory, you could initialize Dozen for offscreen rendering and then
> explicitly marshal the contents out – that would probably work actually.
> There’s 2 more gotchas there though:
>
>1. The ICD loaders (OpenGL32.dll, Vulkan-1.dll) are not available in
>the UWP environment. You could explicitly use the non-ICD version of GL
>(i.e. Mesa’s OpenGL32.dll from the libgl-gdi target), include the
>open-source Vulkan ICD loader, or use the ICD version of either
>(OpenGLOn12.dll/libgallium_wgl.dll for GL – I plan to delete the former at
>some point and just use the latter at some point; vulkan_dzn.dll for VK).
>
> If the objective here is to write a Vulkan app and ship it in UWP, I don't
see any reason why it couldn't be used for that eventually.  Without the
loader, you can still load the driver DLL directly.  You just have to use
vkGetInstance/DeviceProcAddr for everything because drivers don't expose
the Vulkan 1.0 API symbols (the loader does).  We'd also have to come up
with a story for window-system integration, as Jesse alluded.  I'd love to
see Mesa used to bring Vulkan to consoles!  Unfortunately, no one's working
on that currently or has any plans, as far as I know.

--Jason



>
>1.
>2. There’s not currently extensions for D3D12 interop either spec’d or
>implemented.
>
>
>
> There’s one more problem for GL that I don’t think is problematic for VK,
> which is that it uses APIs that are banned from the UWP environment,
> specifically around inserting window hooks for Win32 framebuffer lifetime
> management. So you’d probably have to build a custom version that has all
> of that stuff stripped out to get it to be shippable in a UWP.
>
>
>
> We (Microsoft) don’t really have plans to add this kind of stuff, at least
> not in the near future, but I’d be open to accepting contributions that
> enable this.
>
>
>
> -Jesse
>
>
>
> *From:* mesa-dev  *On Behalf Of 
> *Daniel
> Price
> *Sent:* Monday, June 6, 2022 5:41 AM
> *To:* mesa-dev@lists.freedesktop.org
> *Subject:* [EXTERNAL] Xbox Series S/X UWP
>
>
>
> You don't often get email from riverpr...@hotmail.com. Learn why this is
> important 
>
> Hi, I was wandering if these two layers would work with UWP on Xbox Series
> Console or if not will there be plans to add support?
>
>
>
> https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14766
>
>
>
> https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/14881
>
>
>
> Many Thanks
>
> Dan
>

[PATCH 0/2] dma-buf: Add an API for exporting sync files (v13)

[Jason Ekstrand]

Modern userspace APIs like Vulkan are built on an explicit synchronization
model.  This doesn't always play nicely with the implicit synchronization used
in the kernel and assumed by X11 and Wayland.  The client -> compositor half
of the synchronization isn't too bad, at least on intel, because we can
control whether or not i915 synchronizes on the buffer and whether or not it's
considered written.

The harder part is the compositor -> client synchronization when we get the
buffer back from the compositor.  We're required to be able to provide the
client with a VkSemaphore and VkFence representing the point in time where the
window system (compositor and/or display) finished using the buffer.  With
current APIs, it's very hard to do this in such a way that we don't get
confused by the Vulkan driver's access of the buffer.  In particular, once we
tell the kernel that we're rendering to the buffer again, any CPU waits on the
buffer or GPU dependencies will wait on some of the client rendering and not
just the compositor.

This new IOCTL solves this problem by allowing us to get a snapshot of the
implicit synchronization state of a given dma-buf in the form of a sync file.
It's effectively the same as a poll() or I915_GEM_WAIT only, instead of CPU
waiting directly, it encapsulates the wait operation, at the current moment in
time, in a sync_file so we can check/wait on it later.  As long as the Vulkan
driver does the sync_file export from the dma-buf before we re-introduce it
for rendering, it will only contain fences from the compositor or display.
This allows to accurately turn it into a VkFence or VkSemaphore without any
over-synchronization.

This patch series actually contains two new ioctls.  There is the export one
mentioned above as well as an RFC for an import ioctl which provides the other
half.  The intention is to land the export ioctl since it seems like there's
no real disagreement on that one.  The import ioctl, however, has a lot of
debate around it so it's intended to be RFC-only for now.

Mesa MR: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4037
IGT tests: https://patchwork.freedesktop.org/series/90490/

v10 (Jason Ekstrand, Daniel Vetter):
 - Add reviews/acks
 - Add a patch to rename _rcu to _unlocked
 - Split things better so import is clearly RFC status

v11 (Daniel Vetter):
 - Add more CCs to try and get maintainers
 - Add a patch to document DMA_BUF_IOCTL_SYNC
 - Generally better docs
 - Use separate structs for import/export (easier to document)
 - Fix an issue in the import patch

v12 (Daniel Vetter):
 - Better docs for DMA_BUF_IOCTL_SYNC

v12 (Christian König):
 - Drop the rename patch in favor of Christian's series
 - Add a comment to the commit message for the dma-buf sync_file export
   ioctl saying why we made it an ioctl on dma-buf

v13 (Jason Ekstrand):
 - Rebase on Christian König's fence rework

Cc: Christian König 
Cc: Michel Dänzer 
Cc: Dave Airlie 
Cc: Bas Nieuwenhuizen 
Cc: Daniel Stone 
Cc: mesa-dev@lists.freedesktop.org
Cc: wayland-de...@lists.freedesktop.org

Jason Ekstrand (2):
  dma-buf: Add an API for exporting sync files (v13)
  dma-buf: Add an API for importing sync files (v8)

 drivers/dma-buf/dma-buf.c| 100 +++
 include/uapi/linux/dma-buf.h |  57 
 2 files changed, 157 insertions(+)

-- 
2.36.0

[PATCH 0/2] *dma-buf: Add an API for exporting sync files (v13)

[Jason Ekstrand]

Modern userspace APIs like Vulkan are built on an explicit synchronization
model.  This doesn't always play nicely with the implicit synchronization used
in the kernel and assumed by X11 and Wayland.  The client -> compositor half
of the synchronization isn't too bad, at least on intel, because we can
control whether or not i915 synchronizes on the buffer and whether or not it's
considered written.

The harder part is the compositor -> client synchronization when we get the
buffer back from the compositor.  We're required to be able to provide the
client with a VkSemaphore and VkFence representing the point in time where the
window system (compositor and/or display) finished using the buffer.  With
current APIs, it's very hard to do this in such a way that we don't get
confused by the Vulkan driver's access of the buffer.  In particular, once we
tell the kernel that we're rendering to the buffer again, any CPU waits on the
buffer or GPU dependencies will wait on some of the client rendering and not
just the compositor.

This new IOCTL solves this problem by allowing us to get a snapshot of the
implicit synchronization state of a given dma-buf in the form of a sync file.
It's effectively the same as a poll() or I915_GEM_WAIT only, instead of CPU
waiting directly, it encapsulates the wait operation, at the current moment in
time, in a sync_file so we can check/wait on it later.  As long as the Vulkan
driver does the sync_file export from the dma-buf before we re-introduce it
for rendering, it will only contain fences from the compositor or display.
This allows to accurately turn it into a VkFence or VkSemaphore without any
over-synchronization.

This patch series actually contains two new ioctls.  There is the export one
mentioned above as well as an RFC for an import ioctl which provides the other
half.  The intention is to land the export ioctl since it seems like there's
no real disagreement on that one.  The import ioctl, however, has a lot of
debate around it so it's intended to be RFC-only for now.

Mesa MR: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4037
IGT tests: https://patchwork.freedesktop.org/series/90490/

v10 (Jason Ekstrand, Daniel Vetter):
 - Add reviews/acks
 - Add a patch to rename _rcu to _unlocked
 - Split things better so import is clearly RFC status

v11 (Daniel Vetter):
 - Add more CCs to try and get maintainers
 - Add a patch to document DMA_BUF_IOCTL_SYNC
 - Generally better docs
 - Use separate structs for import/export (easier to document)
 - Fix an issue in the import patch

v12 (Daniel Vetter):
 - Better docs for DMA_BUF_IOCTL_SYNC

v12 (Christian König):
 - Drop the rename patch in favor of Christian's series
 - Add a comment to the commit message for the dma-buf sync_file export
   ioctl saying why we made it an ioctl on dma-buf

v13 (Jason Ekstrand):
 - Rebase on Christian König's fence rework

Cc: Christian König 
Cc: Michel Dänzer 
Cc: Dave Airlie 
Cc: Bas Nieuwenhuizen 
Cc: Daniel Stone 
Cc: mesa-dev@lists.freedesktop.org
Cc: wayland-de...@lists.freedesktop.org

Jason Ekstrand (2):
  dma-buf: Add an API for exporting sync files (v13)
  dma-buf: Add an API for importing sync files (v8)

 drivers/dma-buf/dma-buf.c| 100 +++
 include/uapi/linux/dma-buf.h |  57 
 2 files changed, 157 insertions(+)

-- 
2.36.0

Re: Replacing NIR with SPIR-V?

[Jason Ekstrand]

On Thu, Jan 20, 2022 at 5:49 PM Abel Bernabeu <
abel.berna...@esperantotech.com> wrote:

> In principle, all the properties you highlight in your blog
> <https://www.jlekstrand.net/jason/projects/mesa/nir-notes/> as key points
> of NIR also apply to SPIR-V.
>

First off, that blog post is truly ancient.  Based on the quote from
nir_opt_algebraic.c, it looks like less than 6 months after the original
NIR patches landed which puts it at 5-6 years old.  A lot has changed since
then.


> I was curious to know where in the details that I miss, NIR starts shining
> as a more suitable IR than SPIR-V for the task of communicating front-end
> and back-end. By the way, thanks for putting together that blog post.
>

In terms of what they're capable of communicating, yes, SPIR-V and NIR can
express many of the same things.  But that's not the point.  The point is
that there's a lot that happens between coming out of GLSL or SPIR-V and
going into the back-end.  A lot of what we do with NIR is share as much of
that lowering across drivers as possible.  Yes, we could convert back to
SPIR-V before going into back-ends but there's really no point since they
need their own IRs anyway.  If you're dumping straight into LLVM or
similar, then maybe you don't need any of that, but if you're building a
custom back-end, you really want to let NIR do that lowering and you don't
want to handle it all on your own.


> As it seems clear that the NIR question is well settled within the mesa
> community and I really see value in having mesa drivers, I promise to pay
> as much attention to the NIR use cases as I did with SPIR-V :-)
>
> By the way, we are not planning on supporting with specific RISC-V
> instructions everything that has an instruction on SPIR-V. Regarding the
> two areas you mention:
>
> - Arrays and structs: SPIR-V's OpAccessChain would need to be processed by
> a backend and translated to pointer arithmetic plus dereferencing (kind of
> the same thing as having to process a nir_deref). This translation can be
> done in RISC-V with no issue, whether it is OpAccessChain or nir_deref.
>

A big part of the point of NIR is to get rid of these things so that
drivers don't have to deal with them.  Yes, NIR arrays and struct and
nir_deref to deal with them but, by the time you get into the back-end, all
the nir_derefs are gone and you're left with load/store messages with
actual addresses (either a 64-bit memory address or a index+offset pair for
a bound resource).  Again, unless you're going to dump straight into LLVM,
you really don't want to handle that in your back-end unless you really
have to.

Over-all, I think you're asking the wrong set of questions.  If you're
trying to understand Mesa GPU compilers, looking at NIR from documentation
and blog posts and comparing with SPIR-V is likely to raise more questions
than answers.  I would instead recommend looking at an actual driver and
seeing how things flow through the compiler stack.  That's likely to teach
you a lot more about how the Mesa compiler stack works than reading blogs.
That, or start implementing a NIR back-end and see what you run into and
ask questions on #dri-devel.

--Jason



> - Trigonometric operations: personally I consider that only "sin" and
> "cos" are needed additions for RISC-V. Unclear what precision yet, likely 8
> bits, for serving as initial value for a Newton-Rapson style computation.
>
> Regards.
>
> On Thu, Jan 20, 2022 at 2:36 AM Jason Ekstrand 
> wrote:
>
>> > - Does it make sense to move to SPIR-V?
>>
>> None whatsoever.  SPIR-V is an interchange format, not a set of
>> manipulatable data structures suitable for compiler lowering and
>> optimization.
>>
>> You also don't want to build hardware around consuming SPIR-V.  There are
>> lots of things that the SPIR-V has which you wouldn't want to support
>> natively in hardware such as structures and arrays in SSA values or complex
>> trig ops like atan2().  Part of the purpose of NIR is to lower these things
>> to simpler constructs which are supported in native hardware.
>>
>> --Jason
>>
>> On Wed, Jan 19, 2022 at 7:17 PM Abel Bernabeu <
>> abel.berna...@esperantotech.com> wrote:
>>
>>> Hi,
>>>
>>> My name Abel Bernabeu and I currently chair the Graphics and ML Special
>>> Interest Group within RISC-V.
>>>
>>> As part of my work for RISC-V I am currently looking at what is needed
>>> for supporting a graphics product that uses a (potentially extended) RISC-V
>>> ISA for its shading cores. My initial focus has been on analyzing the
>>> functional gap between RISC-V and SPIR-V, assuming that whatever is needed
>>> for a modern graphics accelerator is inevitably pr

Re: Replacing NIR with SPIR-V?

[Jason Ekstrand]

> - Does it make sense to move to SPIR-V?

None whatsoever.  SPIR-V is an interchange format, not a set of
manipulatable data structures suitable for compiler lowering and
optimization.

You also don't want to build hardware around consuming SPIR-V.  There are
lots of things that the SPIR-V has which you wouldn't want to support
natively in hardware such as structures and arrays in SSA values or complex
trig ops like atan2().  Part of the purpose of NIR is to lower these things
to simpler constructs which are supported in native hardware.

--Jason

On Wed, Jan 19, 2022 at 7:17 PM Abel Bernabeu <
abel.berna...@esperantotech.com> wrote:

> Hi,
>
> My name Abel Bernabeu and I currently chair the Graphics and ML Special
> Interest Group within RISC-V.
>
> As part of my work for RISC-V I am currently looking at what is needed for
> supporting a graphics product that uses a (potentially extended) RISC-V ISA
> for its shading cores. My initial focus has been on analyzing the
> functional gap between RISC-V and SPIR-V, assuming that whatever is needed
> for a modern graphics accelerator is inevitably present on SPIR-V.
>
> Now, the thing is that most of the potential adopters on our committee
> will likely be interested in using mesa for developing their drivers and
> that means using NIR as intermediate representation. Thus, I also need to
> consider NIR when looking at the functional gap, doubling the amount of
> work during the analysis.
>
> Why is mesa using NIR as intermediate representation rather than SPIR-V?
> It would make my life easier if mesa used SPIR-V rather than NIR for
> communicating the front-end and the backends.
>
> I know it is a lot of work to migrate to SPIR-V, but I am interested in
> knowing what is the opinion of the mesa developers:
>
> - My understanding is that when mesa adopted NIR, there was no SPIR-V. Was
> a comparison made after the SPIR-V ratification?
>
> - Does it make sense to move to SPIR-V?
>
> - Is it feasible in terms of functionality supported by SPIR-V?
>
> - Is the cost worth the potential advantage of using a more commonly
> adopted standard?
>
> Thanks in advance for your time and thoughts.
>
> Regards.
>

[PATCH] .mailmap: Switch Jason Ekstrand to @collabora.com

[Jason Ekstrand]

Jason is starting at Collabora on the 14th.  More details at
https://www.jlekstrand.net/jason/blog/2022/01/hello-collabora/
---
 .mailmap | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/.mailmap b/.mailmap
index e8e087ba2d4..e7c1776f306 100644
--- a/.mailmap
+++ b/.mailmap
@@ -297,7 +297,8 @@ Jan Vesely  Jan Vesely 

 
 Jan Zielinski  jzielins 
 
-Jason Ekstrand  
+Jason Ekstrand  
+Jason Ekstrand  
 
 Jeremy Huddleston 
 Jeremy Huddleston  
-- 
2.34.1

Re: Moving code around, post classic

[Jason Ekstrand]

On Thu, Dec 9, 2021 at 11:24 AM Timur Kristóf 
wrote:

> On Tue, 2021-12-07 at 08:19 -0500, Alyssa Rosenzweig wrote
> >
> > If it were just linked lists, I'd say someone should write the
> > Coccinelle to transform the tree to use the one in util and call it a
> > day. It's a bit more complicated though, NIR depends on GLSL types.
> > Though that could probably continue to live in its current location
> > even
> > if glsl moves? Might breed confusion.
>
>
> These GLSL types seem to be used by a lot more than just GLSL. To avoid
> some of the confusion, why not rename them to something like NIR types,
> or something along these lines?
>

First off, they're already in src/compiler, not src/compiler/glsl.

Could/should we rename them?  I'm fine with that and, honestly, the only
reason I haven't yet is because it's a pile of work and because I've not
been able to come up with a better name than `glsl_type` that sounds
generic to all compiler things.  We could go `nir_type`, I guess but that's
already kind-of taken by NIR ALU types.  Then again, those could be renamed
too.  I'm kind-of meh on it.

--Jason

Re: Amber branch plan

[Jason Ekstrand]

+1

On Mon, Dec 6, 2021 at 5:51 PM Dylan Baker  wrote:

> Since classic is gone,  I thought I'd lay out my thinking for Amber.
>
> I'm assuming that we'll branch Amber from the 21.3 branch, after that
> reaches normal EOL. That gives us the benefit of developing on top of a
> known good state for classic drivers, and should minimize friction for
> distros dealing with classic. If anyone wants to backport changes from
> main to amber they can obviously do so.
>
> Are there any objections to that plan?
>
> Dylan

Re: Moving code around, post classic

[Jason Ekstrand]

On Mon, Dec 6, 2021 at 5:50 PM Dylan Baker  wrote:

> Classic is gone, and the cleanups have begun, obviously. There is
> another cleanup that I had in mind, which is moving src/mesa into
> src/gallium/frontends/mesa. This makes the build system a little
> cleaner, as currently we do some bending over backwards to get gallium,
> mesa, and their tests built in the right order. But that's a big ol `git
> mv`, and when I proposed it Dave and Ilia suggested it would be best to
> do all of the post-classic code motion at once. So, let's talk about
> what we want to move, and where we want to move it.
>
> Among the suggestions we had were:
>
> 1. Move src/mesa into src/gallium/frontends/mesa (I have patches for
>this)
>
>Seems like a pretty obvoius thing to do, given that all of the other
>gallium state trackers live there (OpenCL, video, d3d9, etc)
>
> 2. Move src/compiler/glsl into src/gallium/frontends/mesa as well
>
> Given that there are now no? drivers that use GLSL-IR directly, it
> might make sense to move the glsl compiler into the mesa
> state_tracker, and just have that lower to TGSI or NIR, and treat
> GLSL-IR as an implementation detail of the OpenGL frontend.
>
> Unfortunately, there are a lot of code outside of glsl that uses the
> linked list implementation in the glsl compiler, and not the on in
> util.
>

+1 to the first two.


> 3. Move src/gallium* to src/
>
> This was suggested, though given the existance of Vulkan, it wasn't
> clear that this was a good idea or not
>

If we're going to do this, I wonder if we don't want to go even further and
get rid of src/gallium/drivers and move the respective folders to
src/vendor.  So, instead of src/gallium/drivers/(iris|crocus), we'd have
src/intel/gallium/iris and src/intel/gallium/crocus or maybe src/intel/iris
and src/intel/crocus.

--Jason


> 4. What to do about the src/loader, src/glx, src/egl, src/mapi,
>src/glapi
>
> These are all part of OpenGL, but not really part of gallium, but if
> we don't move src/gallium/* to src/ does it make sense to leave them
> in the root?
>
>
> Cheers,
> Dylan

Re: [Mesa-dev] First merge request, should it have been noticed by now?

[Jason Ekstrand]

Also, welcome to the Mesa community and thanks for your contribution! Sorry 
your first experience has been a bit bumpy.  I hope this doesn't put you 
off too badly. We always like when new people show up and dive in. 


--Jason


On November 13, 2021 22:49:44 Autumn  wrote:
All right! Coming in as an outsider and getting no signals is hard to know 
how to interpret, so thanks for letting me know.

-- Autumn
On 11/13/21 19:33, Dave Airlie wrote:



On Sun, 14 Nov 2021, 08:38 Autumn,  wrote:
Hello! I submitted my first merge request to the Mesa GitLab earlier
this month:
https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13671

It hasn't been noticed yet as far as I can tell. I don't need it
reviewed quickly or anything, but I want to check if this is normal.
Should I not have marked it "draft" as a precaution against merging the
two commits I don't have the hardware to test?

I'm aware of it, just haven't had much time to do llvmpipe stuff past few 
weeks, it just requires me paging back in quite a bit of history so I 
sometimes wait until I have a few things in the area.


Dave.

-- Autumn

Re: [Mesa-dev] vulkan video + beta header enables

[Jason Ekstrand]

On Thu, Nov 4, 2021 at 8:53 PM Dave Airlie  wrote:
>
> Just polling for opinions here on how best to move forward with
> enabling the provisional vulkan beta headers in Mesa and the fallout
> from doing so for people importing updated vulkan headers.
>
> Do we want to allow beta/provisional features in main?

IMO, it's fine as long as you never land the final patch which flips
on the extension or, if you do, make sure it's very carefully guarded.

> Do we want to do it only under a meson option that distros never enable?

RADV_I_KNOW_IM_ASKING_FOR_BETA_FEATURES_WHICH_MAY_NOT_WORK=

That way, it's harder to put in your .profile. :-P  Honestly, though,
just leave the last patch which flips it on in an MR that doesn't get
merged until the extension is final.

> If we do, who is responsible for updating the beta code on new header
> import, since the video headers are under quite regular changes?

That needs to be a collaboration between the people working on the
beta feature.  Thanks to all the XML code-gen we do, it's pretty hard
to update it one driver at a time.  Maybe that'd be possible but
ugh

--Jason

Re: [Mesa-dev] GitLab Tag Proposal

[Jason Ekstrand]

Fine with me. 

On October 31, 2021 10:11:44 Michael Blumenkrantz 
 wrote:



Hi,

I'd like to propose adding/changing two tags:

* change `feature_request` -> `Feature` (this is barely used at present, so 
renaming shouldn't affect anyone negatively)

* add `Bug`

I would use these primarily for tagging open MRs so that reviewers can more 
appropriately prioritize bug fixes at a glance over (less urgent) feature 
implementations. They would also service the same feature for issues, 
though those are usually more obvious from the title anyway.



Does anyone else think these would be useful?


Mike

Re: [Mesa-dev] Workflow Proposal

[Jason Ekstrand]

On Tue, Oct 12, 2021 at 3:56 PM apinheiro  wrote:
>
>
> On 12/10/21 13:55, Alyssa Rosenzweig wrote:
>
> I would love to see this be the process across Mesa.  We already don't
> rewrite commit messages for freedreno and i915g, and I only have to do
> the rebase (busy-)work for my projects in other areas of the tree.
>
> Likewise for Panfrost. At least, I don't do the rewriting. Some Panfrost
> devs do, which I'm fine with. But it's not a requirement to merging.
>
> The arguments about "who can help support this years from now?" are moot
> at our scale... the team is small enough that the name on the reviewer
> is likely the code owner / maintainer, and patches regularly go in
> unreviewed for lack of review bandwidth.
>
> There is another reason to the Rb tag, that is to measure the quantity
> of patch review people do.
>
> This was well summarized some years ago by Matt Turner, as it was
> minimized (even suggested to be removed) on a different thread:
>
> https://lists.freedesktop.org/archives/mesa-dev/2019-January/213586.html
>
> I was part of the Intel team when people started doing this r-b
> counting.  I believe that it was being done due to Intel management's
> failure to understand who was doing the work on the team and credit
> them appropriately, and also to encourage those doing less to step up.
>
>
> That's basically the same problem with trying to measure and compare 
> developers just by commit count. In theory commit count is a bad measure for 
> that. In practice it is used somehow.
>
> Unfortunately, the problem with Intel management wasn't a lack of
> available information, and I didn't see publishing the counts change
> reviews either.
>
> 
>
> Upstream should do what's best for upstream, not for Intel's "unique"
> management.
>
>
> Not sure how from Emma explaining how Rb tags were used by Intel management 
> it came the conclusion that it were used in that way only by Intel 
> management. Spoiler: it is not.
>
> Replying both, that's is one of the reasons I pointed original Matt Turner 
> email. He never mentioned explicitly Intel management, neither pointed this 
> as an accurate measure of the use. Quoting:
>
> "The number of R-b tags is not a 100% accurate picture of the
> situation, but it gives at least a good overview of who is doing the
> tedious work of patch review. "
>
> In any case, just to be clear here: Im not saying that the Rb tags main use 
> is this one. Just saying that is one of their uses, and the value for such 
> use can be debatable, but it is not zero.

Negative numbers aren't zero!

--Jason

Re: [Mesa-dev] Workflow Proposal

[Jason Ekstrand]

On Wed, Oct 6, 2021 at 12:37 PM Mike Blumenkrantz
 wrote:
>
> On Wed, Oct 6, 2021 at 1:27 PM Bas Nieuwenhuizen  
> wrote:
>>
>> On Wed, Oct 6, 2021 at 7:07 PM Jason Ekstrand  wrote:
>> >
>> > On Wed, Oct 6, 2021 at 11:24 AM Emma Anholt  wrote:
>> > >
>> > > On Wed, Oct 6, 2021 at 9:20 AM Mike Blumenkrantz
>> > >  wrote:
>> > > >
>> > > > Hi,
>> > > >
>> > > > It's recently come to my attention that gitlab has Approvals. Was 
>> > > > anyone else aware of this feature? You can just click a button and 
>> > > > have your name recorded in the system as having signed off on landing 
>> > > > a patch? Blew my mind.
>> > > >
>> > > > So with that being said, we also have this thing in the Mesa repo 
>> > > > where everyone* has to constantly be adding these esoteric tags like 
>> > > > Reviewed-by (I reviewed it), and Acked-by (I rubber stamped it), or 
>> > > > Tested-by (I compiled it and maybe ran glxgears), and so forth.
>> > > >
>> > > > * Except some incredibly smart people already know where I'm going 
>> > > > with this
>> > > >
>> > > > Instead of continuing to have to manually update each patch with the 
>> > > > appropriate and definitely-unforgeable tags, what if we just used 
>> > > > Approvals in the UI instead? We could then have marge-bot require 
>> > > > approvals as needed in components and bring reviewing into the current 
>> > > > year. Just think: no more rewriting all the commit logs and 
>> > > > force-pushing the branch again before you merge!
>> > > >
>> > > > Anyway, I thought maybe this would be a nice idea to improve 
>> > > > everyone's workflows. What do other people think?
>> >
>> > My primary grip with approvals or the  button is that it's the wrong
>> > granularity.  It's per-MR instead of per-patch.  When people are
>> > regularly posting MRs that touch a bunch of different stuff, per-patch
>> > review is pretty common.  I'm not sure I want to lose that.  :-/
>>
>> Would it be an option to get Marge to not remove existing Rb tags, so
>> we could get the streamlined process where possible and fall back if
>> the MRs turn more complicated?
>
>
> If people really, truly care about per-patch Approval, couldn't they just 
> split out patches from bigger MRs and get Approvals there? Otherwise it 
> should be trivial enough to check the gitlab MR and see who reviewed which 
> patch if it becomes an issue at a later date. Odds are at that point you're 
> already going to the MR to see wtf someone was thinking...

That's a really easy thing to say but this is an actual problem and
one I hit on a regular basis.  Take, for instance, this MR:

https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/13045

What am I supposed to do?  Should I post one MR and merge it as soon
as at least one representative from each touched driver approves?  On
the above MR, I've gotten quite a few people to sign off on the
search-and-replace driver patch but no one has yet to look at the
common bits.  How do I know when those are reviewed?  Or should I
assume everyone who clicks "approve" reviewed every part of the MR
that touches their driver.  That would mean the common bits would get
reviewed 6 times which, while great for code quality, is a waste of
review time.

Or maybe I can split it up.  Make one MR with all the common
improvements, then 6 per-driver MRs and then another big MR that goes
on top of the per-driver MRs?  In that case, because GitLab also
doesn't have a concept of MR dependencies, the initial common patches
are going to be in all 8, and everything's going to be in the last
one.  How the "what did they review?" confusion is even worse.

And, no, I can't trust people to approve the MR they intend to.  Just
the other day, I posted !13184 which explicitly said in the
description that it's based on !13156 and you (Mike) reviewed a patch
from !13156 on !13184.

Or should I post the one MR for common code first and then wait for
that to land before posting the per-driver MRs.  That doesn't work out
well because can be very important, when reviewing common code, to see
how it impacts your driver.

Just to be clear, the above are all genuine questions.  I want the
button-based review process as much as the next person but I have yet
to come up with a scheme that actually works when you start crossing
component boundaries.  The best I've got is typing RB tags in comments
and copy+pasting them into commit messages.  If someone's got a plan

Re: [Mesa-dev] Workflow Proposal

[Jason Ekstrand]

On Wed, Oct 6, 2021 at 11:24 AM Emma Anholt  wrote:
>
> On Wed, Oct 6, 2021 at 9:20 AM Mike Blumenkrantz
>  wrote:
> >
> > Hi,
> >
> > It's recently come to my attention that gitlab has Approvals. Was anyone 
> > else aware of this feature? You can just click a button and have your name 
> > recorded in the system as having signed off on landing a patch? Blew my 
> > mind.
> >
> > So with that being said, we also have this thing in the Mesa repo where 
> > everyone* has to constantly be adding these esoteric tags like Reviewed-by 
> > (I reviewed it), and Acked-by (I rubber stamped it), or Tested-by (I 
> > compiled it and maybe ran glxgears), and so forth.
> >
> > * Except some incredibly smart people already know where I'm going with this
> >
> > Instead of continuing to have to manually update each patch with the 
> > appropriate and definitely-unforgeable tags, what if we just used Approvals 
> > in the UI instead? We could then have marge-bot require approvals as needed 
> > in components and bring reviewing into the current year. Just think: no 
> > more rewriting all the commit logs and force-pushing the branch again 
> > before you merge!
> >
> > Anyway, I thought maybe this would be a nice idea to improve everyone's 
> > workflows. What do other people think?

My primary grip with approvals or the  button is that it's the wrong
granularity.  It's per-MR instead of per-patch.  When people are
regularly posting MRs that touch a bunch of different stuff, per-patch
review is pretty common.  I'm not sure I want to lose that.  :-/

--Jason

> I would love to see this be the process across Mesa.  We already don't
> rewrite commit messages for freedreno and i915g, and I only have to do
> the rebase (busy-)work for my projects in other areas of the tree.
>
> I don't think we should have marge-bot require approvals
> per-component, though.  There are times when an MR only incidentally
> touches a component (for example, changing function signatures in
> gallium), and actually getting a dev from every driver to sign off on
> it would be too much.

Re: [Mesa-dev] [PATCH 0/6] dma-buf: Add an API for exporting sync files (v12)

[Jason Ekstrand]

uf is a different beast, and i915 (and fwiw
> > these other drivers) need to change before they can do dynamic dma-buf.
> >
> >> Otherwise we have an information leak worth a CVE and that is certainly not
> >> something we want.
> > Because yes otherwise we get a CVE. But right now I don't think we have
> > one.
>
> Yeah, agree. But this is just because of coincident and not because of
> good engineering :)
>
> > We do have a quite big confusion on what exactly the signaling ordering is
> > supposed to be between exclusive and the collective set of shared fences,
> > and there's some unifying that needs to happen here. But I think what
> > Jason implements here in the import ioctl is the most defensive version
> > possible, so really can't break any driver. It really works like you have
> > an ad-hoc gpu engine that does nothing itself, but waits for the current
> > exclusive fence and then sets the exclusive fence with its "CS" completion
> > fence.
> >
> > That's imo perfectly legit use-case.
>
> The use case is certainly legit, but I'm not sure if merging this at the
> moment is a good idea.
>
> Your note that drivers are already ignoring the exclusive fence in the
> dma_resv object was eye opening to me. And I now have the very strong
> feeling that the synchronization and the design of the dma_resv object
> is even more messy then I thought it is.
>
> To summarize we can be really lucky that it didn't blow up into our
> faces already.
>
> > Same for the export one. Waiting for a previous snapshot of implicit
> > fences is imo perfectly ok use-case and useful for compositors - client
> > might soon start more rendering, and on some drivers that always results
> > in the exclusive slot being set, so if you dont take a snapshot you
> > oversync real bad for your atomic flip.
>
> The export use case is unproblematic as far as I can see.

Then why are we holding it up?  I'm not asking to have import merged.
That's still labled RFC and I've said over and over that it's not
baked and I'm not convinced it helps all that much.

> >>> Those changes are years in the past.  If we have a real problem here (not 
> >>> sure on
> >>> that yet), then we'll have to figure out how to fix it without nuking
> >>> uAPI.
> >> Well, that was the basic idea of attaching flags to the fences in the
> >> dma_resv object.
> >>
> >> In other words you clearly denote when you have to wait for a fence before
> >> accessing a buffer or you cause a security issue.
> > Replied somewhere else, and I do kinda like the flag idea. But the problem
> > is we first need a ton more encapsulation and review of drivers before we
> > can change the internals. One thing at a time.

I think I'm warming to flags as well.  I didn't like them at first but
I think they actually make quite a bit of sense.  Unlike the explicit
fence where ignoring it can lead to a security hole, the worst that
happens if someone forgets a flag somewhere is a bit of memory
corruption and garbage on-screen.  That seems fairly non-dangerous to
me.

> Ok how should we then proceed?
>
> The large patch set I've send out to convert all users of the shared
> fence list to a for_each API is a step into the right direction I think,
> but there is still a bit more todo.
>
> > And yes for amdgpu this gets triple-hard because you both have the
> > ttm_bo->moving fence _and_ the current uapi of using fence ownership _and_
> > you need to figure out how to support vulkan properly with true opt-in
> > fencing.
>
> Well I have been pondering on that for a bit and I came to the
> conclusion that it is actually not a problem at all.
>
> See radeon, nouveau, msm etc... all implement functions that they don't
> wait for fences from the same timeline, context, engine. That amdgpu
> doesn't wait for fences from the same process can be seen as just a
> special case of this.

That doesn't totally solve the issue, though.  RADV suffers in the WSI
arena today from too much cross-process implicit sharing.  I do think
you want some sort of "ignore implicit sync" API but, in this new
world of flags, it would look more like "don't bother looking for
shared fences flagged write".  You'd still respect the exclusive
fence, if there is one, and you'd still add a shared fence.  You just
wouldn't bother with implicit sync.  Should be safe, right?

--Jason

> >   I'm pretty sure it's doable, I'm just not finding any time
> > anywhere to hack on these patches - too many other fires :-(
>
> Well I'm here. Let's just agree on the direction and I can do the coding.
>
> What I need help with is all the auditing

Re: [Mesa-dev] [PATCH 0/6] dma-buf: Add an API for exporting sync files (v12)

[Jason Ekstrand]

On Tue, Jun 15, 2021 at 3:41 AM Christian König
 wrote:
>
> Hi Jason & Daniel,
>
> maybe I should explain once more where the problem with this approach is
> and why I think we need to get that fixed before we can do something
> like this here.
>
> To summarize what this patch here does is that it copies the exclusive
> fence and/or the shared fences into a sync_file. This alone is totally
> unproblematic.
>
> The problem is what this implies. When you need to copy the exclusive
> fence to a sync_file then this means that the driver is at some point
> ignoring the exclusive fence on a buffer object.

Not necessarily.  Part of the point of this is to allow for CPU waits
on a past point in buffers timeline.  Today, we have poll() and
GEM_WAIT both of which wait for the buffer to be idle from whatever
GPU work is currently happening.  We want to wait on something in the
past and ignore anything happening now.

But, to the broader point, maybe?  I'm a little fuzzy on exactly where
i915 inserts and/or depends on fences.

> When you combine that with complex drivers which use TTM and buffer
> moves underneath you can construct an information leak using this and
> give userspace access to memory which is allocated to the driver, but
> not yet initialized.
>
> This way you can leak things like page tables, passwords, kernel data
> etc... in large amounts to userspace and is an absolutely no-go for
> security.

Ugh...  Unfortunately, I'm really out of my depth on the implications
going on here but I think I see your point.

> That's why I'm said we need to get this fixed before we upstream this
> patch set here and especially the driver change which is using that.

Well, i915 has had uAPI for a while to ignore fences.  Those changes
are years in the past.  If we have a real problem here (not sure on
that yet), then we'll have to figure out how to fix it without nuking
uAPI.

--Jason


> Regards,
> Christian.
>
> Am 10.06.21 um 23:09 schrieb Jason Ekstrand:
> > Modern userspace APIs like Vulkan are built on an explicit
> > synchronization model.  This doesn't always play nicely with the
> > implicit synchronization used in the kernel and assumed by X11 and
> > Wayland.  The client -> compositor half of the synchronization isn't too
> > bad, at least on intel, because we can control whether or not i915
> > synchronizes on the buffer and whether or not it's considered written.
> >
> > The harder part is the compositor -> client synchronization when we get
> > the buffer back from the compositor.  We're required to be able to
> > provide the client with a VkSemaphore and VkFence representing the point
> > in time where the window system (compositor and/or display) finished
> > using the buffer.  With current APIs, it's very hard to do this in such
> > a way that we don't get confused by the Vulkan driver's access of the
> > buffer.  In particular, once we tell the kernel that we're rendering to
> > the buffer again, any CPU waits on the buffer or GPU dependencies will
> > wait on some of the client rendering and not just the compositor.
> >
> > This new IOCTL solves this problem by allowing us to get a snapshot of
> > the implicit synchronization state of a given dma-buf in the form of a
> > sync file.  It's effectively the same as a poll() or I915_GEM_WAIT only,
> > instead of CPU waiting directly, it encapsulates the wait operation, at
> > the current moment in time, in a sync_file so we can check/wait on it
> > later.  As long as the Vulkan driver does the sync_file export from the
> > dma-buf before we re-introduce it for rendering, it will only contain
> > fences from the compositor or display.  This allows to accurately turn
> > it into a VkFence or VkSemaphore without any over- synchronization.
> >
> > This patch series actually contains two new ioctls.  There is the export
> > one mentioned above as well as an RFC for an import ioctl which provides
> > the other half.  The intention is to land the export ioctl since it seems
> > like there's no real disagreement on that one.  The import ioctl, however,
> > has a lot of debate around it so it's intended to be RFC-only for now.
> >
> > Mesa MR: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4037
> > IGT tests: https://patchwork.freedesktop.org/series/90490/
> >
> > v10 (Jason Ekstrand, Daniel Vetter):
> >   - Add reviews/acks
> >   - Add a patch to rename _rcu to _unlocked
> >   - Split things better so import is clearly RFC status
> >
> > v11 (Daniel Vetter):
> >   - Add more CCs to try and get maintainers
> >   - Add a patch to document DMA_BUF_IOCTL_SYNC
> >   - Generally better docs
> &

Re: [Mesa-dev] [RFC] Concrete proposal to split classic

[Jason Ekstrand]

On Tue, Jun 15, 2021 at 8:46 PM Timothy Arceri  wrote:
>
> On 6/16/21 11:03 AM, Jason Ekstrand wrote:
>
> I'm bringing this up via e-mail so it gets a wider audience. Given how will 
> crocus is working at this point, is like to propose we hold off for about 
> three more releases before we drop classic. This next release, 21.2, we'll 
> have crocus as an option with i965 as the default. There will also be a 
> -Dprefer-crocus meson options so distros or individuals can attempt to flip 
> it on. The release after that, 21.3, we'll keep i965 in the tree but have 
> crocus be the default (assuming things are going well.) Some time in 2022, 
> probably after the 22.2 release or so, we'll delete classic.
>
> Why wait so long? Well, it just landed and we don't have a Cherryview story 
> yet so I'm hesitant to make it the default too quickly. Even if it were the 
> default in 21.2, it's already too late, likely, to hit the fall 2021 distro 
> release cycle. If we flip it to the default before the end of the year, 
> that'll get crocus into spring distros. This is good because 22.04 is an 
> Ubuntu LTS release and I think they'd rather bump crocus versions to fix bugs 
> than backport on top of i965. But that's really fort Ubuntu to decide. In any 
> case, we won't see broad-spread usage and the flood of bug reports until next 
> spring so we may want to wait until then to stay deleting code.
>
> If we wanted to accelerate things, one option, once we're ready, would be to 
> ask the person who manages the oibaf PPA to switch to crocus early. That may 
> get some early adopters on board.
>
> Thoughts?
>
> I though the idea was to put everything in a classic branch and let distros 
> run "classic" hardware from that. What happens after 3 releases does i965 
> still go to the classic branch with the other classic drivers? If so is it 
> really worth waiting just because Ubuntu might have to back-port a bug fix?

Yeah, that was the idea.  However, with crocus in good shape and Emma
Anholt working on i915g, it may be that the actual answer is we just
throw away the classic drivers and the only thing you really need the
old branch for is r200 and ancient nouveau.

--Jason


> --Jason
>
> On April 9, 2021 22:09:14 Dylan Baker  wrote:
>
>> Quoting Dylan Baker (2021-03-22 15:15:30)
>>>
>>> Hi list,
>>>
>>> We've talked about it a number of times, but I think it's time time to
>>> discuss splitting the classic drivers off of the main development branch
>>> again, although this time I have a concrete plan for how this would
>>> work.
>>>
>>> First, why? Basically, all of the classic drivers are in maintanence
>>> mode (even i965). Second, many of them rely on code that no one works
>>> on, and very few people still understand. There is no CI for most of
>>> them, and the Intel CI is not integrated with gitlab, so it's easy to
>>> unintentionally break them, and this breakage usually isn't noticed
>>> until just before or just after a release. 21.0 was held up (in small
>>> part, also me just getting behind) because of such breakages.
>>>
>>> I konw there is some interest in getting i915g in good enough shape that
>>> it could replace i915c, at least for the common case. I also am aware
>>> that Dave, Ilia, and Eric (with some pointers from Ken) have been
>>> working on a gallium driver to replace i965. Neither of those things are
>>> ready yet, but I've taken them into account.
>>>
>>> Here's the plan:
>>>
>>> 1) 21.1 release happens
>>> 2) we remove classic from master
>>> 3) 21.1 reaches EOL because of 21.2
>>> 4) we fork the 21.1 branch into a "classic-lts"¹ branch
>>> 5) we disable all vulkan and gallium drivers in said branch, at least at
>>> the Meson level
>>> 6) We change the name and precidence of the glvnd loader file
>>> 7) apply any build fixups (turn of intel generators for versions >= 7.5,
>>> for example
>>> 8) maintain that branch with build and critical bug fixes only
>>>
>>> This gives ditros and end users two options.
>>> 1) then can build *only* the legacy branch in the a normal Mesa provides
>>> libGL interfaces fashion
>>> 2) They can use glvnd and install current mesa and the legacy branch in
>>> parallel
>>>
>>> Because of glvnd, we can control which driver will get loaded first, and
>>> thus if we decide i915g or the i965 replacement is ready and turn it on
>>> by default it will be loaded by default. An end user who doesn't like
>>> this can add a new glvnd lo

Re: [Mesa-dev] [RFC] Concrete proposal to split classic

[Jason Ekstrand]

I'm bringing this up via e-mail so it gets a wider audience. Given how will 
crocus is working at this point, is like to propose we hold off for about 
three more releases before we drop classic. This next release, 21.2, we'll 
have crocus as an option with i965 as the default. There will also be a 
-Dprefer-crocus meson options so distros or individuals can attempt to flip 
it on. The release after that, 21.3, we'll keep i965 in the tree but have 
crocus be the default (assuming things are going well.) Some time in 2022, 
probably after the 22.2 release or so, we'll delete classic.


Why wait so long? Well, it just landed and we don't have a Cherryview story 
yet so I'm hesitant to make it the default too quickly. Even if it were the 
default in 21.2, it's already too late, likely, to hit the fall 2021 distro 
release cycle. If we flip it to the default before the end of the year, 
that'll get crocus into spring distros. This is good because 22.04 is an 
Ubuntu LTS release and I think they'd rather bump crocus versions to fix 
bugs than backport on top of i965. But that's really fort Ubuntu to decide. 
In any case, we won't see broad-spread usage and the flood of bug reports 
until next spring so we may want to wait until then to stay deleting code.


If we wanted to accelerate things, one option, once we're ready, would be 
to ask the person who manages the oibaf PPA to switch to crocus early. That 
may get some early adopters on board.


Thoughts?

--Jason

On April 9, 2021 22:09:14 Dylan Baker  wrote:


Quoting Dylan Baker (2021-03-22 15:15:30)

Hi list,

We've talked about it a number of times, but I think it's time time to
discuss splitting the classic drivers off of the main development branch
again, although this time I have a concrete plan for how this would
work.

First, why? Basically, all of the classic drivers are in maintanence
mode (even i965). Second, many of them rely on code that no one works
on, and very few people still understand. There is no CI for most of
them, and the Intel CI is not integrated with gitlab, so it's easy to
unintentionally break them, and this breakage usually isn't noticed
until just before or just after a release. 21.0 was held up (in small
part, also me just getting behind) because of such breakages.

I konw there is some interest in getting i915g in good enough shape that
it could replace i915c, at least for the common case. I also am aware
that Dave, Ilia, and Eric (with some pointers from Ken) have been
working on a gallium driver to replace i965. Neither of those things are
ready yet, but I've taken them into account.

Here's the plan:

1) 21.1 release happens
2) we remove classic from master
3) 21.1 reaches EOL because of 21.2
4) we fork the 21.1 branch into a "classic-lts"¹ branch
5) we disable all vulkan and gallium drivers in said branch, at least at
   the Meson level
6) We change the name and precidence of the glvnd loader file
7) apply any build fixups (turn of intel generators for versions >= 7.5,
   for example
8) maintain that branch with build and critical bug fixes only

This gives ditros and end users two options.
1) then can build *only* the legacy branch in the a normal Mesa provides
   libGL interfaces fashion
2) They can use glvnd and install current mesa and the legacy branch in
   parallel

Because of glvnd, we can control which driver will get loaded first, and
thus if we decide i915g or the i965 replacement is ready and turn it on
by default it will be loaded by default. An end user who doesn't like
this can add a new glvnd loader file that makes the classic drivers
higher precident and continue to use them.

Why fork from 21.1 instead of master?

First, it allows us to delete classic immediately, which will allow
refactoring to happen earlier in the cycle, and for any fallout to be
caught and hopefully fixed before the release. Second, it means that
when a user is switched from 21.1 to the new classic-lts branch, there
will be no regressions, and no one has to spend time figuring out what
broke and fixing the lts branch.

When you say "build and critical bug fixes", what do you mean?

I mean update Meson if we rely on something that in the future is
deprecated and removed, and would prevent building the branch or an
relying on some compiler behavior that changes, gaping exploitable
security holes, that kind of thing.

footnotes
¹Or whatever color you like your bikeshed


Here is a merge request to remove classic:
https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10153

Dylan


___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

[Mesa-dev] Deleting Android.mk

[Jason Ekstrand]

All,

I've had a MR open to delete Android.mk in its entirety for a while
[1].  This was initially motivated by Google's success doing
meson-based NDK builds of Mesa for the Android container in ChromeOS.
When I posted !9728, a number of developers raised their hands saying
they were still using in-Android-tree builds and they didn't like the
NDK build option for various reasons.  After a bunch of very good work
by Roman Stratiienko and others, !10183 [2] was produced which allows
the normal Android build system to invoke meson and use the generated
Ninja file to build.  Now that we have mechanisms in place to build on
Android with meson both via the NDK and in-Android-tree, I'd like to
go ahead with the removal of Android.mk.

To that end, I'm asking for ACKs or NAKs on !9728.  I'd really
appreciate ACKs from all the major Android stake-holders.  I intend to
merge it this coming Friday (June 18th) unless we get a good reason to
do otherwise.  If you've got a good reason (it'd better be good enough
to justify 12k LOC), please raise it by replying to this e-mail or
starting a thread on !9728.

Thanks,

--Jason Ekstrand


 [1]: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/9728
 [2]: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/10183
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

[Mesa-dev] [PATCH 0/6] dma-buf: Add an API for exporting sync files (v12)

[Jason Ekstrand]

Modern userspace APIs like Vulkan are built on an explicit
synchronization model.  This doesn't always play nicely with the
implicit synchronization used in the kernel and assumed by X11 and
Wayland.  The client -> compositor half of the synchronization isn't too
bad, at least on intel, because we can control whether or not i915
synchronizes on the buffer and whether or not it's considered written.

The harder part is the compositor -> client synchronization when we get
the buffer back from the compositor.  We're required to be able to
provide the client with a VkSemaphore and VkFence representing the point
in time where the window system (compositor and/or display) finished
using the buffer.  With current APIs, it's very hard to do this in such
a way that we don't get confused by the Vulkan driver's access of the
buffer.  In particular, once we tell the kernel that we're rendering to
the buffer again, any CPU waits on the buffer or GPU dependencies will
wait on some of the client rendering and not just the compositor.

This new IOCTL solves this problem by allowing us to get a snapshot of
the implicit synchronization state of a given dma-buf in the form of a
sync file.  It's effectively the same as a poll() or I915_GEM_WAIT only,
instead of CPU waiting directly, it encapsulates the wait operation, at
the current moment in time, in a sync_file so we can check/wait on it
later.  As long as the Vulkan driver does the sync_file export from the
dma-buf before we re-introduce it for rendering, it will only contain
fences from the compositor or display.  This allows to accurately turn
it into a VkFence or VkSemaphore without any over- synchronization.

This patch series actually contains two new ioctls.  There is the export
one mentioned above as well as an RFC for an import ioctl which provides
the other half.  The intention is to land the export ioctl since it seems
like there's no real disagreement on that one.  The import ioctl, however,
has a lot of debate around it so it's intended to be RFC-only for now.

Mesa MR: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4037
IGT tests: https://patchwork.freedesktop.org/series/90490/

v10 (Jason Ekstrand, Daniel Vetter):
 - Add reviews/acks
 - Add a patch to rename _rcu to _unlocked
 - Split things better so import is clearly RFC status

v11 (Daniel Vetter):
 - Add more CCs to try and get maintainers
 - Add a patch to document DMA_BUF_IOCTL_SYNC
 - Generally better docs
 - Use separate structs for import/export (easier to document)
 - Fix an issue in the import patch

v12 (Daniel Vetter):
 - Better docs for DMA_BUF_IOCTL_SYNC

v12 (Christian König):
 - Drop the rename patch in favor of Christian's series
 - Add a comment to the commit message for the dma-buf sync_file export
   ioctl saying why we made it an ioctl on dma-buf

Cc: Christian König 
Cc: Michel Dänzer 
Cc: Dave Airlie 
Cc: Bas Nieuwenhuizen 
Cc: Daniel Stone 
Cc: mesa-dev@lists.freedesktop.org
Cc: wayland-de...@lists.freedesktop.org
Test-with: 20210524205225.872316-1-ja...@jlekstrand.net

Christian König (1):
  dma-buf: Add dma_fence_array_for_each (v2)

Jason Ekstrand (5):
  dma-buf: Add dma_resv_get_singleton (v6)
  dma-buf: Document DMA_BUF_IOCTL_SYNC (v2)
  dma-buf: Add an API for exporting sync files (v12)
  RFC: dma-buf: Add an extra fence to dma_resv_get_singleton_unlocked
  RFC: dma-buf: Add an API for importing sync files (v7)

 Documentation/driver-api/dma-buf.rst |   8 ++
 drivers/dma-buf/dma-buf.c| 103 +
 drivers/dma-buf/dma-fence-array.c|  27 +++
 drivers/dma-buf/dma-resv.c   | 110 +++
 include/linux/dma-fence-array.h  |  17 +
 include/linux/dma-resv.h |   2 +
 include/uapi/linux/dma-buf.h | 103 -
 7 files changed, 369 insertions(+), 1 deletion(-)

-- 
2.31.1

___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] [PATCH 0/7] dma-buf: Add an API for exporting sync files (v11)

[Jason Ekstrand]

On Thu, Jun 10, 2021 at 3:11 PM Chia-I Wu  wrote:
>
> On Tue, May 25, 2021 at 2:18 PM Jason Ekstrand  wrote:
> > Modern userspace APIs like Vulkan are built on an explicit
> > synchronization model.  This doesn't always play nicely with the
> > implicit synchronization used in the kernel and assumed by X11 and
> > Wayland.  The client -> compositor half of the synchronization isn't too
> > bad, at least on intel, because we can control whether or not i915
> > synchronizes on the buffer and whether or not it's considered written.
> We might have an important use case for this half, for virtio-gpu and Chrome 
> OS.
>
> When the guest compositor acts as a proxy to connect guest apps to the
> host compositor, implicit fencing requires the guest compositor to do
> a wait before forwarding the buffer to the host compositor.  With this
> patch, the guest compositor can extract the dma-fence from the buffer,
> and if the fence is a virtio-gpu fence, forward both the fence and the
> buffer to the host compositor.  It will allow us to convert a
> guest-side wait into a host-side wait.

Yeah, I think the first half solves a lot of problems.  I'm rebasing
it now and will send a v12 series shortly.  I don't think there's a
lot standing between the first few patches and merging.  I've got IGT
tests and I'm pretty sure the code is good.  The last review cycle got
distracted with some renaming fun.

--Jason
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Intel: ABI for DRM format modifiers with multi-planar images

[Jason Ekstrand]

I should have said that the minimal support can be for LINEAR, X-tiled
and Y-tiled.  CCS can and probably should come later.

On Wed, Jun 9, 2021 at 6:14 PM Yiwei Zhang  wrote:
>
> On Wed, Jun 9, 2021 at 2:19 PM Jason Ekstrand  wrote:
> >
> > +Nanley
> >
> > We've not defined those yet.  We had some internal talks a couple
> > years ago.  The rough idea we had at the time was to define a modifier
> > for those cases which put the CCS after each main surface at some
> > fixed calculation offset based on width, height, and stride.  Then the
> > one modifier would apply independently to the three different planes.
> >
> > --Jason
> >
> > On Wed, Jun 9, 2021 at 2:11 PM Chad Versace  wrote:
> > >
> > > The Problem: For a given 3-tuple (multi-planar format, DRM format 
> > > modifier, chipset), we need Intel ABI that decides (a) the value of 
> > > VkDrmFormatModifierPropertiesEXT::drmFormatModifierPlaneCount and (b) the 
> > > content of each "modifier" plane.
> > >
> > > For example, suppose drmFormatModifierPlaneCount is 2 for 
> > > (VK_FORMAT_G8_B8R8_2PLANE_420_UNORM, INTEL_FORMAT_MOD_FOO). Is the layout 
> > > (plane1=g8, plane2=b8r8); or is it (plane1=g8_b8r8, plane2=aux)? The 
> > > second choice isn't crazy; iirc, some non-Intel hardware (sorry, NDA) 
> > > uses only 2 modifier planes for color-compressed 3-planar formats, with 
> > > (plane1=r8_g8_b8, plane2=aux).
> > >
> > > I'm asking because Yiwei (cc'd) has begun implementing limited support 
> > > for multi-planar formats in Anvil in order to pass the Android CTS. To 
> > > support more media formats (for imminent Chrome OS projects and future 
> > > vanilla Linux stuff too), we need to decide on some ABI.
>
> Hi,
>
> I have sent a 
> MR(https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/11281)
> to add the minimal multi-planar format support with drm format
> modifier, so that Venus Android AHB extension layered on top of ANV
> VK_EXT_image_drm_format_modifier implementation can pass all the
> interop graphics cts for camera and media.
>
> I'd be interested to follow up about the stable ABI for this to expand
> multi-planar support there.
>
> Best,
> Yiwei
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Intel: ABI for DRM format modifiers with multi-planar images

[Jason Ekstrand]

+Nanley

We've not defined those yet.  We had some internal talks a couple
years ago.  The rough idea we had at the time was to define a modifier
for those cases which put the CCS after each main surface at some
fixed calculation offset based on width, height, and stride.  Then the
one modifier would apply independently to the three different planes.

--Jason

On Wed, Jun 9, 2021 at 2:11 PM Chad Versace  wrote:
>
> The Problem: For a given 3-tuple (multi-planar format, DRM format modifier, 
> chipset), we need Intel ABI that decides (a) the value of 
> VkDrmFormatModifierPropertiesEXT::drmFormatModifierPlaneCount and (b) the 
> content of each "modifier" plane.
>
> For example, suppose drmFormatModifierPlaneCount is 2 for 
> (VK_FORMAT_G8_B8R8_2PLANE_420_UNORM, INTEL_FORMAT_MOD_FOO). Is the layout 
> (plane1=g8, plane2=b8r8); or is it (plane1=g8_b8r8, plane2=aux)? The second 
> choice isn't crazy; iirc, some non-Intel hardware (sorry, NDA) uses only 2 
> modifier planes for color-compressed 3-planar formats, with (plane1=r8_g8_b8, 
> plane2=aux).
>
> I'm asking because Yiwei (cc'd) has begun implementing limited support for 
> multi-planar formats in Anvil in order to pass the Android CTS. To support 
> more media formats (for imminent Chrome OS projects and future vanilla Linux 
> stuff too), we need to decide on some ABI.
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

[Mesa-dev] [PATCH 0/7] dma-buf: Add an API for exporting sync files (v11)

[Jason Ekstrand]

Modern userspace APIs like Vulkan are built on an explicit
synchronization model.  This doesn't always play nicely with the
implicit synchronization used in the kernel and assumed by X11 and
Wayland.  The client -> compositor half of the synchronization isn't too
bad, at least on intel, because we can control whether or not i915
synchronizes on the buffer and whether or not it's considered written.

The harder part is the compositor -> client synchronization when we get
the buffer back from the compositor.  We're required to be able to
provide the client with a VkSemaphore and VkFence representing the point
in time where the window system (compositor and/or display) finished
using the buffer.  With current APIs, it's very hard to do this in such
a way that we don't get confused by the Vulkan driver's access of the
buffer.  In particular, once we tell the kernel that we're rendering to
the buffer again, any CPU waits on the buffer or GPU dependencies will
wait on some of the client rendering and not just the compositor.

This new IOCTL solves this problem by allowing us to get a snapshot of
the implicit synchronization state of a given dma-buf in the form of a
sync file.  It's effectively the same as a poll() or I915_GEM_WAIT only,
instead of CPU waiting directly, it encapsulates the wait operation, at
the current moment in time, in a sync_file so we can check/wait on it
later.  As long as the Vulkan driver does the sync_file export from the
dma-buf before we re-introduce it for rendering, it will only contain
fences from the compositor or display.  This allows to accurately turn
it into a VkFence or VkSemaphore without any over- synchronization.

This patch series actually contains two new ioctls.  There is the export
one mentioned above as well as an RFC for an import ioctl which provides
the other half.  The intention is to land the export ioctl since it seems
like there's no real disagreement on that one.  The import ioctl, however,
has a lot of debate around it so it's intended to be RFC-only for now.

Mesa MR: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4037
IGT tests: https://patchwork.freedesktop.org/series/90490/

v10 (Jason Ekstrand, Daniel Vetter):
 - Add reviews/acks
 - Add a patch to rename _rcu to _unlocked
 - Split things better so import is clearly RFC status

v11 (Daniel Vetter):
 - Add more CCs to try and get maintainers
 - Add a patch to document DMA_BUF_IOCTL_SYNC
 - Generally better docs
 - Use separate structs for import/export (easier to document)
 - Fix an issue in the import patch

Cc: Christian König 
Cc: Michel Dänzer 
Cc: Dave Airlie 
Cc: Bas Nieuwenhuizen 
Cc: Daniel Stone 
Cc: mesa-dev@lists.freedesktop.org
Cc: wayland-de...@lists.freedesktop.org
Test-with: 20210524205225.872316-1-ja...@jlekstrand.net

Christian König (1):
  dma-buf: Add dma_fence_array_for_each (v2)

Jason Ekstrand (6):
  dma-buf: Rename dma_resv helpers from _rcu to _unlocked (v2)
  dma-buf: Add dma_resv_get_singleton_unlocked (v5)
  dma-buf: Document DMA_BUF_IOCTL_SYNC
  dma-buf: Add an API for exporting sync files (v11)
  RFC: dma-buf: Add an extra fence to dma_resv_get_singleton_unlocked
  RFC: dma-buf: Add an API for importing sync files (v7)

 Documentation/driver-api/dma-buf.rst  |   8 +
 drivers/dma-buf/dma-buf.c | 107 +-
 drivers/dma-buf/dma-fence-array.c |  27 
 drivers/dma-buf/dma-resv.c| 139 --
 drivers/gpu/drm/amd/amdgpu/amdgpu_display.c   |   6 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c   |   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c   |   4 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c   |   6 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c|   4 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c|   4 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c   |   6 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c|  14 +-
 .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c |   6 +-
 drivers/gpu/drm/drm_gem.c |  10 +-
 drivers/gpu/drm/drm_gem_atomic_helper.c   |   2 +-
 drivers/gpu/drm/etnaviv/etnaviv_gem.c |   7 +-
 drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c  |   8 +-
 drivers/gpu/drm/i915/display/intel_display.c  |   2 +-
 drivers/gpu/drm/i915/dma_resv_utils.c |   2 +-
 drivers/gpu/drm/i915/gem/i915_gem_busy.c  |   2 +-
 .../gpu/drm/i915/gem/i915_gem_execbuffer.c|   2 +-
 drivers/gpu/drm/i915/gem/i915_gem_object.h|   2 +-
 drivers/gpu/drm/i915/gem/i915_gem_userptr.c   |   4 +-
 drivers/gpu/drm/i915/gem/i915_gem_wait.c  |  10 +-
 drivers/gpu/drm/i915/i915_request.c   |   6 +-
 drivers/gpu/drm/i915/i915_sw_fence.c  |   4 +-
 drivers/gpu/drm/msm/msm_gem.c |   3 +-
 drivers/gpu/drm/nouveau/dispnv50/wndw.c   |   2 +-
 drivers/gpu/drm/nouveau/nouveau_gem.c |   4 +-
 drivers/gpu/drm/panfrost/panfrost_drv.c   |   4 +-
 drivers/gpu/drm/panfrost/panfrost_job.c   |   2 +-
 drivers/g

[Mesa-dev] [PATCH 0/6] dma-buf: Add an API for exporting sync files (v10)

[Jason Ekstrand]

Modern userspace APIs like Vulkan are built on an explicit
synchronization model.  This doesn't always play nicely with the
implicit synchronization used in the kernel and assumed by X11 and
Wayland.  The client -> compositor half of the synchronization isn't too
bad, at least on intel, because we can control whether or not i915
synchronizes on the buffer and whether or not it's considered written.

The harder part is the compositor -> client synchronization when we get
the buffer back from the compositor.  We're required to be able to
provide the client with a VkSemaphore and VkFence representing the point
in time where the window system (compositor and/or display) finished
using the buffer.  With current APIs, it's very hard to do this in such
a way that we don't get confused by the Vulkan driver's access of the
buffer.  In particular, once we tell the kernel that we're rendering to
the buffer again, any CPU waits on the buffer or GPU dependencies will
wait on some of the client rendering and not just the compositor.

This new IOCTL solves this problem by allowing us to get a snapshot of
the implicit synchronization state of a given dma-buf in the form of a
sync file.  It's effectively the same as a poll() or I915_GEM_WAIT only,
instead of CPU waiting directly, it encapsulates the wait operation, at
the current moment in time, in a sync_file so we can check/wait on it
later.  As long as the Vulkan driver does the sync_file export from the
dma-buf before we re-introduce it for rendering, it will only contain
fences from the compositor or display.  This allows to accurately turn
it into a VkFence or VkSemaphore without any over- synchronization.

This patch series actually contains two new ioctls.  There is the export
one mentioned above as well as an RFC for an import ioctl which provides
the other half.  The intention is to land the export ioctl since it seems
like there's no real disagreement on that one.  The import ioctl, however,
has a lot of debate around it so it's intended to be RFC-only for now.

Mesa MR: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4037
IGT tests: https://patchwork.freedesktop.org/series/90490/

v10 (Jason Ekstrand, Daniel Vetter):
 - Add reviews/acks
 - Add a patch to rename _rcu to _unlocked
 - Split things better so import is clearly RFC status

Cc: Christian König 
Cc: Michel Dänzer 
Cc: Dave Airlie 
Cc: Bas Nieuwenhuizen 
Cc: Daniel Stone 
Cc: mesa-dev@lists.freedesktop.org
Cc: wayland-de...@lists.freedesktop.org
Test-with: 20210524205225.872316-1-ja...@jlekstrand.net

Christian König (1):
  dma-buf: add dma_fence_array_for_each (v2)

Jason Ekstrand (5):
  dma-buf: Rename dma_resv helpers from _rcu to _unlocked
  dma-buf: add dma_resv_get_singleton_unlocked (v4)
  dma-buf: Add an API for exporting sync files (v9)
  RFC: dma-buf: Add an extra fence to dma_resv_get_singleton_unlocked
  RFC: dma-buf: Add an API for importing sync files (v6)

 drivers/dma-buf/dma-buf.c | 100 -
 drivers/dma-buf/dma-fence-array.c |  27 
 drivers/dma-buf/dma-resv.c| 140 --
 drivers/gpu/drm/amd/amdgpu/amdgpu_display.c   |   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c   |   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c   |   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c   |   4 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_mn.c|   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c|   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c   |   2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c|   8 +-
 .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c |   2 +-
 drivers/gpu/drm/drm_gem.c |   6 +-
 drivers/gpu/drm/drm_gem_atomic_helper.c   |   2 +-
 drivers/gpu/drm/etnaviv/etnaviv_gem.c |   4 +-
 drivers/gpu/drm/etnaviv/etnaviv_gem_submit.c  |   4 +-
 drivers/gpu/drm/i915/display/intel_display.c  |   2 +-
 drivers/gpu/drm/i915/dma_resv_utils.c |   2 +-
 drivers/gpu/drm/i915/gem/i915_gem_busy.c  |   2 +-
 .../gpu/drm/i915/gem/i915_gem_execbuffer.c|   2 +-
 drivers/gpu/drm/i915/gem/i915_gem_object.h|   2 +-
 drivers/gpu/drm/i915/gem/i915_gem_userptr.c   |   2 +-
 drivers/gpu/drm/i915/gem/i915_gem_wait.c  |  10 +-
 drivers/gpu/drm/i915/i915_request.c   |   4 +-
 drivers/gpu/drm/i915/i915_sw_fence.c  |   4 +-
 drivers/gpu/drm/msm/msm_gem.c |   2 +-
 drivers/gpu/drm/nouveau/dispnv50/wndw.c   |   2 +-
 drivers/gpu/drm/nouveau/nouveau_gem.c |   2 +-
 drivers/gpu/drm/panfrost/panfrost_drv.c   |   2 +-
 drivers/gpu/drm/panfrost/panfrost_job.c   |   2 +-
 drivers/gpu/drm/radeon/radeon_gem.c   |   6 +-
 drivers/gpu/drm/radeon/radeon_mn.c|   2 +-
 drivers/gpu/drm/ttm/ttm_bo.c  |  12 +-
 drivers/gpu/drm/vgem/vgem_fence.c |   2 +-
 drivers/gpu/drm/virtio/virtgpu_ioctl.c|   4 +-
 drivers/gpu/drm/vmwgfx/vmwgfx_bo.c|   2 +-
 include

Re: [Mesa-dev] [Intel-gfx] [RFC 2/2] drm/doc/rfc: i915 new parallel submission uAPI plan

[Jason Ekstrand]

On Thu, May 20, 2021 at 10:46 AM Matthew Brost  wrote:
>
> On Thu, May 20, 2021 at 01:11:59PM +0200, Christian König wrote:
> > Am 19.05.21 um 18:51 schrieb Matthew Brost:
> > > On Wed, May 19, 2021 at 01:45:39PM +0200, Christian König wrote:
> > > > Oh, yeah we call that gang submit on the AMD side.
> > > >
> > > > Had already some internal discussions how to implement this, but so far
> > > > couldn't figure out how to cleanly introduce that into the DRM 
> > > > scheduler.
> > > >
> > > > Can you briefly describe in a few words how that is supposed to work on 
> > > > the
> > > > Intel side?

On Intel, we actually have two cases which don't fit the current
drm/scheduler model well: balanced and bonded.

In the balanced model, we want to submit a batch which can go to any
one of some set of engines and we don't care which.  It's up to the
kernel to pick an engine.  Imagine you had 64 identical HW compute
queues, for instance.  This could be done by making all the identical
engines share a single drm_gpu_scheduler and round-robin around the HW
queues or something.  I don't know that we strictly need drm/scheduler
to be aware of it but it might be nice if it grew support for this
mode so we could maintain a 1:1 relationship between HW queues and
drm_gpu_schedulers.  That said, I'm not sure how this would play with
GuC queues so maybe it doesn't help?

The bonded model is like your ganged, I think.  We want to submit N
batches to run in parallel.  And they actually have to be executing on
the GPU simultaneously and not just sort-of at similar times.  We need
this for video.  There are also potential use-cases in Vulkan or even
GL that might be able to use this.  One difference with the balanced
mode is that bonds don't, strictly speaking, need to be on the same
type of engine.  Imagine, for instance, a 3D batch with a parallel
compute batch doing vertex pre-processing.

I'm pretty sure the bonded case is something that the mobile drivers
(panfrost, etc.) would like as well for doing Vulkan on tilers where
you often have to have two command buffers running in parallel.
They're currently doing it by submitting a giant pile of batches where
they split the batch and add sync primitives every time some GL call
requires them to sync between fragment and vertex pipes.

So, to sum up, I think there's likely some good collaboration to be
had here for everyone. :-)

--Jason

> > > Sure, I've done a quick PoC internally and have been able to hook this
> > > into the DRM scheduler.
> > >
> > > Basically each BB still maps to a single job as each job is somewhat
> > > unique (e.g. each job has its own ring, lrc, seqno, etc...). However all
> > > the jobs configured to run in parallel map to a single sched_entity
> > > which maintains the order each job was generated from the execbuf IOCTL
> > > (1 - N). When the backend receives jobs 1 to N - 1 it basically just
> > > updates some internal state. When the backend sees job N (last job) it
> > > actually does the submit for jobs 1 - N which with GuC submission is a
> > > simple command moving the LRC tail of the N jobs.
> > >
> > > Daniel has suggested that we create a single job for the NN BBs but that
> > > would be huge rework to the internals of the i915 and likely won't
> > > happen by the time this code first lands.
> > >
> > > Also worth noting one way a job isn't really a treated individually is
> > > the excl slot with dma-resv. In that case we create a composite fence of
> > > all jobs (dma_fence_array).
> >
> > Yeah, that's something we have discussed as well.
> >
> > How do you prevent the scheduler from over committing to a single ring
> > buffer in this scenario?
> >
>
> Each job has its own ring, the execbuf IOCTL throttles itself for each
> job if there isn't space in the ring. This is exactly the same as
> non-parallel submits.
>
> I think this is what you were asking? If not, maybe try explaining the
> question a bit more.
>
> Matt
>
> > Christian.
> >
> > >
> > > Matt
> > >
> > > > Thanks,
> > > > Christian.
> > > >
> > > > Am 19.05.21 um 01:58 schrieb Matthew Brost:
> > > > > Add entry fpr i915 new parallel submission uAPI plan.
> > > > >
> > > > > v2:
> > > > >(Daniel Vetter):
> > > > > - Expand logical order explaination
> > > > > - Add dummy header
> > > > > - Only allow N BBs in execbuf IOCTL
> > > > > - Configure parallel submission per slot not per gem 

Re: [Mesa-dev] [RFC] Linux Graphics Next: Explicit fences everywhere and no BO fences - initial proposal

[Jason Ekstrand]

On Tue, May 4, 2021 at 12:16 PM Marek Olšák  wrote:
>
> I see some mentions of XNACK and recoverable page faults. Note that all 
> gaming AMD hw that has userspace queues doesn't have XNACK, so there is no 
> overhead in compute units. My understanding is that recoverable page faults 
> are still supported without XNACK, but instead of the compute unit replaying 
> the faulting instruction, the L1 cache does that. Anyway, the point is that 
> XNACK is totally irrelevant here.
>
> Marek
>
> On Tue., May 4, 2021, 08:48 Christian König, 
>  wrote:
>>
>> Am 04.05.21 um 13:13 schrieb Daniel Vetter:
>> > On Tue, May 4, 2021 at 12:53 PM Christian König
>> >  wrote:
>> >> Am 04.05.21 um 11:47 schrieb Daniel Vetter:
>> >>> [SNIP]
>>  Yeah, it just takes to long for the preemption to complete to be really
>>  useful for the feature we are discussing here.
>> 
>>  As I said when the kernel requests to preempt a queue we can easily 
>>  expect a
>>  timeout of ~100ms until that comes back. For compute that is even in the
>>  multiple seconds range.
>> >>> 100ms for preempting an idle request sounds like broken hw to me. Of
>> >>> course preemting something that actually runs takes a while, that's
>> >>> nothing new. But it's also not the thing we're talking about here. Is 
>> >>> this
>> >>> 100ms actual numbers from hw for an actual idle ringbuffer?
>> >> Well 100ms is just an example of the scheduler granularity. Let me
>> >> explain in a wider context.
>> >>
>> >> The hardware can have X queues mapped at the same time and every Y time
>> >> interval the hardware scheduler checks if those queues have changed and
>> >> only if they have changed the necessary steps to reload them are started.
>> >>
>> >> Multiple queues can be rendering at the same time, so you can have X as
>> >> a high priority queue active and just waiting for a signal to start and
>> >> the client rendering one frame after another and a third background
>> >> compute task mining bitcoins for you.
>> >>
>> >> As long as everything is static this is perfectly performant. Adding a
>> >> queue to the list of active queues is also relatively simple, but taking
>> >> one down requires you to wait until we are sure the hardware has seen
>> >> the change and reloaded the queues.
>> >>
>> >> Think of it as an RCU grace period. This is simply not something which
>> >> is made to be used constantly, but rather just at process termination.
>> > Uh ... that indeed sounds rather broken.
>>
>> Well I wouldn't call it broken. It's just not made for the use case we
>> are trying to abuse it for.
>>
>> > Otoh it's just a dma_fence that'd we'd inject as this unload-fence.
>>
>> Yeah, exactly that's why it isn't much of a problem for process
>> termination or freeing memory.
>>
>> > So by and large everyone should already be able to cope with it taking a
>> > bit longer. So from a design pov I don't see a huge problem, but I
>> > guess you guys wont be happy since it means on amd hw there will be
>> > random unsightly stalls in desktop linux usage.
>> >
>>  The "preemption" feature is really called suspend and made just for the 
>>  case
>>  when we want to put a process to sleep or need to forcefully kill it for
>>  misbehavior or stuff like that. It is not meant to be used in normal
>>  operation.
>> 
>>  If we only attach it on ->move then yeah maybe a last resort 
>>  possibility to
>>  do it this way, but I think in that case we could rather stick with 
>>  kernel
>>  submissions.
>> >>> Well this is a hybrid userspace ring + kernel augmeted submit mode, so 
>> >>> you
>> >>> can keep dma-fences working. Because the dma-fence stuff wont work with
>> >>> pure userspace submit, I think that conclusion is rather solid. Once more
>> >>> even after this long thread here.
>> >> When assisted with unload fences, then yes. Problem is that I can't see
>> >> how we could implement those performant currently.
>> > Is there really no way to fix fw here? Like if process start/teardown
>> > takes 100ms, that's going to suck no matter what.
>>
>> As I said adding the queue is unproblematic and teardown just results in
>> a bit more waiting to free things up.
>>
>> Problematic is more overcommit swapping and OOM situations which need to
>> wait for the hw scheduler to come back and tell us that the queue is now
>> unmapped.
>>
>> > Also, if userspace lies to us and keeps pushing crap into the ring
>> > after it's supposed to be idle: Userspace is already allowed to waste
>> > gpu time. If you're too worried about this set a fairly aggressive
>> > preempt timeout on the unload fence, and kill the context if it takes
>> > longer than what preempting an idle ring should take (because that
>> > would indicate broken/evil userspace).
>>  I think you have the wrong expectation here. It is perfectly valid and
>>  expected for userspace to keep writing commands into the ring buffer.

Re: [Mesa-dev] [RFC] Linux Graphics Next: Explicit fences everywhere and no BO fences - initial proposal

[Jason Ekstrand]

On Mon, May 3, 2021 at 10:16 AM Bas Nieuwenhuizen
 wrote:
>
> On Mon, May 3, 2021 at 5:00 PM Jason Ekstrand  wrote:
> >
> > Sorry for the top-post but there's no good thing to reply to here...
> >
> > One of the things pointed out to me recently by Daniel Vetter that I
> > didn't fully understand before is that dma_buf has a very subtle
> > second requirement beyond finite time completion:  Nothing required
> > for signaling a dma-fence can allocate memory.  Why?  Because the act
> > of allocating memory may wait on your dma-fence.  This, as it turns
> > out, is a massively more strict requirement than finite time
> > completion and, I think, throws out all of the proposals we have so
> > far.
> >
> > Take, for instance, Marek's proposal for userspace involvement with
> > dma-fence by asking the kernel for a next serial and the kernel
> > trusting userspace to signal it.  That doesn't work at all if
> > allocating memory to trigger a dma-fence can blow up.  There's simply
> > no way for the kernel to trust userspace to not do ANYTHING which
> > might allocate memory.  I don't even think there's a way userspace can
> > trust itself there.  It also blows up my plan of moving the fences to
> > transition boundaries.
> >
> > Not sure where that leaves us.
>
> Honestly the more I look at things I think userspace-signalable fences
> with a timeout sound like they are a valid solution for these issues.
> Especially since (as has been mentioned countless times in this email
> thread) userspace already has a lot of ways to cause timeouts and or
> GPU hangs through GPU work already.
>
> Adding a timeout on the signaling side of a dma_fence would ensure:
>
> - The dma_fence signals in finite time
> -  If the timeout case does not allocate memory then memory allocation
> is not a blocker for signaling.
>
> Of course you lose the full dependency graph and we need to make sure
> garbage collection of fences works correctly when we have cycles.
> However, the latter sounds very doable and the first sounds like it is
> to some extent inevitable.
>
> I feel like I'm missing some requirement here given that we
> immediately went to much more complicated things but can't find it.
> Thoughts?

Timeouts are sufficient to protect the kernel but they make the fences
unpredictable and unreliable from a userspace PoV.  One of the big
problems we face is that, once we expose a dma_fence to userspace,
we've allowed for some pretty crazy potential dependencies that
neither userspace nor the kernel can sort out.  Say you have marek's
"next serial, please" proposal and a multi-threaded application.
Between time time you ask the kernel for a serial and get a dma_fence
and submit the work to signal that serial, your process may get
preempted, something else shoved in which allocates memory, and then
we end up blocking on that dma_fence.  There's no way userspace can
predict and defend itself from that.

So I think where that leaves us is that there is no safe place to
create a dma_fence except for inside the ioctl which submits the work
and only after any necessary memory has been allocated.  That's a
pretty stiff requirement.  We may still be able to interact with
userspace a bit more explicitly but I think it throws any notion of
userspace direct submit out the window.

--Jason


> - Bas
> >
> > --Jason
> >
> > On Mon, May 3, 2021 at 9:42 AM Alex Deucher  wrote:
> > >
> > > On Sat, May 1, 2021 at 6:27 PM Marek Olšák  wrote:
> > > >
> > > > On Wed, Apr 28, 2021 at 5:07 AM Michel Dänzer  
> > > > wrote:
> > > >>
> > > >> On 2021-04-28 8:59 a.m., Christian König wrote:
> > > >> > Hi Dave,
> > > >> >
> > > >> > Am 27.04.21 um 21:23 schrieb Marek Olšák:
> > > >> >> Supporting interop with any device is always possible. It depends 
> > > >> >> on which drivers we need to interoperate with and update them. 
> > > >> >> We've already found the path forward for amdgpu. We just need to 
> > > >> >> find out how many other drivers need to be updated and evaluate the 
> > > >> >> cost/benefit aspect.
> > > >> >>
> > > >> >> Marek
> > > >> >>
> > > >> >> On Tue, Apr 27, 2021 at 2:38 PM Dave Airlie  > > >> >> <mailto:airl...@gmail.com>> wrote:
> > > >> >>
> > > >> >> On Tue, 27 Apr 2021 at 22:06, Christian König
> > > >> >>  > > >> >> <mailto:ckoenig.leichtzumer...@gmail.com&

Re: [Mesa-dev] [RFC] Linux Graphics Next: Explicit fences everywhere and no BO fences - initial proposal

[Jason Ekstrand]

On Mon, May 3, 2021 at 10:03 AM Christian König
 wrote:
>
> Am 03.05.21 um 16:59 schrieb Jason Ekstrand:
> > Sorry for the top-post but there's no good thing to reply to here...
> >
> > One of the things pointed out to me recently by Daniel Vetter that I
> > didn't fully understand before is that dma_buf has a very subtle
> > second requirement beyond finite time completion:  Nothing required
> > for signaling a dma-fence can allocate memory.  Why?  Because the act
> > of allocating memory may wait on your dma-fence.  This, as it turns
> > out, is a massively more strict requirement than finite time
> > completion and, I think, throws out all of the proposals we have so
> > far.
> >
> > Take, for instance, Marek's proposal for userspace involvement with
> > dma-fence by asking the kernel for a next serial and the kernel
> > trusting userspace to signal it.  That doesn't work at all if
> > allocating memory to trigger a dma-fence can blow up.  There's simply
> > no way for the kernel to trust userspace to not do ANYTHING which
> > might allocate memory.  I don't even think there's a way userspace can
> > trust itself there.  It also blows up my plan of moving the fences to
> > transition boundaries.
> >
> > Not sure where that leaves us.
>
> Well at least I was perfectly aware of that :)

I'd have been a bit disappointed if this had been news to you. :-P
However, there are a number of us plebeians on the thread who need
things spelled out sometimes. :-)

> I'm currently experimenting with some sample code which would allow
> implicit sync with user fences.
>
> Not that I'm pushing hard into that directly, but I just want to make
> clear how simple or complex the whole thing would be.

I'd like to see that.  It'd be good to know what our options are.
Honestly, if we can get implicit sync somehow without tying our hands
w.r.t. how fences work in modern drivers, that's the opens a lot of
doors.

--Jason

> Christian.
>
> >
> > --Jason
> >
> > On Mon, May 3, 2021 at 9:42 AM Alex Deucher  wrote:
> >> On Sat, May 1, 2021 at 6:27 PM Marek Olšák  wrote:
> >>> On Wed, Apr 28, 2021 at 5:07 AM Michel Dänzer  wrote:
> >>>> On 2021-04-28 8:59 a.m., Christian König wrote:
> >>>>> Hi Dave,
> >>>>>
> >>>>> Am 27.04.21 um 21:23 schrieb Marek Olšák:
> >>>>>> Supporting interop with any device is always possible. It depends on 
> >>>>>> which drivers we need to interoperate with and update them. We've 
> >>>>>> already found the path forward for amdgpu. We just need to find out 
> >>>>>> how many other drivers need to be updated and evaluate the 
> >>>>>> cost/benefit aspect.
> >>>>>>
> >>>>>> Marek
> >>>>>>
> >>>>>> On Tue, Apr 27, 2021 at 2:38 PM Dave Airlie  >>>>>> <mailto:airl...@gmail.com>> wrote:
> >>>>>>
> >>>>>>  On Tue, 27 Apr 2021 at 22:06, Christian König
> >>>>>>   >>>>>> <mailto:ckoenig.leichtzumer...@gmail.com>> wrote:
> >>>>>>  >
> >>>>>>  > Correct, we wouldn't have synchronization between device with 
> >>>>>> and without user queues any more.
> >>>>>>  >
> >>>>>>  > That could only be a problem for A+I Laptops.
> >>>>>>
> >>>>>>  Since I think you mentioned you'd only be enabling this on newer
> >>>>>>  chipsets, won't it be a problem for A+A where one A is a 
> >>>>>> generation
> >>>>>>  behind the other?
> >>>>>>
> >>>>> Crap, that is a good point as well.
> >>>>>
> >>>>>>  I'm not really liking where this is going btw, seems like a ill
> >>>>>>  thought out concept, if AMD is really going down the road of 
> >>>>>> designing
> >>>>>>  hw that is currently Linux incompatible, you are going to have to
> >>>>>>  accept a big part of the burden in bringing this support in to 
> >>>>>> more
> >>>>>>  than just amd drivers for upcoming generations of gpu.
> >>>>>>
> >>>>> Well we don't really like that either, but we have no other option as 
> >>>>> far as I can see.
> >>>> I don't really understan

Re: [Mesa-dev] [RFC] Linux Graphics Next: Explicit fences everywhere and no BO fences - initial proposal

[Jason Ekstrand]

Sorry for the top-post but there's no good thing to reply to here...

One of the things pointed out to me recently by Daniel Vetter that I
didn't fully understand before is that dma_buf has a very subtle
second requirement beyond finite time completion:  Nothing required
for signaling a dma-fence can allocate memory.  Why?  Because the act
of allocating memory may wait on your dma-fence.  This, as it turns
out, is a massively more strict requirement than finite time
completion and, I think, throws out all of the proposals we have so
far.

Take, for instance, Marek's proposal for userspace involvement with
dma-fence by asking the kernel for a next serial and the kernel
trusting userspace to signal it.  That doesn't work at all if
allocating memory to trigger a dma-fence can blow up.  There's simply
no way for the kernel to trust userspace to not do ANYTHING which
might allocate memory.  I don't even think there's a way userspace can
trust itself there.  It also blows up my plan of moving the fences to
transition boundaries.

Not sure where that leaves us.

--Jason

On Mon, May 3, 2021 at 9:42 AM Alex Deucher  wrote:
>
> On Sat, May 1, 2021 at 6:27 PM Marek Olšák  wrote:
> >
> > On Wed, Apr 28, 2021 at 5:07 AM Michel Dänzer  wrote:
> >>
> >> On 2021-04-28 8:59 a.m., Christian König wrote:
> >> > Hi Dave,
> >> >
> >> > Am 27.04.21 um 21:23 schrieb Marek Olšák:
> >> >> Supporting interop with any device is always possible. It depends on 
> >> >> which drivers we need to interoperate with and update them. We've 
> >> >> already found the path forward for amdgpu. We just need to find out how 
> >> >> many other drivers need to be updated and evaluate the cost/benefit 
> >> >> aspect.
> >> >>
> >> >> Marek
> >> >>
> >> >> On Tue, Apr 27, 2021 at 2:38 PM Dave Airlie  >> >> > wrote:
> >> >>
> >> >> On Tue, 27 Apr 2021 at 22:06, Christian König
> >> >>  >> >> > wrote:
> >> >> >
> >> >> > Correct, we wouldn't have synchronization between device with and 
> >> >> without user queues any more.
> >> >> >
> >> >> > That could only be a problem for A+I Laptops.
> >> >>
> >> >> Since I think you mentioned you'd only be enabling this on newer
> >> >> chipsets, won't it be a problem for A+A where one A is a generation
> >> >> behind the other?
> >> >>
> >> >
> >> > Crap, that is a good point as well.
> >> >
> >> >>
> >> >> I'm not really liking where this is going btw, seems like a ill
> >> >> thought out concept, if AMD is really going down the road of 
> >> >> designing
> >> >> hw that is currently Linux incompatible, you are going to have to
> >> >> accept a big part of the burden in bringing this support in to more
> >> >> than just amd drivers for upcoming generations of gpu.
> >> >>
> >> >
> >> > Well we don't really like that either, but we have no other option as 
> >> > far as I can see.
> >>
> >> I don't really understand what "future hw may remove support for kernel 
> >> queues" means exactly. While the per-context queues can be mapped to 
> >> userspace directly, they don't *have* to be, do they? I.e. the kernel 
> >> driver should be able to either intercept userspace access to the queues, 
> >> or in the worst case do it all itself, and provide the existing 
> >> synchronization semantics as needed?
> >>
> >> Surely there are resource limits for the per-context queues, so the kernel 
> >> driver needs to do some kind of virtualization / multi-plexing anyway, or 
> >> we'll get sad user faces when there's no queue available for  >> game>.
> >>
> >> I'm probably missing something though, awaiting enlightenment. :)
> >
> >
> > The hw interface for userspace is that the ring buffer is mapped to the 
> > process address space alongside a doorbell aperture (4K page) that isn't 
> > real memory, but when the CPU writes into it, it tells the hw scheduler 
> > that there are new GPU commands in the ring buffer. Userspace inserts all 
> > the wait, draw, and signal commands into the ring buffer and then "rings" 
> > the doorbell. It's my understanding that the ring buffer and the doorbell 
> > are always mapped in the same GPU address space as the process, which makes 
> > it very difficult to emulate the current protected ring buffers in the 
> > kernel. The VMID of the ring buffer is also not changeable.
> >
>
> The doorbell does not have to be mapped into the process's GPU virtual
> address space.  The CPU could write to it directly.  Mapping it into
> the GPU's virtual address space would allow you to have a device kick
> off work however rather than the CPU.  E.g., the GPU could kick off
> it's own work or multiple devices could kick off work without CPU
> involvement.
>
> Alex
>
>
> > The hw scheduler doesn't do any synchronization and it doesn't see any 
> > dependencies. It only chooses which queue to execute, so it's really just a 
> > simple queue manager handling the virtualization 

Re: [Mesa-dev] [PATCH 1/9] drm/doc/rfc: i915 DG1 uAPI

[Jason Ekstrand]

On Wed, Apr 28, 2021 at 11:41 AM Matthew Auld  wrote:
>
> On 28/04/2021 16:51, Jason Ekstrand wrote:
> > On Mon, Apr 26, 2021 at 4:42 AM Matthew Auld  wrote:
> >>
> >> Add an entry for the new uAPI needed for DG1. Also add the overall
> >> upstream plan, including some notes for the TTM conversion.
> >>
> >> v2(Daniel):
> >>- include the overall upstreaming plan
> >>- add a note for mmap, there are differences here for TTM vs i915
> >>- bunch of other suggestions from Daniel
> >> v3:
> >>   (Daniel)
> >>- add a note for set/get caching stuff
> >>- add some more docs for existing query and extensions stuff
> >>- add an actual code example for regions query
> >>- bunch of other stuff
> >>   (Jason)
> >>- uAPI change(!):
> >>  - try a simpler design with the placements extension
> >>  - rather than have a generic setparam which can cover multiple
> >>use cases, have each extension be responsible for one thing
> >>only
> >> v4:
> >>   (Daniel)
> >>- add some more notes for ttm conversion
> >>- bunch of other stuff
> >>   (Jason)
> >>- uAPI change(!):
> >>  - drop all the extra rsvd members for the region_query and
> >>region_info, just keep the bare minimum needed for padding
> >>
> >> Signed-off-by: Matthew Auld 
> >> Cc: Joonas Lahtinen 
> >> Cc: Thomas Hellström 
> >> Cc: Daniele Ceraolo Spurio 
> >> Cc: Lionel Landwerlin 
> >> Cc: Jon Bloomfield 
> >> Cc: Jordan Justen 
> >> Cc: Daniel Vetter 
> >> Cc: Kenneth Graunke 
> >> Cc: Jason Ekstrand 
> >> Cc: Dave Airlie 
> >> Cc: dri-de...@lists.freedesktop.org
> >> Cc: mesa-dev@lists.freedesktop.org
> >> Acked-by: Daniel Vetter 
> >> Acked-by: Dave Airlie 
> >> ---
> >>   Documentation/gpu/rfc/i915_gem_lmem.h   | 212 
> >>   Documentation/gpu/rfc/i915_gem_lmem.rst | 130 +++
> >>   Documentation/gpu/rfc/index.rst |   4 +
> >>   3 files changed, 346 insertions(+)
> >>   create mode 100644 Documentation/gpu/rfc/i915_gem_lmem.h
> >>   create mode 100644 Documentation/gpu/rfc/i915_gem_lmem.rst
> >>
> >> diff --git a/Documentation/gpu/rfc/i915_gem_lmem.h 
> >> b/Documentation/gpu/rfc/i915_gem_lmem.h
> >> new file mode 100644
> >> index ..7ed59b6202d5
> >> --- /dev/null
> >> +++ b/Documentation/gpu/rfc/i915_gem_lmem.h
> >> @@ -0,0 +1,212 @@
> >> +/**
> >> + * enum drm_i915_gem_memory_class - Supported memory classes
> >> + */
> >> +enum drm_i915_gem_memory_class {
> >> +   /** @I915_MEMORY_CLASS_SYSTEM: System memory */
> >> +   I915_MEMORY_CLASS_SYSTEM = 0,
> >> +   /** @I915_MEMORY_CLASS_DEVICE: Device local-memory */
> >> +   I915_MEMORY_CLASS_DEVICE,
> >> +};
> >> +
> >> +/**
> >> + * struct drm_i915_gem_memory_class_instance - Identify particular memory 
> >> region
> >> + */
> >> +struct drm_i915_gem_memory_class_instance {
> >> +   /** @memory_class: See enum drm_i915_gem_memory_class */
> >> +   __u16 memory_class;
> >> +
> >> +   /** @memory_instance: Which instance */
> >> +   __u16 memory_instance;
> >> +};
> >> +
> >> +/**
> >> + * struct drm_i915_memory_region_info - Describes one region as known to 
> >> the
> >> + * driver.
> >> + *
> >> + * Note that we reserve some stuff here for potential future work. As an 
> >> example
> >> + * we might want expose the capabilities(see @caps) for a given region, 
> >> which
> >> + * could include things like if the region is CPU mappable/accessible, 
> >> what are
> >> + * the supported mapping types etc.
> >> + *
> >> + * Note this is using both struct drm_i915_query_item and struct 
> >> drm_i915_query.
> >> + * For this new query we are adding the new query id 
> >> DRM_I915_QUERY_MEMORY_REGIONS
> >> + * at _i915_query_item.query_id.
> >> + */
> >> +struct drm_i915_memory_region_info {
> >> +   /** @region: The class:instance pair encoding */
> >> +   struct drm_i915_gem_memory_class_instance region;
> >> +
> >> +   /** @pad: MBZ */
> >> 

Re: [Mesa-dev] [PATCH 1/9] drm/doc/rfc: i915 DG1 uAPI

[Jason Ekstrand]

On Mon, Apr 26, 2021 at 4:42 AM Matthew Auld  wrote:
>
> Add an entry for the new uAPI needed for DG1. Also add the overall
> upstream plan, including some notes for the TTM conversion.
>
> v2(Daniel):
>   - include the overall upstreaming plan
>   - add a note for mmap, there are differences here for TTM vs i915
>   - bunch of other suggestions from Daniel
> v3:
>  (Daniel)
>   - add a note for set/get caching stuff
>   - add some more docs for existing query and extensions stuff
>   - add an actual code example for regions query
>   - bunch of other stuff
>  (Jason)
>   - uAPI change(!):
> - try a simpler design with the placements extension
> - rather than have a generic setparam which can cover multiple
>   use cases, have each extension be responsible for one thing
>   only
> v4:
>  (Daniel)
>   - add some more notes for ttm conversion
>   - bunch of other stuff
>  (Jason)
>   - uAPI change(!):
> - drop all the extra rsvd members for the region_query and
>   region_info, just keep the bare minimum needed for padding
>
> Signed-off-by: Matthew Auld 
> Cc: Joonas Lahtinen 
> Cc: Thomas Hellström 
> Cc: Daniele Ceraolo Spurio 
> Cc: Lionel Landwerlin 
> Cc: Jon Bloomfield 
> Cc: Jordan Justen 
> Cc: Daniel Vetter 
> Cc: Kenneth Graunke 
> Cc: Jason Ekstrand 
> Cc: Dave Airlie 
> Cc: dri-de...@lists.freedesktop.org
> Cc: mesa-dev@lists.freedesktop.org
> Acked-by: Daniel Vetter 
> Acked-by: Dave Airlie 
> ---
>  Documentation/gpu/rfc/i915_gem_lmem.h   | 212 
>  Documentation/gpu/rfc/i915_gem_lmem.rst | 130 +++
>  Documentation/gpu/rfc/index.rst |   4 +
>  3 files changed, 346 insertions(+)
>  create mode 100644 Documentation/gpu/rfc/i915_gem_lmem.h
>  create mode 100644 Documentation/gpu/rfc/i915_gem_lmem.rst
>
> diff --git a/Documentation/gpu/rfc/i915_gem_lmem.h 
> b/Documentation/gpu/rfc/i915_gem_lmem.h
> new file mode 100644
> index ..7ed59b6202d5
> --- /dev/null
> +++ b/Documentation/gpu/rfc/i915_gem_lmem.h
> @@ -0,0 +1,212 @@
> +/**
> + * enum drm_i915_gem_memory_class - Supported memory classes
> + */
> +enum drm_i915_gem_memory_class {
> +   /** @I915_MEMORY_CLASS_SYSTEM: System memory */
> +   I915_MEMORY_CLASS_SYSTEM = 0,
> +   /** @I915_MEMORY_CLASS_DEVICE: Device local-memory */
> +   I915_MEMORY_CLASS_DEVICE,
> +};
> +
> +/**
> + * struct drm_i915_gem_memory_class_instance - Identify particular memory 
> region
> + */
> +struct drm_i915_gem_memory_class_instance {
> +   /** @memory_class: See enum drm_i915_gem_memory_class */
> +   __u16 memory_class;
> +
> +   /** @memory_instance: Which instance */
> +   __u16 memory_instance;
> +};
> +
> +/**
> + * struct drm_i915_memory_region_info - Describes one region as known to the
> + * driver.
> + *
> + * Note that we reserve some stuff here for potential future work. As an 
> example
> + * we might want expose the capabilities(see @caps) for a given region, which
> + * could include things like if the region is CPU mappable/accessible, what 
> are
> + * the supported mapping types etc.
> + *
> + * Note this is using both struct drm_i915_query_item and struct 
> drm_i915_query.
> + * For this new query we are adding the new query id 
> DRM_I915_QUERY_MEMORY_REGIONS
> + * at _i915_query_item.query_id.
> + */
> +struct drm_i915_memory_region_info {
> +   /** @region: The class:instance pair encoding */
> +   struct drm_i915_gem_memory_class_instance region;
> +
> +   /** @pad: MBZ */
> +   __u32 pad;
> +
> +   /** @caps: MBZ */
> +   __u64 caps;

As was commented on another thread somewhere, if we're going to have
caps, we should have another __u64 supported_caps which tells
userspace what caps the kernel is capable of advertising.  That way
userspace can tell the difference between a kernel which doesn't
advertise a cap and a kernel which can advertise the cap but where the
cap isn't supported.

> +
> +   /** @probed_size: Memory probed by the driver (-1 = unknown) */
> +   __u64 probed_size;
> +
> +   /** @unallocated_size: Estimate of memory remaining (-1 = unknown) */
> +   __u64 unallocated_size;
> +};
> +
> +/**
> + * struct drm_i915_query_memory_regions
> + *
> + * The region info query enumerates all regions known to the driver by 
> filling
> + * in an array of struct drm_i915_memory_region_info structures.
> + *
> + * Example for getting the list of supported regions:
> + *
> + * .. code-block:: C
> + *
> + * struct drm_i915_query_memory_regions *info;
> + * struct drm_i915_query_ite

Re: [Mesa-dev] [RFC] Linux Graphics Next: Explicit fences everywhere and no BO fences - initial proposal

[Jason Ekstrand]

On Tue, Apr 27, 2021 at 4:59 PM Marek Olšák  wrote:
>
> Jason, both memory-based signalling as well as interrupt-based signalling to 
> the CPU would be supported by amdgpu. External devices don't need to support 
> memory-based sync objects. The only limitation is that they can't convert 
> amdgpu sync objects to dma_fence.

Sure.  I'm not worried about the mechanism.  We just need a word that
means "the new fence thing" and I've been throwing "memory fence"
around for that.  Other mechanisms may work as well.

> The sad thing is that "external -> amdgpu" dependencies are really "external 
> <-> amdgpu" dependencies due to mutually-exclusive access required by 
> non-explicitly-sync'd buffers, so amdgpu-amdgpu interop is the only interop 
> that would initially work with those buffers. Explicitly sync'd buffers also 
> won't work if other drivers convert explicit fences to dma_fence. Thus, both 
> implicit sync and explicit sync might not work with other drivers at all. The 
> only interop that would initially work is explicit fences with memory-based 
> waiting and signalling on the external device to keep the kernel out of the 
> picture.

Yup.  This is where things get hard.  That said, I'm not quite ready
to give up on memory/interrupt fences just yet.

One thought that came to mind which might help would be if we added an
extremely strict concept of memory ownership.  The idea would be that
any given BO would be in one of two states at any given time:

 1. legacy: dma_fences and implicit sync works as normal but it cannot
be resident in any "modern" (direct submission, ULLS, whatever you
want to call it) context

 2. modern: In this mode they should not be used by any legacy
context.  We can't strictly prevent this, unfortunately, but maybe we
can say reading produces garbage and writes may be discarded.  In this
mode, they can be bound to modern contexts.

In theory, when in "modern" mode, you could bind the same buffer in
multiple modern contexts at a time.  However, when that's the case, it
makes ownership really tricky to track.  Therefore, we might want some
sort of dma-buf create flag for "always modern" vs. "switchable" and
only allow binding to one modern context at a time when it's
switchable.

If we did this, we may be able to move any dma_fence shenanigans to
the ownership transition points.  We'd still need some sort of "wait
for fence and transition" which has a timeout.  However, then we'd be
fairly well guaranteed that the application (not just Mesa!) has
really and truly decided it's done with the buffer and we wouldn't (I
hope!) end up with the accidental edges in the dependency graph.

Of course, I've not yet proven any of this correct so feel free to
tell me why it won't work. :-)  It was just one of those "about to go
to bed and had a thunk" type thoughts.

--Jason

P.S.  Daniel was 100% right when he said this discussion needs a glossary.


> Marek
>
>
> On Tue, Apr 27, 2021 at 3:41 PM Jason Ekstrand  wrote:
>>
>> Trying to figure out which e-mail in this mess is the right one to reply 
>> to
>>
>> On Tue, Apr 27, 2021 at 12:31 PM Lucas Stach  wrote:
>> >
>> > Hi,
>> >
>> > Am Dienstag, dem 27.04.2021 um 09:26 -0400 schrieb Marek Olšák:
>> > > Ok. So that would only make the following use cases broken for now:
>> > > - amd render -> external gpu
>>
>> Assuming said external GPU doesn't support memory fences.  If we do
>> amdgpu and i915 at the same time, that covers basically most of the
>> external GPU use-cases.  Of course, we'd want to convert nouveau as
>> well for the rest.
>>
>> > > - amd video encode -> network device
>> >
>> > FWIW, "only" breaking amd render -> external gpu will make us pretty
>> > unhappy, as we have some cases where we are combining an AMD APU with a
>> > FPGA based graphics card. I can't go into the specifics of this use-
>> > case too much but basically the AMD graphics is rendering content that
>> > gets composited on top of a live video pipeline running through the
>> > FPGA.
>>
>> I think it's worth taking a step back and asking what's being here
>> before we freak out too much.  If we do go this route, it doesn't mean
>> that your FPGA use-case can't work, it just means it won't work
>> out-of-the box anymore.  You'll have to separate execution and memory
>> dependencies inside your FPGA driver.  That's still not great but it's
>> not as bad as you maybe made it sound.
>>
>> > > What about the case when we get a buffer from an external device and
>> > > we're supposed to make it "busy&quo

Re: [Mesa-dev] [RFC] Linux Graphics Next: Explicit fences everywhere and no BO fences - initial proposal

[Jason Ekstrand]

On Tue, Apr 27, 2021 at 1:38 PM Dave Airlie  wrote:
>
> On Tue, 27 Apr 2021 at 22:06, Christian König
>  wrote:
> >
> > Correct, we wouldn't have synchronization between device with and without 
> > user queues any more.
> >
> > That could only be a problem for A+I Laptops.
>
> Since I think you mentioned you'd only be enabling this on newer
> chipsets, won't it be a problem for A+A where one A is a generation
> behind the other?
>
> I'm not really liking where this is going btw, seems like a ill
> thought out concept, if AMD is really going down the road of designing
> hw that is currently Linux incompatible, you are going to have to
> accept a big part of the burden in bringing this support in to more
> than just amd drivers for upcoming generations of gpu.

In case my previous e-mail sounded too enthusiastic, I'm also pensive
about this direction.  I'm not sure I'm ready to totally give up on
all of Linux WSI just yet.  We definitely want to head towards memory
fences and direct submission but I'm not convinced that throwing out
all of interop is necessary.  It's certainly a very big hammer and we
should try to figure out something less destructive, if that's
possible.  (I don't know for sure that it is.)

--Jason
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] [RFC] Linux Graphics Next: Explicit fences everywhere and no BO fences - initial proposal

[Jason Ekstrand]

Trying to figure out which e-mail in this mess is the right one to reply to

On Tue, Apr 27, 2021 at 12:31 PM Lucas Stach  wrote:
>
> Hi,
>
> Am Dienstag, dem 27.04.2021 um 09:26 -0400 schrieb Marek Olšák:
> > Ok. So that would only make the following use cases broken for now:
> > - amd render -> external gpu

Assuming said external GPU doesn't support memory fences.  If we do
amdgpu and i915 at the same time, that covers basically most of the
external GPU use-cases.  Of course, we'd want to convert nouveau as
well for the rest.

> > - amd video encode -> network device
>
> FWIW, "only" breaking amd render -> external gpu will make us pretty
> unhappy, as we have some cases where we are combining an AMD APU with a
> FPGA based graphics card. I can't go into the specifics of this use-
> case too much but basically the AMD graphics is rendering content that
> gets composited on top of a live video pipeline running through the
> FPGA.

I think it's worth taking a step back and asking what's being here
before we freak out too much.  If we do go this route, it doesn't mean
that your FPGA use-case can't work, it just means it won't work
out-of-the box anymore.  You'll have to separate execution and memory
dependencies inside your FPGA driver.  That's still not great but it's
not as bad as you maybe made it sound.

> > What about the case when we get a buffer from an external device and
> > we're supposed to make it "busy" when we are using it, and the
> > external device wants to wait until we stop using it? Is it something
> > that can happen, thus turning "external -> amd" into "external <->
> > amd"?
>
> Zero-copy texture sampling from a video input certainly appreciates
> this very much. Trying to pass the render fence through the various
> layers of userspace to be able to tell when the video input can reuse a
> buffer is a great experience in yak shaving. Allowing the video input
> to reuse the buffer as soon as the read dma_fence from the GPU is
> signaled is much more straight forward.

Oh, it's definitely worse than that.  Every window system interaction
is bi-directional.  The X server has to wait on the client before
compositing from it and the client has to wait on X before re-using
that back-buffer.  Of course, we can break that later dependency by
doing a full CPU wait but that's going to mean either more latency or
reserving more back buffers.  There's no good clean way to claim that
any of this is one-directional.

--Jason
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] One more thing to cut from the main branch...

[Jason Ekstrand]

Maybe, maybe not  I mean, normally I'd be all for it but

https://rosenzweig.io/blog/asahi-gpu-part-3.html

--Jason

On Tue, Apr 27, 2021 at 12:32 PM Ian Romanick  wrote:
>
> If we're going to cut all the classic drivers and a handful of older
> Gallium drivers... can we also cut Apple GLX?  Apple comes around every
> couple years to fix breakages that have crept in, and we periodically
> have compile breaks that need fixing (see
> https://gitlab.freedesktop.org/mesa/mesa/-/issues/4702).  As far as I
> can tell, having it in the main branch provides zero value to anyone...
> including Apple.
> ___
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] [PATCH 1/9] drm/doc/rfc: i915 DG1 uAPI

[Jason Ekstrand]

On Mon, Apr 26, 2021 at 10:31 AM Matthew Auld  wrote:
>
> On 26/04/2021 16:11, Jason Ekstrand wrote:
> > On Mon, Apr 26, 2021 at 4:42 AM Matthew Auld  wrote:
> >>
> >> Add an entry for the new uAPI needed for DG1. Also add the overall
> >> upstream plan, including some notes for the TTM conversion.
> >>
> >> v2(Daniel):
> >>- include the overall upstreaming plan
> >>- add a note for mmap, there are differences here for TTM vs i915
> >>- bunch of other suggestions from Daniel
> >> v3:
> >>   (Daniel)
> >>- add a note for set/get caching stuff
> >>- add some more docs for existing query and extensions stuff
> >>- add an actual code example for regions query
> >>- bunch of other stuff
> >>   (Jason)
> >>- uAPI change(!):
> >>  - try a simpler design with the placements extension
> >>  - rather than have a generic setparam which can cover multiple
> >>use cases, have each extension be responsible for one thing
> >>only
> >> v4:
> >>   (Daniel)
> >>- add some more notes for ttm conversion
> >>- bunch of other stuff
> >>   (Jason)
> >>- uAPI change(!):
> >>  - drop all the extra rsvd members for the region_query and
> >>region_info, just keep the bare minimum needed for padding
> >>
> >> Signed-off-by: Matthew Auld 
> >> Cc: Joonas Lahtinen 
> >> Cc: Thomas Hellström 
> >> Cc: Daniele Ceraolo Spurio 
> >> Cc: Lionel Landwerlin 
> >> Cc: Jon Bloomfield 
> >> Cc: Jordan Justen 
> >> Cc: Daniel Vetter 
> >> Cc: Kenneth Graunke 
> >> Cc: Jason Ekstrand 
> >> Cc: Dave Airlie 
> >> Cc: dri-de...@lists.freedesktop.org
> >> Cc: mesa-dev@lists.freedesktop.org
> >> Acked-by: Daniel Vetter 
> >> Acked-by: Dave Airlie 
> >> ---
> >>   Documentation/gpu/rfc/i915_gem_lmem.h   | 212 
> >>   Documentation/gpu/rfc/i915_gem_lmem.rst | 130 +++
> >>   Documentation/gpu/rfc/index.rst |   4 +
> >>   3 files changed, 346 insertions(+)
> >>   create mode 100644 Documentation/gpu/rfc/i915_gem_lmem.h
> >>   create mode 100644 Documentation/gpu/rfc/i915_gem_lmem.rst
> >>
> >> diff --git a/Documentation/gpu/rfc/i915_gem_lmem.h 
> >> b/Documentation/gpu/rfc/i915_gem_lmem.h
> >> new file mode 100644
> >> index ..7ed59b6202d5
> >> --- /dev/null
> >> +++ b/Documentation/gpu/rfc/i915_gem_lmem.h
> >> @@ -0,0 +1,212 @@
> >> +/**
> >> + * enum drm_i915_gem_memory_class - Supported memory classes
> >> + */
> >> +enum drm_i915_gem_memory_class {
> >> +   /** @I915_MEMORY_CLASS_SYSTEM: System memory */
> >> +   I915_MEMORY_CLASS_SYSTEM = 0,
> >> +   /** @I915_MEMORY_CLASS_DEVICE: Device local-memory */
> >> +   I915_MEMORY_CLASS_DEVICE,
> >> +};
> >> +
> >> +/**
> >> + * struct drm_i915_gem_memory_class_instance - Identify particular memory 
> >> region
> >> + */
> >> +struct drm_i915_gem_memory_class_instance {
> >> +   /** @memory_class: See enum drm_i915_gem_memory_class */
> >> +   __u16 memory_class;
> >> +
> >> +   /** @memory_instance: Which instance */
> >> +   __u16 memory_instance;
> >> +};
> >> +
> >> +/**
> >> + * struct drm_i915_memory_region_info - Describes one region as known to 
> >> the
> >> + * driver.
> >> + *
> >> + * Note that we reserve some stuff here for potential future work. As an 
> >> example
> >> + * we might want expose the capabilities(see @caps) for a given region, 
> >> which
> >> + * could include things like if the region is CPU mappable/accessible, 
> >> what are
> >> + * the supported mapping types etc.
> >> + *
> >> + * Note this is using both struct drm_i915_query_item and struct 
> >> drm_i915_query.
> >> + * For this new query we are adding the new query id 
> >> DRM_I915_QUERY_MEMORY_REGIONS
> >> + * at _i915_query_item.query_id.
> >> + */
> >> +struct drm_i915_memory_region_info {
> >> +   /** @region: The class:instance pair encoding */
> >> +   struct drm_i915_gem_memory_class_instance region;
> >> +
> >> +   /** @pad: MBZ */
> >> +   __u32 pad;
&g

Re: [Mesa-dev] [PATCH v3 4/4] drm/doc/rfc: i915 DG1 uAPI

[Jason Ekstrand]

On Wed, Apr 21, 2021 at 2:23 PM Daniel Vetter  wrote:
>
> On Wed, Apr 21, 2021 at 8:28 PM Tvrtko Ursulin
>  wrote:
> > On 21/04/2021 18:17, Jason Ekstrand wrote:
> > > On Wed, Apr 21, 2021 at 9:25 AM Tvrtko Ursulin
> > >  wrote:
> > >> On 21/04/2021 14:54, Jason Ekstrand wrote:
> > >>> On Wed, Apr 21, 2021 at 3:22 AM Tvrtko Ursulin
> > >>>  wrote:
> > >>>> On 20/04/2021 18:00, Jason Ekstrand wrote:
> > >>>> I am not claiming to know memory region query will end up the same, and
> > >>>> I definitely agree we cannot guess the future. I am just saying rsvd
> > >>>> fields are inconsequential really in terms of maintenance burden and
> > >>>> have been proven useful in the past. So I disagree with the drive to
> > >>>> kick them all out.
> > >>>
> > >>> Sure, it doesn't cost anything to have extra zeros in the struct.
> > >>> However, if/when the API grows using rsvd fields, we end up with "if
> > >>> CAP_FOO is set, rsvd[5] means blah" which makes for a horribly
> > >>> confusing API.  As a userspace person who has to remember how to use
> > >>> this stuff, I'd rather make another call or chain in a struct than try
> > >>> to remember and/or figure out what all 8 rsvd fields mean.
> > >>
> > >> Well it's not called rsvd in the uapi which is aware of the new field
> > >> but has a new name.
> > >
> > > Are we allowed to do that?  This is a genuine question.  When I've
> > > tried in the past (cliprects), I was told we couldn't rename it even
> > > though literally no one had used it in code for years.
> >
> > Well we did the union for pad_to_size so I thought we are allowed that
> > trick at least. From my experience backward source level compatibility
> > is not always there even with things like glibc. Despite that, are we
> > generally required to stay backward source compatible I will not claim
> > either way.

I'm starting to lose the will to care about this particular bike shed.
I think I'm fine with keeping some RSVD fields as long as:

 1. We're very clear in the docs with flags and caps about what things
are inputs and what things are outputs and how they interact.
Preferably, everything is an output.
 2. If we already know that caps is useless without supported_caps,
let's either add supported_caps in now or throw out both and use some
rsvd for them when they begin to be needed.
 3. We have a plan for how we're going to use them in a
backwards-compatible way.

On 3, it sounds like we have a rough sketch of a plan but I'm still
unclear on some details.  In particular, we have an rsvd[8] at the end
but, if we're going to replace individual bits of it, we can't ever
shorten or re-name that array.  We could, potentially, do

union {
__u32 rsvd[8];
struct {
__u32 new_field;
};
};

and trust C to put all the fields of our anonymous struct at the top.
Otherwise, we've got to fill out our struct with more rsvd and that
gets to be a mess.

Another option would be to have

__u32 rsvd1;
__u32 rsvd2;
__u32 rsvd3;
/* etc... */

and we can replace them one at a time.

Again, my big point is that I want us to have a PLAN and not end up in
a scenario where we end up with rsvd[5] having magical meaning.  What
I see in DII does not give me confidence.  However, I do believe that
such a plan can exist.

--Jason

> I think the anonymous union with exactly same sized field is ok. We
> also try hard to be source compatible, but we have screwed up in the
> past and shrugged it off. The one example that comes to mind is
> extended structures at the bottom with new field, which the kernel
> automatically zero-extends for old userspace. But when you recompile,
> your new-old userspace might no longer clear the new fields because
> the ioctl code didn't start out by memset()ing the entire struct.

Also, we need to ensure that we memset everything to 0. :-)

--Jason
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] [PATCH 1/9] drm/doc/rfc: i915 DG1 uAPI

[Jason Ekstrand]

On Mon, Apr 26, 2021 at 4:42 AM Matthew Auld  wrote:
>
> Add an entry for the new uAPI needed for DG1. Also add the overall
> upstream plan, including some notes for the TTM conversion.
>
> v2(Daniel):
>   - include the overall upstreaming plan
>   - add a note for mmap, there are differences here for TTM vs i915
>   - bunch of other suggestions from Daniel
> v3:
>  (Daniel)
>   - add a note for set/get caching stuff
>   - add some more docs for existing query and extensions stuff
>   - add an actual code example for regions query
>   - bunch of other stuff
>  (Jason)
>   - uAPI change(!):
> - try a simpler design with the placements extension
> - rather than have a generic setparam which can cover multiple
>   use cases, have each extension be responsible for one thing
>   only
> v4:
>  (Daniel)
>   - add some more notes for ttm conversion
>   - bunch of other stuff
>  (Jason)
>   - uAPI change(!):
> - drop all the extra rsvd members for the region_query and
>   region_info, just keep the bare minimum needed for padding
>
> Signed-off-by: Matthew Auld 
> Cc: Joonas Lahtinen 
> Cc: Thomas Hellström 
> Cc: Daniele Ceraolo Spurio 
> Cc: Lionel Landwerlin 
> Cc: Jon Bloomfield 
> Cc: Jordan Justen 
> Cc: Daniel Vetter 
> Cc: Kenneth Graunke 
> Cc: Jason Ekstrand 
> Cc: Dave Airlie 
> Cc: dri-de...@lists.freedesktop.org
> Cc: mesa-dev@lists.freedesktop.org
> Acked-by: Daniel Vetter 
> Acked-by: Dave Airlie 
> ---
>  Documentation/gpu/rfc/i915_gem_lmem.h   | 212 
>  Documentation/gpu/rfc/i915_gem_lmem.rst | 130 +++
>  Documentation/gpu/rfc/index.rst |   4 +
>  3 files changed, 346 insertions(+)
>  create mode 100644 Documentation/gpu/rfc/i915_gem_lmem.h
>  create mode 100644 Documentation/gpu/rfc/i915_gem_lmem.rst
>
> diff --git a/Documentation/gpu/rfc/i915_gem_lmem.h 
> b/Documentation/gpu/rfc/i915_gem_lmem.h
> new file mode 100644
> index ..7ed59b6202d5
> --- /dev/null
> +++ b/Documentation/gpu/rfc/i915_gem_lmem.h
> @@ -0,0 +1,212 @@
> +/**
> + * enum drm_i915_gem_memory_class - Supported memory classes
> + */
> +enum drm_i915_gem_memory_class {
> +   /** @I915_MEMORY_CLASS_SYSTEM: System memory */
> +   I915_MEMORY_CLASS_SYSTEM = 0,
> +   /** @I915_MEMORY_CLASS_DEVICE: Device local-memory */
> +   I915_MEMORY_CLASS_DEVICE,
> +};
> +
> +/**
> + * struct drm_i915_gem_memory_class_instance - Identify particular memory 
> region
> + */
> +struct drm_i915_gem_memory_class_instance {
> +   /** @memory_class: See enum drm_i915_gem_memory_class */
> +   __u16 memory_class;
> +
> +   /** @memory_instance: Which instance */
> +   __u16 memory_instance;
> +};
> +
> +/**
> + * struct drm_i915_memory_region_info - Describes one region as known to the
> + * driver.
> + *
> + * Note that we reserve some stuff here for potential future work. As an 
> example
> + * we might want expose the capabilities(see @caps) for a given region, which
> + * could include things like if the region is CPU mappable/accessible, what 
> are
> + * the supported mapping types etc.
> + *
> + * Note this is using both struct drm_i915_query_item and struct 
> drm_i915_query.
> + * For this new query we are adding the new query id 
> DRM_I915_QUERY_MEMORY_REGIONS
> + * at _i915_query_item.query_id.
> + */
> +struct drm_i915_memory_region_info {
> +   /** @region: The class:instance pair encoding */
> +   struct drm_i915_gem_memory_class_instance region;
> +
> +   /** @pad: MBZ */
> +   __u32 pad;
> +
> +   /** @caps: MBZ */
> +   __u64 caps;
> +
> +   /** @probed_size: Memory probed by the driver (-1 = unknown) */
> +   __u64 probed_size;
> +
> +   /** @unallocated_size: Estimate of memory remaining (-1 = unknown) */
> +   __u64 unallocated_size;
> +};
> +
> +/**
> + * struct drm_i915_query_memory_regions
> + *
> + * The region info query enumerates all regions known to the driver by 
> filling
> + * in an array of struct drm_i915_memory_region_info structures.
> + *
> + * Example for getting the list of supported regions:
> + *
> + * .. code-block:: C
> + *
> + * struct drm_i915_query_memory_regions *info;
> + * struct drm_i915_query_item item = {
> + * .query_id = DRM_I915_QUERY_MEMORY_REGIONS;
> + * };
> + * struct drm_i915_query query = {
> + * .num_items = 1,
> + * .items_ptr = (uintptr_t),
> + * };
> + * int err, i;
> + *
> + * // First query the size of the blob we need, this needs to be large
&g

Re: [Mesa-dev] [PATCH v3 4/4] drm/doc/rfc: i915 DG1 uAPI

[Jason Ekstrand]

On Wed, Apr 21, 2021 at 9:25 AM Tvrtko Ursulin
 wrote:
>
>
> On 21/04/2021 14:54, Jason Ekstrand wrote:
> > On Wed, Apr 21, 2021 at 3:22 AM Tvrtko Ursulin
> >  wrote:
> >>
> >> On 20/04/2021 18:00, Jason Ekstrand wrote:
> >>> On Tue, Apr 20, 2021 at 11:34 AM Tvrtko Ursulin
> >>>  wrote:
> >>>>
> >>>>
> >>>> On 19/04/2021 16:19, Jason Ekstrand wrote:
> >>>>> On Mon, Apr 19, 2021 at 7:02 AM Matthew Auld  
> >>>>> wrote:
> >>>>>>
> >>>>>> On 16/04/2021 17:38, Jason Ekstrand wrote:
> >>>>>>> On Thu, Apr 15, 2021 at 11:04 AM Matthew Auld 
> >>>>>>>  wrote:
> >>>>>>>>
> >>>>>>>> Add an entry for the new uAPI needed for DG1.
> >>>>>>>>
> >>>>>>>> v2(Daniel):
> >>>>>>>>   - include the overall upstreaming plan
> >>>>>>>>   - add a note for mmap, there are differences here for TTM vs 
> >>>>>>>> i915
> >>>>>>>>   - bunch of other suggestions from Daniel
> >>>>>>>> v3:
> >>>>>>>>  (Daniel)
> >>>>>>>>   - add a note for set/get caching stuff
> >>>>>>>>   - add some more docs for existing query and extensions stuff
> >>>>>>>>   - add an actual code example for regions query
> >>>>>>>>   - bunch of other stuff
> >>>>>>>>  (Jason)
> >>>>>>>>   - uAPI change(!):
> >>>>>>>> - try a simpler design with the placements extension
> >>>>>>>> - rather than have a generic setparam which can cover 
> >>>>>>>> multiple
> >>>>>>>>   use cases, have each extension be responsible for one 
> >>>>>>>> thing
> >>>>>>>>   only
> >>>>>>>>
> >>>>>>>> Signed-off-by: Matthew Auld 
> >>>>>>>> Cc: Joonas Lahtinen 
> >>>>>>>> Cc: Jordan Justen 
> >>>>>>>> Cc: Daniel Vetter 
> >>>>>>>> Cc: Kenneth Graunke 
> >>>>>>>> Cc: Jason Ekstrand 
> >>>>>>>> Cc: Dave Airlie 
> >>>>>>>> Cc: dri-de...@lists.freedesktop.org
> >>>>>>>> Cc: mesa-dev@lists.freedesktop.org
> >>>>>>>> ---
> >>>>>>>>  Documentation/gpu/rfc/i915_gem_lmem.h   | 255 
> >>>>>>>> 
> >>>>>>>>  Documentation/gpu/rfc/i915_gem_lmem.rst | 139 +
> >>>>>>>>  Documentation/gpu/rfc/index.rst |   4 +
> >>>>>>>>  3 files changed, 398 insertions(+)
> >>>>>>>>  create mode 100644 Documentation/gpu/rfc/i915_gem_lmem.h
> >>>>>>>>  create mode 100644 Documentation/gpu/rfc/i915_gem_lmem.rst
> >>>>>>>>
> >>>>>>>> diff --git a/Documentation/gpu/rfc/i915_gem_lmem.h 
> >>>>>>>> b/Documentation/gpu/rfc/i915_gem_lmem.h
> >>>>>>>> new file mode 100644
> >>>>>>>> index ..2a82a452e9f2
> >>>>>>>> --- /dev/null
> >>>>>>>> +++ b/Documentation/gpu/rfc/i915_gem_lmem.h
> >>>>>>>> @@ -0,0 +1,255 @@
> >>>>>>>> +/*
> >>>>>>>> + * Note that drm_i915_query_item and drm_i915_query are existing 
> >>>>>>>> bits of uAPI.
> >>>>>>>> + * For the regions query we are just adding a new query id, so no 
> >>>>>>>> actual new
> >>>>>>>> + * ioctl or anything, but including it here for reference.
> >>>>>>>> + */
> >>>>>>>> +struct drm_i915_query_item {
> >>>>>>>> +#define DRM_I915_QUERY_MEMORY_REGIONS   0xdeadbeaf
> >>>>>>>> +   
> >>>>>>>> +__u64 query_id;
> >>>>>>>> +
> >>>>>>>> +

Re: [Mesa-dev] [PATCH v3 4/4] drm/doc/rfc: i915 DG1 uAPI

[Jason Ekstrand]

On Wed, Apr 21, 2021 at 3:22 AM Tvrtko Ursulin
 wrote:
>
> On 20/04/2021 18:00, Jason Ekstrand wrote:
> > On Tue, Apr 20, 2021 at 11:34 AM Tvrtko Ursulin
> >  wrote:
> >>
> >>
> >> On 19/04/2021 16:19, Jason Ekstrand wrote:
> >>> On Mon, Apr 19, 2021 at 7:02 AM Matthew Auld  
> >>> wrote:
> >>>>
> >>>> On 16/04/2021 17:38, Jason Ekstrand wrote:
> >>>>> On Thu, Apr 15, 2021 at 11:04 AM Matthew Auld  
> >>>>> wrote:
> >>>>>>
> >>>>>> Add an entry for the new uAPI needed for DG1.
> >>>>>>
> >>>>>> v2(Daniel):
> >>>>>>  - include the overall upstreaming plan
> >>>>>>  - add a note for mmap, there are differences here for TTM vs i915
> >>>>>>  - bunch of other suggestions from Daniel
> >>>>>> v3:
> >>>>>> (Daniel)
> >>>>>>  - add a note for set/get caching stuff
> >>>>>>  - add some more docs for existing query and extensions stuff
> >>>>>>  - add an actual code example for regions query
> >>>>>>  - bunch of other stuff
> >>>>>> (Jason)
> >>>>>>  - uAPI change(!):
> >>>>>>- try a simpler design with the placements extension
> >>>>>>- rather than have a generic setparam which can cover 
> >>>>>> multiple
> >>>>>>  use cases, have each extension be responsible for one 
> >>>>>> thing
> >>>>>>  only
> >>>>>>
> >>>>>> Signed-off-by: Matthew Auld 
> >>>>>> Cc: Joonas Lahtinen 
> >>>>>> Cc: Jordan Justen 
> >>>>>> Cc: Daniel Vetter 
> >>>>>> Cc: Kenneth Graunke 
> >>>>>> Cc: Jason Ekstrand 
> >>>>>> Cc: Dave Airlie 
> >>>>>> Cc: dri-de...@lists.freedesktop.org
> >>>>>> Cc: mesa-dev@lists.freedesktop.org
> >>>>>> ---
> >>>>>> Documentation/gpu/rfc/i915_gem_lmem.h   | 255 
> >>>>>> 
> >>>>>> Documentation/gpu/rfc/i915_gem_lmem.rst | 139 +
> >>>>>> Documentation/gpu/rfc/index.rst |   4 +
> >>>>>> 3 files changed, 398 insertions(+)
> >>>>>> create mode 100644 Documentation/gpu/rfc/i915_gem_lmem.h
> >>>>>> create mode 100644 Documentation/gpu/rfc/i915_gem_lmem.rst
> >>>>>>
> >>>>>> diff --git a/Documentation/gpu/rfc/i915_gem_lmem.h 
> >>>>>> b/Documentation/gpu/rfc/i915_gem_lmem.h
> >>>>>> new file mode 100644
> >>>>>> index ..2a82a452e9f2
> >>>>>> --- /dev/null
> >>>>>> +++ b/Documentation/gpu/rfc/i915_gem_lmem.h
> >>>>>> @@ -0,0 +1,255 @@
> >>>>>> +/*
> >>>>>> + * Note that drm_i915_query_item and drm_i915_query are existing bits 
> >>>>>> of uAPI.
> >>>>>> + * For the regions query we are just adding a new query id, so no 
> >>>>>> actual new
> >>>>>> + * ioctl or anything, but including it here for reference.
> >>>>>> + */
> >>>>>> +struct drm_i915_query_item {
> >>>>>> +#define DRM_I915_QUERY_MEMORY_REGIONS   0xdeadbeaf
> >>>>>> +   
> >>>>>> +__u64 query_id;
> >>>>>> +
> >>>>>> +/*
> >>>>>> + * When set to zero by userspace, this is filled with the 
> >>>>>> size of the
> >>>>>> + * data to be written at the data_ptr pointer. The kernel 
> >>>>>> sets this
> >>>>>> + * value to a negative value to signal an error on a 
> >>>>>> particular query
> >>>>>> + * item.
> >>>>>> + */
> >>>>>> +__s32 length;
> >>>>>> +
> >>>>>> +__u32 flags;
> >>>>>> +/*
> >>>>>> + * Data will be written at the location pointed by data_ptr 
&g

Re: [Mesa-dev] [RFC] Linux Graphics Next: Explicit fences everywhere and no BO fences - initial proposal

[Jason Ekstrand]

On Tue, Apr 20, 2021 at 1:54 PM Daniel Vetter  wrote:
>
> On Tue, Apr 20, 2021 at 7:45 PM Daniel Stone  wrote:
>
> > And something more concrete:
> >
> > dma_fence.
> >
> > This already has all of the properties described above. Kernel-wise, it 
> > already devolves to CPU-side signaling when it crosses device boundaries. 
> > We need to support it roughly forever since it's been plumbed so far and so 
> > wide. Any primitive which is acceptable for winsys-like usage which crosses 
> > so many device/subsystem/process/security boundaries has to meet the same 
> > requirements. So why reinvent something which looks so similar, and has the 
> > same requirements of the kernel babysitting completion, providing little to 
> > no benefit for that difference?
>
> So I can mostly get behind this, except it's _not_ going to be
> dma_fence. That thing has horrendous internal ordering constraints
> within the kernel, and the one thing that doesn't allow you is to make
> a dma_fence depend upon a userspace fence.

Let me elaborate on this a bit.  One of the problems I mentioned
earlier is the conflation of fence types inside the kernel.  dma_fence
is used for solving two different semi-related but different problems:
client command synchronization and memory residency synchronization.
In the old implicit GL world, we conflated these two and thought we
were providing ourselves a service.  Not so much

It's all well and good to say that we should turn the memory fence
into a dma_fence and throw a timeout on it.  However, these
window-system sync primitives, as you said, have to be able to be
shared across everything.  In particular, we have to be able to share
them with drivers that don't make a good separation between command
and memory synchronization.

Let's say we're rendering on ANV with memory fences and presenting on
some USB display adapter whose kernel driver is a bit old-school.
When we pass that fence to the other driver via a sync_file or
similar, that driver may shove that dma_fence into the dma_resv on
some buffer somewhere.  Then our client, completely unaware of
internal kernel dependencies, binds that buffer into its address space
and kicks off another command buffer.  So i915 throws in a dependency
on that dma_resv which contains the previously created dma_fence and
refuses to execute any more command buffers until it signals.
Unfortunately, unbeknownst to i915, that command buffer which the
client kicked off after doing that bind was required for signaling the
memory fence on which our first dma_fence depends.  Deadlock.

Sure, we put a timeout on the dma_fence and it will eventually fire
and unblock everything.  However, there's one very important point
that's easy to miss here:  Neither i915 nor the client did anything
wrong in the above scenario.  The Vulkan footgun approach works
because there are a set of rules and, if you follow those rules,
you're guaranteed everything works.  In the above scenario, however,
the client followed all of the rules and got a deadlock anyway.  We
can't have that.


> But what we can do is use the same currently existing container
> objects like drm_syncobj or sync_file (timeline syncobj would fit best
> tbh), and stuff a userspace fence behind it. The only trouble is that
> currently timeline syncobj implement vulkan's spec, which means if you
> build a wait-before-signal deadlock, you'll wait forever. Well until
> the user ragequits and kills your process.

Yeah, it may be that this approach can be made to work.  Instead of
reusing dma_fence, maybe we can reuse syncobj and have another form of
syncobj which is a memory fence, a value to wait on, and a timeout.

--Jason
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] [PATCH v3 4/4] drm/doc/rfc: i915 DG1 uAPI

[Jason Ekstrand]

On Tue, Apr 20, 2021 at 11:34 AM Tvrtko Ursulin
 wrote:
>
>
> On 19/04/2021 16:19, Jason Ekstrand wrote:
> > On Mon, Apr 19, 2021 at 7:02 AM Matthew Auld  wrote:
> >>
> >> On 16/04/2021 17:38, Jason Ekstrand wrote:
> >>> On Thu, Apr 15, 2021 at 11:04 AM Matthew Auld  
> >>> wrote:
> >>>>
> >>>> Add an entry for the new uAPI needed for DG1.
> >>>>
> >>>> v2(Daniel):
> >>>> - include the overall upstreaming plan
> >>>> - add a note for mmap, there are differences here for TTM vs i915
> >>>> - bunch of other suggestions from Daniel
> >>>> v3:
> >>>>(Daniel)
> >>>> - add a note for set/get caching stuff
> >>>> - add some more docs for existing query and extensions stuff
> >>>> - add an actual code example for regions query
> >>>> - bunch of other stuff
> >>>>(Jason)
> >>>> - uAPI change(!):
> >>>>   - try a simpler design with the placements extension
> >>>>   - rather than have a generic setparam which can cover multiple
> >>>> use cases, have each extension be responsible for one thing
> >>>> only
> >>>>
> >>>> Signed-off-by: Matthew Auld 
> >>>> Cc: Joonas Lahtinen 
> >>>> Cc: Jordan Justen 
> >>>> Cc: Daniel Vetter 
> >>>> Cc: Kenneth Graunke 
> >>>> Cc: Jason Ekstrand 
> >>>> Cc: Dave Airlie 
> >>>> Cc: dri-de...@lists.freedesktop.org
> >>>> Cc: mesa-dev@lists.freedesktop.org
> >>>> ---
> >>>>Documentation/gpu/rfc/i915_gem_lmem.h   | 255 
> >>>>Documentation/gpu/rfc/i915_gem_lmem.rst | 139 +
> >>>>Documentation/gpu/rfc/index.rst |   4 +
> >>>>3 files changed, 398 insertions(+)
> >>>>create mode 100644 Documentation/gpu/rfc/i915_gem_lmem.h
> >>>>create mode 100644 Documentation/gpu/rfc/i915_gem_lmem.rst
> >>>>
> >>>> diff --git a/Documentation/gpu/rfc/i915_gem_lmem.h 
> >>>> b/Documentation/gpu/rfc/i915_gem_lmem.h
> >>>> new file mode 100644
> >>>> index ..2a82a452e9f2
> >>>> --- /dev/null
> >>>> +++ b/Documentation/gpu/rfc/i915_gem_lmem.h
> >>>> @@ -0,0 +1,255 @@
> >>>> +/*
> >>>> + * Note that drm_i915_query_item and drm_i915_query are existing bits 
> >>>> of uAPI.
> >>>> + * For the regions query we are just adding a new query id, so no 
> >>>> actual new
> >>>> + * ioctl or anything, but including it here for reference.
> >>>> + */
> >>>> +struct drm_i915_query_item {
> >>>> +#define DRM_I915_QUERY_MEMORY_REGIONS   0xdeadbeaf
> >>>> +   
> >>>> +__u64 query_id;
> >>>> +
> >>>> +/*
> >>>> + * When set to zero by userspace, this is filled with the size 
> >>>> of the
> >>>> + * data to be written at the data_ptr pointer. The kernel sets 
> >>>> this
> >>>> + * value to a negative value to signal an error on a particular 
> >>>> query
> >>>> + * item.
> >>>> + */
> >>>> +__s32 length;
> >>>> +
> >>>> +__u32 flags;
> >>>> +/*
> >>>> + * Data will be written at the location pointed by data_ptr 
> >>>> when the
> >>>> + * value of length matches the length of the data to be written 
> >>>> by the
> >>>> + * kernel.
> >>>> + */
> >>>> +__u64 data_ptr;
> >>>> +};
> >>>> +
> >>>> +struct drm_i915_query {
> >>>> +__u32 num_items;
> >>>> +/*
> >>>> + * Unused for now. Must be cleared to zero.
> >>>> + */
> >>>> +__u32 flags;
> >>>> +/*
> >>>> + * This points to an array of num_items drm_i915_query_item 
> >>>> structures.
> >>>> + */
> >>>> +__u64 items_ptr;
> >>>> +};
> >

Re: [Mesa-dev] [RFC] Linux Graphics Next: Explicit fences everywhere and no BO fences - initial proposal

[Jason Ekstrand]

On Tue, Apr 20, 2021 at 9:10 AM Daniel Vetter  wrote:
>
> On Tue, Apr 20, 2021 at 1:59 PM Christian König
>  wrote:
> >
> > > Yeah. If we go with userspace fences, then userspace can hang itself. Not
> > > the kernel's problem.
> >
> > Well, the path of inner peace begins with four words. “Not my fucking
> > problem.”
> >
> > But I'm not that much concerned about the kernel, but rather about
> > important userspace processes like X, Wayland, SurfaceFlinger etc...
> >
> > I mean attaching a page to a sync object and allowing to wait/signal
> > from both CPU as well as GPU side is not so much of a problem.
> >
> > > You have to somehow handle that, e.g. perhaps with conditional
> > > rendering and just using the old frame in compositing if the new one
> > > doesn't show up in time.
> >
> > Nice idea, but how would you handle that on the OpenGL/Glamor/Vulkan level.
>
> For opengl we do all the same guarantees, so if you get one of these
> you just block until the fence is signalled. Doing that properly means
> submit thread to support drm_syncobj like for vulkan.
>
> For vulkan we probably want to represent these as proper vk timeline
> objects, and the vulkan way is to just let the application (well
> compositor) here deal with it. If they import timelines from untrusted
> other parties, they need to handle the potential fallback of being
> lied at. How is "not vulkan's fucking problem", because that entire
> "with great power (well performance) comes great responsibility" is
> the entire vk design paradigm.

The security aspects are currently an unsolved problem in Vulkan.  The
assumption is that everyone trusts everyone else to be careful with
the scissors.  It's a great model!

I think we can do something in Vulkan to allow apps to protect
themselves a bit but it's tricky and non-obvious.

--Jason


> Glamour will just rely on GL providing nice package of the harsh
> reality of gpus, like usual.
>
> So I guess step 1 here for GL would be to provide some kind of
> import/export of timeline syncobj, including properly handling this
> "future/indefinite fences" aspect of them with submit thread and
> everything.
>
> -Daniel
>
> >
> > Regards,
> > Christian.
> >
> > Am 20.04.21 um 13:16 schrieb Daniel Vetter:
> > > On Tue, Apr 20, 2021 at 07:03:19AM -0400, Marek Olšák wrote:
> > >> Daniel, are you suggesting that we should skip any deadlock prevention in
> > >> the kernel, and just let userspace wait for and signal any fence it has
> > >> access to?
> > > Yeah. If we go with userspace fences, then userspace can hang itself. Not
> > > the kernel's problem. The only criteria is that the kernel itself must
> > > never rely on these userspace fences, except for stuff like implementing
> > > optimized cpu waits. And in those we must always guarantee that the
> > > userspace process remains interruptible.
> > >
> > > It's a completely different world from dma_fence based kernel fences,
> > > whether those are implicit or explicit.
> > >
> > >> Do you have any concern with the deprecation/removal of BO fences in the
> > >> kernel assuming userspace is only using explicit fences? Any concern with
> > >> the submit and return fences for modesetting and other 
> > >> producer<->consumer
> > >> scenarios?
> > > Let me work on the full replay for your rfc first, because there's a lot
> > > of details here and nuance.
> > > -Daniel
> > >
> > >> Thanks,
> > >> Marek
> > >>
> > >> On Tue, Apr 20, 2021 at 6:34 AM Daniel Vetter  wrote:
> > >>
> > >>> On Tue, Apr 20, 2021 at 12:15 PM Christian König
> > >>>  wrote:
> > >>>> Am 19.04.21 um 17:48 schrieb Jason Ekstrand:
> > >>>>> Not going to comment on everything on the first pass...
> > >>>>>
> > >>>>> On Mon, Apr 19, 2021 at 5:48 AM Marek Olšák  wrote:
> > >>>>>> Hi,
> > >>>>>>
> > >>>>>> This is our initial proposal for explicit fences everywhere and new
> > >>> memory management that doesn't use BO fences. It's a redesign of how 
> > >>> Linux
> > >>> graphics drivers work, and it can coexist with what we have now.
> > >>>>>>
> > >>>>>> 1. Introduction
> > >>>>>> (skip this if you are already sold on explicit fences)
> > >>>

Re: [Mesa-dev] [RFC] Linux Graphics Next: Explicit fences everywhere and no BO fences - initial proposal

[Jason Ekstrand]

Sorry for the mega-reply but timezones...

On Tue, Apr 20, 2021 at 6:59 AM Christian König
 wrote:
>
> > Yeah. If we go with userspace fences, then userspace can hang itself. Not
> > the kernel's problem.
>
> Well, the path of inner peace begins with four words. “Not my fucking
> problem.”

律

> But I'm not that much concerned about the kernel, but rather about
> important userspace processes like X, Wayland, SurfaceFlinger etc...
>
> I mean attaching a page to a sync object and allowing to wait/signal
> from both CPU as well as GPU side is not so much of a problem.

Yup... Sorting out these issues is what makes this a hard problem.


> > You have to somehow handle that, e.g. perhaps with conditional
> > rendering and just using the old frame in compositing if the new one
> > doesn't show up in time.
>
> Nice idea, but how would you handle that on the OpenGL/Glamor/Vulkan level.

"Just handle it with conditional rendering" is a pretty trite answer.
If we have memory fences, we could expose a Vulkan extension to allow
them to be read by conditional rendering or by a shader.  However, as
Daniel has pointed out multiple times, composition pipelines are long
and complex and cheap tricks like that aren't something we can rely on
for solving the problem.  If we're going to solve the problem, we need
to make driver-internal stuff nice while still providing something
that looks very much like a sync_file with finite time semantics to
the composition pipeline.  How?  That's the question.


> Regards,
> Christian.
>
> Am 20.04.21 um 13:16 schrieb Daniel Vetter:
> > On Tue, Apr 20, 2021 at 07:03:19AM -0400, Marek Olšák wrote:
> >> Daniel, are you suggesting that we should skip any deadlock prevention in
> >> the kernel, and just let userspace wait for and signal any fence it has
> >> access to?
> > Yeah. If we go with userspace fences, then userspace can hang itself. Not
> > the kernel's problem. The only criteria is that the kernel itself must
> > never rely on these userspace fences, except for stuff like implementing
> > optimized cpu waits. And in those we must always guarantee that the
> > userspace process remains interruptible.
> >
> > It's a completely different world from dma_fence based kernel fences,
> > whether those are implicit or explicit.
> >
> >> Do you have any concern with the deprecation/removal of BO fences in the
> >> kernel assuming userspace is only using explicit fences? Any concern with
> >> the submit and return fences for modesetting and other producer<->consumer
> >> scenarios?
> > Let me work on the full replay for your rfc first, because there's a lot
> > of details here and nuance.
> > -Daniel
> >
> >> Thanks,
> >> Marek
> >>
> >> On Tue, Apr 20, 2021 at 6:34 AM Daniel Vetter  wrote:
> >>
> >>> On Tue, Apr 20, 2021 at 12:15 PM Christian König
> >>>  wrote:
> >>>> Am 19.04.21 um 17:48 schrieb Jason Ekstrand:
> >>>>> Not going to comment on everything on the first pass...
> >>>>>
> >>>>> On Mon, Apr 19, 2021 at 5:48 AM Marek Olšák  wrote:
> >>>>>> Hi,
> >>>>>>
> >>>>>> This is our initial proposal for explicit fences everywhere and new
> >>> memory management that doesn't use BO fences. It's a redesign of how Linux
> >>> graphics drivers work, and it can coexist with what we have now.
> >>>>>>
> >>>>>> 1. Introduction
> >>>>>> (skip this if you are already sold on explicit fences)
> >>>>>>
> >>>>>> The current Linux graphics architecture was initially designed for
> >>> GPUs with only one graphics queue where everything was executed in the
> >>> submission order and per-BO fences were used for memory management and
> >>> CPU-GPU synchronization, not GPU-GPU synchronization. Later, multiple
> >>> queues were added on top, which required the introduction of implicit
> >>> GPU-GPU synchronization between queues of different processes using per-BO
> >>> fences. Recently, even parallel execution within one queue was enabled
> >>> where a command buffer starts draws and compute shaders, but doesn't wait
> >>> for them, enabling parallelism between back-to-back command buffers.
> >>> Modesetting also uses per-BO fences for scheduling flips. Our GPU 
> >>> scheduler
> >>> was created to enable all those use cases, and it's the only reason why 
> >>> the
> >>> scheduler exists.
> &

Re: [Mesa-dev] [RFC] Linux Graphics Next: Explicit fences everywhere and no BO fences - initial proposal

[Jason Ekstrand]

It's still early in the morning here and I'm not awake yet so sorry if
this comes out in bits and pieces...

On Tue, Apr 20, 2021 at 7:43 AM Daniel Stone  wrote:
>
> Hi Marek,
>
> On Mon, 19 Apr 2021 at 11:48, Marek Olšák  wrote:
>>
>> 2. Explicit synchronization for window systems and modesetting
>>
>> The producer is an application and the consumer is a compositor or a 
>> modesetting driver.
>>
>> 2.1. The Present request
>
>
> So the 'present request' is an ioctl, right? Not a userspace construct like 
> it is today? If so, how do we correlate the two?
>
> The terminology is pretty X11-centric so I'll assume that's what you've 
> designed against, but Wayland and even X11 carry much more auxiliary 
> information attached to a present request than just 'this buffer, this 
> swapchain'. Wayland latches a lot of data on presentation, including 
> non-graphics data such as surface geometry (so we can have resizes which 
> don't suck), window state (e.g. fullscreen or not, also so we can have 
> resizes which don't suck), and these requests can also cascade through a tree 
> of subsurfaces (so we can have embeds which don't suck). X11 mostly just 
> carries timestamps, which is more tractable.
>
> Given we don't want to move the entirety of Wayland into kernel-visible 
> objects, how do we synchronise the two streams so they aren't incoherent? 
> Taking a rough stab at it whilst assuming we do have 
> DRM_IOCTL_NONMODE_PRESENT, this would create a present object somewhere in 
> kernel space, which the producer would create and ?? export a FD from, that 
> the compositor would ?? import.
>
>> As part of the Present request, the producer will pass 2 fences (sync 
>> objects) to the consumer alongside the presented DMABUF BO:
>> - The submit fence: Initially unsignalled, it will be signalled when the 
>> producer has finished drawing into the presented buffer.
>
>
> We have already have this in Wayland through dma_fence. I'm relaxed about 
> this becoming drm_syncobj or drm_newmappedysncobjthing, it's just a matter of 
> typing. X11 has patches to DRI3 to support dma_fence, but they never got 
> merged because it was far too invasive to a server which is no longer 
> maintained.
>
>>
>> - The return fence: Initially unsignalled, it will be signalled when the 
>> consumer has finished using the presented buffer.
>
>
> Currently in Wayland the return fence (again a dma_fence) is generated by the 
> compositor and sent as an event when it's done, because we can't have 
> speculative/empty/future fences. drm_syncobj would make this possible, but so 
> far I've been hesitant because I don't see the benefit to it (more below).
>
>>
>> Deadlock mitigation to recover from segfaults:
>> - The kernel knows which process is obliged to signal which fence. This 
>> information is part of the Present request and supplied by userspace.
>
>
> Same as today with dma_fence. Less true with drm_syncobj if we're using 
> timelines.
>
>>
>> - If the producer crashes, the kernel signals the submit fence, so that the 
>> consumer can make forward progress.
>
>
> This is only a change if the producer is now allowed to submit a fence before 
> it's flushed the work which would eventually fulfill that fence. Using 
> dma_fence has so far isolated us from this.
>
>>
>> - If the consumer crashes, the kernel signals the return fence, so that the 
>> producer can reclaim the buffer.
>
>
> 'The consumer' is problematic, per below. I think the wording you want is 'if 
> no references are held to the submitted present object'.
>
>>
>> - A GPU hang signals all fences. Other deadlocks will be handled like GPU 
>> hangs.
>>
>> Other window system requests can follow the same idea.
>
>
> Which other window system requests did you have in mind? Again, moving the 
> entirety of Wayland's signaling into the kernel is a total non-starter. 
> Partly because it means our entire protocol would be subject to the kernel's 
> ABI rules, partly because the rules and interdependencies between the 
> requests are extremely complex, but mostly because the kernel is just a 
> useless proxy: it would be forced to do significant work to reason about what 
> those requests do and when they should happen, but wouldn't be able to make 
> those decisions itself so would have to just punt everything to userspace. 
> Unless we have eBPF compositors.
>
>>
>> Merged fences where one fence object contains multiple fences will be 
>> supported. A merged fence is signalled only when its fences are signalled. 
>> The consumer will have the option to redefine the unsignalled return fence 
>> to a merged fence.
>
>
> An elaboration of how this differed from drm_syncobj would be really helpful 
> here. I can make some guesses based on the rest of the mail, but I'm not sure 
> how accurate they are.
>
>>
>> 2.2. Modesetting
>>
>> Since a modesetting driver can also be the consumer, the present ioctl will 
>> contain a submit fence and a return fence too. One small problem with 

Re: [Mesa-dev] [RFC] Linux Graphics Next: Explicit fences everywhere and no BO fences - initial proposal

[Jason Ekstrand]

Not going to comment on everything on the first pass...

On Mon, Apr 19, 2021 at 5:48 AM Marek Olšák  wrote:
>
> Hi,
>
> This is our initial proposal for explicit fences everywhere and new memory 
> management that doesn't use BO fences. It's a redesign of how Linux graphics 
> drivers work, and it can coexist with what we have now.
>
>
> 1. Introduction
> (skip this if you are already sold on explicit fences)
>
> The current Linux graphics architecture was initially designed for GPUs with 
> only one graphics queue where everything was executed in the submission order 
> and per-BO fences were used for memory management and CPU-GPU 
> synchronization, not GPU-GPU synchronization. Later, multiple queues were 
> added on top, which required the introduction of implicit GPU-GPU 
> synchronization between queues of different processes using per-BO fences. 
> Recently, even parallel execution within one queue was enabled where a 
> command buffer starts draws and compute shaders, but doesn't wait for them, 
> enabling parallelism between back-to-back command buffers. Modesetting also 
> uses per-BO fences for scheduling flips. Our GPU scheduler was created to 
> enable all those use cases, and it's the only reason why the scheduler exists.
>
> The GPU scheduler, implicit synchronization, BO-fence-based memory 
> management, and the tracking of per-BO fences increase CPU overhead and 
> latency, and reduce parallelism. There is a desire to replace all of them 
> with something much simpler. Below is how we could do it.
>
>
> 2. Explicit synchronization for window systems and modesetting
>
> The producer is an application and the consumer is a compositor or a 
> modesetting driver.
>
> 2.1. The Present request
>
> As part of the Present request, the producer will pass 2 fences (sync 
> objects) to the consumer alongside the presented DMABUF BO:
> - The submit fence: Initially unsignalled, it will be signalled when the 
> producer has finished drawing into the presented buffer.
> - The return fence: Initially unsignalled, it will be signalled when the 
> consumer has finished using the presented buffer.

I'm not sure syncobj is what we want.  In the Intel world we're trying
to go even further to something we're calling "userspace fences" which
are a timeline implemented as a single 64-bit value in some
CPU-mappable BO.  The client writes a higher value into the BO to
signal the timeline.  The kernel then provides some helpers for
waiting on them reliably and without spinning.  I don't expect
everyone to support these right away but, If we're going to re-plumb
userspace for explicit synchronization, I'd like to make sure we take
this into account so we only have to do it once.


> Deadlock mitigation to recover from segfaults:
> - The kernel knows which process is obliged to signal which fence. This 
> information is part of the Present request and supplied by userspace.

This isn't clear to me.  Yes, if we're using anything dma-fence based
like syncobj, this is true.  But it doesn't seem totally true as a
general statement.


> - If the producer crashes, the kernel signals the submit fence, so that the 
> consumer can make forward progress.
> - If the consumer crashes, the kernel signals the return fence, so that the 
> producer can reclaim the buffer.
> - A GPU hang signals all fences. Other deadlocks will be handled like GPU 
> hangs.

What do you mean by "all"?  All fences that were supposed to be
signaled by the hung context?


>
> Other window system requests can follow the same idea.
>
> Merged fences where one fence object contains multiple fences will be 
> supported. A merged fence is signalled only when its fences are signalled. 
> The consumer will have the option to redefine the unsignalled return fence to 
> a merged fence.
>
> 2.2. Modesetting
>
> Since a modesetting driver can also be the consumer, the present ioctl will 
> contain a submit fence and a return fence too. One small problem with this is 
> that userspace can hang the modesetting driver, but in theory, any later 
> present ioctl can override the previous one, so the unsignalled presentation 
> is never used.
>
>
> 3. New memory management
>
> The per-BO fences will be removed and the kernel will not know which buffers 
> are busy. This will reduce CPU overhead and latency. The kernel will not need 
> per-BO fences with explicit synchronization, so we just need to remove their 
> last user: buffer evictions. It also resolves the current OOM deadlock.

Is this even really possible?  I'm no kernel MM expert (trying to
learn some) but my understanding is that the use of per-BO dma-fence
runs deep.  I would like to stop using it for implicit synchronization
to be sure, but I'm not sure I believe the claim that we can get rid
of it entirely.  Happy to see someone try, though.


> 3.1. Evictions
>
> If the kernel wants to move a buffer, it will have to wait for everything to 
> go idle, halt all userspace command submissions, move the 

Re: [Mesa-dev] [PATCH v3 4/4] drm/doc/rfc: i915 DG1 uAPI

[Jason Ekstrand]

On Mon, Apr 19, 2021 at 7:02 AM Matthew Auld  wrote:
>
> On 16/04/2021 17:38, Jason Ekstrand wrote:
> > On Thu, Apr 15, 2021 at 11:04 AM Matthew Auld  
> > wrote:
> >>
> >> Add an entry for the new uAPI needed for DG1.
> >>
> >> v2(Daniel):
> >>- include the overall upstreaming plan
> >>- add a note for mmap, there are differences here for TTM vs i915
> >>- bunch of other suggestions from Daniel
> >> v3:
> >>   (Daniel)
> >>- add a note for set/get caching stuff
> >>- add some more docs for existing query and extensions stuff
> >>- add an actual code example for regions query
> >>- bunch of other stuff
> >>   (Jason)
> >>- uAPI change(!):
> >>  - try a simpler design with the placements extension
> >>  - rather than have a generic setparam which can cover multiple
> >>use cases, have each extension be responsible for one thing
> >>only
> >>
> >> Signed-off-by: Matthew Auld 
> >> Cc: Joonas Lahtinen 
> >> Cc: Jordan Justen 
> >> Cc: Daniel Vetter 
> >> Cc: Kenneth Graunke 
> >> Cc: Jason Ekstrand 
> >> Cc: Dave Airlie 
> >> Cc: dri-de...@lists.freedesktop.org
> >> Cc: mesa-dev@lists.freedesktop.org
> >> ---
> >>   Documentation/gpu/rfc/i915_gem_lmem.h   | 255 
> >>   Documentation/gpu/rfc/i915_gem_lmem.rst | 139 +
> >>   Documentation/gpu/rfc/index.rst |   4 +
> >>   3 files changed, 398 insertions(+)
> >>   create mode 100644 Documentation/gpu/rfc/i915_gem_lmem.h
> >>   create mode 100644 Documentation/gpu/rfc/i915_gem_lmem.rst
> >>
> >> diff --git a/Documentation/gpu/rfc/i915_gem_lmem.h 
> >> b/Documentation/gpu/rfc/i915_gem_lmem.h
> >> new file mode 100644
> >> index ..2a82a452e9f2
> >> --- /dev/null
> >> +++ b/Documentation/gpu/rfc/i915_gem_lmem.h
> >> @@ -0,0 +1,255 @@
> >> +/*
> >> + * Note that drm_i915_query_item and drm_i915_query are existing bits of 
> >> uAPI.
> >> + * For the regions query we are just adding a new query id, so no actual 
> >> new
> >> + * ioctl or anything, but including it here for reference.
> >> + */
> >> +struct drm_i915_query_item {
> >> +#define DRM_I915_QUERY_MEMORY_REGIONS   0xdeadbeaf
> >> +   
> >> +__u64 query_id;
> >> +
> >> +/*
> >> + * When set to zero by userspace, this is filled with the size of 
> >> the
> >> + * data to be written at the data_ptr pointer. The kernel sets 
> >> this
> >> + * value to a negative value to signal an error on a particular 
> >> query
> >> + * item.
> >> + */
> >> +__s32 length;
> >> +
> >> +__u32 flags;
> >> +/*
> >> + * Data will be written at the location pointed by data_ptr when 
> >> the
> >> + * value of length matches the length of the data to be written 
> >> by the
> >> + * kernel.
> >> + */
> >> +__u64 data_ptr;
> >> +};
> >> +
> >> +struct drm_i915_query {
> >> +__u32 num_items;
> >> +/*
> >> + * Unused for now. Must be cleared to zero.
> >> + */
> >> +__u32 flags;
> >> +/*
> >> + * This points to an array of num_items drm_i915_query_item 
> >> structures.
> >> + */
> >> +__u64 items_ptr;
> >> +};
> >> +
> >> +#define DRM_IOCTL_I915_QUERY   DRM_IOWR(DRM_COMMAND_BASE + 
> >> DRM_I915_QUERY, struct drm_i915_query)
> >> +
> >> +/**
> >> + * enum drm_i915_gem_memory_class
> >> + */
> >> +enum drm_i915_gem_memory_class {
> >> +   /** @I915_MEMORY_CLASS_SYSTEM: system memory */
> >> +   I915_MEMORY_CLASS_SYSTEM = 0,
> >> +   /** @I915_MEMORY_CLASS_DEVICE: device local-memory */
> >> +   I915_MEMORY_CLASS_DEVICE,
> >> +};
> >> +
> >> +/**
> >> + * struct drm_i915_gem_memory_class_instance
> >> + */
> >> +struct drm_i915_gem_memory_class_instance {
> >> +   /** @memory_class: see enum drm_i915_gem_memory_class */
> >> +   __u16 memory_class;
> >> +
> >> +   /** @memory_inst

Re: [Mesa-dev] [PATCH 4/4] drm/i915/uapi: convert i915_query and friend to kernel doc

[Jason Ekstrand]

Reviewed-by: Jason Ekstrand 

On April 16, 2021 05:37:55 Matthew Auld  wrote:


Add a note about the two-step process.

v2(Tvrtko):
 - Also document the other method of just passing in a buffer which is
   large enough, which avoids two ioctl calls. Can make sense for
   smaller query items.

Suggested-by: Daniel Vetter 
Signed-off-by: Matthew Auld 
Cc: Joonas Lahtinen 
Cc: Tvrtko Ursulin 
Cc: Jordan Justen 
Cc: Daniel Vetter 
Cc: Kenneth Graunke 
Cc: Jason Ekstrand 
Cc: Dave Airlie 
Cc: dri-de...@lists.freedesktop.org
Cc: mesa-dev@lists.freedesktop.org
Reviewed-by: Daniel Vetter 
---
include/uapi/drm/i915_drm.h | 61 ++---
1 file changed, 50 insertions(+), 11 deletions(-)

diff --git a/include/uapi/drm/i915_drm.h b/include/uapi/drm/i915_drm.h
index d79b51c12ff2..12f375c52317 100644
--- a/include/uapi/drm/i915_drm.h
+++ b/include/uapi/drm/i915_drm.h
@@ -2218,14 +2218,23 @@ struct drm_i915_perf_oa_config {
__u64 flex_regs_ptr;
};

+/**
+ * struct drm_i915_query_item - An individual query for the kernel to process.
+ *
+ * The behaviour is determined by the @query_id. Note that exactly what
+ * @data_ptr is also depends on the specific @query_id.
+ */
struct drm_i915_query_item {
+   /** @query_id: The id for this query */
__u64 query_id;
#define DRM_I915_QUERY_TOPOLOGY_INFO1
#define DRM_I915_QUERY_ENGINE_INFO  2
#define DRM_I915_QUERY_PERF_CONFIG  3
/* Must be kept compact -- no holes and well documented */

-   /*
+   /**
+* @length:
+*
 * When set to zero by userspace, this is filled with the size of the
 * data to be written at the data_ptr pointer. The kernel sets this
 * value to a negative value to signal an error on a particular query
@@ -2233,21 +2242,26 @@ struct drm_i915_query_item {
 */
__s32 length;

-   /*
+   /**
+* @flags:
+*
 * When query_id == DRM_I915_QUERY_TOPOLOGY_INFO, must be 0.
 *
 * When query_id == DRM_I915_QUERY_PERF_CONFIG, must be one of the
-* following :
-* - DRM_I915_QUERY_PERF_CONFIG_LIST
-* - DRM_I915_QUERY_PERF_CONFIG_DATA_FOR_UUID
-* - DRM_I915_QUERY_PERF_CONFIG_FOR_UUID
+* following:
+*
+*  - DRM_I915_QUERY_PERF_CONFIG_LIST
+*  - DRM_I915_QUERY_PERF_CONFIG_DATA_FOR_UUID
+*  - DRM_I915_QUERY_PERF_CONFIG_FOR_UUID
 */
__u32 flags;
#define DRM_I915_QUERY_PERF_CONFIG_LIST  1
#define DRM_I915_QUERY_PERF_CONFIG_DATA_FOR_UUID 2
#define DRM_I915_QUERY_PERF_CONFIG_DATA_FOR_ID   3

-   /*
+   /**
+* @data_ptr:
+*
 * Data will be written at the location pointed by data_ptr when the
 * value of length matches the length of the data to be written by the
 * kernel.
@@ -2255,16 +2269,41 @@ struct drm_i915_query_item {
__u64 data_ptr;
};

+/**
+ * struct drm_i915_query - Supply an array of drm_i915_query_item for the 
kernel

+ * to fill out.
+ *
+ * Note that this is generally a two step process for each drm_i915_query_item
+ * in the array:
+ *
+ * 1. Call the DRM_IOCTL_I915_QUERY, giving it our array of 
drm_i915_query_item,

+ *with drm_i915_query_item.size set to zero. The kernel will then fill in
+ *the size, in bytes, which tells userspace how memory it needs to 
allocate

+ *for the blob(say for an array of properties).
+ *
+ * 2. Next we call DRM_IOCTL_I915_QUERY again, this time with the
+ *drm_i915_query_item.data_ptr equal to our newly allocated blob. Note 
that

+ *the i915_query_item.size should still be the same as what the kernel
+ *previously set. At this point the kernel can fill in the blob.
+ *
+ * Note that for some query items it can make sense for userspace to just pass
+ * in a buffer/blob equal to or larger than the required size. In this 
case only

+ * a single ioctl call is needed. For some smaller query items this can work
+ * quite well.
+ *
+ */
struct drm_i915_query {
+   /** @num_items: The number of elements in the @items_ptr array */
__u32 num_items;

-   /*
-* Unused for now. Must be cleared to zero.
+   /**
+* @flags: Unused for now. Must be cleared to zero.
 */
__u32 flags;

-   /*
-* This points to an array of num_items drm_i915_query_item structures.
+   /**
+* @items_ptr: This points to an array of num_items drm_i915_query_item
+* structures.
 */
__u64 items_ptr;
};
--
2.26.3


___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] [PATCH 3/4] drm/i915/uapi: convert i915_user_extension to kernel doc

[Jason Ekstrand]

On April 16, 2021 05:37:52 Matthew Auld  wrote:


Add some example usage for the extension chaining also, which is quite
nifty.

v2: (Daniel)
 - clarify that the name is just some integer, also document that the
   name space is not global

Suggested-by: Daniel Vetter 
Signed-off-by: Matthew Auld 
Cc: Joonas Lahtinen 
Cc: Jordan Justen 
Cc: Daniel Vetter 
Cc: Kenneth Graunke 
Cc: Jason Ekstrand 
Cc: Dave Airlie 
Cc: dri-de...@lists.freedesktop.org
Cc: mesa-dev@lists.freedesktop.org
Reviewed-by: Daniel Vetter 
---
include/uapi/drm/i915_drm.h | 54 ++---
1 file changed, 50 insertions(+), 4 deletions(-)

diff --git a/include/uapi/drm/i915_drm.h b/include/uapi/drm/i915_drm.h
index 92da48e935d1..d79b51c12ff2 100644
--- a/include/uapi/drm/i915_drm.h
+++ b/include/uapi/drm/i915_drm.h
@@ -62,8 +62,8 @@ extern "C" {
#define I915_ERROR_UEVENT "ERROR"
#define I915_RESET_UEVENT "RESET"

-/*
- * i915_user_extension: Base class for defining a chain of extensions
+/**
+ * struct i915_user_extension - Base class for defining a chain of extensions
 *
 * Many interfaces need to grow over time. In most cases we can simply
 * extend the struct and have userspace pass in more data. Another option,
@@ -76,12 +76,58 @@ extern "C" {
 * increasing complexity, and for large parts of that interface to be
 * entirely optional. The downside is more pointer chasing; chasing across
 * the __user boundary with pointers encapsulated inside u64.
+ *
+ * Example chaining:
+ *
+ * .. code-block:: C
+ *
+ * struct i915_user_extension ext3 {
+ * .next_extension = 0, // end
+ * .name = ...,
+ * };
+ * struct i915_user_extension ext2 {
+ * .next_extension = (uintptr_t),
+ * .name = ...,
+ * };
+ * struct i915_user_extension ext1 {
+ * .next_extension = (uintptr_t),
+ * .name = ...,
+ * };
+ *
+ * Typically the i915_user_extension would be embedded in some uAPI 
struct, and

+ * in this case we would feed it the head of the chain(i.e ext1), which would
+ * then apply all of the above extensions.
+ *
 */
struct i915_user_extension {
+ /**
+ * @next_extension:
+ *
+ * Pointer to the next struct i915_user_extension, or zero if the end.
+ */
 __u64 next_extension;
+ /**
+ * @name: Name of the extension.
+ *
+ * Note that the name here is just some integer.
+ *
+ * Also note that the name space for this is not global for the whole
+ * driver, but rather its scope/meaning is limited to the specific piece
+ * of uAPI which has embedded the struct i915_user_extension.


We may want to rethink this decision. In Vulkan, we have several cases 
where we use the same extension multiple places.  Given that the only 
extensible thing currently landed is context creation, we still could make 
this global.  Then again, forcing us to go through the exercise of 
redefining every time has its advantages too.


In any case, this is a correct description of the current state of affairs, so

Reviewed-by: Jason Ekstrand 



+ */
 __u32 name;
- __u32 flags; /* All undefined bits must be zero. */
- __u32 rsvd[4]; /* Reserved for future use; must be zero. */
+ /**
+ * @flags: MBZ
+ *
+ * All undefined bits must be zero.
+ */
+ __u32 flags;
+ /**
+ * @rsvd: MBZ
+ *
+ * Reserved for future use; must be zero.
+ */
+ __u32 rsvd[4];
};

/*
--
2.26.3


___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] [PATCH v3 4/4] drm/doc/rfc: i915 DG1 uAPI

[Jason Ekstrand]

On Thu, Apr 15, 2021 at 11:04 AM Matthew Auld  wrote:
>
> Add an entry for the new uAPI needed for DG1.
>
> v2(Daniel):
>   - include the overall upstreaming plan
>   - add a note for mmap, there are differences here for TTM vs i915
>   - bunch of other suggestions from Daniel
> v3:
>  (Daniel)
>   - add a note for set/get caching stuff
>   - add some more docs for existing query and extensions stuff
>   - add an actual code example for regions query
>   - bunch of other stuff
>  (Jason)
>   - uAPI change(!):
> - try a simpler design with the placements extension
> - rather than have a generic setparam which can cover multiple
>   use cases, have each extension be responsible for one thing
>   only
>
> Signed-off-by: Matthew Auld 
> Cc: Joonas Lahtinen 
> Cc: Jordan Justen 
> Cc: Daniel Vetter 
> Cc: Kenneth Graunke 
> Cc: Jason Ekstrand 
> Cc: Dave Airlie 
> Cc: dri-de...@lists.freedesktop.org
> Cc: mesa-dev@lists.freedesktop.org
> ---
>  Documentation/gpu/rfc/i915_gem_lmem.h   | 255 
>  Documentation/gpu/rfc/i915_gem_lmem.rst | 139 +
>  Documentation/gpu/rfc/index.rst |   4 +
>  3 files changed, 398 insertions(+)
>  create mode 100644 Documentation/gpu/rfc/i915_gem_lmem.h
>  create mode 100644 Documentation/gpu/rfc/i915_gem_lmem.rst
>
> diff --git a/Documentation/gpu/rfc/i915_gem_lmem.h 
> b/Documentation/gpu/rfc/i915_gem_lmem.h
> new file mode 100644
> index ..2a82a452e9f2
> --- /dev/null
> +++ b/Documentation/gpu/rfc/i915_gem_lmem.h
> @@ -0,0 +1,255 @@
> +/*
> + * Note that drm_i915_query_item and drm_i915_query are existing bits of 
> uAPI.
> + * For the regions query we are just adding a new query id, so no actual new
> + * ioctl or anything, but including it here for reference.
> + */
> +struct drm_i915_query_item {
> +#define DRM_I915_QUERY_MEMORY_REGIONS   0xdeadbeaf
> +   
> +__u64 query_id;
> +
> +/*
> + * When set to zero by userspace, this is filled with the size of the
> + * data to be written at the data_ptr pointer. The kernel sets this
> + * value to a negative value to signal an error on a particular query
> + * item.
> + */
> +__s32 length;
> +
> +__u32 flags;
> +/*
> + * Data will be written at the location pointed by data_ptr when the
> + * value of length matches the length of the data to be written by 
> the
> + * kernel.
> + */
> +__u64 data_ptr;
> +};
> +
> +struct drm_i915_query {
> +__u32 num_items;
> +/*
> + * Unused for now. Must be cleared to zero.
> + */
> +__u32 flags;
> +/*
> + * This points to an array of num_items drm_i915_query_item 
> structures.
> + */
> +__u64 items_ptr;
> +};
> +
> +#define DRM_IOCTL_I915_QUERY   DRM_IOWR(DRM_COMMAND_BASE + DRM_I915_QUERY, 
> struct drm_i915_query)
> +
> +/**
> + * enum drm_i915_gem_memory_class
> + */
> +enum drm_i915_gem_memory_class {
> +   /** @I915_MEMORY_CLASS_SYSTEM: system memory */
> +   I915_MEMORY_CLASS_SYSTEM = 0,
> +   /** @I915_MEMORY_CLASS_DEVICE: device local-memory */
> +   I915_MEMORY_CLASS_DEVICE,
> +};
> +
> +/**
> + * struct drm_i915_gem_memory_class_instance
> + */
> +struct drm_i915_gem_memory_class_instance {
> +   /** @memory_class: see enum drm_i915_gem_memory_class */
> +   __u16 memory_class;
> +
> +   /** @memory_instance: which instance */
> +   __u16 memory_instance;
> +};
> +
> +/**
> + * struct drm_i915_memory_region_info
> + *
> + * Describes one region as known to the driver.
> + *
> + * Note that we reserve quite a lot of stuff here for potential future work. 
> As
> + * an example we might want expose the capabilities(see caps) for a given
> + * region, which could include things like if the region is CPU
> + * mappable/accessible etc.

I get caps but I'm seriously at a loss as to what the rest of this
would be used for.  Why are caps and flags both there and separate?
Flags are typically something you set, not query.  Also, what's with
rsvd1 at the end?  This smells of substantial over-building to me.

I thought to myself, "maybe I'm missing a future use-case" so I looked
at the internal tree and none of this is being used there either.
This indicates to me that either I'm missing something and there's
code somewhere I don't know about or, with three years of building on
internal branches, we still haven't proven that any of this is needed.
If it's the latter, which I strongly suspect, maybe we should drop the
unn

Re: [Mesa-dev] [Intel-gfx] [RFC PATCH v2] drm/doc/rfc: i915 DG1 uAPI

[Jason Ekstrand]

+mesa-dev and some Intel mesa people.

On Wed, Apr 14, 2021 at 5:23 AM Daniel Vetter  wrote:
>
> On Tue, Apr 13, 2021 at 12:47:06PM +0100, Matthew Auld wrote:
> > Add an entry for the new uAPI needed for DG1.
> >
> > v2(Daniel):
> >   - include the overall upstreaming plan
> >   - add a note for mmap, there are differences here for TTM vs i915
> >   - bunch of other suggestions from Daniel
> >
> > Signed-off-by: Matthew Auld 
> > Cc: Joonas Lahtinen 
> > Cc: Daniel Vetter 
> > Cc: Dave Airlie 
>
> Bunch more thoughts below, I think we're getting there. Thanks for doing
> this.
>
> > ---
> >  Documentation/gpu/rfc/i915_gem_lmem.h   | 151 
> >  Documentation/gpu/rfc/i915_gem_lmem.rst | 119 +++
> >  Documentation/gpu/rfc/index.rst |   4 +
> >  3 files changed, 274 insertions(+)
> >  create mode 100644 Documentation/gpu/rfc/i915_gem_lmem.h
> >  create mode 100644 Documentation/gpu/rfc/i915_gem_lmem.rst
> >
> > diff --git a/Documentation/gpu/rfc/i915_gem_lmem.h 
> > b/Documentation/gpu/rfc/i915_gem_lmem.h
> > new file mode 100644
> > index ..6ae13209d7ef
> > --- /dev/null
> > +++ b/Documentation/gpu/rfc/i915_gem_lmem.h
> > @@ -0,0 +1,151 @@
> > +/* The new query_id for struct drm_i915_query_item */
> > +#define DRM_I915_QUERY_MEMORY_REGIONS   0xdeadbeaf
> > +
> > +/**
> > + * enum drm_i915_gem_memory_class
> > + */
> > +enum drm_i915_gem_memory_class {
>
> Are we really going with enum in uapi? I thought that was frought with
> peril since the integer type of enum is quite a bit up to compilers. But
> maybe I'm just scared.

It looks to me like it's a __u16 below.  That should be fine.  We
don't really need to give the enum type a name in that case, though.

> > + /** @I915_MEMORY_CLASS_SYSTEM: system memory */
> > + I915_MEMORY_CLASS_SYSTEM = 0,
> > + /** @I915_MEMORY_CLASS_DEVICE: device local-memory */
> > + I915_MEMORY_CLASS_DEVICE,
> > +};
> > +
> > +/**
> > + * struct drm_i915_gem_memory_class_instance
> > + */
> > +struct drm_i915_gem_memory_class_instance {
> > + /** @memory_class: see enum drm_i915_gem_memory_class */
> > + __u16 memory_class;
> > +
> > + /** @memory_instance: which instance */
> > + __u16 memory_instance;
> > +};
> > +
> > +/**
> > + * struct drm_i915_memory_region_info
> > + *
> > + * Describes one region as known to the driver.
> > + */
> > +struct drm_i915_memory_region_info {
> > + /** @region: class:instance pair encoding */
> > + struct drm_i915_gem_memory_class_instance region;
> > +
> > + /** @rsvd0: MBZ */
> > + __u32 rsvd0;
> > +
> > + /** @caps: MBZ */
> > + __u64 caps;
> > +
> > + /** @flags: MBZ */
> > + __u64 flags;
> > +
> > + /** @probed_size: Memory probed by the driver (-1 = unknown) */
> > + __u64 probed_size;
> > +
> > + /** @unallocated_size: Estimate of memory remaining (-1 = unknown) */
> > + __u64 unallocated_size;
> > +
> > + /** @rsvd1: MBZ */
> > + __u64 rsvd1[8];
>
> I guess this is for future stuff that becomes relevant with multi-tile?
> Might be worth explaining in 1-2 words why we reserve a pile here. Also
> it doesn't matter ofc for performance here :-)
>
> > +};
> > +
> > +/**
> > + * struct drm_i915_query_memory_regions
> > + *
> > + * Region info query enumerates all regions known to the driver by filling 
> > in
> > + * an array of struct drm_i915_memory_region_info structures.
>
> I guess this works with the usual 1. query number of regions 2. get them
> all two-step ioctl flow? Worth explaining here.
>
> > + */
> > +struct drm_i915_query_memory_regions {
> > + /** @num_regions: Number of supported regions */
> > + __u32 num_regions;
> > +
> > + /** @rsvd: MBZ */
> > + __u32 rsvd[3];
> > +
> > + /** @regions: Info about each supported region */
> > + struct drm_i915_memory_region_info regions[];
> > +};
>
> Hm don't we need a query ioctl for this too?
>
> > +
> > +#define DRM_I915_GEM_CREATE_EXT  0xdeadbeaf
> > +#define DRM_IOCTL_I915_GEM_CREATE_EXTDRM_IOWR(DRM_COMMAND_BASE + 
> > DRM_I915_GEM_CREATE_EXT, struct drm_i915_gem_create_ext)
> > +
> > +/**
> > + * struct drm_i915_gem_create_ext
>
> I think some explanation here that all new bo flags will be added here,
> and that in general we're phasing out the various SET/GET ioctls.
>
> > + */
> > +struct drm_i915_gem_create_ext {
> > + /**
> > +  * @size: Requested size for the object.
> > +  *
> > +  * The (page-aligned) allocated size for the object will be returned.
> > +  */
> > + __u64 size;
> > + /**
> > +  * @handle: Returned handle for the object.
> > +  *
> > +  * Object handles are nonzero.
> > +  */
> > + __u32 handle;
> > + /** @flags: MBZ */
> > + __u32 flags;
> > + /**
> > +  * @extensions:
> > +  * For I915_GEM_CREATE_EXT_SETPARAM extension usage see both:
> > +  *  struct drm_i915_gem_create_ext_setparam.
> > +  *  

Re: [Mesa-dev] [RFC] Concrete proposal to split classic

[Jason Ekstrand]

On Mon, Mar 29, 2021 at 5:59 PM Ilia Mirkin  wrote:
>
> Probably nv30 would do well to "move on" as well. But it also presents
> an interesting question -- the nv30 driver has lots of problems. I
> have no plans to fix them, nor am I aware of anyone else with such
> plans. However if such a developer were to turn up, would it be
> reasonable to assume that their work would ultimately land in this
> "lts" branch/tree/whatever? Some of the "fixes" will require large-ish
> changes to the driver...

I think that's fine.  I've said before that my definition of "LTS" is
more "it no longer picks up bugs and improvements to core from the
active dev work on master" more than "long-term stable".  I'm fine if
someone wants to fix things or even add features on the LTS branch.
And if a driver goes from virtually zero maintenance to active
development again, it can move back to main; it just may need a little
updating.

--Jason


> Cheers,
>
>   -ilia
>
> On Mon, Mar 29, 2021 at 6:48 PM Marek Olšák  wrote:
> >
> > Alright that's r300 and swr that are going to find a new home in the lts 
> > branch. Do any other gallium drivers want to join them?
> >
> > Marek
> >
> > On Mon., Mar. 29, 2021, 13:51 Zielinski, Jan,  
> > wrote:
> >>
> >> On Thursday, March 25, 2021 8:47 Marek Olšák wrote:
> >> > Same thinking could be applied to other gallium drivers for old hardware 
> >> > that don't receive new development and are becoming more and more 
> >> > irrelevant every year due to their age.
> >>
> >> Can we also keep Gallium for OpenSWR driver on -lts branch? We currently 
> >> are focusing effort on other OSS projects, and want to maintain OpenSWR at 
> >> its current feature level, but we are often seeing Mesa core changes 
> >> causing problems in OpenSWR, that we can’t always address right away. So, 
> >> we would like to point our users to a stable branch, that limits the 
> >> amount of effort required for OpenSWR to support its existing users.
> >>
> >> Jan
> >>
> >> On Wed, Mar 24, 2021, at 09:15, Jason Ekstrand wrote:
> >> > On Wed, Mar 24, 2021 at 10:28 AM Rob Clark <mailto:robdcl...@gmail.com> 
> >> > wrote:
> >> > >
> >> > > On Mon, Mar 22, 2021 at 3:15 PM Dylan Baker 
> >> > > <mailto:dy...@pnwbakers.com> wrote:
> >> > > >
> >> > > > Hi list,
> >> > > >
> >> > > > We've talked about it a number of times, but I think it's time time 
> >> > > > to
> >> > > > discuss splitting the classic drivers off of the main development 
> >> > > > branch
> >> > > > again, although this time I have a concrete plan for how this would
> >> > > > work.
> >> > > >
> >> > > > First, why? Basically, all of the classic drivers are in maintanence
> >> > > > mode (even i965). Second, many of them rely on code that no one works
> >> > > > on, and very few people still understand. There is no CI for most of
> >> > > > them, and the Intel CI is not integrated with gitlab, so it's easy to
> >> > > > unintentionally break them, and this breakage usually isn't noticed
> >> > > > until just before or just after a release. 21.0 was held up (in small
> >> > > > part, also me just getting behind) because of such breakages.
> >> > > >
> >> > > > I konw there is some interest in getting i915g in good enough shape 
> >> > > > that
> >> > > > it could replace i915c, at least for the common case. I also am aware
> >> > > > that Dave, Ilia, and Eric (with some pointers from Ken) have been
> >> > > > working on a gallium driver to replace i965. Neither of those things 
> >> > > > are
> >> > > > ready yet, but I've taken them into account.
> >> > > >
> >> > > > Here's the plan:
> >> > > >
> >> > > > 1) 21.1 release happens
> >> > > > 2) we remove classic from master
> >> > > > 3) 21.1 reaches EOL because of 21.2
> >> > > > 4) we fork the 21.1 branch into a "classic-lts"¹ branch
> >> > > > 5) we disable all vulkan and gallium drivers in said branch, at 
> >> > > > least at
> >> > > >the Meson level
> >> > >
> >> > &g

Re: [Mesa-dev] [RFC] Concrete proposal to split classic

[Jason Ekstrand]

On Thu, Mar 25, 2021 at 4:32 PM Kenneth Graunke  wrote:
>
> On Thursday, March 25, 2021 2:04:45 PM PDT Ian Romanick wrote:
> > On 3/25/21 10:49 AM, Jason Ekstrand wrote:
> > Can you be more specific? Also, is there a reason why that work can't
> > or shouldn't be done directly in the LTS branch?  As Ken pointed out,
>
> The bulk of things that I had going were to enable some extensions and
> make those extensions non-optional.  ARB_framebuffer_object was at the
> top of the list, but there were a couple of others.  I think ARB_fbo
> also affected the Gallium nouveau driver.  Some of that was derailed
> when I wasn't able to remove (classic) support for NV04 and NV05... and
> I don't remember exactly where I left it.  I would expect some of those
> kinds of changes to also happen in post-fork mainline too.

Hrm... That is a genuinely interesting extension on those platforms.
From a "make it non-optional and dead-code" perspective, we can do
that on mainline immediately post-fork easily enough.  Enabling it on
legacy could be done separately.  If we wanted to also clean up the
core on the legacy branch then, yeah, it'd have to be done twice.

> > > I'm not sure we want to totally declare those drivers dead. People can
> > > still do feature or enhancement development of they want to, it just
> > > happens in a different branch.
> > >
> > > I think we need to be more clear about what "LTS mode" means for
> > > developers and users here. It isn't that they can never change our be
> > > improved. It's that we've gotten to the point where what they're
> > > getting from being in the active development tree is breakage, not
> > > "free" improvements. We're suggesting changing the social contract
> > > with users, so to speak, from "these drivers still pick up
> > > improvements from core" to "we won't break these drivers when we make
> > > improvements to core in master."
> >
> > That is interesting.  I doesn't sound very much like "long term stable"
> > as in the original proposal.  What would the versioning scheme be?  In
> > the original proposal, I would have expected versions go to 21.1.∞.  How
> > would that work if some version adds a feature?  Would the versions (and
> > branches from the branch?) follow the usual Mesa Year.Quarter.x scheme?

I think we should make a distinction between what users should expect
and what devs are allowed to do.  It should be LTS from the
perspective that users shouldn't expect new features.  Then again,
they really shouldn't expect new features on those drivers anyway.
¯\_(ツ)_/¯

> That's a good point.  It's a bit expanded from "put out to pasture" so
> maybe "-lts" isn't great.  We could call it "-classic", unless Marek
> wants to include r300g in there too.  "-legacy" seems reasonable.  It
> looks like NVIDIA has various "legacy" branches with a version number
> based on the original version where things branched off from.
>
> So maybe we could do:
>
>- mesa-legacy21 21.1.X
>
> where X increments on every release without worrying whether it adds
> features or simply bug fixes.  We figure true features and major
> development will be pretty rare anyway.

Yeah, I don't know that we need to worry too much about stable vs.
feature releases on the legacy branch.  If we still wanted a concept
of major releases, we could slow it down to 1/year or something.
Really, whatever makes it easiest for the release maintainers is fine
with me.

> > > So, unless there's a solid reason why such work needs to happen in the
> > > master branch, I don't see a reason to hold this up for it.  As long
> > > as you're committed to test r200 and i915 when you make said changes,
> > > we can do a feature release on the LTS branch. Users and distros just
> > > shouldn't expect that to be a common thing.
> >
> > This inverts the current testing problem.  Right now, r200 and i915 are
> > poorly tested, but 965G through Sandybridge are very well tested.  While
> > I can totally test core changes on r200 and i915, how would I verify
> > that those changes don't break, say, Ironlake?
>
> Post-fork, Intel CI would stop testing i965 on mainline obviously,
> since it wouldn't exist there any longer.
>
> But I imagine it would add a new mesa_legacy21 job (like mesa_master)
> which would still test i965 and i915, per-commit.  Clayton/Nico could
> also add "dev/idr/legacy" branches which trigger testing on i965/i915
> only.  The existing expected results files would continue to work.

I think this should work fine.  It would end up being more like once
per piglit MR rather than once per mesa MR since the cadence on the
legacy branch should slow to a crawl.

If anything, it might make testing easier since you'll have the whole
pre-gen8 farm to yourself if you're hacking on the legacy branch. 

--Jason
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] [RFC] Concrete proposal to split classic

[Jason Ekstrand]

Can you be more specific? Also, is there a reason why that work can't
or shouldn't be done directly in the LTS branch?  As Ken pointed out,
I'm not sure we want to totally declare those drivers dead. People can
still do feature or enhancement development of they want to, it just
happens in a different branch.

I think we need to be more clear about what "LTS mode" means for
developers and users here. It isn't that they can never change our be
improved. It's that we've gotten to the point where what they're
getting from being in the active development tree is breakage, not
"free" improvements. We're suggesting changing the social contract
with users, so to speak, from "these drivers still pick up
improvements from core" to "we won't break these drivers when we make
improvements to core in master."

So, unless there's a solid reason why such work needs to happen in the
master branch, I don't see a reason to hold this up for it.  As long
as you're committed to test r200 and i915 when you make said changes,
we can do a feature release on the LTS branch. Users and distros just
shouldn't expect that to be a common thing.

--Jason

On Tue, Mar 23, 2021 at 3:26 AM Ian Romanick  wrote:
>
> I would like to wait a couple more releases to do this.  I have a couple
> things that I've been gradually working on for some of the non-i965
> classic drivers that I'd like to land before they're put out to pasture.
>  I talked to ajax about this a few weeks ago, and he was amenable at the
> time.
>
> I can step up on testing at least r200 to make sure core changes aren't
> making things explode.  That should also cover most of the problems that
> could hit i915.  i965 gets good coverage in the Intel CI, so I don't
> think that's as big of a problem.
>
> On 3/22/21 3:15 PM, Dylan Baker wrote:
> > Hi list,
> >
> > We've talked about it a number of times, but I think it's time time to
> > discuss splitting the classic drivers off of the main development branch
> > again, although this time I have a concrete plan for how this would
> > work.
> >
> > First, why? Basically, all of the classic drivers are in maintanence
> > mode (even i965). Second, many of them rely on code that no one works
> > on, and very few people still understand. There is no CI for most of
> > them, and the Intel CI is not integrated with gitlab, so it's easy to
> > unintentionally break them, and this breakage usually isn't noticed
> > until just before or just after a release. 21.0 was held up (in small
> > part, also me just getting behind) because of such breakages.
> >
> > I konw there is some interest in getting i915g in good enough shape that
> > it could replace i915c, at least for the common case. I also am aware
> > that Dave, Ilia, and Eric (with some pointers from Ken) have been
> > working on a gallium driver to replace i965. Neither of those things are
> > ready yet, but I've taken them into account.
> >
> > Here's the plan:
> >
> > 1) 21.1 release happens
> > 2) we remove classic from master
> > 3) 21.1 reaches EOL because of 21.2
> > 4) we fork the 21.1 branch into a "classic-lts"¹ branch
> > 5) we disable all vulkan and gallium drivers in said branch, at least at
> >the Meson level
> > 6) We change the name and precidence of the glvnd loader file
> > 7) apply any build fixups (turn of intel generators for versions >= 7.5,
> >for example
> > 8) maintain that branch with build and critical bug fixes only
> >
> > This gives ditros and end users two options.
> > 1) then can build *only* the legacy branch in the a normal Mesa provides
> >libGL interfaces fashion
> > 2) They can use glvnd and install current mesa and the legacy branch in
> >parallel
> >
> > Because of glvnd, we can control which driver will get loaded first, and
> > thus if we decide i915g or the i965 replacement is ready and turn it on
> > by default it will be loaded by default. An end user who doesn't like
> > this can add a new glvnd loader file that makes the classic drivers
> > higher precident and continue to use them.
> >
> > Why fork from 21.1 instead of master?
> >
> > First, it allows us to delete classic immediately, which will allow
> > refactoring to happen earlier in the cycle, and for any fallout to be
> > caught and hopefully fixed before the release. Second, it means that
> > when a user is switched from 21.1 to the new classic-lts branch, there
> > will be no regressions, and no one has to spend time figuring out what
> > broke and fixing the lts branch.
> >
> > When you say "build and critical bug fixes", what do you mean?
> >
> > I mean update Meson if we rely on something that in the future is
> > deprecated and removed, and would prevent building the branch or an
> > relying on some compiler behavior that changes, gaping exploitable
> > security holes, that kind of thing.
> >
> > footnotes
> > ¹Or whatever color you like your bikeshed
> >
> >
> > ___
> > mesa-dev mailing list
> > 

Re: [Mesa-dev] Rename "master" branch to "main"?

[Jason Ekstrand]

On Thu, Mar 25, 2021 at 9:41 AM Rob Clark  wrote:
>
> On Wed, Mar 24, 2021 at 9:15 PM Jason Ekstrand  wrote:
> >
> > On March 24, 2021 22:25:10 Rob Clark  wrote:
> >
> >> On Wed, Mar 24, 2021 at 3:52 PM Jordan Justen  
> >> wrote:
> >>>
> >>>
> >>> On 2021-03-23 09:38:59, Eric Anholt wrote:
> >>>>
> >>>> On Tue, Mar 23, 2021 at 7:02 AM Jason Ekstrand  
> >>>> wrote:
> >>>>>
> >>>>>
> >>>>> Trying to pick this discussion back up.  Daniel Stone thinks it's a
> >>>>> half hour of API bashing to retarget all the MRs so, if the fd.o
> >>>>> admins have some heads up, it should be tractable.  Should we do this
> >>>>> right after branching 21.1 along with the LTS branch?
> >>>>
> >>>>
> >>>> +1
> >>>
> >>>
> >>> Jason,
> >>>
> >>> I opened a related Mesa issue:
> >>> https://gitlab.freedesktop.org/mesa/mesa/-/issues/4501
> >>>
> >>> I made this change in crucible, and used a script to update the 7 MR
> >>> target branches:
> >>> https://gitlab.freedesktop.org/mesa/crucible/-/issues/5
> >>>
> >>> As mentioned in the Mesa issue, I think we should use piglit as
> >>> another test run before changing Mesa:
> >>> https://gitlab.freedesktop.org/mesa/piglit/-/issues/50
> >>>
> >>> Piglit currently has 60 open merge requests.
> >>
> >>
> >> I'm in favor of branch rename, but was in the camp of "hope gitlab
> >> comes up with a way to make this easy for us".. but as far as fallback
> >> plan, converting trees with fewer outstanding MRs first seems like a
> >> pretty good idea so solid +1 for that
> >
> >
> > If you read enough off the things, you'll see that Jordan wrote a python 
> > script that re-targets all the open MRs so that's not a manual process. 
> > It's not a GitLab-sanctioned solution but it's the next best thing. The one 
> > downside is that all the MRs will get their last updated timestamp reset 
> > but that seems like a pretty small price to pay.
> >
>
> yeah, I meant to convert a repo w/ more MRs than 7 but less than mesa
> using the script to beta test that, wasn't suggesting to do it by hand

Plan is to convert piglit next.  It's sitting at about 60 open MRs.

--Jason
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Rename "master" branch to "main"?

[Jason Ekstrand]

On March 24, 2021 22:25:10 Rob Clark  wrote:

On Wed, Mar 24, 2021 at 3:52 PM Jordan Justen  
wrote:


On 2021-03-23 09:38:59, Eric Anholt wrote:

On Tue, Mar 23, 2021 at 7:02 AM Jason Ekstrand  wrote:


Trying to pick this discussion back up.  Daniel Stone thinks it's a
half hour of API bashing to retarget all the MRs so, if the fd.o
admins have some heads up, it should be tractable.  Should we do this
right after branching 21.1 along with the LTS branch?


+1


Jason,

I opened a related Mesa issue:
https://gitlab.freedesktop.org/mesa/mesa/-/issues/4501

I made this change in crucible, and used a script to update the 7 MR
target branches:
https://gitlab.freedesktop.org/mesa/crucible/-/issues/5

As mentioned in the Mesa issue, I think we should use piglit as
another test run before changing Mesa:
https://gitlab.freedesktop.org/mesa/piglit/-/issues/50

Piglit currently has 60 open merge requests.


I'm in favor of branch rename, but was in the camp of "hope gitlab
comes up with a way to make this easy for us".. but as far as fallback
plan, converting trees with fewer outstanding MRs first seems like a
pretty good idea so solid +1 for that


If you read enough off the things, you'll see that Jordan wrote a python 
script that re-targets all the open MRs so that's not a manual process. 
It's not a GitLab-sanctioned solution but it's the next best thing. The one 
downside is that all the MRs will get their last updated timestamp reset 
but that seems like a pretty small price to pay.


Jordan, is there any way you can make the script sort by last updated 
before it re-targets the MRs so they at least stay in the same order? 
Updating them in MR # order wouldn't be bad either, I guess.


--Jason
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] [RFC] Concrete proposal to split classic

[Jason Ekstrand]

On Wed, Mar 24, 2021 at 10:28 AM Rob Clark  wrote:
>
> On Mon, Mar 22, 2021 at 3:15 PM Dylan Baker  wrote:
> >
> > Hi list,
> >
> > We've talked about it a number of times, but I think it's time time to
> > discuss splitting the classic drivers off of the main development branch
> > again, although this time I have a concrete plan for how this would
> > work.
> >
> > First, why? Basically, all of the classic drivers are in maintanence
> > mode (even i965). Second, many of them rely on code that no one works
> > on, and very few people still understand. There is no CI for most of
> > them, and the Intel CI is not integrated with gitlab, so it's easy to
> > unintentionally break them, and this breakage usually isn't noticed
> > until just before or just after a release. 21.0 was held up (in small
> > part, also me just getting behind) because of such breakages.
> >
> > I konw there is some interest in getting i915g in good enough shape that
> > it could replace i915c, at least for the common case. I also am aware
> > that Dave, Ilia, and Eric (with some pointers from Ken) have been
> > working on a gallium driver to replace i965. Neither of those things are
> > ready yet, but I've taken them into account.
> >
> > Here's the plan:
> >
> > 1) 21.1 release happens
> > 2) we remove classic from master
> > 3) 21.1 reaches EOL because of 21.2
> > 4) we fork the 21.1 branch into a "classic-lts"¹ branch
> > 5) we disable all vulkan and gallium drivers in said branch, at least at
> >the Meson level
>
> I'm +1 for the -lts branch.. the layering between mesa "classic" and
> gallium is already starting to get poked thru in the name of
> performance, and we've already discovered cases of classic drivers
> being broken for multiple months with no one noticing.  I think a
> slower moving -lts branch is the best approach to keeping things
> working for folks with older hw.
>
> But possibly there is some value in not completely disabling gallium
> completely in the -lts branch.  We do have some older gallium drivers
> which do not have CI coverage and I think are not used frequently by
> developers who are tracking the latest main/master branch.  I'm not
> suggesting that we remove them from the main (non-lts) branch but it
> might be useful to be able to recommend users of those drivers stick
> with the -lts version for better stability?

I agree with this.  Generally, I don't think we should delete anything
from the -lts branch.  Doing so only risks more breakage.  We probably
want to change some meson build defaults to not build anything but old
drivers but that's it.

--Jason

> BR,
> -R
>
> > 6) We change the name and precidence of the glvnd loader file
> > 7) apply any build fixups (turn of intel generators for versions >= 7.5,
> >for example
> > 8) maintain that branch with build and critical bug fixes only
> >
> > This gives ditros and end users two options.
> > 1) then can build *only* the legacy branch in the a normal Mesa provides
> >libGL interfaces fashion
> > 2) They can use glvnd and install current mesa and the legacy branch in
> >parallel
> >
> > Because of glvnd, we can control which driver will get loaded first, and
> > thus if we decide i915g or the i965 replacement is ready and turn it on
> > by default it will be loaded by default. An end user who doesn't like
> > this can add a new glvnd loader file that makes the classic drivers
> > higher precident and continue to use them.
> >
> > Why fork from 21.1 instead of master?
> >
> > First, it allows us to delete classic immediately, which will allow
> > refactoring to happen earlier in the cycle, and for any fallout to be
> > caught and hopefully fixed before the release. Second, it means that
> > when a user is switched from 21.1 to the new classic-lts branch, there
> > will be no regressions, and no one has to spend time figuring out what
> > broke and fixing the lts branch.
> >
> > When you say "build and critical bug fixes", what do you mean?
> >
> > I mean update Meson if we rely on something that in the future is
> > deprecated and removed, and would prevent building the branch or an
> > relying on some compiler behavior that changes, gaping exploitable
> > security holes, that kind of thing.
> >
> > footnotes
> > ¹Or whatever color you like your 
> > bikeshed___
> > mesa-dev mailing list
> > mesa-dev@lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/mesa-dev
> ___
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] [PATCH 3/3] dma-buf: Add an API for exporting sync files (v8)

[Jason Ekstrand]

On Tue, Mar 23, 2021 at 2:06 PM Simon Ser  wrote:
>
> From a user-space point-of-view, this looks super useful! The uAPI sounds
> good to me.
>
> Acked-by: Simon Ser 
>
> Would be nice to have some short docs as well. Here's an example of a
> patch adding some docs for an ioctl [1], if you aren't familiar with
> that. I think just some basic stuff (description, which parameters are
> in/out, what the flags are) would be great.

Good call.  v9 will have docs.

--Jason
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] [PATCH 3/3] dma-buf: Add an API for exporting sync files (v8)

[Jason Ekstrand]

Adding mesa-dev and wayland-devel for broader circulation.

On Wed, Mar 17, 2021 at 5:19 PM Jason Ekstrand  wrote:
>
> Modern userspace APIs like Vulkan are built on an explicit
> synchronization model.  This doesn't always play nicely with the
> implicit synchronization used in the kernel and assumed by X11 and
> Wayland.  The client -> compositor half of the synchronization isn't too
> bad, at least on intel, because we can control whether or not i915
> synchronizes on the buffer and whether or not it's considered written.
>
> The harder part is the compositor -> client synchronization when we get
> the buffer back from the compositor.  We're required to be able to
> provide the client with a VkSemaphore and VkFence representing the point
> in time where the window system (compositor and/or display) finished
> using the buffer.  With current APIs, it's very hard to do this in such
> a way that we don't get confused by the Vulkan driver's access of the
> buffer.  In particular, once we tell the kernel that we're rendering to
> the buffer again, any CPU waits on the buffer or GPU dependencies will
> wait on some of the client rendering and not just the compositor.
>
> This new IOCTL solves this problem by allowing us to get a snapshot of
> the implicit synchronization state of a given dma-buf in the form of a
> sync file.  It's effectively the same as a poll() or I915_GEM_WAIT only,
> instead of CPU waiting directly, it encapsulates the wait operation, at
> the current moment in time, in a sync_file so we can check/wait on it
> later.  As long as the Vulkan driver does the sync_file export from the
> dma-buf before we re-introduce it for rendering, it will only contain
> fences from the compositor or display.  This allows to accurately turn
> it into a VkFence or VkSemaphore without any over- synchronization.
>
> v2 (Jason Ekstrand):
>  - Use a wrapper dma_fence_array of all fences including the new one
>when importing an exclusive fence.
>
> v3 (Jason Ekstrand):
>  - Lock around setting shared fences as well as exclusive
>  - Mark SIGNAL_SYNC_FILE as a read-write ioctl.
>  - Initialize ret to 0 in dma_buf_wait_sync_file
>
> v4 (Jason Ekstrand):
>  - Use the new dma_resv_get_singleton helper
>
> v5 (Jason Ekstrand):
>  - Rename the IOCTLs to import/export rather than wait/signal
>  - Drop the WRITE flag and always get/set the exclusive fence
>
> v6 (Jason Ekstrand):
>  - Drop the sync_file import as it was all-around sketchy and not nearly
>as useful as import.
>  - Re-introduce READ/WRITE flag support for export
>  - Rework the commit message
>
> v7 (Jason Ekstrand):
>  - Require at least one sync flag
>  - Fix a refcounting bug: dma_resv_get_excl() doesn't take a reference
>  - Use _rcu helpers since we're accessing the dma_resv read-only
>
> v8 (Jason Ekstrand):
>  - Return -ENOMEM if the sync_file_create fails
>  - Predicate support on IS_ENABLED(CONFIG_SYNC_FILE)
>
> Signed-off-by: Jason Ekstrand 
> ---
>  drivers/dma-buf/dma-buf.c| 62 
>  include/uapi/linux/dma-buf.h |  6 
>  2 files changed, 68 insertions(+)
>
> diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
> index f264b70c383eb..a5e4b0b6a049c 100644
> --- a/drivers/dma-buf/dma-buf.c
> +++ b/drivers/dma-buf/dma-buf.c
> @@ -20,6 +20,7 @@
>  #include 
>  #include 
>  #include 
> +#include 
>  #include 
>  #include 
>  #include 
> @@ -362,6 +363,62 @@ static long dma_buf_set_name(struct dma_buf *dmabuf, 
> const char __user *buf)
> return ret;
>  }
>
> +#if IS_ENABLED(CONFIG_SYNC_FILE)
> +static long dma_buf_export_sync_file(struct dma_buf *dmabuf,
> +void __user *user_data)
> +{
> +   struct dma_buf_sync_file arg;
> +   struct dma_fence *fence = NULL;
> +   struct sync_file *sync_file;
> +   int fd, ret;
> +
> +   if (copy_from_user(, user_data, sizeof(arg)))
> +   return -EFAULT;
> +
> +   if (arg.flags & ~DMA_BUF_SYNC_RW)
> +   return -EINVAL;
> +
> +   if ((arg.flags & DMA_BUF_SYNC_RW) == 0)
> +   return -EINVAL;
> +
> +   fd = get_unused_fd_flags(O_CLOEXEC);
> +   if (fd < 0)
> +   return fd;
> +
> +   if (arg.flags & DMA_BUF_SYNC_WRITE) {
> +   ret = dma_resv_get_singleton_rcu(dmabuf->resv, );
> +   if (ret)
> +   goto err_put_fd;
> +   } else if (arg.flags & DMA_BUF_SYNC_READ) {
> +   fence = dma_resv_get_excl_rcu(dmabuf->resv);
> +   }
> +
> +   if (!fence)
> +   fence = dma_fenc

Re: [Mesa-dev] [RFC] Concrete proposal to split classic

[Jason Ekstrand]

On Tue, Mar 23, 2021 at 8:39 AM Kenneth Graunke  wrote:
>
> On Tuesday, March 23, 2021 6:28:23 AM PDT Jason Ekstrand wrote:
> > On March 23, 2021 01:46:54 Kenneth Graunke  wrote:
> [snip]
> > > One extra thought: can we also fork off anv Gen7.x support at the same
> > > time?  If distros are already going to be building i965 for Gen7.x from
> > > that branch, building Vulkan from there should be easy as well.
> >
> > I'm not sure how I feel about that one. Here are some thoughts:
> >
> >  1. I'd love to have it out of the way
> >  2. Unlike i965, it is still picking up features. Part of what makes
> > dropping i965 a reasonable idea is that OpenGL is also in maintenance mode.
> > Vulkan isn't.
> >  3. Generally, things are architected well enough that relocations aren't a
> > problem.
> >  4. Being able to always do batch chaining would be nice.
> >  5. The only new feature still in development for i965 is
> > GL_EXT_external_objects. That would be more reliable if Gen7 ANV and i965
> > were in the same branch.
> >  6. If crocus gets GL_EXT_external_objects, it'll be better if Gen7 ANV is
> > in the master branch.
> >  7. I'd love to stop caring about vec4.
>
> Point 2 is true.  I'm not sure I agree about 3, the anv execbuf handling
> is a lot more complicated than I think it needs to be.

It's not great, I'll grant.  But it's not too awful and, until we get
rid of VM_BIND, we have to at least keep BO usage tracking even if we
were able to drop relocations.

> It would be really nice to cull vec4 and MRF support from the compiler
> as well---and all of the vec4/fs code sharing and base classes.  That
> would be really nice.  But, I guess that if crocus comes into existence,
> then we need all that again.  That's unfortunate :(

Me too.  There is some stuff I think we can drop.  We can drop
shader_time, for instance, as well as the param array and all the push
constant re-arranging code.

And, yeah, I'd love to drop vec4 but yeah...  One advantage to keeping
vec4 in the tree for some stuff is that it means we have full-featured
hardware able to test vec4 NIR.  That seems like a feature.

--Jason
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Rename "master" branch to "main"?

[Jason Ekstrand]

Trying to pick this discussion back up.  Daniel Stone thinks it's a
half hour of API bashing to retarget all the MRs so, if the fd.o
admins have some heads up, it should be tractable.  Should we do this
right after branching 21.1 along with the LTS branch?

--Jason

On Fri, Aug 7, 2020 at 3:38 AM Timur Kristóf  wrote:
>
> On Thu, 2020-08-06 at 14:17 +0200, Eric Engestrom wrote:
> >
> > There is an upstream issue about having gitlab handle the branch
> > renaming and provide redirections, MR re-targeting, etc.
> >
> > https://gitlab.com/gitlab-org/gitlab/-/issues/233427
> >
> > If we wait for this feature instead of doing it by hand, it could be
> > much less disruptive to devs and everyone downstream from us, but
> > there's also no telling how long this will take.
>
> Sounds like this would save us a lot of hassle. If we are not in a
> hurry, I think the best option for us is to wait for this upstream
> issue.
>
>
>
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] [RFC] Concrete proposal to split classic

[Jason Ekstrand]

On March 23, 2021 01:46:54 Kenneth Graunke  wrote:


On Monday, March 22, 2021 3:15:30 PM PDT Dylan Baker wrote:

Hi list,

We've talked about it a number of times, but I think it's time time to
discuss splitting the classic drivers off of the main development branch
again, although this time I have a concrete plan for how this would
work.

First, why? Basically, all of the classic drivers are in maintanence
mode (even i965). Second, many of them rely on code that no one works
on, and very few people still understand. There is no CI for most of
them, and the Intel CI is not integrated with gitlab, so it's easy to
unintentionally break them, and this breakage usually isn't noticed
until just before or just after a release. 21.0 was held up (in small
part, also me just getting behind) because of such breakages.

I konw there is some interest in getting i915g in good enough shape that
it could replace i915c, at least for the common case. I also am aware
that Dave, Ilia, and Eric (with some pointers from Ken) have been
working on a gallium driver to replace i965. Neither of those things are
ready yet, but I've taken them into account.

Here's the plan:

1) 21.1 release happens
2) we remove classic from master
3) 21.1 reaches EOL because of 21.2
4) we fork the 21.1 branch into a "classic-lts"¹ branch
5) we disable all vulkan and gallium drivers in said branch, at least at
the Meson level
6) We change the name and precidence of the glvnd loader file
7) apply any build fixups (turn of intel generators for versions >= 7.5,
for example
8) maintain that branch with build and critical bug fixes only

This gives ditros and end users two options.
1) then can build *only* the legacy branch in the a normal Mesa provides
libGL interfaces fashion
2) They can use glvnd and install current mesa and the legacy branch in
parallel

Because of glvnd, we can control which driver will get loaded first, and
thus if we decide i915g or the i965 replacement is ready and turn it on
by default it will be loaded by default. An end user who doesn't like
this can add a new glvnd loader file that makes the classic drivers
higher precident and continue to use them.

Why fork from 21.1 instead of master?

First, it allows us to delete classic immediately, which will allow
refactoring to happen earlier in the cycle, and for any fallout to be
caught and hopefully fixed before the release. Second, it means that
when a user is switched from 21.1 to the new classic-lts branch, there
will be no regressions, and no one has to spend time figuring out what
broke and fixing the lts branch.

When you say "build and critical bug fixes", what do you mean?

I mean update Meson if we rely on something that in the future is
deprecated and removed, and would prevent building the branch or an
relying on some compiler behavior that changes, gaping exploitable
security holes, that kind of thing.

footnotes
¹Or whatever color you like your bikeshed


Hi Dylan,

I largely like this plan.  i915 and r200 and novueau_vieux definitely
need to be forked off, to keep them working as we do core refactors.

In the past, we weren't working on core much, and so they largely kept
working with some pain, but not -too- much effort.  But these days,
we're seeing a lot of work from Marek and others to clean up and rework
a lot of core drawing code, state upload code, and so on.  I remember
all the work Tim did to rework uniform handling and ancient classic was
a pain point for him as well.  I had to track down like 5 overlapping
Piglit regressions on i915 this release cycle, just to get the driver
working...no worse than it was before.  And i915 is tested regularly
in the Intel CI.  r200 isn't tested regularly.  Doubtful about vieux.

I had thought about also forking other old drivers like i915g or r300g.
But it sounds like there's still some interest in i915g, and those
tend to get fixed up as the Gallium infrastructure is still actively
being maintained (unlike things like swrast and tnl).  So I guess we
can leave those in mainline.

I'm hesitant about i965.  One thing I will say is that, if i965 is
included in the plan, there would probably be more interest in working
on that branch than mere "critical bug fixes" as you defined.  We might
discover new games that don't work; people might write bug fixes for
i965, at which point we should merge and ship them.  But we could still
do on-demand releases when enough interesting bug fixes have piled up,
rather than doing them on a schedule.  That should hopefully not be
too burdensome.

While a part of me hates the idea of forking i965 off, I think it may
actually be the best call.  We haven't done any new interesting feature
development on those platforms in ages, and they're complete in terms of
OpenGL spec support.  We aren't spending any time optimizing performance
on those platforms.  They're pretty much in bug-fix-only mode already.
Whether that bug fixing happens on master or on a "classic-lts" branch
doesn't matter; it 

Re: [Mesa-dev] [RFC] Concrete proposal to split classic

[Jason Ekstrand]

+1. I'd we think GLVND and X are ready for this, I think it's a good plan.

On March 22, 2021 17:34:09 Eric Anholt  wrote:


On Mon, Mar 22, 2021 at 3:27 PM Dylan Baker  wrote:


Hi list,

We've talked about it a number of times, but I think it's time time to
discuss splitting the classic drivers off of the main development branch
again, although this time I have a concrete plan for how this would
work.

First, why? Basically, all of the classic drivers are in maintanence
mode (even i965). Second, many of them rely on code that no one works
on, and very few people still understand. There is no CI for most of
them, and the Intel CI is not integrated with gitlab, so it's easy to
unintentionally break them, and this breakage usually isn't noticed
until just before or just after a release. 21.0 was held up (in small
part, also me just getting behind) because of such breakages.

I konw there is some interest in getting i915g in good enough shape that
it could replace i915c, at least for the common case. I also am aware
that Dave, Ilia, and Eric (with some pointers from Ken) have been
working on a gallium driver to replace i965. Neither of those things are
ready yet, but I've taken them into account.

Here's the plan:

1) 21.1 release happens
2) we remove classic from master
3) 21.1 reaches EOL because of 21.2
4) we fork the 21.1 branch into a "classic-lts"¹ branch
5) we disable all vulkan and gallium drivers in said branch, at least at
the Meson level
6) We change the name and precidence of the glvnd loader file
7) apply any build fixups (turn of intel generators for versions >= 7.5,
for example
8) maintain that branch with build and critical bug fixes only


I would like it if Intel could avoid garbage-collecting older-HW
shared code for at least a release due aforementioned WIPs, but I
think it's time to call the classic i965_dri.so and i915_dri.so done.


I'm happy to leave older hardware support in the tree for now. We need to 
keep the vec4 compiler for HSW Vulkan support for now and there's no harm 
in keeping old hardware support in ISL. I'm a little tempted to let HSW 
Vulkan bitrot in the LTS branch too but it does still pick up features here 
and there so I'm unsure if that's a good idea or not.


--Jason



+1 from me assuming that we validate that one can actually get a
working X server with the mesa-legacy set installed.
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] docs: consistent language

[Jason Ekstrand]

On March 16, 2021 05:06:53 Daniel Stone  wrote:

On Mon, 15 Mar 2021 at 20:54, Jason Ekstrand  wrote:

Three comments:

1. +1
2. Khronos has generally standardized on American English in their
specs so I guess that provides some sort of prior art or something.
3. I'd generally be a fan of encouraging American spellings in common
code as well.  As someone who doesn't use auto-complete, having to
remember which variable is named "color" and which is "colour" would
get painful.  Fortunately, we don't have a whole lot of variation here
but it'd be good to keep consistency.

4. UK English is unarguably correct and better.




5. That doesn't change point #3 in particular; just like coding style, one 
'bad' choice is better than no choice at all.


I have three words for you: three space tabs.  Yeah... It may be a good 
style but consistency is good. 浪


--Jason
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] docs: consistent language

[Jason Ekstrand]

Three comments:

 1. +1
 2. Khronos has generally standardized on American English in their
specs so I guess that provides some sort of prior art or something.
 3. I'd generally be a fan of encouraging American spellings in common
code as well.  As someone who doesn't use auto-complete, having to
remember which variable is named "color" and which is "colour" would
get painful.  Fortunately, we don't have a whole lot of variation here
but it'd be good to keep consistency.

--Jason

On Mon, Mar 15, 2021 at 12:59 PM Ian Romanick  wrote:
>
> On 3/15/21 5:44 AM, Erik Faye-Lund wrote:
> > TLDR; I'm proposing to standardize on US English in our public-facing
> > documentation.
> >
> > I proposed an MR a while back that changed the only occurrence of the
> > UK English spelling "optimisation" for the US English spelling
> > "optimization", which is already used 34 times in the docs. I've done
> > similar changes in the past.
> >
> > But this time Ian brought up the good point that picking a preferred
> > language should probably be done using group consensus on the mailing
> > list than just picking what's currently most popular.
>
> Over the years we've had quite a few contributions from folks in many
> different English spelling parts of the world where the UK spelling is
> the norm... though I don't think there have been that many from the UK
> itself. :)  I suggested getting some group consensus because I didn't
> want any of those people to feel left out or undervalued.
>
> If anyone doesn't feel comfortable speaking out publicly about this,
> please feel free to contact Erik or me privately.
>
> > So, I hereby propose to pick US English as our default language for
> > user-facing documentation.
> >
> > I'm not proposing to force anyone to use a particular English variant
> > for things like comments, commit messages or variable names. I don't
> > think there's much value in enforcing that, and I think it'd be
> > perfectly fine if any particular driver wanted to pick a particular
> > variant of English.
> >
> > The main reason I want to standardize on an English variant is that I'm
> > trying create a word-list for domain-specific/technical language to
> > detect spelling mistakes in the docs more easily. And not having to add
> > both US and UK English words makes this list easier to maintain. I'm
> > not planning on going 100% systematically though the dictionary to
> > detect UK English words that might be in my system dictionary, just to
> > fix the words that currently cause me (a tiny amount of) pain ;)
> >
> > The main reason why I'm proposing US English over for instance UK
> > English is that this seems to be the dominant variant currently in the
> > documentation. So it seems like the pragmatic choice to me.
> >
> > Thoughts? Any objections to sticking to US English in the docs?
> >
> > The MR in question:
> > https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8572
> >
> > Ian's response:
> > https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/8572#note_808593
> >
> > Previous changes:
> > https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6864
> > https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/6894
> >
> >
> > ___
> > mesa-dev mailing list
> > mesa-dev@lists.freedesktop.org
> > https://lists.freedesktop.org/mailman/listinfo/mesa-dev
> ___
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] libGL not build but opengl=true option is set

[Jason Ekstrand]

On March 13, 2021 04:18:26 Quentin SCHIBLER  wrote:

GLVND depends on several X librairies. Does it means you cannot have OpenGL 
on wayland without X ?


Yes and no. libGL.so on Linux depends on X11 for historical reasons. The 
short version is that, long ago, libGL.so exposed all its symbols directly, 
including GLX symbols, and a dependency on X11. We've since learned better 
and any new/recentish entrypoints are only exposed via glGetProcAddress. 
Unfortunately, we can't really clean up they mess without breaking 
backwards compatibility.


With GLVND, we did do the next best thing. There is now a new libOpenGL.so 
which has a much reduced set of symbols and, IIRC, no X11 dependency. This 
means that it is possible to run desktop OpenGL without pulling in an X11 
dependency, assuming the app is built to link against the new library.


More importantly than all that, however, is that your really shouldn't 
worry about the X11 dependency. If you have a pure Wayland app which uses 
OpenGL, it'll work fine. It might load Xlib or XCB into it's address space 
but that code will never be executed. The above are only really issues for 
people who are desperate to run a system without any X11 installed which, 
IMO, is a pretty pointless goal.


--Jason



Onn Mar 12, 2021 11:48 AM, Michel Dänzer  wrote:


On 2021-03-11 12:14 a.m., Quentin SCHIBLER wrote:
I have built mesa with wayland platform, gbm and egl enabled, glx disabled, 
gles1 disabled and gles2 enabled.
The build is successful, but I cannot find libGL.so. libGLES2.so is 
present, and GL includes files are also present.


libGL includes GLX APIs, so it cannot be built with -Dglx=false.

If you want to avoid GLX, you can use GLVND's libOpenGL instead.


--
Earthling Michel Dänzer   |   https://redhat.com
Libre software enthusiast | Mesa and X developer

___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev


___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] NIR Debugging Tips

[Jason Ekstrand]

Hi Tommy,

As you've probably noticed already, Mesa documentation is sparse at
best.  For NIR, the best places to go are:

 1. For ALU ops, nir_opcodes.py defines the opcodes as well as a C
snippet which describes the opcode's semantics
 2. For all intrinsics, nir_intrinsics.py defines them and typically
has some sort of description for each.  How good it is very much
depends on the op.

> Also, what is the debug strategy that is used to debug NIR shaders to check 
> if the implementation is correct?

What do you mean?  Debugging back-end drivers?  That's typically done
by running piglit (OpenGL test suite) or CTS tests from one of the
Khronos test suites.

Debugging NIR passes?  Typically the same on your favorite hardware.
The Mesa GitLab CI system will test on a pretty broad spread of
hardware so you can use that once you have it working locally.

--Jason

On Thu, Feb 25, 2021 at 3:39 AM Tommy Chou  wrote:
>
> Hi,
>
> Could I get some tips on figuring out what the NIR intrinsics do, 
> specifically the vulkan related ones? Also, what is the debug strategy that 
> is used to debug NIR shaders to check if the implementation is correct?
>
> Thanks,
> Tommy
> ___
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] lavapipe vulkan driver win32 port: call for help

[Jason Ekstrand]

Drive-by comment:  I'm afraid getting window-system stuff working
isn't "the last push".  It's probably 50-80% of the project.  I have
no idea how any of this works for LLVMpipe; maybe we can steal some
code from there?  In any case, you're unlikely to find a lot of win32
experts here.

--Jason

On Thu, Feb 11, 2021 at 3:09 AM Michel Zou  wrote:
>
> hello,
>
> I started to work on the software vulkan driver (lavapipe) win32 port:
> https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7208
> up to the point it builds and "runs" but nothing is displayed with the vkcube 
> example (from Vulkan-Tools).
> It is probably an issue with the buffer not being properly copied or 
> something (I'm not really good with these windows apis)
>
> I dont know if I'll have time to work on it anymore.
> I'm looking for someone willing to help to give it the last push.
>
> Just in case, here is how to enable it:
> meson build -Dgallium-drivers=swrast -Dvulkan-drivers=swrast
> So far I only used mingw, not msvc.
>
>
>
>
>
>
> ___
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Developer access

[Jason Ekstrand]

To help smooth things along, I've created a new label "Project Access
Requests" for use with these things.  I've subscribed to it myself and
would encourage the other maintainers to do so as well.  That way we
see these issues instead of them getting lost in the pile.

--Jason

On Thu, Jan 21, 2021 at 11:07 AM Mike Blumenkrantz
 wrote:
>
> +1 always in favor of getting more people into the review pipeline
>
> On Thu, Jan 21, 2021 at 10:05 AM Alyssa Rosenzweig 
>  wrote:
>>
>> Hi all,
>>
>> Icecream95[1], a long-time Mesa/Panfrost contributor, has requested
>> developer access to mesa on the GitLab issue tracker [2]. Quoting here
>> for convenience of those who monitor the mailing list but not the
>> tracker:
>>
>> > @alyssa keeps complaining about getting poked to assign stuff to Marge
>> > for me, so I think it's finally time to ask for Mesa developer access,
>> > so I can do the endless retries until CI stops flaking myself.
>> >
>> > Assuming a platform with char as unsigned, I think my 210 patches in
>> > Mesa is more than the 32 commits some have suggested to be required
>> > for the lofty privilege of labelling MRs and bothering the merge bot.
>>
>> All else being equal, with my Panfrost maintainer hat, I am in favour.
>> Marge Bot access would reduce merge friction for their frequent
>> contributions, given the substantial timezone difference with our team
>> at Collabora. Additionally, labeling access would ease triage for us;
>> although I subscribe specifically to Panfrost MRs, I have to monitor all
>> MRs to ensure I haven't missed any from them, a frequent occurrence.
>>
>> [ The cynic reads: This is more for my benefit than theirs ;-) ]
>>
>> Thanks,
>>
>> Alyssa
>>
>> [1] https://gitlab.freedesktop.org/icecream95
>> [2] https://gitlab.freedesktop.org/mesa/mesa/-/issues/4135
>> ___
>> mesa-dev mailing list
>> mesa-dev@lists.freedesktop.org
>> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
>
> ___
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

[Mesa-dev] Mesa main repor branch clean-up

[Jason Ekstrand]

All,

At suggestion from several people on IRC, I've done a bit of house
cleaning of the main Mesa repo.  I created a new mesa/mesa-archive
repo and moved all of the stale feature branches to that repo and
removed them from mesa/mesa.  Many of those branches haven't seen a
commit in 10-20 years so all they're doing for most mesa devs is
adding clutter.

I left all the release branches since 7.8.  Dylan suggested I get rid
of those too but it seems like a good idea to leave them for archival
purposes.  If others think they should be removed, I'm happy to do
that.

If someone objects to moving stale branches to an archive repo, feel
free to say so and I can always put them back.

--Jason
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] fp/int64 on gen11/12?

[Jason Ekstrand]

On Tue, Nov 17, 2020 at 6:10 PM Jordan Justen  wrote:
>
> On 2020-11-17 16:03:31, Brian Paul wrote:
> > Another Intel question:  It looks like gen11/gen12 don't have fp/int64
> > enabled in the Vulkan driver. From gen_device_info.c:
> >
> > #define GEN11_FEATURES(_gt, _slices, _subslices, _l3) \
> > GEN8_FEATURES, \
> > GEN11_HW_INFO, \
> > .has_64bit_float = false,  \
> > .has_64bit_int = false,
> > ...
> >
> > #define GEN12_FEATURES(_gt, _slices, _l3)   \
> > GEN8_FEATURES,   \
> > GEN12_HW_INFO,   \
> > .has_64bit_float = false,\
> > .has_64bit_int = false,
> >
> > But gen8/9 do support it.  Is this a driver and/or hardware issue?
> >
>
> It matches the hardware.

That ^^

Expounding a bit, we do have emulated int64 and fp64 for OpenGL.  I've
got an MR open (!7329) to provide emulated int64 support in Vulkan.
We have no plans for emulated fp64 in Vulkan.  It's so painful that it
really doesn't play well with the Vulkan philosophy of "expose what's
in hardware".  Also, the lowering code we have in Mesa today really
requires the GLSL compiler.

--Jason
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] llvmpipe is OpenGL 4.5 conformant.

[Jason Ekstrand]

Good work!

On Fri, Oct 30, 2020 at 3:24 PM Dave Airlie  wrote:
>
> Just to let everyone know, a month ago I submitted the 20.2 llvmpipe
> driver for OpenGL 4.5 conformance under the SPI/X.org umbrella, and it
> is now official[1].
>
> Thanks to everyone who helped me drive this forward, and to all the
> contributors both to llvmpipe and the general Mesa stack that enabled
> this.
>
> Big shout out to Roland Scheidegger for helping review the mountain of
> patches I produced in this effort.
>
> My next plans involved submitting lavapipe for Vulkan 1.0, it's at 99%
> or so CTS, but there are line drawing, sampler accuracy and some snorm
> blending failure I have to work out.
> I also ran the OpenCL 3.0 conformance suite against clover/llvmpipe
> yesterday and have some vague hopes of driving that to some sort of
> completion.
>
> (for GL 4.6 only texture anisotropy is really missing, I've got
> patches for SPIR-V support, in case someone was feeling adventurous).
>
> Dave.
>
> [1] 
> https://www.khronos.org/conformance/adopters/conformant-products/opengl#submission_272
> ___
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Landing ray-tracing support in ANV

[Jason Ekstrand]

Quick clarification about VK_NV_ray_tracing:  I have no intention of
supporting VK_NV_ray_tracing in ANV.  If new games want cross-vendor
ray-tracing, they should use the KHR extension.  However, there are
some games which have already shipped which I'm sure Linux gamers
would like to be able to play with the shiney new features.  I'd like
to support them somehow but it likely won't be directly in the driver.
Making a layer sounds like a fantastic GSOC project.

--Jason

On Wed, Oct 28, 2020 at 5:26 PM Jason Ekstrand  wrote:
>
> All,
>
> Some of you may be curious about what I've been up to for most of
> 2020, why I've not been working on IBC, and why I suddenly decided to
> start caring about ray-tracing.  Well... it's all because I've been
> working on this little project to implement VK_KHR_ray_tracing in ANV.
> I finally have the approval to start talking publicly about the Intel
> ray-tracing implementation and can start landing code upstream.
>
> To kick things off, I've posted a merge request which contains the
> bulk of the compiler work required to transform ray-tracing shaders
> into Intel's upcoming bindless dispatch model and compile them:
>
> https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7356
>
> This is only the first of what will likely be many MRs.  I've also
> created a tracking issue which will be used to help link it all
> together and keep me sane.  You can find it here:
>
> https://gitlab.freedesktop.org/mesa/mesa/-/issues/3709
>
> Apart from the core ray-tracing compiler bits, my next focus for
> upstream will be landing more OpenCL support patches.  In order to do
> GPU-side BVH building, I've got a pile of kernels written in OpenCL C
> that we need to be able to compile with our stack.  In order for those
> to work, we need a few more things:
>
>  - Generic pointer support (!6332)
>  - SPV_INTEL_subgroups (!7145)
>  - ANV support for compiling and dispatching OpenCL kernels (coming soon)
>  - Misc NIR clean-ups and fixes.
>
> The API bits for ray-tracing will be coming later.  I've got most of
> the API implemented but it's currently all written against the latest
> version of the KHR ray-tracing spec which hasn't been released
> publicly.  Once that spec goes public, I can start dropping patches.
> If the public KHR gets delayed too long and I run out of other bits to
> upstream, I may rebase the API bits on the provisional KHR from
> January or even the NV spec but I'd really rather not do that as it'd
> be a pile of churn in my internal branch for very little benefit.
>
> LAQ (Likely Asked Questions):
>
> Q: Do the patches work?  Have you tested them?
> A: Yes.  Don't worry, I'm not sending a bunch of untested garbage.
> Out of several thousand CTS tests, I've got 3 that are failing due to
> undiagnosed compiler bugs.  I'd say that's pretty good. :-)
>
> Q: Will this work on older Intel GPUs?
> A: No.  While a pure SW emulation on top of compute shaders would be
> possible, this is not it.  This implementation requires new Xe HPG
> hardware that has not yet been released.
>
> Q: Which upcoming Intel GPUs will support ray-tracing and when can I buy one?
> A: We’ve previously disclosed plans to release Xe HPG GPUs with
> dedicated hardware for ray tracing in 2021. I’m not authorized to tell
> you more. Sorry. Please don't ask. It just makes for an awkward
> conversation.
>
> Q: Most of the new compiler code is in src/intel.  Can it be
> generalized for implementing ray-tracing in RADV or other Mesa Vulkan
> drivers?
> A: Yes, likely some of it can.  In particular, there is a shader
> splitting pass (for splitting shader calls into continuations) as well
> as a pass to inline any-hit shaders into intersection shaders.  With a
> little work, both of those could be generalized and pulled into
> src/compiler/nir.  The rest is mostly lowering code to translate to
> Intel's ray-tracing model.  Likely other hardware vendors models will
> be different enough that those passes won't be directly applicable.
>
> Q: What about games that have already shipped ray-tracing support
> using VK_NV_ray_tracing?  Will you support those?
> A: I would eventually like to support any Vulkan games that support
> ray-tracing.  However, it's still a bit TBD just how we'll deal with
> games that use the NV extension.  Ideally, once VK_KHR_ray_tracing
> ships publicly, we would convince the game vendors who have already
> shipped ray-tracing support to push an update that switches their game
> over to the KHR extension.  If that fails, it would be pretty easy to
> write a layer which translates from the NV extension to the KHR one.
>
> That's all for now, folks!  I'll be on IRC and e-mail if you have any
> questions.  I'd love to get so

[Mesa-dev] Landing ray-tracing support in ANV

[Jason Ekstrand]

All,

Some of you may be curious about what I've been up to for most of
2020, why I've not been working on IBC, and why I suddenly decided to
start caring about ray-tracing.  Well... it's all because I've been
working on this little project to implement VK_KHR_ray_tracing in ANV.
I finally have the approval to start talking publicly about the Intel
ray-tracing implementation and can start landing code upstream.

To kick things off, I've posted a merge request which contains the
bulk of the compiler work required to transform ray-tracing shaders
into Intel's upcoming bindless dispatch model and compile them:

https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/7356

This is only the first of what will likely be many MRs.  I've also
created a tracking issue which will be used to help link it all
together and keep me sane.  You can find it here:

https://gitlab.freedesktop.org/mesa/mesa/-/issues/3709

Apart from the core ray-tracing compiler bits, my next focus for
upstream will be landing more OpenCL support patches.  In order to do
GPU-side BVH building, I've got a pile of kernels written in OpenCL C
that we need to be able to compile with our stack.  In order for those
to work, we need a few more things:

 - Generic pointer support (!6332)
 - SPV_INTEL_subgroups (!7145)
 - ANV support for compiling and dispatching OpenCL kernels (coming soon)
 - Misc NIR clean-ups and fixes.

The API bits for ray-tracing will be coming later.  I've got most of
the API implemented but it's currently all written against the latest
version of the KHR ray-tracing spec which hasn't been released
publicly.  Once that spec goes public, I can start dropping patches.
If the public KHR gets delayed too long and I run out of other bits to
upstream, I may rebase the API bits on the provisional KHR from
January or even the NV spec but I'd really rather not do that as it'd
be a pile of churn in my internal branch for very little benefit.

LAQ (Likely Asked Questions):

Q: Do the patches work?  Have you tested them?
A: Yes.  Don't worry, I'm not sending a bunch of untested garbage.
Out of several thousand CTS tests, I've got 3 that are failing due to
undiagnosed compiler bugs.  I'd say that's pretty good. :-)

Q: Will this work on older Intel GPUs?
A: No.  While a pure SW emulation on top of compute shaders would be
possible, this is not it.  This implementation requires new Xe HPG
hardware that has not yet been released.

Q: Which upcoming Intel GPUs will support ray-tracing and when can I buy one?
A: We’ve previously disclosed plans to release Xe HPG GPUs with
dedicated hardware for ray tracing in 2021. I’m not authorized to tell
you more. Sorry. Please don't ask. It just makes for an awkward
conversation.

Q: Most of the new compiler code is in src/intel.  Can it be
generalized for implementing ray-tracing in RADV or other Mesa Vulkan
drivers?
A: Yes, likely some of it can.  In particular, there is a shader
splitting pass (for splitting shader calls into continuations) as well
as a pass to inline any-hit shaders into intersection shaders.  With a
little work, both of those could be generalized and pulled into
src/compiler/nir.  The rest is mostly lowering code to translate to
Intel's ray-tracing model.  Likely other hardware vendors models will
be different enough that those passes won't be directly applicable.

Q: What about games that have already shipped ray-tracing support
using VK_NV_ray_tracing?  Will you support those?
A: I would eventually like to support any Vulkan games that support
ray-tracing.  However, it's still a bit TBD just how we'll deal with
games that use the NV extension.  Ideally, once VK_KHR_ray_tracing
ships publicly, we would convince the game vendors who have already
shipped ray-tracing support to push an update that switches their game
over to the KHR extension.  If that fails, it would be pretty easy to
write a layer which translates from the NV extension to the KHR one.

That's all for now, folks!  I'll be on IRC and e-mail if you have any
questions.  I'd love to get some outside review on the compiler
patches if I could.  The shader splitting pass, in particular, is a
bit on the gnarley side.

--Jason
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Mesa 20.2.x and GL_RG8_SNORM/GL_NONE

[Jason Ekstrand]

Generally, you need to be careful with forcing no_error.  Some apps
rely on gl errors to check for features and other things.
Force-disabling errors may break the app.  Mesa's implementation of
the no_error extension has been a gradual process where people have
been removing the error checking paths one at a time as time permits.
It's entirely possible that an error checking path that the app relies
on got removed from Mesa.  The solution to this is to stop forcing
no_error.

--Jason

On Thu, Oct 15, 2020 at 9:17 PM Daniel Mota Leite  wrote:
>
> >   Since i updated to mesa 20.2.0 and then to 20.2.1, i'm unable to
> > load war thunder game, it now just returns:
> >
> > Unsupported format/type: GL_RG8_SNORM/GL_NONE
>
> The other use found that running as a different user the game run,
> so after some debug, we found that we had the mesa_no_error" option
> enabled, from past tests. With this option disabled, the game runs again
> without any problem.
>
> Still, do not know if the problem was trigger by a game update or
> by the mesa 20.2.x update. The no_error did give a small performance
> increase in the past.
>
> Best regards
> higuita
> --
> Naturally the common people don't want war... but after all it is the
> leaders of a country who determine the policy, and it is always a
> simple matter to drag the people along, whether it is a democracy, or
> a fascist dictatorship, or a parliament, or a communist dictatorship.
> Voice or no voice, the people can always be brought to the bidding of
> the leaders. That is easy. All you have to do is tell them they are
> being attacked, and denounce the pacifists for lack of patriotism and
> exposing the country to danger.  It works the same in every country.
>-- Hermann Goering, Nazi and war criminal, 1883-1946
> ___
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Question on ~/mesa/src/amd/llvm/ac_nir_to_llvm.c visit_load_var

[Jason Ekstrand]

First off, I should point out that the AMD NIR -> LLVM translator is,
as far as I know, the only NIR back-end that consumes variables at
all.  Most back-ends assume that all variable access is completely
lowered away before the back-end ever sees it.  How this is done
depends on the variable mode.

For nir_var_shader_temp, these are typically converted to
nir_var_function_temp by nir_lower_global_vars_to_local after function
inlining has completed.  For nir_var_function_temp, it's some
combination of nir_lower_vars_to_ssa, nir_lower_indirect_derefs, and
nir_lower_locals_to_regs.  If the back-end wants to be able to handle
indirect access (such as with a non-constant array index) directly, it
will typically use nir_lower_locals_to_regs.  If the back-end doesn't
want to handle indirects, it will use nir_lower_indirect_derefs to get
rid of any indirects at which point nir_lower_vars_to_ssa will get rid
of all access to nir_var_function_temp variables assuming complete
deref chains.  (Incomplete deref chains can happen with OpenCL kernel
so they require a different approach.)

Next, we have driver-internal I/O for things like vertex attributes,
varyings, and uniforms, i.e. nir_var_shader_in, nir_var_shader_out,
and nir_var_uniform.  This is typically handled by nir_lower_io.  This
call takes a type_size callback function which it ses to lay out the
I/O data in some sort of index space.  For nir_var_uniform, this is
often bytes (but doesn't have to be).  For nir_var_shader_in and
nir_var_shader_out, it's typically in units of vec4 locations.  The
result of this lowering is a bunch of load/store_input/output
intrinsics.  For FS inputs, nir_lower_io can optionally produce
interpolation intrinsics instead.

The final major category is external I/O.  For this, we use
nir_lower_explicit_io and friends.  One difference between
nir_lower_io and nir_lower_explicit_io is that nir_lower_explicit_io
expects types to have explicit layout information (which is always in
units of bytes) rather than taking it from a callback.  Another
difference is that nir_lower_explicit_io is capable of handling
incomplete deref chains where pointer casts, pointer arithmetic, and
other weirdness may exist while nir_lower_io assumes full deref chains
all the time.  For Vulkan, we need explicit type layout information
because that's what we get from SPIR-V and we need partial deref
chains because of extensions like VK_KHR_variable_pointers.  For
OpenGL, we have neither of those things but we use
nir_lower_explicit_io anyway because we want the consistency and
because the std140/430 layouts are a little too complex to describe
with the callback used by nir_lower_io.

Lastly, we have nir_var_system_value which is lowered by
nir_lower_system_values and nir_lower_cs_system_values.

I hope that helps.  I should really turn this into a blog post or,
better yet, real documentation

--Jason

On Sun, Oct 11, 2020 at 6:27 AM vivek pandya  wrote:
>
> I see that
> visit_load_var() in ~/mesa/src/amd/llvm/ac_nir_to_llvm.c
>
> assumes that nir_variable used in this intrinsic can have few specific mods 
> only.
>
> For example variable can not have nir_var_mem_shared , if such mod 
> encountered it will execute unreachable() code.
>
> Is there any nir pass that needs to be run before nir_to_llvm translation?
>
> Sincerely,
> Vivek
> ___
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Rust drivers in Mesa

[Jason Ekstrand]

On Fri, Oct 2, 2020 at 5:51 PM  wrote:
>
> On Fri, 2020-10-02 at 12:53 -0500, Jason Ekstrand wrote:
> > On Fri, Oct 2, 2020 at 11:34 AM Eric Anholt  wrote:
> > > On Thu, Oct 1, 2020 at 6:36 PM Alyssa Rosenzweig
> > >  wrote:
> > > > Hi all,
> > > >
> > > > Recently I've been thinking about the potential for the Rust
> > > > programming
> > > > language in Mesa. Rust bills itself a safe system programming
> > > > language
> > > > with comparable performance to C [0], which is a naturally fit
> > > > for
> > > > graphics driver development.
> > > >
> >
> > [...]
> >
> > That said, I really do like Rust as a language.  I'm not sure I'd go
> > quite as far as Anholt but I feel like it has everything I like about
> > C++ without most of what I dislike.  I'm not sure how we would do
> > things like ANV's GenXML multi-compile magic but I'm sure we could
> > figure out a way.  I'd love to see someone build something in Rust
> > and
> > figure out some best practices.
> >
> > --Jason
>
> The good thing about C++ is that you can pick a subset of features that
> you like, and stick to them. Then you'll never have to deal with other
> the stuff if you don't want to. There exists a subset of the language
> which helps get rid of most memory management issues.

The difficulty in Mesa has always been deciding what that subset is.
This thread is proof of that.  So far we've had people extole
everything from basic type safety and memory management to template
meta-programming.  Features which one developer enjoys another
loathes.  This is why we tend to stick to C for the core: It's limited
but those limitations we can sort-of all agree on it.

> I would also argue that there exist a bunch of tools that can help make
> C/C++ code stay safe, for example our collegaue Tony has recently
> addressed some findings by an undefined behaviour sanitizer. There are
> also free and open source static analyzers.
>
> I like the idea of Rust, especially their intention to focus on safety,
> but honestly, I don't like how they realized it.
>
> The Rust syntax is slightly annoying. They departed from C/C++ enough
> to make Rust look different, but then they got lazy and for some reason
> they chose to keep the most annoying parts from C/C++ like curly braces
> and semicolons, so even if we switch to Rust we can still enjoy
> debating important topics like where to put curly braces, how many
> spaces or tabs are best, and so on.

IMO: Like many modern languages, Rust has a defined default coding
style and It's pretty reasonable.  We should just go with it.

> My main concern is that memory is just one of many resources that we
> need to use safely, but it's the only one that Rust concerns itself
> with. Memory safety can give you a false sense of security. It's
> possible to write insecure apps that are perfectly memory safe.
>
> That said, I think it's perfectly okay to give it a try, write a
> backend compiler in Rust and see what happens. However, I don't think
> it is feasible or desireable to rewrite complicated parts from C/C++ to
> Rust. I think Daniel Steinberg's thoughts apply:
> https://daniel.haxx.se/blog/2017/03/27/curl-is-c/

Same.  Different developers/teams make different choices in their
drivers and back-ends all the time.  Some use C++ while others stickk
to C.  They all have different coding styles and choose different C++
feature sets (when they use C++).  IMO, the choice to use a language
like Rust is no different.  As long as it doesn't cause a giant
project-wide problem, I'm fine with it.

Even bits in core like the SPIR-V parser, if all the developers who
are working in that area want to write it in Rust, I'd say it's ok.  I
doubt it will actually happen because, has has been pointed out, those
bits of the code base are all large and aging and rewriting the SPIR-V
parser, as much as it might be fun from a "make it more secure"
perspective, would likely introduce far more bugs in the short term
than it would solve long-term.

--Jason
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Rust drivers in Mesa

[Jason Ekstrand]

On Fri, Oct 2, 2020 at 11:34 AM Eric Anholt  wrote:
>
> On Thu, Oct 1, 2020 at 6:36 PM Alyssa Rosenzweig
>  wrote:
> >
> > Hi all,
> >
> > Recently I've been thinking about the potential for the Rust programming
> > language in Mesa. Rust bills itself a safe system programming language
> > with comparable performance to C [0], which is a naturally fit for
> > graphics driver development.
> >
> > Mesa today is written primarily in C, a notoriously low-level language,
> > with some components in C++. To handle the impedance mismatch, we've
> > built up a number of abstractions in-tree, including multiple ad hoc
> > code generators (GenXML, NIR algebraic passes, Bifrost disassembler). A
> > higher level language can help avoid the web of metaprogramming and
> > effect code that is simpler and easier to reason about. Similarly, a
> > better type system can aid static analysis.
> >
> > Beyond abstraction, Rust's differentiating feature is the borrow checker
> > to guarantee memory safety. Historically, safety has not been a primary
> > concern of graphics drivers, since drivers are implemented as regular
> > userspace code running in the process of the application calling them.
> > Unfortunately, now that OpenGL is being exposed to untrusted code via
> > WebGL, the driver does become an attack vector.
> >
> > For the time being, Mesa attempts to minimize memory bugs with defensive
> > programming, safe in-tree abstractions (including ralloc), and static
> > analysis via Coverity. Nevertheless, these are all heuristic solutions.
> > Static analysis is imperfect and in our case, proprietary software.
> > Ideally, the bugs we've been fixing via Coverity could be caught at
> > compile-time with a free and open source toolchain.
> >
> > As Rust would allow exactly this, I see the primary benefit of Rust in
> > verifying correctness and robustness, rather than security concerns per
> > se.  Indeed, safety guarantees do translate well beyond WebGL.
> >
> > Practically, how would Rust fit in with our existing C codebase?
> > Obviously I'm not suggesting a rewrite of Mesa's more than 15 million
> > lines of C. Instead, I see value in introducing Rust in targeted parts
> > of the tree. In particular, I envision backend compilers written in part
> > in Rust. While creating an idiomatic Rust wrapper for NIR or Gallium
> > would be prohibitively costly for now, a backend compiler could be
> > written in Rust with IR builders exported for use of the NIR -> backend
> > IR translator written in C.
> >
> > This would have minimal impact on the tree. Users that are not building
> > such a driver would be unaffected. For those who _are_ building Rust
> > code, the Rust compiler would be added as a build-time dependency and
> > the (statically linked) Rust standard library would be added as a
> > runtime dependency. There is concern about the Rust compiler requiring
> > LLVM as a dependency, but again this is build-time, and no worse than
> > Mesa already requiring LLVM as a runtime dependency for llvmpipe and
> > clover. As for the standard library, it is possible to eliminate the
> > dependency as embedded Rust does, perhaps calling out to the C standard
> > library via the FFI, but this is likely quixotic. I do regret the binary
> > size increase, however.
> >
> > Implications for the build system vary. Rust prefers to be built by its
> > own package manager, Cargo, which is tricky to integrate with other
> > build systems. Actually, Meson has native support for Rust, invoking the
> > compiler directly and skipping Cargo, as if it were C code. This support
> > is not widely adopted as it prevents linking with external libraries
> > ("crates", in Rust parlance), with discussions between Rust and Meson
> > developers ending in a stand-still [1]. For Mesa, this might be just
> > fine. Our out-of-tree run-time dependencies are minimal for the C code,
> > and Rust's standard library largely avoids the need for us to maintain a
> > Rust version of util/ in-tree. If this proves impractical in the
> > long-term, it is possible to integrate Cargo with Meson on our end [2].
> >
> > One outstanding concern is build-time, which has been a notorious
> > growing pain for Rust due to both language design and LLVM itself [3],
> > although there is active work to improve both fronts [4][5]. I build
> > Mesa on my Arm laptop, so I suppose I'd be hurt more than many of us.
> > There's also awkward bootstrapping questions, but there is work here too
> > [6].
> >
> > If this is of interest, please discuss. It's clear to me Rust is not
> > going away any time soon, and I see value in Mesa embracing the new
> > technology. I'd like to hear other Mesa developers' thoughts.
>
> For me, every day I write C code, I wish I was writing rust.  I've
> written hobby rust (https://crates.io/crates/gpu-trace-perf) and also
> dabbled in a huge project (https://servo.org/), and I've gone through
> a bit of the struggles with the borrow checker and come out the 

Re: [Mesa-dev] Rust drivers in Mesa

[Jason Ekstrand]

On Thu, Oct 1, 2020 at 10:56 PM Rob Clark  wrote:
>
> On Thu, Oct 1, 2020 at 6:36 PM Alyssa Rosenzweig
>  wrote:
> >
> > Implications for the build system vary. Rust prefers to be built by its
> > own package manager, Cargo, which is tricky to integrate with other
> > build systems. Actually, Meson has native support for Rust, invoking the
> > compiler directly and skipping Cargo, as if it were C code. This support
> > is not widely adopted as it prevents linking with external libraries
> > ("crates", in Rust parlance), with discussions between Rust and Meson
> > developers ending in a stand-still [1]. For Mesa, this might be just
> > fine. Our out-of-tree run-time dependencies are minimal for the C code,
> > and Rust's standard library largely avoids the need for us to maintain a
> > Rust version of util/ in-tree. If this proves impractical in the
> > long-term, it is possible to integrate Cargo with Meson on our end [2].
> >
>
> drive-by comment: for something like a gl driver that a lot of other
> things depend on, making it harder for us to depend on other external
> things is actually a good thing

Generally, I'm a fan in concept.  Generally, I feel like rust has most
of what I like from C++ without most of what I don't like.  I
particularly like it's error handling mechanism, for instance.  That
said, when it comes to things like the borrow checker, my little bit
of rust experience says that it doesn't actually remove bugs so much
as move them around.

What's been stopping me is practicalities.  Not only build systems but
the way in which everything in mesa is interconnected.  Your
suggestion for building the entire back-end compiler with C-wrapped
helpers for NIR->compiler translation may be workable.  We might also
be able to write NIR wrappers sufficient for reading NIR.  It's not
actually that much API surface if all you want to do is read the data
structure.

I've also thought of breaking off a component like ISL and converting
it.  However, all the nicely contained pieces are also the ones that
wouldn't benefit as much. :-/

--Jason
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] XDC 2020 feedback and comments

[Jason Ekstrand]

First off, I think you all did a fantastic job.  I felt that things
ran very smoothly and, as far as the talks themselves go, I think it
went almost as smoothly as an in-person XDC.  I'm really quite
impressed.  I do have a couple pieces of more nuanced feedback:

 1. I think we were maybe a bit too scared of overloading jitsi.
Having more people in the instance for questions might have made that
portion go better.  As it was, there was only one or two talks that
had any live questions.  That said, there are a few advantages to
having things funneled through IRC, the most obvious of which being
that people can ask their question mid-talk and have it handled at the
end instead of having to remember it for 20 minutes.

 2. I really miss the hallway track.  On Thursday, after the
conference, Bas, Connor, and I used jitsi to have a chat about
ray-tracing.  That was really fun and I wish I'd done something like
that every day of XDC.  Maybe it's my own fault for not setting up
said chats but I think it could have been made more accessible (I had
no idea how to fork off a jitsi instance) and/or encouraged somehow.

--Jason

On Mon, Sep 21, 2020 at 3:07 AM Samuel Iglesias Gonsálvez
 wrote:
>
> Hi all,
>
> Huge thanks again to the entire team from Intel, for their great work
> organizing XDC 2020, our first virtual conference!
>
> As usual we're looking for feedback on both XDC itself, and the CFP
> process and program selection. Both about what was great and should be
> kept for next year's edition, and where there's room for improvement.
>
> The board does keep some notes, for those interested in what we have
> already:
>
> - XDC notes for prospective organizers:
> https://www.x.org/wiki/Events/RFP/
>
> - CFP notes: https://www.x.org/wiki/Events/PapersCommittee/
>
> If you want to send in your comments in private, please send them to
> the X.org Foundation board: bo...@foundation.x.org
>
> Cheers,
>
> Sam
> ___
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Plan for merging v3dv in mesa

[Jason Ekstrand]

FYI: I just opened this issue:
https://gitlab.freedesktop.org/mesa/mesa/-/issues/3534

On Thu, Sep 17, 2020 at 12:13 PM Jason Ekstrand  wrote:
>
> One more comment:  Could you make a big WIP MR with the whole driver
> to act as a pointer to the branch for now?  Then it can be the thing
> that gets merged once the stuff in a and b land.
>
> --Jason
>
> On Thu, Sep 17, 2020 at 12:10 PM Jason Ekstrand  wrote:
> >
> > Good work, all!  I'm super-happy to see another Vulkan driver land in Mesa.
> >
> > On Thu, Sep 17, 2020 at 8:52 AM apinheiro  wrote:
> > > Our development branch is ~525 patches on top of master, categorized as 
> > > follows:
> > >a) Patches that touch common Mesa infrastructure (NIR, Vulkan WSI, 
> > > Meson, etc):  ~5 patches.
> > >b) Patches that touch common Broadcom infrastructure under 
> > > src/broadcom (V3D backend compiler mostly): ~20 patches
> > >c) Patches that are independent and specific to the V3D Vulkan driver 
> > > (under src/broadcom/vulkan): ~500 patches.
> > >
> > > Since we are talking about a very large amount of patches, we are 
> > > expecting that we can merge most of them without a review, particularly 
> > > those in c) that implement the bulk of the Vulkan driver.
> >
> > I think that's fine.
> >
> > > The patches in b) are mostly about extending our compiler backend to 
> > > support Vulkan intrinsics and requirements as well a a few more general 
> > > fixes or improvements. Our plan is to at least have someone in our team 
> > > review them internally and grant Rbs, I think the only other person who 
> > > might want to review these would be Eric if he has the time and is 
> > > interested in doing so. We have sent some of these for early review 
> > > [1][2] when we found they were more generic fixes or improvements to the 
> > > compiler, but we might not want to do this for each and every one of them 
> > > unless we see there is interest in reviewing them separately.
> >
> > An ACK from Eric or someone other v3d devs would be good.  Not my
> > area, though, so I'll let others talk.
> >
> > > For the patches in a) we would like to at least get an Ack from other 
> > > Mesa devs. They are mostly very simple things that just add an option to 
> > > a NIR pass or a new intrinsic for use in our driver backend, so maybe it 
> > > is not needed to create dedicated MRs and it is fine to just ping 
> > > specific Mesa devs for reviews on those patches when we propose the large 
> > > MR for the bulk of the driver. We did send one of these as an RFC some 
> > > time ago [3] and it would be nice to get some more feedback there if 
> > > possible.
> >
> > I'd definitely like to see the patches in a) get real review.  I don't
> > know how many MRs you want to make there; do what makes sense.
> >
> > > Does all this sound sensible?
> >
> > Yup.
> >
> > --Jason
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Plan for merging v3dv in mesa

[Jason Ekstrand]

One more comment:  Could you make a big WIP MR with the whole driver
to act as a pointer to the branch for now?  Then it can be the thing
that gets merged once the stuff in a and b land.

--Jason

On Thu, Sep 17, 2020 at 12:10 PM Jason Ekstrand  wrote:
>
> Good work, all!  I'm super-happy to see another Vulkan driver land in Mesa.
>
> On Thu, Sep 17, 2020 at 8:52 AM apinheiro  wrote:
> > Our development branch is ~525 patches on top of master, categorized as 
> > follows:
> >a) Patches that touch common Mesa infrastructure (NIR, Vulkan WSI, 
> > Meson, etc):  ~5 patches.
> >b) Patches that touch common Broadcom infrastructure under src/broadcom 
> > (V3D backend compiler mostly): ~20 patches
> >c) Patches that are independent and specific to the V3D Vulkan driver 
> > (under src/broadcom/vulkan): ~500 patches.
> >
> > Since we are talking about a very large amount of patches, we are expecting 
> > that we can merge most of them without a review, particularly those in c) 
> > that implement the bulk of the Vulkan driver.
>
> I think that's fine.
>
> > The patches in b) are mostly about extending our compiler backend to 
> > support Vulkan intrinsics and requirements as well a a few more general 
> > fixes or improvements. Our plan is to at least have someone in our team 
> > review them internally and grant Rbs, I think the only other person who 
> > might want to review these would be Eric if he has the time and is 
> > interested in doing so. We have sent some of these for early review [1][2] 
> > when we found they were more generic fixes or improvements to the compiler, 
> > but we might not want to do this for each and every one of them unless we 
> > see there is interest in reviewing them separately.
>
> An ACK from Eric or someone other v3d devs would be good.  Not my
> area, though, so I'll let others talk.
>
> > For the patches in a) we would like to at least get an Ack from other Mesa 
> > devs. They are mostly very simple things that just add an option to a NIR 
> > pass or a new intrinsic for use in our driver backend, so maybe it is not 
> > needed to create dedicated MRs and it is fine to just ping specific Mesa 
> > devs for reviews on those patches when we propose the large MR for the bulk 
> > of the driver. We did send one of these as an RFC some time ago [3] and it 
> > would be nice to get some more feedback there if possible.
>
> I'd definitely like to see the patches in a) get real review.  I don't
> know how many MRs you want to make there; do what makes sense.
>
> > Does all this sound sensible?
>
> Yup.
>
> --Jason
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Plan for merging v3dv in mesa

[Jason Ekstrand]

Good work, all!  I'm super-happy to see another Vulkan driver land in Mesa.

On Thu, Sep 17, 2020 at 8:52 AM apinheiro  wrote:
> Our development branch is ~525 patches on top of master, categorized as 
> follows:
>a) Patches that touch common Mesa infrastructure (NIR, Vulkan WSI, Meson, 
> etc):  ~5 patches.
>b) Patches that touch common Broadcom infrastructure under src/broadcom 
> (V3D backend compiler mostly): ~20 patches
>c) Patches that are independent and specific to the V3D Vulkan driver 
> (under src/broadcom/vulkan): ~500 patches.
>
> Since we are talking about a very large amount of patches, we are expecting 
> that we can merge most of them without a review, particularly those in c) 
> that implement the bulk of the Vulkan driver.

I think that's fine.

> The patches in b) are mostly about extending our compiler backend to support 
> Vulkan intrinsics and requirements as well a a few more general fixes or 
> improvements. Our plan is to at least have someone in our team review them 
> internally and grant Rbs, I think the only other person who might want to 
> review these would be Eric if he has the time and is interested in doing so. 
> We have sent some of these for early review [1][2] when we found they were 
> more generic fixes or improvements to the compiler, but we might not want to 
> do this for each and every one of them unless we see there is interest in 
> reviewing them separately.

An ACK from Eric or someone other v3d devs would be good.  Not my
area, though, so I'll let others talk.

> For the patches in a) we would like to at least get an Ack from other Mesa 
> devs. They are mostly very simple things that just add an option to a NIR 
> pass or a new intrinsic for use in our driver backend, so maybe it is not 
> needed to create dedicated MRs and it is fine to just ping specific Mesa devs 
> for reviews on those patches when we propose the large MR for the bulk of the 
> driver. We did send one of these as an RFC some time ago [3] and it would be 
> nice to get some more feedback there if possible.

I'd definitely like to see the patches in a) get real review.  I don't
know how many MRs you want to make there; do what makes sense.

> Does all this sound sensible?

Yup.

--Jason
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Loading Vulkan Driver

[Jason Ekstrand]

The error says pretty clearly what went wrong.  The loader looked for
the `vk_icdGetInstanceProcAddr` symbol and couldn't find it in your
so.  You need at least the basic Get*ProcAddr symbols or else the
loader can't do anything.  You'll also need device and instance
creation functions and possibly some of the queries before anything
will work.

On Thu, Aug 20, 2020 at 10:43 AM vivek pandya  wrote:
>
> Hello,
>
> I have started building mesa Vulkan driver.
> I have started by copying amd/vulkan driver however I have just kept only one 
> file in build
> libresoc_pipeline.c
> I have only one method
>
> VkResult libresoc_CreateGraphicsPipelines(
> VkDevice_device,
> VkPipelineCache pipelineCache,
> uint32_tcount,
> const VkGraphicsPipelineCreateInfo* pCreateInfos,
> const VkAllocationCallbacks*pAllocator,
> VkPipeline* pPipelines)
> {
> return VK_ERROR_UNKNOWN;
> }
>
> with few edits/commenting out code into files I am able to build 
> libvulkan_libresoc.so
>  but when I forced loaded driver with VK_ICD_FILENAMES I am getting following 
> error:
> however I was expecting to hit VK_ERROR_UNKNOWN. Anyone have any ideas? Am I 
> missing any file in the build setting?
>
> vivek@vivek-VirtualBox:~/install/share/vulkan/icd.d$ vulkaninfo
> ERROR: [Loader Message] Code 0 : loader_scanned_icd_add: Attempt to retrieve 
> either 'vkGetInstanceProcAddr' or 'vk_icdGetInstanceProcAddr' from ICD 
> /home/vivek/install/lib/x86_64-linux-gnu/libvulkan_libresoc.so failed.
> Cannot create Vulkan instance.
> This problem is often caused by a faulty installation of the Vulkan driver or 
> attempting to use a GPU that does not support Vulkan.
> /build/vulkan-tools-KEbD_A/vulkan-tools-1.2.131.1+dfsg1/vulkaninfo/vulkaninfo.h:371:
>  failed with ERROR_INCOMPATIBLE_DRIVER
>
> Thanks,
> Vivek
> ___
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Mesa Intel vulkan driver on VirtualBox

[Jason Ekstrand]

3D graphics virtualization is not as simple as just running the driver
required by the host GPU inside the guest OS.  For desktop
virtualization, it's typically done by the guest providing a different
3D graphics driver which somehow passes through the graphics commands
to the 3D driver on the host.  If VirtualBox doesn't implement this
for Vulkan, you're out of luck.  Simply running ANV in a VM won't
work.

--Jason

On Sat, Aug 15, 2020 at 7:31 AM vivek pandya  wrote:
>
> Hello,
>
> Is it possible to run a vulkan program on virtual box through 
> libvulkan_intel.so ?
>
> I have build and installed latest mesa from git source but I get following 
> output in vulkaninfo and vulkan-smoketest just fails with -3 error code.
>
> vivek@vivek-VirtualBox:~/mesa/dbg_opt$ vulkaninfo
> ===
> VULKAN INFO
> ===
>
> Vulkan API Version: 1.0.61
>
> INFO: [loader] Code 0 : Found manifest file 
> /usr/share/vulkan/explicit_layer.d/VkLayer_unique_objects.json, version 
> "1.1.0"
> INFO: [loader] Code 0 : Found manifest file 
> /usr/share/vulkan/explicit_layer.d/VkLayer_core_validation.json, version 
> "1.1.0"
> INFO: [loader] Code 0 : Found manifest file 
> /usr/share/vulkan/explicit_layer.d/VkLayer_standard_validation.json, version 
> "1.1.1"
> INFO: [loader] Code 0 : Encountered meta-layer 
> VK_LAYER_LUNARG_standard_validation
> INFO: [loader] Code 0 : Found manifest file 
> /usr/share/vulkan/explicit_layer.d/VkLayer_object_tracker.json, version 
> "1.1.0"
> INFO: [loader] Code 0 : Found manifest file 
> /usr/share/vulkan/explicit_layer.d/VkLayer_parameter_validation.json, version 
> "1.1.0"
> INFO: [loader] Code 0 : Found manifest file 
> /usr/share/vulkan/explicit_layer.d/VkLayer_threading.json, version "1.1.0"
> INFO: [loader] Code 0 : Meta-layer VK_LAYER_LUNARG_standard_validation all 5 
> component layers appear to be valid.
> INFO: [loader] Code 0 : Found ICD manifest file 
> /usr/local/share/vulkan/icd.d/radeon_icd.x86_64.json, version "1.0.0"
> INFO: [loader] Code 0 : Found ICD manifest file 
> /usr/local/share/vulkan/icd.d/intel_icd.x86_64.json, version "1.0.0"
> INFO: [loader] Code 0 : Found ICD manifest file 
> /usr/share/vulkan/icd.d/radeon_icd.x86_64.json, version "1.0.0"
> INFO: [loader] Code 0 : Found ICD manifest file 
> /usr/share/vulkan/icd.d/intel_icd.x86_64.json, version "1.0.0"
>
> Instance Extensions:
> 
> Instance Extensions count = 14
> VK_KHR_external_fence_capabilities  : extension revision  1
> VK_KHR_external_memory_capabilities : extension revision  1
> VK_KHR_external_semaphore_capabilities: extension revision  1
> VK_KHR_get_physical_device_properties2: extension revision  1
> VK_KHR_get_surface_capabilities2: extension revision  1
> VK_KHR_surface  : extension revision 25
> VK_KHR_wayland_surface  : extension revision  6
> VK_KHR_xcb_surface  : extension revision  6
> VK_KHR_xlib_surface : extension revision  6
> VK_KHR_display  : extension revision 23
> VK_EXT_direct_mode_display  : extension revision  1
> VK_EXT_acquire_xlib_display : extension revision  1
> VK_EXT_display_surface_counter  : extension revision  1
> VK_EXT_debug_report : extension revision  9
> /build/vulkan-YO5iw0/vulkan-1.0.61.1+dfsg1/demos/vulkaninfo.c:1722: failed 
> with VK_ERROR_INITIALIZATION_FAILED
> vivek@vivek-VirtualBox:~/mesa/dbg_opt$ vulkan-smoketest
> terminate called after throwing an instance of 'std::runtime_error'
>   what():  VkResult -3 returned
> Aborted (core dumped)
>
>
> Any help?
>
> Thanks,
> Vivek
> ___
> mesa-dev mailing list
> mesa-dev@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/mesa-dev
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Rename "master" branch to "main"?

[Jason Ekstrand]

On Tue, Aug 4, 2020 at 5:54 AM Daniel Stone  wrote:
>
> Hi,
>
> On Mon, 3 Aug 2020 at 17:16, Jason Ekstrand  wrote:
> > On Mon, Aug 3, 2020 at 11:12 AM Kenneth Graunke  
> > wrote:
> > > Seems reasonable to me...in the old Subversion days, we called it
> > > 'trunk'...then 'master' with Git...but calling the main development
> > > branch 'main' is arguably the simplest and most descriptive term.
> > >
> > > One thing we'll have to coordinate: getting Gitlab CI / Marge and the
> > > Intel Mesa CI to switch over at the right time, so we don't end up
> > > breaking/interrupting those services.  Should be easy, just requires
> > > a bit of coordination.
> >
> > Yup, I threw Daniel onto the CC of this e-mail explicitly for that
> > reason.  We may also want to coordinate with the rest of fd.o so that
> > everyone chooses the same new mainline branch name.  I just added
> > Michel to the CC as he's doing lots of CI stuff and might be a good
> > person to help coordinate there.  I certainly don't want to pull the
> > rug out from under anyone.
>
> That's fine by me. I think 'main' is a perfectly fine name, and we'd
> be happy to encode that in whatever useful way. I suppose the main
> problem with a global nature is the very disparate nature of the
> projects - getting everyone to move at once before we throw the switch
> would require a great deal of effort. But we can figure it out.

I don't think we need to get everyone to sync up necessarily.  It
probably would be good if we had consistency across projects as to
what the primary branch is called.  It may also matter from a GitLab
configuration perspective.  There's some chatter on the GitLab issue
tracker about allowing different default branch names and I could
imagine them even adding a per-install default default name.  I don't
think it's a big deal but something to consider.  Here's the GitLab
issue for this:

https://gitlab.com/gitlab-org/gitlab/-/issues/220906

> As for retargeting MRs; if it can be done manually, then it can be
> done automatically as well. We can figure out a way to just
> automatically retarget all the outstanding MRs, but a couple of weeks'
> leadtime would be good right now.

It sounds like people are ok with manually re-targetting if they have
to.  However, if there's a script I can run or coordinate with you to
run, that'd probably make the process smoother.  As I've said before,
I don't think we need to rush so if you think that's something someone
could get put together in a couple weeks or a month, I think it's fine
to wait for it.  Likely, Mesa isn't the only project on fd.o that's
going to make a change like this so such a script would probably be
pretty useful.

--Jason
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Rename "master" branch to "main"?

[Jason Ekstrand]

On Mon, Aug 3, 2020 at 1:38 PM Eric Engestrom  wrote:
>
> On Monday, 2020-08-03 13:31:19 -0500, Jason Ekstrand wrote:
> > On Mon, Aug 3, 2020 at 1:24 PM Eric Engestrom  wrote:
> > >
> > > On Monday, 2020-08-03 10:30:29 -0500, Jason Ekstrand wrote:
> > > > All,
> > > >
> > > > I'm sure by now you've all seen the articles, LKML mails, and other
> > > > chatter around inclusive language in software.  While mesa doesn't
> > > > provide a whole lot of documentation (hah!), we do have a website, a
> > > > code-base, and a git repo and this is something that we, as a project
> > > > should consider.
> > > >
> > > > What I'm proposing today is simply re-naming the primary Git branch
> > > > from "master" to "main".  Why "main"?  Because that's what GitHub has
> > > > chosen "main" as their new default branch name and so it sounds to me
> > > > like the most likely new default.
> > > >
> > > > As far as impact on the project goes, if and when we rename the
> > > > primary branch, the old "master" branch will be locked (no
> > > > pushing/merging allowed) and all MRs will have to be re-targeted
> > > > against the new branch.  Fortunately, that's very easy to do.  You
> > > > just edit the MR and there's a little drop-down box at the top for
> > > > which branch it targets.  I just tested this with one of mine and it
> > > > seems to work ok.
> > > >
> > > > As far as other bits of language in the code-base, I'm happy to see
> > > > those cleaned up as people have opportunity.  I'm not aware of any
> > > > particularly egregious offenders.  However, changing the name of the
> > > > primary branch is something which will cause a brief hiccup in
> > > > people's development process and so warrants broader discussion.
> > > >
> > > > Thoughts?
> > >
> > > Definite +1 for me on the idea, but we do have a lot of tools and
> > > processes with `master` baked in. I'll try and have a look at everything
> > > to make sure everything supports the transition (some things will need
> > > to support both the old and new names), but assuming no issue there this
> > > would be a really good thing to do, and `main` is a good name.
> >
> > I did some grepping and I noticed that as well.  Some of the tools
> > such as the khronos sync scripts will have to change if/when Khronos
> > repos make a similar transition.  I expect that to happen but don't
> > have a timeline.  I'll try to keep you posted on those.
>
> The external things like Khronos should be easy enough to handle, I was
> more concerned about internal things like the stable branches.
>
> >
> > For the internal ones, if you wanted to make a MR for it, we can
> > either land it with support for both ahead of the switch or we can
> > make it the first commit that goes on the new "main" branch.  In any
> > case, I'm not in so much of a hurry that I think we need to make the
> > switch ahead of getting tooling ready.
>
> No hurry either... except a branchpoint like the one happening in 2
> days is the perfect time to minimize issues, as we could have eg.
> `master` for VERSION<20.2 and `main` for VERSION>=20.2 which would
> make it trivial for tools to know which branch name to use based on
> the VERSION file.

Pardon me for being a bit daft but what's the issue there?  "main"
will contain all of the commits in "master" with exactly the same
SHAs.  The tools shouldn't have to make a distinction, I wouldn't
think.  What am I missing?

--Jason
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Rename "master" branch to "main"?

[Jason Ekstrand]

On Mon, Aug 3, 2020 at 1:24 PM Eric Engestrom  wrote:
>
> On Monday, 2020-08-03 10:30:29 -0500, Jason Ekstrand wrote:
> > All,
> >
> > I'm sure by now you've all seen the articles, LKML mails, and other
> > chatter around inclusive language in software.  While mesa doesn't
> > provide a whole lot of documentation (hah!), we do have a website, a
> > code-base, and a git repo and this is something that we, as a project
> > should consider.
> >
> > What I'm proposing today is simply re-naming the primary Git branch
> > from "master" to "main".  Why "main"?  Because that's what GitHub has
> > chosen "main" as their new default branch name and so it sounds to me
> > like the most likely new default.
> >
> > As far as impact on the project goes, if and when we rename the
> > primary branch, the old "master" branch will be locked (no
> > pushing/merging allowed) and all MRs will have to be re-targeted
> > against the new branch.  Fortunately, that's very easy to do.  You
> > just edit the MR and there's a little drop-down box at the top for
> > which branch it targets.  I just tested this with one of mine and it
> > seems to work ok.
> >
> > As far as other bits of language in the code-base, I'm happy to see
> > those cleaned up as people have opportunity.  I'm not aware of any
> > particularly egregious offenders.  However, changing the name of the
> > primary branch is something which will cause a brief hiccup in
> > people's development process and so warrants broader discussion.
> >
> > Thoughts?
>
> Definite +1 for me on the idea, but we do have a lot of tools and
> processes with `master` baked in. I'll try and have a look at everything
> to make sure everything supports the transition (some things will need
> to support both the old and new names), but assuming no issue there this
> would be a really good thing to do, and `main` is a good name.

I did some grepping and I noticed that as well.  Some of the tools
such as the khronos sync scripts will have to change if/when Khronos
repos make a similar transition.  I expect that to happen but don't
have a timeline.  I'll try to keep you posted on those.

For the internal ones, if you wanted to make a MR for it, we can
either land it with support for both ahead of the switch or we can
make it the first commit that goes on the new "main" branch.  In any
case, I'm not in so much of a hurry that I think we need to make the
switch ahead of getting tooling ready.

--Jason
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Rename "master" branch to "main"?

[Jason Ekstrand]

On Mon, Aug 3, 2020 at 12:51 PM Erik Faye-Lund
 wrote:
>
> On Mon, 2020-08-03 at 12:48 -0500, Jason Ekstrand wrote:
> > On Mon, Aug 3, 2020 at 12:42 PM Erik Faye-Lund
> >  wrote:
> > > On Mon, 2020-08-03 at 10:30 -0500, Jason Ekstrand wrote:
> > > > All,
> > > >
> > > > I'm sure by now you've all seen the articles, LKML mails, and
> > > > other
> > > > chatter around inclusive language in software.  While mesa
> > > > doesn't
> > > > provide a whole lot of documentation (hah!), we do have a
> > > > website, a
> > > > code-base, and a git repo and this is something that we, as a
> > > > project
> > > > should consider.
> > > >
> > > > What I'm proposing today is simply re-naming the primary Git
> > > > branch
> > > > from "master" to "main".  Why "main"?  Because that's what GitHub
> > > > has
> > > > chosen "main" as their new default branch name and so it sounds
> > > > to me
> > > > like the most likely new default.
> > > >
> > > > As far as impact on the project goes, if and when we rename the
> > > > primary branch, the old "master" branch will be locked (no
> > > > pushing/merging allowed) and all MRs will have to be re-targeted
> > > > against the new branch.  Fortunately, that's very easy to
> > > > do.  You
> > > > just edit the MR and there's a little drop-down box at the top
> > > > for
> > > > which branch it targets.  I just tested this with one of mine and
> > > > it
> > > > seems to work ok.
> > > >
> > > > As far as other bits of language in the code-base, I'm happy to
> > > > see
> > > > those cleaned up as people have opportunity.  I'm not aware of
> > > > any
> > > > particularly egregious offenders.  However, changing the name of
> > > > the
> > > > primary branch is something which will cause a brief hiccup in
> > > > people's development process and so warrants broader discussion.
> > > >
> > > > Thoughts?
> > > >
> > >
> > > I'm all for renaming it, but I'm a bit worried about doing it in a
> > > way
> > > where we don't break all merge-requests...
> > >
> > > As far as I know, GitLab doesn't allow changing the target-branch
> > > of a
> > > merge-request, so all pending merge-requests would all of a sudden
> > > point to the wrong branch.
> >
> > No and yes.  It doesn't have any way to mass-edit merge requests to
> > re-target them.  (It does have a mass-edit tool but it doesn't
> > support
> > that).  However, it does support re-targetting individual merge
> > requests.  I even gave instructions for doing so in my first e-mail.
> > :-)
> >
>
> Thanks for pointing that out, I feel kinda stupid now. I was looking
> for that feature in the past, and couldn't find it, but surely there it
> is!

No worries.  I was 100% sure it wasn't there too until I looked for it
today.  Maybe it was added recently?

> I guess I should have read more carefully. Updating every merge request
> manually should be a totally OK compromise IMO.

Cool.

--Jason
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Rename "master" branch to "main"?

[Jason Ekstrand]

On Mon, Aug 3, 2020 at 12:42 PM Erik Faye-Lund
 wrote:
>
> On Mon, 2020-08-03 at 10:30 -0500, Jason Ekstrand wrote:
> > All,
> >
> > I'm sure by now you've all seen the articles, LKML mails, and other
> > chatter around inclusive language in software.  While mesa doesn't
> > provide a whole lot of documentation (hah!), we do have a website, a
> > code-base, and a git repo and this is something that we, as a project
> > should consider.
> >
> > What I'm proposing today is simply re-naming the primary Git branch
> > from "master" to "main".  Why "main"?  Because that's what GitHub has
> > chosen "main" as their new default branch name and so it sounds to me
> > like the most likely new default.
> >
> > As far as impact on the project goes, if and when we rename the
> > primary branch, the old "master" branch will be locked (no
> > pushing/merging allowed) and all MRs will have to be re-targeted
> > against the new branch.  Fortunately, that's very easy to do.  You
> > just edit the MR and there's a little drop-down box at the top for
> > which branch it targets.  I just tested this with one of mine and it
> > seems to work ok.
> >
> > As far as other bits of language in the code-base, I'm happy to see
> > those cleaned up as people have opportunity.  I'm not aware of any
> > particularly egregious offenders.  However, changing the name of the
> > primary branch is something which will cause a brief hiccup in
> > people's development process and so warrants broader discussion.
> >
> > Thoughts?
> >
>
> I'm all for renaming it, but I'm a bit worried about doing it in a way
> where we don't break all merge-requests...
>
> As far as I know, GitLab doesn't allow changing the target-branch of a
> merge-request, so all pending merge-requests would all of a sudden
> point to the wrong branch.

No and yes.  It doesn't have any way to mass-edit merge requests to
re-target them.  (It does have a mass-edit tool but it doesn't support
that).  However, it does support re-targetting individual merge
requests.  I even gave instructions for doing so in my first e-mail.
:-)

--Jason
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Rename "master" branch to "main"?

[Jason Ekstrand]

On Mon, Aug 3, 2020 at 11:12 AM Kenneth Graunke  wrote:
>
> Seems reasonable to me...in the old Subversion days, we called it
> 'trunk'...then 'master' with Git...but calling the main development
> branch 'main' is arguably the simplest and most descriptive term.
>
> One thing we'll have to coordinate: getting Gitlab CI / Marge and the
> Intel Mesa CI to switch over at the right time, so we don't end up
> breaking/interrupting those services.  Should be easy, just requires
> a bit of coordination.

Yup, I threw Daniel onto the CC of this e-mail explicitly for that
reason.  We may also want to coordinate with the rest of fd.o so that
everyone chooses the same new mainline branch name.  I just added
Michel to the CC as he's doing lots of CI stuff and might be a good
person to help coordinate there.  I certainly don't want to pull the
rug out from under anyone.

--Jason
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] Rename "master" branch to "main"?

[Jason Ekstrand]

If it's helpful (I think it is), Google has published the following
document as part of their Android developer documentation:

https://source.android.com/setup/contribute/respectful-code

It provides not only lists of terms but also suggestions for
replacements.  It's all context-specific, of course, so doing a simple
search-and-replace likely isn't going to work.  However, I think such
documents are helpful for guiding the discussion.

--Jason

On Mon, Aug 3, 2020 at 10:30 AM Jason Ekstrand  wrote:
>
> All,
>
> I'm sure by now you've all seen the articles, LKML mails, and other
> chatter around inclusive language in software.  While mesa doesn't
> provide a whole lot of documentation (hah!), we do have a website, a
> code-base, and a git repo and this is something that we, as a project
> should consider.
>
> What I'm proposing today is simply re-naming the primary Git branch
> from "master" to "main".  Why "main"?  Because that's what GitHub has
> chosen "main" as their new default branch name and so it sounds to me
> like the most likely new default.
>
> As far as impact on the project goes, if and when we rename the
> primary branch, the old "master" branch will be locked (no
> pushing/merging allowed) and all MRs will have to be re-targeted
> against the new branch.  Fortunately, that's very easy to do.  You
> just edit the MR and there's a little drop-down box at the top for
> which branch it targets.  I just tested this with one of mine and it
> seems to work ok.
>
> As far as other bits of language in the code-base, I'm happy to see
> those cleaned up as people have opportunity.  I'm not aware of any
> particularly egregious offenders.  However, changing the name of the
> primary branch is something which will cause a brief hiccup in
> people's development process and so warrants broader discussion.
>
> Thoughts?
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

[Mesa-dev] Rename "master" branch to "main"?

[Jason Ekstrand]

All,

I'm sure by now you've all seen the articles, LKML mails, and other
chatter around inclusive language in software.  While mesa doesn't
provide a whole lot of documentation (hah!), we do have a website, a
code-base, and a git repo and this is something that we, as a project
should consider.

What I'm proposing today is simply re-naming the primary Git branch
from "master" to "main".  Why "main"?  Because that's what GitHub has
chosen "main" as their new default branch name and so it sounds to me
like the most likely new default.

As far as impact on the project goes, if and when we rename the
primary branch, the old "master" branch will be locked (no
pushing/merging allowed) and all MRs will have to be re-targeted
against the new branch.  Fortunately, that's very easy to do.  You
just edit the MR and there's a little drop-down box at the top for
which branch it targets.  I just tested this with one of mine and it
seems to work ok.

As far as other bits of language in the code-base, I'm happy to see
those cleaned up as people have opportunity.  I'm not aware of any
particularly egregious offenders.  However, changing the name of the
primary branch is something which will cause a brief hiccup in
people's development process and so warrants broader discussion.

Thoughts?
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev

Re: [Mesa-dev] RFC: Memory allocation on Mesa

[Jason Ekstrand]

I have given some thought as to how we could do it in our compiler
stack.  It basically comes down to adding a new type of ralloc context
which takes a Vulkan allocator struct.  If the parent context has such
a struct, that allocator gets used.  It wouldn't be that hard; I've
just not gone to the effort of wiring it up.

--Jason

On Tue, May 12, 2020 at 9:14 AM Jose Fonseca  wrote:
>
> You raise a good point about LLVM.  It can easily be the biggest memory 
> consumer (at least transiently) for any driver that uses it, so the value of 
> implementing Vulkan allocation callbacks without is indeed dubious.
>
> Jose
>
> ____
> From: Jason Ekstrand 
> Sent: Monday, May 11, 2020 17:29
> To: Jose Fonseca 
> Cc: ML mesa-dev ; 
> erik.faye-l...@collabora.com 
> Subject: Re: [Mesa-dev] RFC: Memory allocation on Mesa
>
> Sorry for the top-post.
>
> Very quick comment:  If part of your objective is to fulfill Vulkan's
> requirements, we need a LOT more plumbing than just
> MALLOC/CALLOC/FREE.  The Vulkan callbacks aren't set at a global level
> when the driver is loaded but are provided to every call that
> allocates anything and we're expected to track these sorts of
> "domains" that things are allocated from.  The reality, however, is
> that the moment you get into the compiler, all bets are off.  This is
> also true on other drivers; I don't think anyone has plumbed the
> Vulkan allocation callbacks into LLVM. :-)
>
> --Jason
>
> On Mon, May 11, 2020 at 11:13 AM Jose Fonseca  wrote:
> >
> > Hi,
> >
> > To give everybody a bit of background context, this email comes from 
> > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgitlab.freedesktop.org%2Fmesa%2Fmesa%2F-%2Fissues%2F2911data=02%7C01%7Cjfonseca%40vmware.com%7C6565468f840241a093ae08d7f5c877d3%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637248113667594798sdata=hYmA5dMivC0jGjAx9cA9MwF81FjQSoo5plQBvHEDYes%3Dreserved=0
> >  .
> >
> > The short story is that Gallium components (but not Mesa) used to have 
> > their malloc/free calls intercepted, to satisfy certain needs: 1) memory 
> > debugging on Windows, 2) memory accounting on embedded systems.  But with 
> > the unification of Gallium into Mesa, the gallium vs non-gallium division 
> > got blurred, leading to some mallocs being intercepted but not the 
> > respective frees, and vice-versa.
> >
> >
> > I admit that trying to intercept mallocs/frees for some components and not 
> > others is error prone.  We could get this going on again, it's doable, but 
> > it's possible it would keep come causing troubles, for us or others, over 
> > and over again.
> >
> >
> > The two needs mentioned above were mostly VMware's needs.  So I've 
> > reevaluated, and I think that with some trickery we satisfy those two needs 
> > differently.  (Without wide spread source code changes.)
> >
> >
> > On the other hand, VMware is probably not the only one to have such needs.  
> > In fact Vulkan spec added memory callbacks precisely with the same use 
> > cases as ours, as seen 
> > https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.khronos.org%2Fregistry%2Fvulkan%2Fspecs%2F1.2%2Fhtml%2Fchap10.html%23memory-hostdata=02%7C01%7Cjfonseca%40vmware.com%7C6565468f840241a093ae08d7f5c877d3%7Cb39138ca3cee4b4aa4d6cd83d9dd62f0%7C0%7C0%7C637248113667594798sdata=SarQxeuRUMOm%2FZHogUQKo64rh7K7uLn5UOlIRPDe1jM%3Dreserved=0
> >  which states:
> >
> > Vulkan provides applications the opportunity to perform host memory 
> > allocations on behalf of the Vulkan implementation. If this feature is not 
> > used, the implementation will perform its own memory allocations. Since 
> > most memory allocations are off the critical path, this is not meant as a 
> > performance feature. Rather, this can be useful for certain embedded 
> > systems, for debugging purposes (e.g. putting a guard page after all host 
> > allocations), or for memory allocation logging.
> >
> >
> > And I know there were people interested in having Mesa drivers on embedded 
> > devices on the past (the old Tunsten Graphics having even been multiple 
> > times hired to do so), and I'm pretty sure they exist again.
> >
> >
> >
> > Therefore, rather than shying away from memory allocation abstractions now, 
> > I wonder if now it's not the time to actually double down on them and 
> > ensure we do so comprehensively throughout the whole mesa, all drivers?
> >
> > After all Mesa traditionally always had MALLOC*/CALLOC*/FREE wrappers 
> > around malloc/free.  As so many other projects do.
> >
> &

Re: [Mesa-dev] RFC: Memory allocation on Mesa

[Jason Ekstrand]

Sorry for the top-post.

Very quick comment:  If part of your objective is to fulfill Vulkan's
requirements, we need a LOT more plumbing than just
MALLOC/CALLOC/FREE.  The Vulkan callbacks aren't set at a global level
when the driver is loaded but are provided to every call that
allocates anything and we're expected to track these sorts of
"domains" that things are allocated from.  The reality, however, is
that the moment you get into the compiler, all bets are off.  This is
also true on other drivers; I don't think anyone has plumbed the
Vulkan allocation callbacks into LLVM. :-)

--Jason

On Mon, May 11, 2020 at 11:13 AM Jose Fonseca  wrote:
>
> Hi,
>
> To give everybody a bit of background context, this email comes from 
> https://gitlab.freedesktop.org/mesa/mesa/-/issues/2911 .
>
> The short story is that Gallium components (but not Mesa) used to have their 
> malloc/free calls intercepted, to satisfy certain needs: 1) memory debugging 
> on Windows, 2) memory accounting on embedded systems.  But with the 
> unification of Gallium into Mesa, the gallium vs non-gallium division got 
> blurred, leading to some mallocs being intercepted but not the respective 
> frees, and vice-versa.
>
>
> I admit that trying to intercept mallocs/frees for some components and not 
> others is error prone.  We could get this going on again, it's doable, but 
> it's possible it would keep come causing troubles, for us or others, over and 
> over again.
>
>
> The two needs mentioned above were mostly VMware's needs.  So I've 
> reevaluated, and I think that with some trickery we satisfy those two needs 
> differently.  (Without wide spread source code changes.)
>
>
> On the other hand, VMware is probably not the only one to have such needs.  
> In fact Vulkan spec added memory callbacks precisely with the same use cases 
> as ours, as seen 
> https://www.khronos.org/registry/vulkan/specs/1.2/html/chap10.html#memory-host
>  which states:
>
> Vulkan provides applications the opportunity to perform host memory 
> allocations on behalf of the Vulkan implementation. If this feature is not 
> used, the implementation will perform its own memory allocations. Since most 
> memory allocations are off the critical path, this is not meant as a 
> performance feature. Rather, this can be useful for certain embedded systems, 
> for debugging purposes (e.g. putting a guard page after all host 
> allocations), or for memory allocation logging.
>
>
> And I know there were people interested in having Mesa drivers on embedded 
> devices on the past (the old Tunsten Graphics having even been multiple times 
> hired to do so), and I'm pretty sure they exist again.
>
>
>
> Therefore, rather than shying away from memory allocation abstractions now, I 
> wonder if now it's not the time to actually double down on them and ensure we 
> do so comprehensively throughout the whole mesa, all drivers?
>
> After all Mesa traditionally always had MALLOC*/CALLOC*/FREE wrappers around 
> malloc/free.  As so many other projects do.
>
>
>
> More concretely, I'd like to propose that we:
>
> ensure all components use MALLOC*/CALLOC*/FREE and never malloc/calloc/free 
> directly (unless interfacing with a 3rd party which expects memory to be 
> allocated/freed with malloc/free directly)
> Perhaps consider renaming MALLOC -> _mesa_malloc etc while we're at it
> introduce a mechanisms to quickly catch any mistaken use of 
> malloc/calloc/free, regardless compiler/OS used:
>
> #define malloc/free/calloc as malloc_do_not_use/free_do_not_use to trigger 
> compilation errors, except on files which explicely opt out of this (source 
> files which need to interface with 3rd party, or source files which implement 
> the callbacks)
> Add a cookie to MALLOC/CALLOC/FREE memory to ensure it's not inadvertently 
> mixed with malloc/calloc/free
>
> The end goal is that we should be able to have a memory allocation 
> abstraction which can be used for all the needs above: memory debugging, 
> memory accounting, and satisfying Vulkan host memory callbacks.
>
>
> Some might retort: why not just play some tricks with the linker, and 
> intercept all malloc/free calls, without actually having to modify any source 
> code?
>
> Yes, that's indeed technically feasible.  And is precisely the sort of trick 
> I was planing to resort to satisfy VMware needs without having to further 
> modify the source code.  But for these tricks to work, it is absolutely 
> imperative that one statically links C++ library and STL.  The problem being 
> that if one doesn't then there's an imbalance: the malloc/free/new/delete 
> calls done in inline code on C++ headers will be intercepted, where as 
> malloc/free/new/delete calls done in code from the shared object which is not 
> inlined will not, causing havoc.  This is OK for us VMware (we do it already 
> for many other reasons, including avoiding DLL hell.)  But I doubt it will be 
> palatable for everybody else, particularly Linux distros, to 

Re: [Mesa-dev] New Mesa3D.org website proposal

[Jason Ekstrand]

On Thu, May 7, 2020 at 9:07 AM Erik Faye-Lund
 wrote:
>
> On Thu, 2020-05-07 at 09:05 -0500, Jason Ekstrand wrote:
> > Looks shiny but
> >
> > We need to be very careful here. One of the big no-nos with Khronos
> > trademark rules is using logos for things where implementations
> > aren't
> > conformant.  Mesa has been living in a mirky in-between land for a
> > while and everything has been mostly fine.  We've been very careful
> > to
> > *not* say that Mesa is an implementation of any Khronos APIs.
> > Instead, we say that Mesa is an open-source software project that
> > contains implementations and that some of those implementations are
> > conformant.  That page full of logos makes me concerned that we're
> > going to risk getting into trouble.  Khronos marketing cares A LOT
> > about logos.
>
> As I wrote in the e-mail, I've already clarified the logo-usage with
> Khronos. They are happy with it as-is.

Sorry.  I should have done more reading.  Thanks for checking with them!

I think the new thing looks great!

--Jason



> > On Thu, May 7, 2020 at 5:33 AM Erik Faye-Lund
> >  wrote:
> > > Hey Brian
> > >
> > > TLDR; are you OK with me moving forward with the rework of
> > > mesa3d.org?
> > >
> > > As you hopefully are aware of, I've been working on a new website
> > > for
> > > mesa3d.org, split into a "marketing"-frontpage and a documentation
> > > page.
> > >
> > > You can read more about the structure and details here if you
> > > haven't
> > > already:
> > > https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4630
> > >
> > > That MR only deals with the marketing website, so I think it's time
> > > to
> > > talk more about the main website. What I currently have is located
> > > here:
> > >
> > > https://gitlab.freedesktop.org/kusma/kusma.pages.freedesktop.org/
> > >
> > > This builds on CI and currently gets served here:
> > >
> > > https://kusma.pages.freedesktop.org/
> > >
> > >
> > > Since last time I published any details about this, I've:
> > > 1. Switched the static-site-generator from Jekyll to Hugo. This
> > > lead to
> > > a build-time decrease from about two minutes to about 200ms on my
> > > machine. This makes iteration times a lot more bearable.
> > > 2. Back-paddled on moving some pages. I think we can hold off on
> > > moving
> > > pages to the website until the initial work lands.
> > > 3. Gotten approval from the Khronos marketing team about the usage
> > > of
> > > their logos / trademarks. They had some minor requests for changes,
> > > which has been taken into account.
> > >
> > > The news articles and redirects for URL compatibility is converted
> > > from
> > > a branch in my Mesa fork, here:
> > >
> > > https://gitlab.freedesktop.org/kusma/mesa/-/tree/docs-convert-news-hugo
> > >
> > > So, after a long introduction (sorry if a lot of this is stuff you
> > > already know), what I really want is for you Brian, as the project
> > > originator and listed webmaster, to say how you feel about all of
> > > this.
> > >
> > > If you're happy to move forward with this, I will start the
> > > technical
> > > discussions with the FDO admins and other people who might need to
> > > be
> > > involved in setting this up for production.
> > >
> > >
> > > ___
> > > mesa-dev mailing list
> > > mesa-dev@lists.freedesktop.org
> > > https://lists.freedesktop.org/mailman/listinfo/mesa-dev
>
___
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev