Hi,
On 13.09.2017 19:08, Jason Ekstrand wrote:
On Wed, Sep 13, 2017 at 8:42 AM, Nicolai Hähnle <nhaeh...@gmail.com
<mailto:nhaeh...@gmail.com>> wrote:
On 13.09.2017 11:54, Eero Tamminen wrote:
On 12.09.2017 09:55, Jordan Justen wrote:
On 2017-09-11 21:44:32, Timothy Arceri wrote:
On 12/09/17 14:23, Ian Romanick wrote:
On 09/08/2017 01:59 AM, Kenneth Graunke wrote:
We shouldn't use SPIR-V for the shader cache.
The compilation process for GLSL is: GLSL ->
GLSL IR -> NIR -> i965 IRs.
Storing the content at one of those points, and
later loading it and
resuming the normal compilation process from
that point...that's totally
reasonable.
Having a fallback for "some things in the cache
but not all the variants
we needed" suddenly take a different compilation
pipeline, i.e. SPIR-V
-> NIR -> ... seems risky. It's a different
compilation path that we
don't normally use. And one you'd only hit in
limited circumstances.
There's a lot of potential for really obscure bugs.
Since we're going to expose exactly that path for
GL_ARB_spirv / OpenGL
4.6, we'd better make sure it works always. Right?
One nice thing about SPIR-V is that all of the
handling of uniform
layouts, initial uniform values, attribute
locations, etc. is already
serialized. If I'm not mistaken, that was one of
the big pain points
for all of the existing on-disk storage methods.
All of that has been
sorted out for SPIR-V, and we have to make it work
anyway.
Correct these are the main issues for the fallback path,
however this is
only used by i965 (exactly because an intermediate cache
is missing).
Using SPIR-V as the intermediate cache means we still
need to convert to
NIR and run all the opts, so I don't really see the
advantage of caching
to SPIR-V over NIR.
For shader cache, hopefully we'll normally have the final
program in
the cache, which means the 're-run opt passes' is probably
not a big
factor. But, it still seems a fair point.
I think the biggest advantage of having either nir or spir-v
would be
not having to fallback to running the glsl compiler, right?
Shader cache is performance optimization aimed at reducing
compile times.
When I earlier profiled it, 2/3 of the shader compilation work
is done at linking stage.
Sorry, I should have mentioned that this is ancient data from before NIR
came into picture. Most of the time in linking was due to the i965
backend code, not intermediate representation manipulation.
Caching NIR would completely eliminate the linking stage for shader
variant compiles, since the NIR is already linked.
Quick check of latest Mesa code with Valgrind/Callgrind & perf and
SynMark v7 DrvShComp shader compilation test shows most of the time
(1/3) going to copy propagation in
./src/intel/compiler/brw_fs_copy_propagation.cpp.
Btw. I think one of the main things that shader compiler cache should
fix, is the huge amount of time spent with spilling shaders. Here's
some 1/2-1 year old profiling info about where they spend their time:
https://bugs.freedesktop.org/show_bug.cgi?id=93840#c33
https://bugs.freedesktop.org/show_bug.cgi?id=98455#c3
https://bugs.freedesktop.org/show_bug.cgi?id=97035#c3
- Eero
Yes, so there goes 2/3 of our compile time.
Cheers,
Nicolai
Caching done at higher level than the linked binary shaders, may
be of questionable performance value, because the caching itself
has also some cost (extra code, disk access cache lookups etc).
Some profiling showing that doing caching at NIR/SPIRV level
could measurable help performance (compared to caching
overhead), would be appreciated.
However, if the main purpose of caching NIR/SPIRV is something
else than performance[1], then that's a different matter. It
just should be clear what is the aim of this change.
There are a couple of reasons:
1) Caching the linked NIR should git rid of 2/3 to 3/4 of your shader
compile time. That's very significant. Sure, we can do better, but 70%
is a big perf boost. :-) I highly doubt that nir deserialization will
be a significant cost.
2) Provide a better fall-back when we don't have an Intel GEN binary.
Falling back all the way to GLSL is very expensive and potentially
unreliable due to the way OpenGL lets you change bindings post-linking
and the changes don't apply until you re-link. Falling back to linked
NIR should have similar reliability to the TGSI cache which seems to be
working very well.
--Jason
Maybe we'll see which becomes available first? :)
- Eero
[1] Such as:
* better code validation, or
* ability to replace / tweak the cached files to manually test
impact of compiler optimizations before implementing them
_______________________________________________
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
<mailto:mesa-dev@lists.freedesktop.org>
https://lists.freedesktop.org/mailman/listinfo/mesa-dev
<https://lists.freedesktop.org/mailman/listinfo/mesa-dev>
_______________________________________________
mesa-dev mailing list
mesa-dev@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/mesa-dev
