On 05/26/2017 08:57 AM, Jason Ekstrand wrote: > This is something that I have considered doing on multiple occasions and > have given a significant amount of thought. Unfortunately, it has never > made it particularly high on my priority list so no code has been > written. However, there have been a number of people who have brought > this up lately so I thought I would do a brain-dump on the topic in the > hopes of saving someone some effort.
I've thought about this a lot too. I agree, to at least some extent, with everything you say below. > Why? > > That's as good a place to start as any. I think there a re a number of > reasons why this may be a useful thing to do: > > 1) Provide another GLSL -> SPIR-V compiler. Right now, the only thing > in that space is GLSLang and, while it seems to be servicing the need > ok, it also means that drivers tend not to implement SPIR-V support so > much as GLSLang support. SPIR-V provides a decent amount of freedom in > how you do things and GLSLang tends to only do them one way. This means > that the other 10 ways are untested in most drivers. Having another > SPIR-V producer out there would help the ecosystem. > > 2) Optimizations. One thing that I've heard developers asking for is > some basic platform-agnostic optimizations specifically to reduce SPIR-V > code size. Our compiler stack already has quite a few optimizations > that developers may want such as copy propagation, dead code > elimination, constant folding, loop unrolling, CSE, etc. In a GLSL -> > SPIR-V translator, we would probably want these to be optional, but it > wouldn't be hard to provide them as flags. So... isn't that going to be a problem? Pretty much all of our optimization infrastructure is now in NIR, and we're working towards removing at least some of the existing GLSL IR passes. > 3) Bootstrapping GL_ARB_spirv. The biggest barrier here is going to be > testing and convincing ourselves that we've done everything right. > However, if we did GLSL -> SPIR-V -> NIR -> back-end, then we would be > able to exercise both the GLSL -> SPIR-V path and the SPIR-V -> NIR path > with the full piglit suite. Some people have suggested modifying > shader_runner. However, I think that if we simply did the SPIR-V > translation inside the driver, it would be much easier to test things > like non-SSO because we could go into SPIR-V after linking. > > 4) Better testing spirv_to_nir. See above. This would augment the existing testing for spriv_to_nir (and not-yet-existing testing for GL_ARB_spriv), but we will probably need to start using a SPIR-V assembler to really hit all the corners. > 5) Better spec compliance. I don't know the glslang code base well > enough to speak particularly strongly about it's correctness. What I do > know, however, is that the GLSL compiler in mesa is used in > production-grade drivers that pass piglit, the Open GL 4.5 CTS, the Open > GL ES 3.2 CTS, and the Android CTS. That's not something you can say > about GLSLang. I think we should make that available to game devs > wishing to target Vulkan. > > Where? > > This is a question that I have internally debated for some time. When > we first started working on Vulkan, we boot-strapped it by adding some > hacks to the GLSL compiler to provide the descriptor set bindings and > just used GLSL. When it came time for SPIR-V support, I chose to go > directly into NIR. At the time, I thought that SPIR-V was going to end > up being a fairly low-level IR and that NIR would be a better match. > However, as things turned out, SPIR-V (at least for graphics) is more of > a binary form of GLSL than anything else. In retrospect, there are a > number of things (dealing with built-ins comes to mind) which would have > been easier had we gone SPIR-V -> GLSL. However, I still stand by my > original decision for two reasons: > > 1) As we try to add some of these "advanced compute" features that > people keep talking about, I think NIR will be a better and better fit. > Trying to retrofit some of those things into GLSL may be more trouble > than it's worth. > > 2) Going directly into NIR meant that we didn't have to pull in > mtypes.h. NIR is a fairly stand-alone chunk of the mesa tree and we > have pulled our i965 back-end compiler out so it only depends on NIR. > This makes the Vulkan driver much smaller and keeps it separate from the > GL state tracker. > > When considering a GLSL -> SPIR-V path there is some question about > whether it should be GLSL -> NIR -> SPIR-V or GLSL -> SPIR-V directly. > There are pros and cons to both: > > 1) SPIR-V would actually make a reasonable serialization format for > NIR. It's maybe not as compact as we could do if we made our own, but > it's not bad. Yeah, I think I suggested that to Nicolai in the SPIR-V for Radeon thread. > 2) NIR is, in some ways simpler than GLSL IR so it may be a bit > easier. Unfortunately, that's not always a help... > > 3) Because NIR is simpler, it requires a lot of lowering. There are > several things such as block variables (for UBOs and SSBOs) that NIR > doesn't know how to handle. It assumes that it just gets > load_ubo(index, offset). NIR also doesn't handle GLSL built-ins. For > SPIR-V -> NIR, those things are lowered a way on-the-fly. This means > that we would have to add some things to NIR and the SPIR-V code you get > out would necessarily be lower-level than what glslang produces. > > 4) As I pointed out above, GLSL is a bit better fit than NIR in a lot > of ways. > > 5) If we did NIR -> SPIR-V, we could also compile ARB programs to > SPIR-V. Maybe that's a downside? I'm somewhat ambivalent about that. I think an interesting standalone project would be a compiler that could take some of the "advanced" NV assembly shaders to SPRI-V. I don't really want to see any of that support come back into Mesa. > All in all, my recommendation would be to do GLSL -> SPIR-V directly. I > think there's less of an impedance mismatch there and I think it would > be easier to handle some of the higher-level things such as built-ins > and block variables. > > How? > > This is something that I've also given quite a bit of thought. :-) You > could even say that I've written a decent chunk of it in my head. Too > bad I haven't typed it into a computer yet. :-( > > The first thing I would do would be to write some sort of SPIR-V builder > and put it in src/compiler/spirv. It have multiple dword streams going > at a time (SPIR-V has multiple sections) to handle things such as types, > constants, etc. as well as function bodies. Then you would have An alternative would be to do it in two passes: the first pass figures out the sizes of the sections, and the second pass fills in the data. Or perhaps I'm misunderstanding. > functions which would generate SPIR-V opcodes similar to nir_builder.h. > For Types, you would have a single spv_type function that turned a > glsl_type pointer into a SPIR-V ID. You would want to have this backed > by a hash map so that they aren't re-emitted all the time. For the rest > of the opcodes, SPIRV-Headers project on the Khronos github project has > a JSON representation of SPIR-V so it shouldn't be hard to write a > little python code that generates most of the "build an opcode" > functions similar to what we do for nir_builder_opcodes.h. > > Finally, you would write the GLSL -> SPIR-V (or NIR -> SPIR-V) pass on > top of that. If you did GLSL -> SPIR-V, you could probably look at > glsl_to_nir for inspiration. > > Ok, there's my brain-dump. It's on the list now so anyone who wants to > pick up the project has it. I'm happy to chat further via e-mail or IRC > about different approaches but I think I've given a fairly complete > picture of my thoughts at the moment. Happy Hacking! > > --Jason _______________________________________________ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
