Re: [Mesa-dev] Rust drivers in Mesa
On Thu, Oct 1, 2020 at 10:56 PM Rob Clark wrote: > > On Thu, Oct 1, 2020 at 6:36 PM Alyssa Rosenzweig > wrote: > > > > Implications for the build system vary. Rust prefers to be built by its > > own package manager, Cargo, which is tricky to integrate with other > > build systems. Actually, Meson has native support for Rust, invoking the > > compiler directly and skipping Cargo, as if it were C code. This support > > is not widely adopted as it prevents linking with external libraries > > ("crates", in Rust parlance), with discussions between Rust and Meson > > developers ending in a stand-still [1]. For Mesa, this might be just > > fine. Our out-of-tree run-time dependencies are minimal for the C code, > > and Rust's standard library largely avoids the need for us to maintain a > > Rust version of util/ in-tree. If this proves impractical in the > > long-term, it is possible to integrate Cargo with Meson on our end [2]. > > > > drive-by comment: for something like a gl driver that a lot of other > things depend on, making it harder for us to depend on other external > things is actually a good thing Generally, I'm a fan in concept. Generally, I feel like rust has most of what I like from C++ without most of what I don't like. I particularly like it's error handling mechanism, for instance. That said, when it comes to things like the borrow checker, my little bit of rust experience says that it doesn't actually remove bugs so much as move them around. What's been stopping me is practicalities. Not only build systems but the way in which everything in mesa is interconnected. Your suggestion for building the entire back-end compiler with C-wrapped helpers for NIR->compiler translation may be workable. We might also be able to write NIR wrappers sufficient for reading NIR. It's not actually that much API surface if all you want to do is read the data structure. I've also thought of breaking off a component like ISL and converting it. However, all the nicely contained pieces are also the ones that wouldn't benefit as much. :-/ --Jason ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Rust drivers in Mesa
On Thu, Oct 1, 2020 at 6:36 PM Alyssa Rosenzweig wrote: > > Implications for the build system vary. Rust prefers to be built by its > own package manager, Cargo, which is tricky to integrate with other > build systems. Actually, Meson has native support for Rust, invoking the > compiler directly and skipping Cargo, as if it were C code. This support > is not widely adopted as it prevents linking with external libraries > ("crates", in Rust parlance), with discussions between Rust and Meson > developers ending in a stand-still [1]. For Mesa, this might be just > fine. Our out-of-tree run-time dependencies are minimal for the C code, > and Rust's standard library largely avoids the need for us to maintain a > Rust version of util/ in-tree. If this proves impractical in the > long-term, it is possible to integrate Cargo with Meson on our end [2]. > drive-by comment: for something like a gl driver that a lot of other things depend on, making it harder for us to depend on other external things is actually a good thing BR, -R ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Rust drivers in Mesa
Alyssa Rosenzweig writes: > Hi all, > > Recently I've been thinking about the potential for the Rust programming > language in Mesa. Rust bills itself a safe system programming language > with comparable performance to C [0], which is a naturally fit for > graphics driver development. > > Mesa today is written primarily in C, a notoriously low-level language, > with some components in C++. To handle the impedance mismatch, we've > built up a number of abstractions in-tree, including multiple ad hoc > code generators (GenXML, NIR algebraic passes, Bifrost disassembler). A > higher level language can help avoid the web of metaprogramming and > effect code that is simpler and easier to reason about. Similarly, a > better type system can aid static analysis. > > Beyond abstraction, Rust's differentiating feature is the borrow checker > to guarantee memory safety. Historically, safety has not been a primary > concern of graphics drivers, since drivers are implemented as regular > userspace code running in the process of the application calling them. > Unfortunately, now that OpenGL is being exposed to untrusted code via > WebGL, the driver does become an attack vector. > > For the time being, Mesa attempts to minimize memory bugs with defensive > programming, safe in-tree abstractions (including ralloc), and static > analysis via Coverity. Nevertheless, these are all heuristic solutions. > Static analysis is imperfect and in our case, proprietary software. > Ideally, the bugs we've been fixing via Coverity could be caught at > compile-time with a free and open source toolchain. > > As Rust would allow exactly this, I see the primary benefit of Rust in > verifying correctness and robustness, rather than security concerns per > se. Indeed, safety guarantees do translate well beyond WebGL. > > Practically, how would Rust fit in with our existing C codebase? > Obviously I'm not suggesting a rewrite of Mesa's more than 15 million > lines of C. Instead, I see value in introducing Rust in targeted parts > of the tree. In particular, I envision backend compilers written in part > in Rust. While creating an idiomatic Rust wrapper for NIR or Gallium > would be prohibitively costly for now, a backend compiler could be > written in Rust with IR builders exported for use of the NIR -> backend > IR translator written in C. > > This would have minimal impact on the tree. Users that are not building > such a driver would be unaffected. For those who _are_ building Rust > code, the Rust compiler would be added as a build-time dependency and > the (statically linked) Rust standard library would be added as a > runtime dependency. There is concern about the Rust compiler requiring > LLVM as a dependency, but again this is build-time, and no worse than > Mesa already requiring LLVM as a runtime dependency for llvmpipe and > clover. As for the standard library, it is possible to eliminate the > dependency as embedded Rust does, perhaps calling out to the C standard > library via the FFI, but this is likely quixotic. I do regret the binary > size increase, however. > > Implications for the build system vary. Rust prefers to be built by its > own package manager, Cargo, which is tricky to integrate with other > build systems. Actually, Meson has native support for Rust, invoking the > compiler directly and skipping Cargo, as if it were C code. This support > is not widely adopted as it prevents linking with external libraries > ("crates", in Rust parlance), with discussions between Rust and Meson > developers ending in a stand-still [1]. For Mesa, this might be just > fine. Our out-of-tree run-time dependencies are minimal for the C code, > and Rust's standard library largely avoids the need for us to maintain a > Rust version of util/ in-tree. If this proves impractical in the > long-term, it is possible to integrate Cargo with Meson on our end [2]. > > One outstanding concern is build-time, which has been a notorious > growing pain for Rust due to both language design and LLVM itself [3], > although there is active work to improve both fronts [4][5]. I build > Mesa on my Arm laptop, so I suppose I'd be hurt more than many of us. > There's also awkward bootstrapping questions, but there is work here too > [6]. > > If this is of interest, please discuss. It's clear to me Rust is not > going away any time soon, and I see value in Mesa embracing the new > technology. I'd like to hear other Mesa developers' thoughts. > > Thanks, > > Alyssa > I fully agree with the memory safety, generic programming and type system benefits of Rust over C you're talking about, particularly while writing any minimally complex piece of code like a compiler back-end. I have no objection to new back-ends being written in Rust instead of C. But just saying, most of those benefits can be reasonably achieved with modern dialects of C++ (modern as in >10 years old), which we already have hooked up to the build system and doesn't require any additional
[Mesa-dev] Rust drivers in Mesa
Hi all, Recently I've been thinking about the potential for the Rust programming language in Mesa. Rust bills itself a safe system programming language with comparable performance to C [0], which is a naturally fit for graphics driver development. Mesa today is written primarily in C, a notoriously low-level language, with some components in C++. To handle the impedance mismatch, we've built up a number of abstractions in-tree, including multiple ad hoc code generators (GenXML, NIR algebraic passes, Bifrost disassembler). A higher level language can help avoid the web of metaprogramming and effect code that is simpler and easier to reason about. Similarly, a better type system can aid static analysis. Beyond abstraction, Rust's differentiating feature is the borrow checker to guarantee memory safety. Historically, safety has not been a primary concern of graphics drivers, since drivers are implemented as regular userspace code running in the process of the application calling them. Unfortunately, now that OpenGL is being exposed to untrusted code via WebGL, the driver does become an attack vector. For the time being, Mesa attempts to minimize memory bugs with defensive programming, safe in-tree abstractions (including ralloc), and static analysis via Coverity. Nevertheless, these are all heuristic solutions. Static analysis is imperfect and in our case, proprietary software. Ideally, the bugs we've been fixing via Coverity could be caught at compile-time with a free and open source toolchain. As Rust would allow exactly this, I see the primary benefit of Rust in verifying correctness and robustness, rather than security concerns per se. Indeed, safety guarantees do translate well beyond WebGL. Practically, how would Rust fit in with our existing C codebase? Obviously I'm not suggesting a rewrite of Mesa's more than 15 million lines of C. Instead, I see value in introducing Rust in targeted parts of the tree. In particular, I envision backend compilers written in part in Rust. While creating an idiomatic Rust wrapper for NIR or Gallium would be prohibitively costly for now, a backend compiler could be written in Rust with IR builders exported for use of the NIR -> backend IR translator written in C. This would have minimal impact on the tree. Users that are not building such a driver would be unaffected. For those who _are_ building Rust code, the Rust compiler would be added as a build-time dependency and the (statically linked) Rust standard library would be added as a runtime dependency. There is concern about the Rust compiler requiring LLVM as a dependency, but again this is build-time, and no worse than Mesa already requiring LLVM as a runtime dependency for llvmpipe and clover. As for the standard library, it is possible to eliminate the dependency as embedded Rust does, perhaps calling out to the C standard library via the FFI, but this is likely quixotic. I do regret the binary size increase, however. Implications for the build system vary. Rust prefers to be built by its own package manager, Cargo, which is tricky to integrate with other build systems. Actually, Meson has native support for Rust, invoking the compiler directly and skipping Cargo, as if it were C code. This support is not widely adopted as it prevents linking with external libraries ("crates", in Rust parlance), with discussions between Rust and Meson developers ending in a stand-still [1]. For Mesa, this might be just fine. Our out-of-tree run-time dependencies are minimal for the C code, and Rust's standard library largely avoids the need for us to maintain a Rust version of util/ in-tree. If this proves impractical in the long-term, it is possible to integrate Cargo with Meson on our end [2]. One outstanding concern is build-time, which has been a notorious growing pain for Rust due to both language design and LLVM itself [3], although there is active work to improve both fronts [4][5]. I build Mesa on my Arm laptop, so I suppose I'd be hurt more than many of us. There's also awkward bootstrapping questions, but there is work here too [6]. If this is of interest, please discuss. It's clear to me Rust is not going away any time soon, and I see value in Mesa embracing the new technology. I'd like to hear other Mesa developers' thoughts. Thanks, Alyssa [0] https://www.rust-lang.org/ [1] https://github.com/mesonbuild/meson/issues/2173 [2] https://gitlab.gnome.org/GNOME/fractal/-/blob/master/meson.build [3] https://pingcap.com/blog/rust-compilation-model-calamity/ [4] https://blog.mozilla.org/nnethercote/2020/04/24/how-to-speed-up-the-rust-compiler-in-2020/ [5] https://github.com/bjorn3/rustc_codegen_cranelift [6] https://github.com/thepowersgang/mrustc signature.asc Description: PGP signature ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev