Re: [PATCH 4/4] build: rust: provide an option to inline C helpers into Rust
Gary Guo writes:
> On Tue, 02 Dec 2025 20:27:59 +
> Alice Ryhl wrote:
>
>> From: Gary Guo
>>
>> A new experimental Kconfig option, `RUST_INLINE_HELPERS` is added to
>> allow C helpers (which were created to allow Rust to call into
>> inline/macro C functions without having to re-implement the logic in
>> Rust) to be inlined into Rust crates without performing global LTO.
>>
>> If the option is enabled, the following is performed:
>> * For helpers, instead of compiling them to an object file to be linked
>> into vmlinux, they're compiled to LLVM IR.
>> * The LLVM IR is compiled to bitcode (This is step is not necessary, but
>> is a performance optimisation to prevent LLVM from always have to
>> reparse the same IR).
>> * When a Rust crate is compiled, instead of generating an object file, we
>> ask LLVM bitcode to be generated.
>> * llvm-link is invoked with --internalize to combine the helper bitcode
>> with the crate bitcode. This step is similar to LTO, but this is much
>> faster since it only needs to inline the helpers.
>> * clang is invoked to turn the combined bitcode into a final object file.
>>
>> The --internalize flag tells llvm-link to treat all symbols in
>> helpers.bc using `internal` linkage. This matches the behavior of
>> `clang` on `static inline` functions, and avoids exporting the symbol
>> from the object file.
>
> Huge shout-out to Matthew for discovering this option, this makes thing
> so much nicer and it feels way less hacky and more reliable now.
>
>>
>> To ensure that RUST_INLINE_HELPERS is not incompatible with BTF, we pass
>> the -g0 flag when building helpers. See commit 5daa0c35a1f0 ("rust:
>> Disallow BTF generation with Rust + LTO") for details.
>
> Again this is a much nicer way to workaround the issue than changing
> the language name in the DWARF info using text replacement that I come
> up with :)
>
>>
>> We have an intended triple mismatch of `aarch64-unknown-none` vs
>> `aarch64-unknown-linux-gnu`, so we suppress the warning.
>>
>> Co-developed-by: Boqun Feng
>> Signed-off-by: Boqun Feng
>> Co-developed-by: Matthew Maurer
>> Signed-off-by: Matthew Maurer
>> Signed-off-by: Gary Guo
>> Co-developed-by: Alice Ryhl
>> Signed-off-by: Alice Ryhl
>> ---
>> Makefile | 4 +++-
>> lib/Kconfig.debug | 15 +++
>> rust/Makefile | 26 ++
>> rust/exports.c | 5 -
>> scripts/Makefile.build | 5 -
>> 5 files changed, 48 insertions(+), 7 deletions(-)
>>
>> diff --git a/Makefile b/Makefile
>> index
>> 96ddbaae7e12de71bcfabef4639de3a13a6e4815..5834bfd568548d1bee34b328dccce5d60f85526f
>> 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -517,6 +517,8 @@ OBJCOPY =
>> $(LLVM_PREFIX)llvm-objcopy$(LLVM_SUFFIX)
>> OBJDUMP = $(LLVM_PREFIX)llvm-objdump$(LLVM_SUFFIX)
>> READELF = $(LLVM_PREFIX)llvm-readelf$(LLVM_SUFFIX)
>> STRIP = $(LLVM_PREFIX)llvm-strip$(LLVM_SUFFIX)
>> +LLVM_LINK = $(LLVM_PREFIX)llvm-link$(LLVM_SUFFIX)
>> +LLVM_AS = $(LLVM_PREFIX)llvm-as$(LLVM_SUFFIX)
>
> I think this is no longer needed as we only use LLVM BC now without
> having to round-trip between LLVM IR.
>
We should update the commit message to reflect this as well.
Best regards,
Andreas Hindborg
Re: [PATCH 4/4] build: rust: provide an option to inline C helpers into Rust
On Tue, 02 Dec 2025 20:27:59 +
Alice Ryhl wrote:
> From: Gary Guo
>
> A new experimental Kconfig option, `RUST_INLINE_HELPERS` is added to
> allow C helpers (which were created to allow Rust to call into
> inline/macro C functions without having to re-implement the logic in
> Rust) to be inlined into Rust crates without performing global LTO.
>
> If the option is enabled, the following is performed:
> * For helpers, instead of compiling them to an object file to be linked
> into vmlinux, they're compiled to LLVM IR.
> * The LLVM IR is compiled to bitcode (This is step is not necessary, but
> is a performance optimisation to prevent LLVM from always have to
> reparse the same IR).
> * When a Rust crate is compiled, instead of generating an object file, we
> ask LLVM bitcode to be generated.
> * llvm-link is invoked with --internalize to combine the helper bitcode
> with the crate bitcode. This step is similar to LTO, but this is much
> faster since it only needs to inline the helpers.
> * clang is invoked to turn the combined bitcode into a final object file.
>
> The --internalize flag tells llvm-link to treat all symbols in
> helpers.bc using `internal` linkage. This matches the behavior of
> `clang` on `static inline` functions, and avoids exporting the symbol
> from the object file.
Huge shout-out to Matthew for discovering this option, this makes thing
so much nicer and it feels way less hacky and more reliable now.
>
> To ensure that RUST_INLINE_HELPERS is not incompatible with BTF, we pass
> the -g0 flag when building helpers. See commit 5daa0c35a1f0 ("rust:
> Disallow BTF generation with Rust + LTO") for details.
Again this is a much nicer way to workaround the issue than changing
the language name in the DWARF info using text replacement that I come
up with :)
>
> We have an intended triple mismatch of `aarch64-unknown-none` vs
> `aarch64-unknown-linux-gnu`, so we suppress the warning.
>
> Co-developed-by: Boqun Feng
> Signed-off-by: Boqun Feng
> Co-developed-by: Matthew Maurer
> Signed-off-by: Matthew Maurer
> Signed-off-by: Gary Guo
> Co-developed-by: Alice Ryhl
> Signed-off-by: Alice Ryhl
> ---
> Makefile | 4 +++-
> lib/Kconfig.debug | 15 +++
> rust/Makefile | 26 ++
> rust/exports.c | 5 -
> scripts/Makefile.build | 5 -
> 5 files changed, 48 insertions(+), 7 deletions(-)
>
> diff --git a/Makefile b/Makefile
> index
> 96ddbaae7e12de71bcfabef4639de3a13a6e4815..5834bfd568548d1bee34b328dccce5d60f85526f
> 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -517,6 +517,8 @@ OBJCOPY =
> $(LLVM_PREFIX)llvm-objcopy$(LLVM_SUFFIX)
> OBJDUMP = $(LLVM_PREFIX)llvm-objdump$(LLVM_SUFFIX)
> READELF = $(LLVM_PREFIX)llvm-readelf$(LLVM_SUFFIX)
> STRIP= $(LLVM_PREFIX)llvm-strip$(LLVM_SUFFIX)
> +LLVM_LINK= $(LLVM_PREFIX)llvm-link$(LLVM_SUFFIX)
> +LLVM_AS = $(LLVM_PREFIX)llvm-as$(LLVM_SUFFIX)
I think this is no longer needed as we only use LLVM BC now without
having to round-trip between LLVM IR.
> else
> CC = $(CROSS_COMPILE)gcc
> LD = $(CROSS_COMPILE)ld
> @@ -625,7 +627,7 @@ export RUSTC_BOOTSTRAP := 1
> export CLIPPY_CONF_DIR := $(srctree)
>
> export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS CROSS_COMPILE
> LD CC HOSTPKG_CONFIG
> -export RUSTC RUSTDOC RUSTFMT RUSTC_OR_CLIPPY_QUIET RUSTC_OR_CLIPPY BINDGEN
> +export RUSTC RUSTDOC RUSTFMT RUSTC_OR_CLIPPY_QUIET RUSTC_OR_CLIPPY BINDGEN
> LLVM_LINK LLVM_AS
> export HOSTRUSTC KBUILD_HOSTRUSTFLAGS
> export CPP AR NM STRIP OBJCOPY OBJDUMP READELF PAHOLE RESOLVE_BTFIDS LEX
> YACC AWK INSTALLKERNEL
> export PERL PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX
Best,
Gary
Re: [PATCH 4/4] build: rust: provide an option to inline C helpers into Rust
On 12/4/25 12:57 PM, Miguel Ojeda wrote: On Thu, Dec 4, 2025 at 12:11 PM Peter Zijlstra wrote: Right. Earlier I also proposed using libclang to parse the C header and inject that. This might be a little simpler, in that.. Yeah, that would be closer to the `bindgen` route in that `libclang` gets already involved. Yeah, so... there are existing tools (c2rust [0] being the actively maintained one IIUC) that in theory could do something like that (translate the bodies of the functions from C to Rust so that rustc could consume them directly rather than via LLVM LTO). I think the intended use case is more "translate a whole C project into rust", but it could be interesting to test how well / poorly it performs with the kernel helpers / with a single header translated to Rust. I personally haven't tried it because for work I need to deal with C++, which means that automatic translation to Rust is a lot harder / probably impossible in general. So for Firefox we end up relying on bindgen + cross-language LTO for this kind of thing, and it works well for us. If I'm understanding correctly, it seems the kernel needs this extra bit of help (__always_inline) to push LLVM to inline C functions into rust, which is a bit unfortunate... But this approach seems sensible to me, for now at least. FWIW Bindgen recently gained an option to generate inline functions [1], which could help avoid at least the bindgen ifdef in the patch series? Anyways, it might be interesting to give c2rust a go on the kernel helpers if nobody has done so, and see how well / poorly it works in practice? Of course probably introducing a new dependency would be kind of a pain, but could be a good data point for pushing into adding something like it built into rustc... Thanks, -- Emilio [0]: https://github.com/immunant/c2rust [1]: https://docs.rs/bindgen/latest/bindgen/struct.Builder.html#method.generate_inline_functions
Re: [PATCH 4/4] build: rust: provide an option to inline C helpers into Rust
On Thu, Dec 4, 2025 at 12:11 PM Peter Zijlstra wrote: > > Right. Earlier I also proposed using libclang to parse the C header and > inject that. This might be a little simpler, in that.. Yeah, that would be closer to the `bindgen` route in that `libclang` gets already involved. > ... if you build rustc against libclang they are necessarily from the > same LLVM build. So currently there are 3 "LLVMs" that get involved: - The one Clang uses (in LLVM=1 builds). - The one `rustc` uses (the LLVM backend). - The one `bindgen` uses (via libclang). If that is all done within `rustc` (so no `bindgen`), then there may still be `rustc` vs. Clang mismatches, which are harder to resolve in the Rust side at least (it is easier to pick another Clang version to match). For those using builds from distros, that shouldn't be a problem. Others using external `rustc` builds, e.g. from `rustup` (e.g. for testing different Rust versions) it would be harder. But I mean, anything approach that gets us into a better position is welcome and I think requiring people to match LLVM everywhere should be easier now that distributions are starting to enable Rust (even Debian). We have been talking about this since the very beginning of the project -- e.g. I remember Wedson and I talking to Josh et al. about improving the situation here (in particular, talking about integrating a solution into `rustc` directly) long before Rust was merged into the kernel. Even on things like a `rustc cc` or `cImport` like Zig (but Zig moved on the other direction since then), which I recall Gary having opinions about too. There is also the question about GCC. A deeper integration into `rustc` would ideally need to have a way (perhaps depending on the backend picked?) to support GCC builds properly (to read the header and flags as expected, as you mention). And finally there is the question of what GCC Rust would do in such a case. Things have substantially changed on the GCC Rust in the last years, and they are now closer to build the kernel, thus I think their side of things is getting important to consider too. Cc'ing Emilio (`bindgen`), Antoni (GCC backend) and Arthur (GCC Rust) so that they are in the loop -- context at: https://lore.kernel.org/rust-for-linux/[email protected]/ Cheers, Miguel
Re: [PATCH 4/4] build: rust: provide an option to inline C helpers into Rust
On Thu, Dec 4, 2025 at 6:27 AM Peter Zijlstra wrote: > > On Thu, Dec 04, 2025 at 01:15:34PM +, Alice Ryhl wrote: > > On Thu, Dec 04, 2025 at 01:49:28PM +0100, Emilio Cobos Álvarez wrote: > > > On 12/4/25 12:57 PM, Miguel Ojeda wrote: > > > > On Thu, Dec 4, 2025 at 12:11 PM Peter Zijlstra > > > > wrote: > > > > > > > > > > Right. Earlier I also proposed using libclang to parse the C header > > > > > and > > > > > inject that. This might be a little simpler, in that.. > > > > > > > > Yeah, that would be closer to the `bindgen` route in that `libclang` > > > > gets already involved. > > > > > > Yeah, so... there are existing tools (c2rust [0] being the actively > > > maintained one IIUC) that in theory could do something like that > > > (translate > > > the bodies of the functions from C to Rust so that rustc could consume > > > them > > > directly rather than via LLVM LTO). > > > > > > I think the intended use case is more "translate a whole C project into > > > rust", but it could be interesting to test how well / poorly it performs > > > with the kernel helpers / with a single header translated to Rust. > > > > > > I personally haven't tried it because for work I need to deal with C++, > > > which means that automatic translation to Rust is a lot harder / probably > > > impossible in general. So for Firefox we end up relying on bindgen + > > > cross-language LTO for this kind of thing, and it works well for us. > > > > > > If I'm understanding correctly, it seems the kernel needs this extra bit > > > of > > > help (__always_inline) to push LLVM to inline C functions into rust, which > > > is a bit unfortunate... But this approach seems sensible to me, for now at > > > least. > > > > > > FWIW Bindgen recently gained an option to generate inline functions [1], > > > which could help avoid at least the bindgen ifdef in the patch series? > > > > > > Anyways, it might be interesting to give c2rust a go on the kernel helpers > > > if nobody has done so, and see how well / poorly it works in practice? Of > > > course probably introducing a new dependency would be kind of a pain, but > > > could be a good data point for pushing into adding something like it built > > > into rustc... > > > > I already tried c2rust as an alternative to this patch. It works okay > > for many functions, but it's missing support for some features such as > > asm goto, though this is fixable. But a larger issue is that some things > > simply do not translate to Rust right now. For example: > > > > * Atomics use the Ir operand. > > * static_branch uses the i operand. > > > > neither of which translate directly to Rust. I took a shot at `c2rust` as well, the summary was that `c2rust` was missing: * Asm goto support * __builtin_types_compatible_p * Deduced types * __count * _Generic I was adding hacky implementations along the way to see if things could be made to work, but the %l and %i missing in Rust itself that Alice pointed out were not things that could really be worked around. I also think that the bitcode-based approach is more future proof - the kernel may pick up new C features that `c2rust` hasn't yet learned about. If `clang` is in charge of generating the bitcode for the to-be-inlined helpers, we don't risk any desync in terms of what features are supported either by `c2rust` or by `rustc` relative to what's in use by the rest of the kernel. > > Right this. AFAIK Rust simply does not have feature parity with inline > asm. Them having picked a wildly different syntax for inline asm didn't > help either of course. But Rust is Rust, must have terrible syntax :-)
Re: [PATCH 4/4] build: rust: provide an option to inline C helpers into Rust
On Thu, Dec 04, 2025 at 01:15:34PM +, Alice Ryhl wrote: > On Thu, Dec 04, 2025 at 01:49:28PM +0100, Emilio Cobos Álvarez wrote: > > On 12/4/25 12:57 PM, Miguel Ojeda wrote: > > > On Thu, Dec 4, 2025 at 12:11 PM Peter Zijlstra > > > wrote: > > > > > > > > Right. Earlier I also proposed using libclang to parse the C header and > > > > inject that. This might be a little simpler, in that.. > > > > > > Yeah, that would be closer to the `bindgen` route in that `libclang` > > > gets already involved. > > > > Yeah, so... there are existing tools (c2rust [0] being the actively > > maintained one IIUC) that in theory could do something like that (translate > > the bodies of the functions from C to Rust so that rustc could consume them > > directly rather than via LLVM LTO). > > > > I think the intended use case is more "translate a whole C project into > > rust", but it could be interesting to test how well / poorly it performs > > with the kernel helpers / with a single header translated to Rust. > > > > I personally haven't tried it because for work I need to deal with C++, > > which means that automatic translation to Rust is a lot harder / probably > > impossible in general. So for Firefox we end up relying on bindgen + > > cross-language LTO for this kind of thing, and it works well for us. > > > > If I'm understanding correctly, it seems the kernel needs this extra bit of > > help (__always_inline) to push LLVM to inline C functions into rust, which > > is a bit unfortunate... But this approach seems sensible to me, for now at > > least. > > > > FWIW Bindgen recently gained an option to generate inline functions [1], > > which could help avoid at least the bindgen ifdef in the patch series? > > > > Anyways, it might be interesting to give c2rust a go on the kernel helpers > > if nobody has done so, and see how well / poorly it works in practice? Of > > course probably introducing a new dependency would be kind of a pain, but > > could be a good data point for pushing into adding something like it built > > into rustc... > > I already tried c2rust as an alternative to this patch. It works okay > for many functions, but it's missing support for some features such as > asm goto, though this is fixable. But a larger issue is that some things > simply do not translate to Rust right now. For example: > > * Atomics use the Ir operand. > * static_branch uses the i operand. > > neither of which translate directly to Rust. Right this. AFAIK Rust simply does not have feature parity with inline asm. Them having picked a wildly different syntax for inline asm didn't help either of course. But Rust is Rust, must have terrible syntax :-)
Re: [PATCH 4/4] build: rust: provide an option to inline C helpers into Rust
On Thu, Dec 04, 2025 at 01:49:28PM +0100, Emilio Cobos Álvarez wrote: > On 12/4/25 12:57 PM, Miguel Ojeda wrote: > > On Thu, Dec 4, 2025 at 12:11 PM Peter Zijlstra wrote: > > > > > > Right. Earlier I also proposed using libclang to parse the C header and > > > inject that. This might be a little simpler, in that.. > > > > Yeah, that would be closer to the `bindgen` route in that `libclang` > > gets already involved. > > Yeah, so... there are existing tools (c2rust [0] being the actively > maintained one IIUC) that in theory could do something like that (translate > the bodies of the functions from C to Rust so that rustc could consume them > directly rather than via LLVM LTO). > > I think the intended use case is more "translate a whole C project into > rust", but it could be interesting to test how well / poorly it performs > with the kernel helpers / with a single header translated to Rust. > > I personally haven't tried it because for work I need to deal with C++, > which means that automatic translation to Rust is a lot harder / probably > impossible in general. So for Firefox we end up relying on bindgen + > cross-language LTO for this kind of thing, and it works well for us. > > If I'm understanding correctly, it seems the kernel needs this extra bit of > help (__always_inline) to push LLVM to inline C functions into rust, which > is a bit unfortunate... But this approach seems sensible to me, for now at > least. > > FWIW Bindgen recently gained an option to generate inline functions [1], > which could help avoid at least the bindgen ifdef in the patch series? > > Anyways, it might be interesting to give c2rust a go on the kernel helpers > if nobody has done so, and see how well / poorly it works in practice? Of > course probably introducing a new dependency would be kind of a pain, but > could be a good data point for pushing into adding something like it built > into rustc... I already tried c2rust as an alternative to this patch. It works okay for many functions, but it's missing support for some features such as asm goto, though this is fixable. But a larger issue is that some things simply do not translate to Rust right now. For example: * Atomics use the Ir operand. * static_branch uses the i operand. neither of which translate directly to Rust. Alice
Re: [PATCH 4/4] build: rust: provide an option to inline C helpers into Rust
On Thu, Dec 04, 2025 at 01:39:06PM +0100, Peter Zijlstra wrote: > On Thu, Dec 04, 2025 at 12:57:31PM +0100, Miguel Ojeda wrote: > > On Thu, Dec 4, 2025 at 12:11 PM Peter Zijlstra wrote: > > > > > > Right. Earlier I also proposed using libclang to parse the C header and > > > inject that. This might be a little simpler, in that.. > > > > Yeah, that would be closer to the `bindgen` route in that `libclang` > > gets already involved. > > > > > ... if you build rustc against libclang they are necessarily from the > > > same LLVM build. > > > > So currently there are 3 "LLVMs" that get involved: > > > > - The one Clang uses (in LLVM=1 builds). > > Well, being on Debian, I'm more likely to be using LLVM=-22 (or whatever > actual version is required, 22 just being the latest shipped by Debian > at this point in time). > > > - The one `rustc` uses (the LLVM backend). > > - The one `bindgen` uses (via libclang). > > These are not necessarily the same? That is, is not bindgen part of the > rustc project and so would be built against the same LLVM? Rustc and bindgen are both part of the Rust project, but they're two different repos: https://github.com/rust-lang/rust https://github.com/rust-lang/rust-bindgen Generally you need to ensure that bindgen matches clang. This ensures that bindgen and clang agree on the interpretation of C headers. > > If that is all done within `rustc` (so no `bindgen`), then there may > > still be `rustc` vs. Clang mismatches, which are harder to resolve in > > the Rust side at least (it is easier to pick another Clang version to > > match). > > > > For those using builds from distros, that shouldn't be a problem. > > Others using external `rustc` builds, e.g. from `rustup` (e.g. for > > testing different Rust versions) it would be harder. > > Make rust part of LLVM and get them all built and distributed > together... such that LLVM=-23 will get me a coherent set of tools. > > /me runs like crazeh ;-) Maybe clang itself should have an option to emit Rust headers, taking over the role of bindgen? /me runs like crazeh ;-) Alice
Re: [PATCH 4/4] build: rust: provide an option to inline C helpers into Rust
On Thu, Dec 04, 2025 at 12:57:31PM +0100, Miguel Ojeda wrote: > On Thu, Dec 4, 2025 at 12:11 PM Peter Zijlstra wrote: > > > > Right. Earlier I also proposed using libclang to parse the C header and > > inject that. This might be a little simpler, in that.. > > Yeah, that would be closer to the `bindgen` route in that `libclang` > gets already involved. > > > ... if you build rustc against libclang they are necessarily from the > > same LLVM build. > > So currently there are 3 "LLVMs" that get involved: > > - The one Clang uses (in LLVM=1 builds). Well, being on Debian, I'm more likely to be using LLVM=-22 (or whatever actual version is required, 22 just being the latest shipped by Debian at this point in time). > - The one `rustc` uses (the LLVM backend). > - The one `bindgen` uses (via libclang). These are not necessarily the same? That is, is not bindgen part of the rustc project and so would be built against the same LLVM? > If that is all done within `rustc` (so no `bindgen`), then there may > still be `rustc` vs. Clang mismatches, which are harder to resolve in > the Rust side at least (it is easier to pick another Clang version to > match). > > For those using builds from distros, that shouldn't be a problem. > Others using external `rustc` builds, e.g. from `rustup` (e.g. for > testing different Rust versions) it would be harder. Make rust part of LLVM and get them all built and distributed together... such that LLVM=-23 will get me a coherent set of tools. /me runs like crazeh ;-) > There is also the question about GCC. A deeper integration into > `rustc` would ideally need to have a way (perhaps depending on the > backend picked?) to support GCC builds properly (to read the header > and flags as expected, as you mention). Right, so the backend that spits out C could obviously just pass through any C headers. But otherwise, inlining C headers (and inline functions) would be something that is independent of the C files. At the end of the day all that really matters is the architecture C ABI. That is, if rustc inlines a C function from a header, it doesn't matter it used libclang to do so, even if the C files are then compiled with GCC. > And finally there is the question of what GCC Rust would do in such a > case. Things have substantially changed on the GCC Rust in the last > years, and they are now closer to build the kernel, thus I think their > side of things is getting important to consider too. > > Cc'ing Emilio (`bindgen`), Antoni (GCC backend) and Arthur (GCC Rust) > so that they are in the loop -- context at: Right, so clearly GCC has the capability to parse C headers :-) So I would imagine their Rust front-end would be able to hand off C headers and get back IR much like LLVM based projects can using libclang.
Re: [PATCH 4/4] build: rust: provide an option to inline C helpers into Rust
On Thu, Dec 04, 2025 at 10:23:17AM +, Alice Ryhl wrote: > > The other day [*] I proposed extending Rust such that it would be able > > to consume a clang precompiled header directly, this would allow doing > > away with most of all this. No more helpers and no more bindgen. > > > > Would that not be a much saner approach to all this? > > > > [*] > > https://lkml.kernel.org/r/[email protected] > > I have actually discussed similar ideas in the past with Josh Triplett, > so you are not the only one who thinks it is a good idea. Unfortunately, > the road to get there is long. Right. Earlier I also proposed using libclang to parse the C header and inject that. This might be a little simpler, in that.. > Another option to get rid of the helpers is that bindgen is working on a > --wrap-static-fns flag, which would generate the helpers for us. > However, that route would still require this patch for them to be > inlined. > > One detail that makes the precompiled clang header really tricky is > that IMO we should not require RUSTC_CLANG_LLVM_COMPATIBLE for the > build. With bindgen, you just need bindgen and clang to match LLVMs. > That's easy since bindgen loads a dylib from your clang install. But if > you build this logic into rustc, then you need to be really careful to > get rustc and clang from the same source, and that same source must use > the same LLVM to build both. ... if you build rustc against libclang they are necessarily from the same LLVM build. But that might be more tricky in that you might need ways to specify C specific build flags. Anyway, good to know people are in fact pondering this, because IMO the whole interoperability thing with C is quite terrible.
Re: [PATCH 4/4] build: rust: provide an option to inline C helpers into Rust
On Thu, Dec 04, 2025 at 11:07:25AM +0100, Peter Zijlstra wrote:
> On Tue, Dec 02, 2025 at 08:27:59PM +, Alice Ryhl wrote:
> > From: Gary Guo
> >
> > A new experimental Kconfig option, `RUST_INLINE_HELPERS` is added to
> > allow C helpers (which were created to allow Rust to call into
> > inline/macro C functions without having to re-implement the logic in
> > Rust) to be inlined into Rust crates without performing global LTO.
> >
> > If the option is enabled, the following is performed:
> > * For helpers, instead of compiling them to an object file to be linked
> > into vmlinux, they're compiled to LLVM IR.
> > * The LLVM IR is compiled to bitcode (This is step is not necessary, but
> > is a performance optimisation to prevent LLVM from always have to
> > reparse the same IR).
> > * When a Rust crate is compiled, instead of generating an object file, we
> > ask LLVM bitcode to be generated.
> > * llvm-link is invoked with --internalize to combine the helper bitcode
> > with the crate bitcode. This step is similar to LTO, but this is much
> > faster since it only needs to inline the helpers.
> > * clang is invoked to turn the combined bitcode into a final object file.
> >
> > The --internalize flag tells llvm-link to treat all symbols in
> > helpers.bc using `internal` linkage. This matches the behavior of
> > `clang` on `static inline` functions, and avoids exporting the symbol
> > from the object file.
> >
> > To ensure that RUST_INLINE_HELPERS is not incompatible with BTF, we pass
> > the -g0 flag when building helpers. See commit 5daa0c35a1f0 ("rust:
> > Disallow BTF generation with Rust + LTO") for details.
> >
> > We have an intended triple mismatch of `aarch64-unknown-none` vs
> > `aarch64-unknown-linux-gnu`, so we suppress the warning.
>
> So if I understand this correctly, it will consume the helpers twice,
> once for bindgen to generate the rust ffi glue, and then a second time
> to 'compile' to IR.
>
> Then the IR is 'linked' into the rust translation units allowing the
> actual inlining to take place once 'LTO' runs.
>
> And while this works, this still has the downside of requiring those
> rust helper files and using bindgen.
That's right.
> The other day [*] I proposed extending Rust such that it would be able
> to consume a clang precompiled header directly, this would allow doing
> away with most of all this. No more helpers and no more bindgen.
>
> Would that not be a much saner approach to all this?
>
> [*]
> https://lkml.kernel.org/r/[email protected]
I have actually discussed similar ideas in the past with Josh Triplett,
so you are not the only one who thinks it is a good idea. Unfortunately,
the road to get there is long.
Another option to get rid of the helpers is that bindgen is working on a
--wrap-static-fns flag, which would generate the helpers for us.
However, that route would still require this patch for them to be
inlined.
One detail that makes the precompiled clang header really tricky is
that IMO we should not require RUSTC_CLANG_LLVM_COMPATIBLE for the
build. With bindgen, you just need bindgen and clang to match LLVMs.
That's easy since bindgen loads a dylib from your clang install. But if
you build this logic into rustc, then you need to be really careful to
get rustc and clang from the same source, and that same source must use
the same LLVM to build both.
Alice
Re: [PATCH 4/4] build: rust: provide an option to inline C helpers into Rust
On Tue, Dec 02, 2025 at 08:27:59PM +, Alice Ryhl wrote:
> From: Gary Guo
>
> A new experimental Kconfig option, `RUST_INLINE_HELPERS` is added to
> allow C helpers (which were created to allow Rust to call into
> inline/macro C functions without having to re-implement the logic in
> Rust) to be inlined into Rust crates without performing global LTO.
>
> If the option is enabled, the following is performed:
> * For helpers, instead of compiling them to an object file to be linked
> into vmlinux, they're compiled to LLVM IR.
> * The LLVM IR is compiled to bitcode (This is step is not necessary, but
> is a performance optimisation to prevent LLVM from always have to
> reparse the same IR).
> * When a Rust crate is compiled, instead of generating an object file, we
> ask LLVM bitcode to be generated.
> * llvm-link is invoked with --internalize to combine the helper bitcode
> with the crate bitcode. This step is similar to LTO, but this is much
> faster since it only needs to inline the helpers.
> * clang is invoked to turn the combined bitcode into a final object file.
>
> The --internalize flag tells llvm-link to treat all symbols in
> helpers.bc using `internal` linkage. This matches the behavior of
> `clang` on `static inline` functions, and avoids exporting the symbol
> from the object file.
>
> To ensure that RUST_INLINE_HELPERS is not incompatible with BTF, we pass
> the -g0 flag when building helpers. See commit 5daa0c35a1f0 ("rust:
> Disallow BTF generation with Rust + LTO") for details.
>
> We have an intended triple mismatch of `aarch64-unknown-none` vs
> `aarch64-unknown-linux-gnu`, so we suppress the warning.
So if I understand this correctly, it will consume the helpers twice,
once for bindgen to generate the rust ffi glue, and then a second time
to 'compile' to IR.
Then the IR is 'linked' into the rust translation units allowing the
actual inlining to take place once 'LTO' runs.
And while this works, this still has the downside of requiring those
rust helper files and using bindgen.
The other day [*] I proposed extending Rust such that it would be able
to consume a clang precompiled header directly, this would allow doing
away with most of all this. No more helpers and no more bindgen.
Would that not be a much saner approach to all this?
[*]
https://lkml.kernel.org/r/[email protected]
Re: [PATCH 4/4] build: rust: provide an option to inline C helpers into Rust
On Wed, Dec 03, 2025 at 02:25:58PM -0700, Nathan Chancellor wrote: > Hi Alice, > > On Tue, Dec 02, 2025 at 08:27:59PM +, Alice Ryhl wrote: > ... > > diff --git a/Makefile b/Makefile > > index > > 96ddbaae7e12de71bcfabef4639de3a13a6e4815..5834bfd568548d1bee34b328dccce5d60f85526f > > 100644 > > --- a/Makefile > > +++ b/Makefile > > @@ -517,6 +517,8 @@ OBJCOPY = > > $(LLVM_PREFIX)llvm-objcopy$(LLVM_SUFFIX) > > OBJDUMP= $(LLVM_PREFIX)llvm-objdump$(LLVM_SUFFIX) > > READELF= $(LLVM_PREFIX)llvm-readelf$(LLVM_SUFFIX) > > STRIP = $(LLVM_PREFIX)llvm-strip$(LLVM_SUFFIX) > > +LLVM_LINK = $(LLVM_PREFIX)llvm-link$(LLVM_SUFFIX) > > +LLVM_AS= $(LLVM_PREFIX)llvm-as$(LLVM_SUFFIX) > > Please keep the binutils alphabetized (i.e, put LLVM_LINK between AR and > NM). > > Other than that, this seems fine from a Kbuild perspective (but I did > not look too hard). Sure will reorder. > > else > > CC = $(CROSS_COMPILE)gcc > > LD = $(CROSS_COMPILE)ld > > @@ -625,7 +627,7 @@ export RUSTC_BOOTSTRAP := 1 > > export CLIPPY_CONF_DIR := $(srctree) > > > > export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS > > CROSS_COMPILE LD CC HOSTPKG_CONFIG > > -export RUSTC RUSTDOC RUSTFMT RUSTC_OR_CLIPPY_QUIET RUSTC_OR_CLIPPY BINDGEN > > +export RUSTC RUSTDOC RUSTFMT RUSTC_OR_CLIPPY_QUIET RUSTC_OR_CLIPPY BINDGEN > > LLVM_LINK LLVM_AS > > export HOSTRUSTC KBUILD_HOSTRUSTFLAGS > > export CPP AR NM STRIP OBJCOPY OBJDUMP READELF PAHOLE RESOLVE_BTFIDS LEX > > YACC AWK INSTALLKERNEL > > export PERL PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX > > diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug > > index > > 3034e294d50df55c4003c5fa5df442f59e711bd8..e63c5eb57b049aff988419ccd12dfd99d59f5080 > > 100644 > > --- a/lib/Kconfig.debug > > +++ b/lib/Kconfig.debug > > @@ -3427,6 +3427,21 @@ config RUST_KERNEL_DOCTESTS > > > > If unsure, say N. > > > > +config RUST_INLINE_HELPERS > > +bool "Inline C helpers into Rust crates (EXPERIMENTAL)" > > +depends on RUST && RUSTC_CLANG_LLVM_COMPATIBLE > > +depends on EXPERT > > +help > > +Links C helpers into Rust crates through LLVM IR. > > + > > +If this option is enabled, instead of generating object files > > directly, > > +rustc is asked to produce LLVM IR, which is then linked together > > with > > +the LLVM IR of C helpers, before object file is generated. > > + > > +This requires a matching LLVM version for Clang and rustc. > > + > > +If unsure, say N. > > + > > I am just curious, why would someone want (or not) to do this? This help > text does not really indicate the point of the option, just what it > does. Is it just the standard tradeoffs with inlining (potential > improvements in performance due to better optimization opportunities > versus text size increase and icache pressure) or something else? Basically you probably want this on if you can enable it, but RUSTC_CLANG_LLVM_COMPATIBLE is actually a pretty harsh requirement. If you just install rustc and clang, it's likely they wont be sufficiently compatible for LTO to take place. I'll expand on this in the help text. Alice
Re: [PATCH 4/4] build: rust: provide an option to inline C helpers into Rust
> I am just curious, why would someone want (or not) to do this? This help > text does not really indicate the point of the option, just what it > does. Is it just the standard tradeoffs with inlining (potential > improvements in performance due to better optimization opportunities > versus text size increase and icache pressure) or something else? The main situations where someone would want this off are: * Not using `clang` (should already be covered in config logic) * Out of tree module build without the whole kernel build tree (the `.bc` file produced here would need to be shipped to your out-of-tree module build environment - it essentially becomes like a header file for purposes of building an out-of-tree / DKMS Rust module) * Don't have matching `rustc` and `clang` LLVM (kind of covered in config logic - if anyone is using a non-release version of LLVM, the config may indicate them as compatible incorrectly). * Requires out-of-tree / DKMS Rust modules to build with the same LLVM revision as the kernel was built with - may be a packaging concern While the usual inlining tradeoffs apply, all of these functions have been explicitly marked `static inline`, which indicates those tradeoffs have already been thought through. I think that if we had a reliable signal of "`clang` and `rustc` use compatible bitcode", turning this on by default would be reasonable. As-is, we have a mostly-reliable signal, so defaulting it to off seems reasonable so that people don't get surprise miscompilations if they use a `clang` or `rustc` which are not using precisely a release-boundary LLVM version. People who know their toolchain story for x-lang is squared away can turn it on. > > Cheers, > Nathan
Re: [PATCH 4/4] build: rust: provide an option to inline C helpers into Rust
Hi Alice, On Tue, Dec 02, 2025 at 08:27:59PM +, Alice Ryhl wrote: ... > diff --git a/Makefile b/Makefile > index > 96ddbaae7e12de71bcfabef4639de3a13a6e4815..5834bfd568548d1bee34b328dccce5d60f85526f > 100644 > --- a/Makefile > +++ b/Makefile > @@ -517,6 +517,8 @@ OBJCOPY = > $(LLVM_PREFIX)llvm-objcopy$(LLVM_SUFFIX) > OBJDUMP = $(LLVM_PREFIX)llvm-objdump$(LLVM_SUFFIX) > READELF = $(LLVM_PREFIX)llvm-readelf$(LLVM_SUFFIX) > STRIP= $(LLVM_PREFIX)llvm-strip$(LLVM_SUFFIX) > +LLVM_LINK= $(LLVM_PREFIX)llvm-link$(LLVM_SUFFIX) > +LLVM_AS = $(LLVM_PREFIX)llvm-as$(LLVM_SUFFIX) Please keep the binutils alphabetized (i.e, put LLVM_LINK between AR and NM). Other than that, this seems fine from a Kbuild perspective (but I did not look too hard). > else > CC = $(CROSS_COMPILE)gcc > LD = $(CROSS_COMPILE)ld > @@ -625,7 +627,7 @@ export RUSTC_BOOTSTRAP := 1 > export CLIPPY_CONF_DIR := $(srctree) > > export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS CROSS_COMPILE > LD CC HOSTPKG_CONFIG > -export RUSTC RUSTDOC RUSTFMT RUSTC_OR_CLIPPY_QUIET RUSTC_OR_CLIPPY BINDGEN > +export RUSTC RUSTDOC RUSTFMT RUSTC_OR_CLIPPY_QUIET RUSTC_OR_CLIPPY BINDGEN > LLVM_LINK LLVM_AS > export HOSTRUSTC KBUILD_HOSTRUSTFLAGS > export CPP AR NM STRIP OBJCOPY OBJDUMP READELF PAHOLE RESOLVE_BTFIDS LEX > YACC AWK INSTALLKERNEL > export PERL PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX > diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug > index > 3034e294d50df55c4003c5fa5df442f59e711bd8..e63c5eb57b049aff988419ccd12dfd99d59f5080 > 100644 > --- a/lib/Kconfig.debug > +++ b/lib/Kconfig.debug > @@ -3427,6 +3427,21 @@ config RUST_KERNEL_DOCTESTS > > If unsure, say N. > > +config RUST_INLINE_HELPERS > +bool "Inline C helpers into Rust crates (EXPERIMENTAL)" > +depends on RUST && RUSTC_CLANG_LLVM_COMPATIBLE > +depends on EXPERT > +help > +Links C helpers into Rust crates through LLVM IR. > + > +If this option is enabled, instead of generating object files > directly, > +rustc is asked to produce LLVM IR, which is then linked together with > +the LLVM IR of C helpers, before object file is generated. > + > +This requires a matching LLVM version for Clang and rustc. > + > +If unsure, say N. > + I am just curious, why would someone want (or not) to do this? This help text does not really indicate the point of the option, just what it does. Is it just the standard tradeoffs with inlining (potential improvements in performance due to better optimization opportunities versus text size increase and icache pressure) or something else? Cheers, Nathan
Re: [PATCH 4/4] build: rust: provide an option to inline C helpers into Rust
On Tue, Dec 2, 2025 at 12:28 PM Alice Ryhl wrote:
>
> From: Gary Guo
>
> A new experimental Kconfig option, `RUST_INLINE_HELPERS` is added to
> allow C helpers (which were created to allow Rust to call into
> inline/macro C functions without having to re-implement the logic in
> Rust) to be inlined into Rust crates without performing global LTO.
>
> If the option is enabled, the following is performed:
> * For helpers, instead of compiling them to an object file to be linked
> into vmlinux, they're compiled to LLVM IR.
> * The LLVM IR is compiled to bitcode (This is step is not necessary, but
> is a performance optimisation to prevent LLVM from always have to
> reparse the same IR).
> * When a Rust crate is compiled, instead of generating an object file, we
> ask LLVM bitcode to be generated.
> * llvm-link is invoked with --internalize to combine the helper bitcode
> with the crate bitcode. This step is similar to LTO, but this is much
> faster since it only needs to inline the helpers.
> * clang is invoked to turn the combined bitcode into a final object file.
>
> The --internalize flag tells llvm-link to treat all symbols in
> helpers.bc using `internal` linkage. This matches the behavior of
> `clang` on `static inline` functions, and avoids exporting the symbol
> from the object file.
I've filed a PR with LLVM [1] to clarify that this is the intended
operation of the tool.
[1]: https://github.com/llvm/llvm-project/pull/170397
>
> To ensure that RUST_INLINE_HELPERS is not incompatible with BTF, we pass
> the -g0 flag when building helpers. See commit 5daa0c35a1f0 ("rust:
> Disallow BTF generation with Rust + LTO") for details.
>
> We have an intended triple mismatch of `aarch64-unknown-none` vs
> `aarch64-unknown-linux-gnu`, so we suppress the warning.
>
> Co-developed-by: Boqun Feng
> Signed-off-by: Boqun Feng
> Co-developed-by: Matthew Maurer
> Signed-off-by: Matthew Maurer
> Signed-off-by: Gary Guo
> Co-developed-by: Alice Ryhl
> Signed-off-by: Alice Ryhl
> ---
> Makefile | 4 +++-
> lib/Kconfig.debug | 15 +++
> rust/Makefile | 26 ++
> rust/exports.c | 5 -
> scripts/Makefile.build | 5 -
> 5 files changed, 48 insertions(+), 7 deletions(-)
>
> diff --git a/Makefile b/Makefile
> index
> 96ddbaae7e12de71bcfabef4639de3a13a6e4815..5834bfd568548d1bee34b328dccce5d60f85526f
> 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -517,6 +517,8 @@ OBJCOPY =
> $(LLVM_PREFIX)llvm-objcopy$(LLVM_SUFFIX)
> OBJDUMP= $(LLVM_PREFIX)llvm-objdump$(LLVM_SUFFIX)
> READELF= $(LLVM_PREFIX)llvm-readelf$(LLVM_SUFFIX)
> STRIP = $(LLVM_PREFIX)llvm-strip$(LLVM_SUFFIX)
> +LLVM_LINK = $(LLVM_PREFIX)llvm-link$(LLVM_SUFFIX)
> +LLVM_AS= $(LLVM_PREFIX)llvm-as$(LLVM_SUFFIX)
> else
> CC = $(CROSS_COMPILE)gcc
> LD = $(CROSS_COMPILE)ld
> @@ -625,7 +627,7 @@ export RUSTC_BOOTSTRAP := 1
> export CLIPPY_CONF_DIR := $(srctree)
>
> export ARCH SRCARCH CONFIG_SHELL BASH HOSTCC KBUILD_HOSTCFLAGS CROSS_COMPILE
> LD CC HOSTPKG_CONFIG
> -export RUSTC RUSTDOC RUSTFMT RUSTC_OR_CLIPPY_QUIET RUSTC_OR_CLIPPY BINDGEN
> +export RUSTC RUSTDOC RUSTFMT RUSTC_OR_CLIPPY_QUIET RUSTC_OR_CLIPPY BINDGEN
> LLVM_LINK LLVM_AS
> export HOSTRUSTC KBUILD_HOSTRUSTFLAGS
> export CPP AR NM STRIP OBJCOPY OBJDUMP READELF PAHOLE RESOLVE_BTFIDS LEX
> YACC AWK INSTALLKERNEL
> export PERL PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX
> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index
> 3034e294d50df55c4003c5fa5df442f59e711bd8..e63c5eb57b049aff988419ccd12dfd99d59f5080
> 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -3427,6 +3427,21 @@ config RUST_KERNEL_DOCTESTS
>
> If unsure, say N.
>
> +config RUST_INLINE_HELPERS
> +bool "Inline C helpers into Rust crates (EXPERIMENTAL)"
> +depends on RUST && RUSTC_CLANG_LLVM_COMPATIBLE
> +depends on EXPERT
> +help
> +Links C helpers into Rust crates through LLVM IR.
> +
> +If this option is enabled, instead of generating object files
> directly,
> +rustc is asked to produce LLVM IR, which is then linked together with
> +the LLVM IR of C helpers, before object file is generated.
> +
> +This requires a matching LLVM version for Clang and rustc.
> +
> +If unsure, say N.
> +
> endmenu # "Rust"
>
> endmenu # Kernel hacking
> diff --git a/rust/Makefile b/rust/Makefile
> index
> d7d19c21b671dea10242b1772a8bcf0bf5dcc1cd..2344e2662ce29280582215954132c09f63cd8c9d
> 100644
> --- a/rust/Makefile
> +++ b/rust/Makefile
> @@ -6,15 +6,19 @@ rustdoc_output :=
> $(objtree)/Documentation/output/rust/rustdoc
> obj-$(CONFIG_RUST) += core.o compiler_builtins.o ffi.o
> always-$(CONFIG_RUST) += exports_core_generated.h
>
> +ifdef CONFIG_RUST_INLINE_HELPERS
> +always-$(CONFIG_RUST) += helpers/helpers.bc
> +else
> +obj-$(CONFIG_RUST) += helpe
