Hi,
I recently spent some time digging into what's needed for proper eBPF build support in OpenWrt. Here's what I found so far: Most out-of-tree eBPF based projects fork some of the BPF related kernel headers from various different kernel versions and manually maintain those forks. These header files are usually very incomplete and tailored specifically for the project that uses them. To make things even worse, they typically explicitly rely on including headers from the local x86 host toolchain header files when building for the BPF target. The in-kernel build of BPF modules is weird in a different way. It explicitly includes all the header files from the arch that the kernel is being built for. And because a lot of assembly stuff in there is completely incompatible with building for the BPF target, the build system actually targets clang to the same arch that the kernel is being built for and only emits un-optimized LLVM bitcode without running any of the LLVM passes. It then passes that bitcode to the optimizer and target code generator with bpf specified as target arch. It's a weird hack, but it seems to work properly even when cross-compiling to non-x86 targets. When writing ebpf modules myself, I definitely don't want to rely on the crappy header fork mess that most out-of-tree projects use. In my tests it was simply too fragile, and I couldn't get it to work on my macOS build host either. Even if we could make it work for more use cases, it would still be a maintenance hell when comes to supporting more features and newer kernel versions. I think staying close to the way that in-tree BPF module builds work is the way to go. Unfortunately, this means that we will not be able to use GCC for BPF (which Daniel has been working on), because targeting the frontend and the rest of the compiler to different architectures is only supported by clang/LLVM. I just did a test build of LLVM with reduced features and targetting only BPF, and it takes around 42 minutes to build on my 2018 macbook. That's more than the time needed to build a typical basic OpenWrt build from scratch. The way I see it, these are our options for eBPF support: 1. Add a host dependency on a recent enough LLVM version for eBPF. 1a) disable it by default 1b) hide eBPF packages unless host support is available 2. Add llvm to tools/ to ensure that it is only built once, even when switching between targets 3. Add llvm as a host package and use build-dependencies in eBPF 4. Ship precompiled big and little endian eBPF binaries + scripts to recompile in packages, and rely on CO-RE (compile once, run everywhere) What do you think? - Felix _______________________________________________ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel