Hi all, hope everyone is staying healthy out there.
I've been working on building BPF programs, and have run into a few issues that
I think might be clang (vs gcc) based.
It seems that either clang isn't the most friendly of compilers when it comes
to building Linux-native programs, or my lack of experience makes it seem so.
I've been trying to build the simple BPF program below:
# include " bpf_helpers.h "
# include < linux/bpf.h >
# include < linux/version.h >
# include < linux/types.h >
# include < linux/tcp.h >
# include < net/sock.h >
struct inet_sock_set_state_args {
long long pad ;
const void * skaddr ;
int oldstate ;
int newstate ;
u16 sport ;
u16 dport ;
u16 family ;
u8 protocol ;
u8 saddr [ 4 ] ;
u8 daddr [ 4 ] ;
u8 saddr_v6 [ 16 ] ;
u8 daddr_v6 [ 16 ] ;
} ;
SEC ( " tracepoint/sock/inet_sock_set_state " )
int bpf_prog ( struct inet_sock_set_state_args * args ) {
struct sock * sk = ( struct sock * ) args - > skaddr ;
short lport = args - > sport ;
char msg [ ] = " lport: %d \n " ;
bpf_trace_printk ( msg , sizeof ( msg ) , lport ) ;
return 0 ;
}
char _license [ ] SEC ( " license " ) = " GPL " ;
I've been looking through selftests/bpf/, samples/bpf/, and examples on various
blogs and articles.
>From this, I've come up with the following makefile:
## Build tools
LLC := llc
CC := clang
HOSTCC := clang
CLANGINC := / usr / lib / llvm-10 / lib / clang / 10. 0. 0 / include
## Some useful flags
INC_FLAGS := -nostdinc -isystem $( CLANGINC )
EXTRA_FLAGS := -O3 -emit-llvm
## Includes
linuxhdrs := / usr / src / linux-headers- $( shell uname -r )
LINUXINCLUDE := -include $( linuxhdrs ) / include / linux / kconfig. h \
-include / usr / include / linux / bpf. h \
-I $( linuxhdrs ) / arch / x86 / include / \
-I $( linuxhdrs ) / arch / x86 / include / uapi
\
-I $( linuxhdrs ) / arch / x86 / include /
generated \
-I $( linuxhdrs ) / arch / x86 / include /
generated / uapi \
-I $( linuxhdrs ) / include \
-I $( linuxhdrs ) / include / uapi \
-I $( linuxhdrs ) / include / generated / uapi \
LIBBPF : = -I / home / vagrant / libbpf / src /
OBJS := tcptest. bpf. o
$( OBJS ) : %. o : %. c
$( CC ) $( INC_FLAGS ) \
-target bpf -D__KERNEL__ -D __ASM_SYSREG_H \
-D__BPF_TRACING__ -D__TARGET_ARCH_ $( ARCH ) \
-Wno-unused-value -Wno-pointer-sign \
-Wno-compare-distinct-pointer-types \
-Wno-gnu-variable-sized-type-not-at-end \
-Wno-address-of-packed-member \
-Wno-tautological-compare \
-Wno-unknown-warning-option \
-Wall -v \
$( LINUXINCLUDE ) $( LIBBPF ) \
$( EXTRA_FLAGS ) -c $< -o - | $( LLC ) -march=bpf -filetype obj
-o $
Unfortunately, I keep running into what seems to be asm errors. I've tried
reorganizing the list of include statements, taking out "-target bpf", not
including some files, including other files, etc etc.
This stackoverflow post suggests that it's a kconfig.h error, but I seem to be
including the file just fine
(https://stackoverflow.com/questions/56975861/error-compiling-ebpf-c-code-out-of-kernel-tree/56990939#56990939).
I'm not really sure where to go from here with building BPF programs and
including files that have the kernel datatypes. Maybe I'm missing something
that's obvious that I'm just ignorant of?
As additional information, and regarding kernel persistence, I am working on a
monitoring project that uses BPF programs to continuously monitor the system
without the bulky dependencies that BCC includes. I'm concurrently working on a
BTF/CO-RE solution but I'm emphasizing a non-CO-RE approach at the moment. I
can load and run BPF programs but upon termination of my userspace loader the
BPF programs themselves also terminate.
I would like to have the BPF program persist in the kernel even after the user
space loader has completed its execution. I read in various documentation and
in a 2015 LWN article that persistent BPF programs can be created by pinning
programs and maps to the BPF vfs so as to keep the fds open. I have attempted
pinning the entire BPF object, various programs and various maps, and no matter
what I've tried the kernel BPF program terminates when the userspace process
terminates. Using bpftool I have verified that the BPF files are pinned to the
location and that BPF programs themselves all work. I know that persistent BPF
programs are a part of projects like XDP and tc. Is there a way to do this for
a generic BPF loader without having to implement customized kernel functions?
Below I have included a simplified version of my code. In which I outline the
basic steps I take to load the compiled bpf programs and attempt to make
persistent instances of them.
#include < stdio.h >
#include < stdlib.h >
#include < string.h >
#include < errno.h >
#include < getopt.h >
#include < dirent.h >
#include < sys/stat.h >
#include < unistd.h >
#include < assert.h >
#include < linux/version.h >
#include " libbpf.h "
#include " bpf.h "
#include " loader_helpers.h "
#include < stdbool.h >
#include < fcntl.h >
#include < poll.h >
#include < linux/perf_event.h >
#include < assert.h >
#include < sys/syscall.h >
#include < sys/ioctl.h >
#include < sys/mman.h >
#include < time.h >
#include < signal.h >
#include < linux/ptrace.h >
*int* main ( *int* argc , *char* ** argv ) {
*struct* bpf_object * bpf_obj ;
*struct* bpf_program * bpf_prog ;
*struct* bpf_map * map ;
*char* * license = " GPL " ;
__u32 kernelvers = LINUX_VERSION_CODE ;
*struct* bpf_link * link ;
*int* err ;
*int* prog_fd ;
bpf_obj = bpf_object__open ( " test_file.bpf.o " ) ;
bpf_prog = bpf_program__next ( NULL , bpf_obj ) ;
err = bpf_program__set_tracepoint ( bpf_prog ) ;
*if* ( err ) {
fprintf ( stderr , " ERR couldn't setup program type \n " ) ;
*return* - 1 ;
}
err = bpf_program__load ( bpf_prog , license , kernelvers ) ;
*if* ( err ) {
fprintf ( stderr , " ERR couldn't setup program phase \n " ) ;
*return* - 1 ;
}
prog_fd = bpf_program__fd ( bpf_prog ) ;
link = bpf_program__attach_tracepoint ( bpf_prog , " syscalls " , "
sys_enter_openat " ) ;
*if* (! link ) {
fprintf ( stderr , " ERROR ATTACHING TRACEPOINT \n " ) ;
*return* - 1 ;
}
assert ( bpf_program__is_tracepoint ( bpf_prog )) ;
pin:
err = bpf_program__pin ( bpf_prog , " /sys/fs/bpf/tpprogram " ) ;
*if* ( err ) {
*if* ( err == - 17 ) {
printf ( " Program exists...trying to unpin and retry! \n " ) ;
err = bpf_program__unpin ( bpf_prog , " /sys/fs/bpf/tpprogram " ) ;
*if* (! err ) {
goto pin ;
}
printf ( " The pining already exists but it couldn't be removed... \n " ) ;
*return* - 1 ;
}
printf ( " We couldn't pin... %d \n " , err ) ;
*return* - 1 ;
}
printf ( " Program pinned and working... \n " ) ;
*return* 0 ;
}
Thanks for having a look and I hope these issues can be cleared up. Seems like
building is the last major hurdle I have to get rolling with better engineering
solutions than manually including structs in my files.
Hope everyone stays well!
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#1847): https://lists.iovisor.org/g/iovisor-dev/message/1847
Mute This Topic: https://lists.iovisor.org/mt/74140628/21656
Group Owner: [email protected]
Unsubscribe: https://lists.iovisor.org/g/iovisor-dev/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
