Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
On 6/15/18 7:24 AM, Arnaldo Carvalho de Melo wrote:
Em Thu, Jun 14, 2018 at 09:56:43PM -0700, Yonghong Song escreveu:
I really want to get rid of this option as well. To make pahole work
with the default default format, I need to add bpf support to
libdwfl in elfutils repo. I will work on that.
Right, I haven't looked into detail, but perhaps we can do like we do in
tools/perf/ where we add a feature test to check if some function is
present in a library (elfutils even) and if so, use it, otherwise, use a
copy that we carry in pahole.git.
For instance:
tools/perf/util/symbol-elf.c
#ifndef HAVE_ELF_GETPHDRNUM_SUPPORT
static int elf_getphdrnum(Elf *elf, size_t *dst)
{
GElf_Ehdr gehdr;
GElf_Ehdr *ehdr;
ehdr = gelf_getehdr(elf, );
if (!ehdr)
return -1;
*dst = ehdr->e_phnum;
return 0;
}
#endif
And we have a feature test to check if that is present, simple one, if
that builds and links, we have it, then the tools build Makefile magic
will end up defining HAVE_ELF_GETPHDRNUM_SUPPORT and our copy doesn't
get included, using what is in elfutils:
[acme@jouet perf]$ cat tools/build/feature/test-libelf-getphdrnum.c
// SPDX-License-Identifier: GPL-2.0
#include
int main(void)
{
size_t dst;
return elf_getphdrnum(0, );
}
[acme@jouet perf]$
[acme@jouet perf]$ grep elf /tmp/build/perf/FEATURE-DUMP
feature-libelf=1
feature-libelf-getphdrnum=1
feature-libelf-gelf_getnote=1
feature-libelf-getshdrstrndx=1
feature-libelf-mmap=1
[acme@jouet perf]$
This way a new pahole version won't get to wait till places where it
gets built have these new functions and we stop using it as soon as the
library get it.
Agreed that later on we can use feature testing to check pahole output
is good or not without dwarfris option. If it is good, compilation does
not need this option and we could deprecate dwarfris option and
eventually remove it once BPF support is in all major elf libraries.
BTW, I have pushed the following commit
https://reviews.llvm.org/rL334839
to clang so now you can compile with attribute dwarfris directly with
clang -tartget bpf.
-bash-4.2$ cat t.c
struct tt {
int a;
char b;
int c;
};
int test(struct tt *a) {
return a->a;
}
-bash-4.2$ clang -target bpf -O2 -g -c -Xclang -target-feature -Xclang
+dwarfris t.c
-bash-4.2$ llvm-objdump -S -d t.o
t.o:file format ELF64-BPF
Disassembly of section .text:
test:
; int test(struct tt *a) {
0: 61 10 00 00 00 00 00 00 r0 = *(u32 *)(r1 + 0)
; return a->a;
1: 95 00 00 00 00 00 00 00 exit
-bash-4.2$ pahole t.o
struct tt {
inta;/* 0 4 */
char b;/* 4 1 */
/* XXX 3 bytes hole, try to pack */
intc;/* 8 4 */
/* size: 12, cachelines: 1, members: 3 */
/* sum members: 9, holes: 1, sum holes: 3 */
/* last cacheline: 12 bytes */
};
-bash-4.2$
Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
Em Thu, Jun 14, 2018 at 09:56:43PM -0700, Yonghong Song escreveu:
> I really want to get rid of this option as well. To make pahole work
> with the default default format, I need to add bpf support to
> libdwfl in elfutils repo. I will work on that.
Right, I haven't looked into detail, but perhaps we can do like we do in
tools/perf/ where we add a feature test to check if some function is
present in a library (elfutils even) and if so, use it, otherwise, use a
copy that we carry in pahole.git.
For instance:
tools/perf/util/symbol-elf.c
#ifndef HAVE_ELF_GETPHDRNUM_SUPPORT
static int elf_getphdrnum(Elf *elf, size_t *dst)
{
GElf_Ehdr gehdr;
GElf_Ehdr *ehdr;
ehdr = gelf_getehdr(elf, );
if (!ehdr)
return -1;
*dst = ehdr->e_phnum;
return 0;
}
#endif
And we have a feature test to check if that is present, simple one, if
that builds and links, we have it, then the tools build Makefile magic
will end up defining HAVE_ELF_GETPHDRNUM_SUPPORT and our copy doesn't
get included, using what is in elfutils:
[acme@jouet perf]$ cat tools/build/feature/test-libelf-getphdrnum.c
// SPDX-License-Identifier: GPL-2.0
#include
int main(void)
{
size_t dst;
return elf_getphdrnum(0, );
}
[acme@jouet perf]$
[acme@jouet perf]$ grep elf /tmp/build/perf/FEATURE-DUMP
feature-libelf=1
feature-libelf-getphdrnum=1
feature-libelf-gelf_getnote=1
feature-libelf-getshdrstrndx=1
feature-libelf-mmap=1
[acme@jouet perf]$
This way a new pahole version won't get to wait till places where it
gets built have these new functions and we stop using it as soon as the
library get it.
- Arnaldo
Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
Hi, On Thu, Jun 07, 2018 at 04:30:29PM -0300, Arnaldo Carvalho de Melo wrote: Em Thu, Jun 07, 2018 at 12:05:10PM -0700, Martin KaFai Lau escreveu: dump-obj = true clang-opt = -g [root@jouet bpf]# perf trace -e open*,hello.c touch /tmp/hello.BTF LLVM: dumping hello.o 0.185 ( ): __bpf_stdout__:Hello, world 0.188 ( 0.009 ms): touch/19670 openat(dfd: CWD, filename: /etc/ld.so.cache, flags: CLOEXEC ) = 3 0.219 ( ): __bpf_stdout__:Hello, world 0.220 ( 0.011 ms): touch/19670 openat(dfd: CWD, filename: /lib64/libc.so.6, flags: CLOEXEC ) = 3 0.480 ( 0.095 ms): touch/19670 open(filename: /usr/lib/locale/locale-archive, flags: CLOEXEC ) = 3 0.624 ( ): __bpf_stdout__:Hello, world 0.626 ( 0.011 ms): touch/19670 openat(dfd: CWD, filename: /tmp/hello.BTF, flags: CREAT|NOCTTY|NONBLOCK|WRONLY, mode: IRUGO|IWUGO) = 3 [root@jouet bpf]# file hello.o hello.o: ELF 64-bit LSB relocatable, *unknown arch 0xf7* version 1 (SYSV), with debug_info, not stripped [root@jouet bpf]# Much better: [root@jouet bpf]# readelf -SW hello.o There are 25 section headers, starting at offset 0xc70: Section Headers: [Nr] Name TypeAddress OffSize ES Flg Lk Inf Al [ 0] NULL 00 00 00 0 0 0 [ 1] .strtab STRTAB 000b50 000119 00 0 0 1 [ 2] .text PROGBITS 40 00 00 AX 0 0 4 [ 3] syscalls:sys_enter_openat PROGBITS 40 88 00 AX 0 0 8 [ 4] .relsyscalls:sys_enter_openat REL 000910 10 10 24 3 8 [ 5] license PROGBITS c8 04 00 WA 0 0 1 [ 6] version PROGBITS cc 04 00 WA 0 0 4 [ 7] maps PROGBITS d0 10 00 WA 0 0 4 [ 8] .rodata.str1.1PROGBITS e0 0e 01 AMS 0 0 1 [ 9] .debug_strPROGBITS ee 00010e 01 MS 0 0 1 [10] .debug_locPROGBITS 0001fc 23 00 0 0 1 [11] .debug_abbrev PROGBITS 00021f e3 00 0 0 1 [12] .debug_info PROGBITS 000302 00015e 00 0 0 1 [13] .rel.debug_info REL 000920 0001e0 10 24 12 8 [14] .debug_ranges PROGBITS 000460 30 00 0 0 1 [15] .debug_macinfoPROGBITS 000490 01 00 0 0 1 [16] .debug_pubnames PROGBITS 000491 6e 00 0 0 1 [17] .rel.debug_pubnames REL 000b00 10 10 24 16 8 [18] .debug_pubtypes PROGBITS 0004ff 5a 00 0 0 1 [19] .rel.debug_pubtypes REL 000b10 10 10 24 18 8 [20] .debug_frame PROGBITS 000560 28 00 0 0 8 [21] .rel.debug_frame REL 000b20 20 10 24 20 8 [22] .debug_line PROGBITS 000588 6e 00 0 0 1 [23] .rel.debug_line REL 000b40 10 10 24 22 8 [24] .symtab SYMTAB 0005f8 000318 18 1 29 8 Key to Flags: W (write), A (alloc), X (execute), M (merge), S (strings), I (info), L (link order), O (extra OS processing required), G (group), T (TLS), C (compressed), x (unknown), o (OS specific), E (exclude), p (processor specific) [root@jouet bpf]# [root@jouet bpf]# readelf -s hello.o Symbol table '.symtab' contains 33 entries: Num:Value Size TypeBind Vis Ndx Name 0: 0 NOTYPE LOCAL DEFAULT UND 1: 0 NOTYPE LOCAL DEFAULT9 2: 002d 0 NOTYPE LOCAL DEFAULT9 3: 0044 0 NOTYPE LOCAL DEFAULT9 4: 0053 0 NOTYPE LOCAL DEFAULT9 5: 005c 0 NOTYPE LOCAL DEFAULT9 6: 0061 0 NOTYPE LOCAL DEFAULT9 7: 006a 0 NOTYPE LOCAL DEFAULT9 8: 0073 0 NOTYPE LOCAL DEFAULT9 9: 0077 0 NOTYPE LOCAL DEFAULT9 10: 0086 0 NOTYPE LOCAL DEFAULT9 11: 008b 0 NOTYPE LOCAL DEFAULT9 12: 0098 0 NOTYPE LOCAL DEFAULT9 13: 00a1 0 NOTYPE LOCAL DEFAULT9 14: 00ac 0 NOTYPE LOCAL DEFAULT9 15: 00b8 0 NOTYPE LOCAL DEFAULT9 16: 00c4 0 NOTYPE LOCAL DEFAULT9 17: 00d6 0 NOTYPE LOCAL DEFAULT9 18: 00e8 0
Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
On 6/14/18 11:00 AM, Arnaldo Carvalho de Melo wrote:
Em Thu, Jun 14, 2018 at 02:47:59PM -0300, Arnaldo Carvalho de Melo escreveu:
Em Thu, Jun 14, 2018 at 10:21:30AM -0700, Alexei Starovoitov escreveu:
On 6/14/18 10:18 AM, Arnaldo Carvalho de Melo wrote:
Just out of curiosity, is there any plan to have this as a clang option?
I think
clang ... -mllvm -mattr=dwarfris
should work.
The message "(LLVM option parsing)" implies what you suggest, but didn't
worked :-\
-mllvm Additional arguments to forward to LLVM's option
processing
Almost there tho :-\
So I thought that this -mattr=dwarfris would be available only after I
set the target, because I tried 'llc -mattr=help' and dwarfris wasn't
there:
[acme@jouet perf]$ llc -mattr=help |& grep dwarf
[acme@jouet perf]$
Only after I set the arch it appears:
[acme@jouet perf]$ llc -march=bpf -mattr=help |& grep dwarf
dwarfris - Disable MCAsmInfo DwarfUsesRelocationsAcrossSections.
dwarfris - Disable MCAsmInfo DwarfUsesRelocationsAcrossSections.
dwarfris - Disable MCAsmInfo DwarfUsesRelocationsAcrossSections.
[acme@jouet perf]$
But even after moving the '-mllvm -mattr=dwarfris' to after '-target
bpf' it still can't grok it :-\
/usr/local/bin/clang -D__KERNEL__ -D__NR_CPUS__=4 -DLINUX_VERSION_CODE=0x41100
-g -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/7/include
-I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated
-I/home/acme/git/linux/include -I./include
-I/home/acme/git/linux/arch/x86/include/uapi
-I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi
-I./include/generated/uapi -include
/home/acme/git/linux/include/linux/kconfig.h -I/home/acme/lib/include/perf/bpf
-Wno-unused-value -Wno-pointer-sign -working-directory
/lib/modules/4.17.0-rc5/build -c /home/acme/bpf/hello.c -target bpf -mllvm
-mattr=dwarfris -O2 -o hello.o
So onlye with 'clang ... -target bpf -emit-llvm -O2 -o - | llc -march=bpf
-mattr=dwarfris ...'
things work as we expect.
Right. Currently, the only way to use option -mattr=dwarfris is through
llc. The "clang -mllvm -mattr=dwarfris" won't work since
-mllvm Additional arguments to forward to LLVM's option
processing
and -mattr=dwarfris is not in LLVM auto option processing system.
Those options, in llvm source code, typically have a pattern like below:
===
static cl::opt MemCmpEqZeroNumLoadsPerBlock(
"memcmp-num-loads-per-block", cl::Hidden, cl::init(1),
cl::desc("The number of loads per basic block for inline expansion of "
"memcmp that is only being compared against zero."));
===
I really want to get rid of this option as well. To make pahole work
with the default default format, I need to add bpf support to
libdwfl in elfutils repo. I will work on that.
- Arnaldo
Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
Em Thu, Jun 14, 2018 at 02:47:59PM -0300, Arnaldo Carvalho de Melo escreveu: > Em Thu, Jun 14, 2018 at 10:21:30AM -0700, Alexei Starovoitov escreveu: > > On 6/14/18 10:18 AM, Arnaldo Carvalho de Melo wrote: > > > Just out of curiosity, is there any plan to have this as a clang option? > > I think > > clang ... -mllvm -mattr=dwarfris > > should work. > The message "(LLVM option parsing)" implies what you suggest, but didn't > worked :-\ > -mllvm Additional arguments to forward to LLVM's option > processing > Almost there tho :-\ So I thought that this -mattr=dwarfris would be available only after I set the target, because I tried 'llc -mattr=help' and dwarfris wasn't there: [acme@jouet perf]$ llc -mattr=help |& grep dwarf [acme@jouet perf]$ Only after I set the arch it appears: [acme@jouet perf]$ llc -march=bpf -mattr=help |& grep dwarf dwarfris - Disable MCAsmInfo DwarfUsesRelocationsAcrossSections. dwarfris - Disable MCAsmInfo DwarfUsesRelocationsAcrossSections. dwarfris - Disable MCAsmInfo DwarfUsesRelocationsAcrossSections. [acme@jouet perf]$ But even after moving the '-mllvm -mattr=dwarfris' to after '-target bpf' it still can't grok it :-\ /usr/local/bin/clang -D__KERNEL__ -D__NR_CPUS__=4 -DLINUX_VERSION_CODE=0x41100 -g -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/7/include -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated -I/home/acme/git/linux/include -I./include -I/home/acme/git/linux/arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi -I./include/generated/uapi -include /home/acme/git/linux/include/linux/kconfig.h -I/home/acme/lib/include/perf/bpf -Wno-unused-value -Wno-pointer-sign -working-directory /lib/modules/4.17.0-rc5/build -c /home/acme/bpf/hello.c -target bpf -mllvm -mattr=dwarfris -O2 -o hello.o So onlye with 'clang ... -target bpf -emit-llvm -O2 -o - | llc -march=bpf -mattr=dwarfris ...' things work as we expect. - Arnaldo
Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
Em Thu, Jun 14, 2018 at 10:21:30AM -0700, Alexei Starovoitov escreveu: > On 6/14/18 10:18 AM, Arnaldo Carvalho de Melo wrote: > > Just out of curiosity, is there any plan to have this as a clang option? > I think > clang ... -mllvm -mattr=dwarfris > should work. [root@jouet bpf]# cat ~/.perfconfig [llvm] dump-obj = true clang-opt = -g -mllvm -mattr=dwarfris [root@jouet bpf]# trace -e openat,hello.c touch /tmp/kafai clang (LLVM option parsing): Unknown command line argument '-mattr=dwarfris'. Try: 'clang (LLVM option parsing) -help' clang (LLVM option parsing): Did you mean '-mxgot=dwarfris'? ERROR: unable to compile hello.c Hint: Check error message shown above. Hint: You can also pre-compile it into .o using: clang -target bpf -O2 -c hello.c with proper -I and -D options. event syntax error: 'hello.c' \___ Failed to load hello.c from source: Error when compiling BPF scriptlet (add -v to see detail) [root@jouet bpf]# [root@jouet bpf]# trace -e openat,hello.c touch /tmp/kafai |& grep clang clang (LLVM option parsing): Unknown command line argument '-mattr=dwarfris'. Try: 'clang (LLVM option parsing) -help' clang (LLVM option parsing): Did you mean '-mxgot=dwarfris'? clang -target bpf -O2 -c hello.c [root@jouet bpf]# trace -v -e openat,hello.c touch /tmp/kafai |& grep clang set env: CLANG_EXEC=/usr/local/bin/clang llvm compiling command : /usr/local/bin/clang -D__KERNEL__ -D__NR_CPUS__=4 -DLINUX_VERSION_CODE=0x41100 -g -mllvm -mattr=dwarfris -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/7/include -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated -I/home/acme/git/linux/include -I./include -I/home/acme/git/linux/arch/x86/include/uapi -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi -I./include/generated/uapi -include /home/acme/git/linux/include/linux/kconfig.h -I/home/acme/lib/include/perf/bpf -Wno-unused-value -Wno-pointer-sign -working-directory /lib/modules/4.17.0-rc5/build -c /home/acme/bpf/hello.c -target bpf -O2 -o - clang (LLVM option parsing): Unknown command line argument '-mattr=dwarfris'. Try: 'clang (LLVM option parsing) -help' clang (LLVM option parsing): Did you mean '-mxgot=dwarfris'? clang -target bpf -O2 -c hello.c [root@jouet bpf]# The message "(LLVM option parsing)" implies what you suggest, but didn't worked :-\ -mllvm Additional arguments to forward to LLVM's option processing Almost there tho :-\ - Arnaldo
Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
Em Thu, Jun 14, 2018 at 10:21:30AM -0700, Alexei Starovoitov escreveu: > On 6/14/18 10:18 AM, Arnaldo Carvalho de Melo wrote: > > > > Just out of curiosity, is there any plan to have this as a clang option? > > I think > clang ... -mllvm -mattr=dwarfris thanks, trying... - Arnaldo
Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
On 6/14/18 10:18 AM, Arnaldo Carvalho de Melo wrote: Just out of curiosity, is there any plan to have this as a clang option? I think clang ... -mllvm -mattr=dwarfris should work.
Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
Em Thu, Jun 14, 2018 at 09:22:27AM -0700, Martin KaFai Lau escreveu:
> On Thu, Jun 14, 2018 at 12:03:34PM -0300, Arnaldo Carvalho de Melo wrote:
>
> > > > > > 1. The tools/testing/selftests/bpf/Makefile has the CLANG_FLAGS and
> > > > > >LLC_FLAGS needed to compile the bpf prog. It requires a new
> > > > > >"-mattr=dwarf" llc option which was added to the future
> > > > > >llvm 7.0.
>
> [ ... ]
>
> > I tried it, but it didn't work, see:
> >
> > [root@jouet bpf]# cat hello.c
> > #include "stdio.h"
> >
> > int syscall_enter(openat)(void *ctx)
> > {
> > puts("Hello, world\n");
> > return 0;
> > }
> > [root@jouet bpf]# trace -e openat,hello.c touch /tmp/kafai
> > clang-6.0: error: unknown argument: '-mattr=dwarf'
> "-mattr=dwarf" is currently a llc only option.
>
> tools/testing/selftests/bpf/Makefile has example on how to pipe clang to llc.
> e.g.:
> clang -g -O2 -target bpf -emit-llvm -c hello.c -o - | llc -march=bpf
> -mcpu=generic -mattr=dwarfris -filetype=obj -o hello.o
Ok, so I'll probably add a llvm.opts .perfconfig entry that, if present
will tell tools/perf/util/llvm-utils.c that piping the output of clang
to llvm, so that we can use llvm specific options, needs to be done.
Probably, for the time being I'll check for -g in llvm.clang-opt and if
it is there, set up the piping...
Just out of curiosity, is there any plan to have this as a clang option?
Just to finish this thing here, lemme try a slightly modified version of
your command line:
[root@jouet bpf]# clang -D__KERNEL__ -D__NR_CPUS__=4
-DLINUX_VERSION_CODE=0x41100 -g -nostdinc -isystem
/usr/lib/gcc/x86_64-redhat-linux/7/include
-I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated
-I/home/acme/git/linux/include -I./include
-I/home/acme/git/linux/arch/x86/include/uapi
-I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi
-I./include/generated/uapi -include
/home/acme/git/linux/include/linux/kconfig.h -I/home/acme/lib/include/perf/bpf
-Wno-unused-value -Wno-pointer-sign -working-directory
/lib/modules/4.17.0-rc5/build -c /home/acme/bpf/hello.c -target bpf -emit-llvm
-O2 -o - | llc -march=bpf -mcpu=generic -mattr=dwarfris -filetype=obj -o
hello2.o
[root@jouet bpf]#
[root@jouet bpf]# file hello2.o
hello2.o: ELF 64-bit LSB relocatable, *unknown arch 0xf7* version 1 (SYSV),
with debug_info, not stripped
[root@jouet bpf]# pahole hello2.o
struct bpf_map_def {
unsigned int type; /* 0 4 */
unsigned int key_size; /* 4 4 */
unsigned int value_size; /* 8 4 */
unsigned int max_entries; /*12 4 */
/* size: 16, cachelines: 1, members: 4 */
/* last cacheline: 16 bytes */
};
[root@jouet bpf]#
Finally works, thanks.
Thanks,
- Arnaldo
> > ERROR: unable to compile hello.c
> > Hint: Check error message shown above.
> > Hint: You can also pre-compile it into .o using:
> > clang -target bpf -O2 -c hello.c
> > with proper -I and -D options.
> > event syntax error: 'hello.c'
> > \___ Failed to load hello.c from source: Error when
> > compiling BPF scriptlet
> >
> > (add -v to see detail)
> > Run 'perf list' for a list of valid events
> >
> > Usage: perf trace [] []
> > or: perf trace [] -- []
> > or: perf trace record [] []
> > or: perf trace record [] -- []
> >
> > -e, --eventevent/syscall selector. use 'perf list' to list
> > available events
> > [root@jouet bpf]#
> >
> > The full command line with that is:
> >
> > [root@jouet bpf]# trace -v -e openat,hello.c touch /tmp/kafai |& grep mattr
> > set env: CLANG_OPTIONS=-g -mattr=dwarf
> > llvm compiling command : /usr/local/bin/clang -D__KERNEL__ -D__NR_CPUS__=4
> > -DLINUX_VERSION_CODE=0x41100 -g -mattr=dwarf -nostdinc -isystem
> > /usr/lib/gcc/x86_64-redhat-linux/7/include
> > -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated
> > -I/home/acme/git/linux/include -I./include
> > -I/home/acme/git/linux/arch/x86/include/uapi
> > -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi
> > -I./include/generated/uapi -include
> > /home/acme/git/linux/include/linux/kconfig.h
> > -I/home/acme/lib/include/perf/bpf -Wno-unused-value -Wno-pointer-sign
> > -working-directory /lib/modules/4.17.0-rc5/build -c /home/acme/bpf/hello.c
> > -target bpf -O2 -o -
> > clang-6.0: error: unknown argument: '-mattr=dwarf'
> > [root@jouet bpf]#
> >
> > This is with these llvm and clang trees:
> >
> > [root@jouet llvm]# git log --oneline -5
> > 98c78e82f54 (HEAD -> master, origin/master, origin/HEAD) [asan] Instrument
> > comdat globals on COFF targets
> > 6ad988b5998 [DAGCombiner] clean up comments; NFC
> > a735ba5b795 [X86][SSE] Support v8i16/v16i16 rotations
> > 1503b9f6fe8 [x86] add tests for node-level FMF; NFC
> >
Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
On Thu, Jun 14, 2018 at 12:03:34PM -0300, Arnaldo Carvalho de Melo wrote:
> > > > > 1. The tools/testing/selftests/bpf/Makefile has the CLANG_FLAGS and
> > > > >LLC_FLAGS needed to compile the bpf prog. It requires a new
> > > > >"-mattr=dwarf" llc option which was added to the future
> > > > >llvm 7.0.
[ ... ]
> I tried it, but it didn't work, see:
>
> [root@jouet bpf]# cat hello.c
> #include "stdio.h"
>
> int syscall_enter(openat)(void *ctx)
> {
> puts("Hello, world\n");
> return 0;
> }
> [root@jouet bpf]# trace -e openat,hello.c touch /tmp/kafai
> clang-6.0: error: unknown argument: '-mattr=dwarf'
"-mattr=dwarf" is currently a llc only option.
tools/testing/selftests/bpf/Makefile has example on how to pipe clang to llc.
e.g.:
clang -g -O2 -target bpf -emit-llvm -c hello.c -o - | llc -march=bpf
-mcpu=generic -mattr=dwarfris -filetype=obj -o hello.o
> ERROR:unable to compile hello.c
> Hint: Check error message shown above.
> Hint: You can also pre-compile it into .o using:
> clang -target bpf -O2 -c hello.c
> with proper -I and -D options.
> event syntax error: 'hello.c'
> \___ Failed to load hello.c from source: Error when
> compiling BPF scriptlet
>
> (add -v to see detail)
> Run 'perf list' for a list of valid events
>
> Usage: perf trace [] []
> or: perf trace [] -- []
> or: perf trace record [] []
> or: perf trace record [] -- []
>
> -e, --eventevent/syscall selector. use 'perf list' to list
> available events
> [root@jouet bpf]#
>
> The full command line with that is:
>
> [root@jouet bpf]# trace -v -e openat,hello.c touch /tmp/kafai |& grep mattr
> set env: CLANG_OPTIONS=-g -mattr=dwarf
> llvm compiling command : /usr/local/bin/clang -D__KERNEL__ -D__NR_CPUS__=4
> -DLINUX_VERSION_CODE=0x41100 -g -mattr=dwarf -nostdinc -isystem
> /usr/lib/gcc/x86_64-redhat-linux/7/include
> -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated
> -I/home/acme/git/linux/include -I./include
> -I/home/acme/git/linux/arch/x86/include/uapi
> -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi
> -I./include/generated/uapi -include
> /home/acme/git/linux/include/linux/kconfig.h
> -I/home/acme/lib/include/perf/bpf -Wno-unused-value -Wno-pointer-sign
> -working-directory /lib/modules/4.17.0-rc5/build -c /home/acme/bpf/hello.c
> -target bpf -O2 -o -
> clang-6.0: error: unknown argument: '-mattr=dwarf'
> [root@jouet bpf]#
>
> This is with these llvm and clang trees:
>
> [root@jouet llvm]# git log --oneline -5
> 98c78e82f54 (HEAD -> master, origin/master, origin/HEAD) [asan] Instrument
> comdat globals on COFF targets
> 6ad988b5998 [DAGCombiner] clean up comments; NFC
> a735ba5b795 [X86][SSE] Support v8i16/v16i16 rotations
> 1503b9f6fe8 [x86] add tests for node-level FMF; NFC
> 4a49826736f [x86] regenerate test checks; NFC
> [root@jouet llvm]#
>
> [root@jouet llvm]# cd tools/clang/
> [root@jouet clang]# git log --oneline -5
> 8c873daccc (HEAD -> master, origin/master, origin/HEAD) [X86] Add builtins
> for vpermq/vpermpd instructions to enable target feature checking.
> a344be6ba4 [X86] Change immediate type for some builtins from char to int.
> dcdd53793e [CUDA] Fix emission of constant strings in sections
> a90c85acaf [X86] Add builtins for shufps and shufpd to enable target feature
> and immediate range checking.
> ff71c0eccc [X86] Add builtins for pshufd, pshuflw, and pshufhw to enable
> target feature and immediate range checking.
> [root@jouet clang]#
>
> [root@jouet clang]# git log | grep mattr=dwarf
> [root@jouet clang]# cd -
> /home/acme/git.tmp/git/llvm
> [root@jouet llvm]# git log | grep mattr=dwarf
> bpf: introduce -mattr=dwarfris to disable
> DwarfUsesRelocationsAcrossSections
> This patch introduces a new flag -mattr=dwarfris
> [root@jouet llvm]#
>
> Humm, so its -mattr=dwarfris and not -attr=dwarf?
>
> Didn't help :-\
>
> commit 0e0047f8c9ada2f0fe0c5f01579a80e2455b8df5
> Author: Yonghong Song
> Date: Thu Mar 1 23:04:59 2018 +
>
> bpf: introduce -mattr=dwarfris to disable
> DwarfUsesRelocationsAcrossSections
>
> Commit e4507fb8c94b ("bpf: disable DwarfUsesRelocationsAcrossSections")
> disables MCAsmInfo DwarfUsesRelocationsAcrossSections unconditionally
> so that dwarf will not use cross section (between dwarf and symbol table)
> relocations. This new debug format enables pahole to dump structures
> correctly as libdwarves.so does not have BPF backend support yet.
>
> This new debug format, however, breaks bcc
> (https://github.com/iovisor/bcc)
> source debug output as llvm in-memory Dwarf support has some issues to
> handle it. More specifically, with DwarfUsesRelocationsAcrossSections
> disabled, JIT compiler does not generate .debug_abbrev and Dwarf
> DIE (debug info entry) processing is not happy about this.
>
> This patch introduces a new flag
Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
Em Wed, Jun 13, 2018 at 04:26:38PM -0700, Martin KaFai Lau escreveu:
> On Tue, Jun 12, 2018 at 05:41:26PM -0300, Arnaldo Carvalho de Melo wrote:
> > Em Tue, Jun 12, 2018 at 05:31:24PM -0300, Arnaldo Carvalho de Melo escreveu:
> > > Em Thu, Jun 07, 2018 at 01:07:01PM -0700, Martin KaFai Lau escreveu:
> > > > On Thu, Jun 07, 2018 at 04:30:29PM -0300, Arnaldo Carvalho de Melo
> > > > wrote:
> > > > > So this must be available in a newer llvm version? Which one?
> > > > I should have put in the details in my last email or
> > > > in the commit message, my bad.
> > > > 1. The tools/testing/selftests/bpf/Makefile has the CLANG_FLAGS and
> > > >LLC_FLAGS needed to compile the bpf prog. It requires a new
> > > >"-mattr=dwarf" llc option which was added to the future
> > > >llvm 7.0.
> > > [root@jouet bpf]# pahole hello.o
> > > struct clang version 5.0.1 (tags/RELEASE_501/final) {
> > > clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1
> > > (tags/RELEASE_501/final); /* 0 4 */
> > > clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1
> > > (tags/RELEASE_501/final); /* 4 4 */
> > > clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1
> > > (tags/RELEASE_501/final); /* 8 4 */
> > > clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1
> > > (tags/RELEASE_501/final); /*12 4 */
> > > /* size: 16, cachelines: 1, members: 4 */
> > > /* last cacheline: 16 bytes */
> > > };
> > > [root@jouet bpf]#
> > >
> > > Ok, I guess I saw this case in the llvm/clang git logs, so this one was
> > > generated with the older clang, will regenerate and add that
> > > "-mattr=dwarf"
> > > part.
> > [root@jouet bpf]# pahole hello.o
> > struct clang version 7.0.0
> > /* size: 16, cachelines: 1, members: 4 */
> > /* last cacheline: 16 bytes */
> > };
> That means the "-mattr=dwarf" is not effective.
> Can you share your clang and llc command to create hello.o?
I tried it, but it didn't work, see:
[root@jouet bpf]# cat hello.c
#include "stdio.h"
int syscall_enter(openat)(void *ctx)
{
puts("Hello, world\n");
return 0;
}
[root@jouet bpf]# trace -e openat,hello.c touch /tmp/kafai
clang-6.0: error: unknown argument: '-mattr=dwarf'
ERROR: unable to compile hello.c
Hint: Check error message shown above.
Hint: You can also pre-compile it into .o using:
clang -target bpf -O2 -c hello.c
with proper -I and -D options.
event syntax error: 'hello.c'
\___ Failed to load hello.c from source: Error when
compiling BPF scriptlet
(add -v to see detail)
Run 'perf list' for a list of valid events
Usage: perf trace [] []
or: perf trace [] -- []
or: perf trace record [] []
or: perf trace record [] -- []
-e, --eventevent/syscall selector. use 'perf list' to list
available events
[root@jouet bpf]#
The full command line with that is:
[root@jouet bpf]# trace -v -e openat,hello.c touch /tmp/kafai |& grep mattr
set env: CLANG_OPTIONS=-g -mattr=dwarf
llvm compiling command : /usr/local/bin/clang -D__KERNEL__ -D__NR_CPUS__=4
-DLINUX_VERSION_CODE=0x41100 -g -mattr=dwarf -nostdinc -isystem
/usr/lib/gcc/x86_64-redhat-linux/7/include
-I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated
-I/home/acme/git/linux/include -I./include
-I/home/acme/git/linux/arch/x86/include/uapi
-I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi
-I./include/generated/uapi -include
/home/acme/git/linux/include/linux/kconfig.h -I/home/acme/lib/include/perf/bpf
-Wno-unused-value -Wno-pointer-sign -working-directory
/lib/modules/4.17.0-rc5/build -c /home/acme/bpf/hello.c -target bpf -O2 -o -
clang-6.0: error: unknown argument: '-mattr=dwarf'
[root@jouet bpf]#
This is with these llvm and clang trees:
[root@jouet llvm]# git log --oneline -5
98c78e82f54 (HEAD -> master, origin/master, origin/HEAD) [asan] Instrument
comdat globals on COFF targets
6ad988b5998 [DAGCombiner] clean up comments; NFC
a735ba5b795 [X86][SSE] Support v8i16/v16i16 rotations
1503b9f6fe8 [x86] add tests for node-level FMF; NFC
4a49826736f [x86] regenerate test checks; NFC
[root@jouet llvm]#
[root@jouet llvm]# cd tools/clang/
[root@jouet clang]# git log --oneline -5
8c873daccc (HEAD -> master, origin/master, origin/HEAD) [X86] Add builtins for
vpermq/vpermpd instructions to enable target feature checking.
a344be6ba4 [X86] Change immediate type for some builtins from char to int.
dcdd53793e [CUDA] Fix emission of constant strings in sections
a90c85acaf [X86] Add builtins for shufps and shufpd to enable target feature
and immediate range checking.
ff71c0eccc [X86] Add builtins for pshufd, pshuflw, and pshufhw to enable target
feature and immediate range checking.
[root@jouet clang]#
[root@jouet clang]# git log | grep mattr=dwarf
[root@jouet clang]# cd -
/home/acme/git.tmp/git/llvm
[root@jouet llvm]# git log | grep mattr=dwarf
bpf: introduce
Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
On Tue, Jun 12, 2018 at 05:41:26PM -0300, Arnaldo Carvalho de Melo wrote:
> Em Tue, Jun 12, 2018 at 05:31:24PM -0300, Arnaldo Carvalho de Melo escreveu:
> > Em Thu, Jun 07, 2018 at 01:07:01PM -0700, Martin KaFai Lau escreveu:
> > > On Thu, Jun 07, 2018 at 04:30:29PM -0300, Arnaldo Carvalho de Melo wrote:
> > > > So this must be available in a newer llvm version? Which one?
> >
> > > I should have put in the details in my last email or
> > > in the commit message, my bad.
> >
> > > 1. The tools/testing/selftests/bpf/Makefile has the CLANG_FLAGS and
> > >LLC_FLAGS needed to compile the bpf prog. It requires a new
> > >"-mattr=dwarf" llc option which was added to the future
> > >llvm 7.0.
>
> > [root@jouet bpf]# pahole hello.o
> > struct clang version 5.0.1 (tags/RELEASE_501/final) {
> > clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1
> > (tags/RELEASE_501/final); /* 0 4 */
> > clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1
> > (tags/RELEASE_501/final); /* 4 4 */
> > clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1
> > (tags/RELEASE_501/final); /* 8 4 */
> > clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1
> > (tags/RELEASE_501/final); /*12 4 */
> >
> > /* size: 16, cachelines: 1, members: 4 */
> > /* last cacheline: 16 bytes */
> > };
> > [root@jouet bpf]#
> >
> > Ok, I guess I saw this case in the llvm/clang git logs, so this one was
> > generated with the older clang, will regenerate and add that "-mattr=dwarf"
> > part.
>
> [root@jouet bpf]# pahole hello.o
> struct clang version 7.0.0
> (https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_git_clang.git=DwIBAg=5VD0RTtNlTh3ycd41b3MUw=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4=4d495SlcvobgBOFahId75gM-V2su4Qq2wiLOGkU-adI=_Qzsu689xEjjl9JvYCvJsIZLZZKDLB6rM-Uc0gqQvyg=
> 8c873daccce7ee5339b9fd82c81fe02b73543b65)
> (https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_git_llvm.git=DwIBAg=5VD0RTtNlTh3ycd41b3MUw=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4=4d495SlcvobgBOFahId75gM-V2su4Qq2wiLOGkU-adI=cFz6VP_YIYy_hubsx05WDqpTDyXl0Wnx_RAmAl1dbpg=
> 98c78e82f54be8fb0bb5f02e3ca674fbde10ef34) {
> clang version 7.0.0
> (https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_git_clang.git=DwIBAg=5VD0RTtNlTh3ycd41b3MUw=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4=4d495SlcvobgBOFahId75gM-V2su4Qq2wiLOGkU-adI=_Qzsu689xEjjl9JvYCvJsIZLZZKDLB6rM-Uc0gqQvyg=
> 8c873daccce7ee5339b9fd82c81fe02b73543b65)
> (https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_git_llvm.git=DwIBAg=5VD0RTtNlTh3ycd41b3MUw=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4=4d495SlcvobgBOFahId75gM-V2su4Qq2wiLOGkU-adI=cFz6VP_YIYy_hubsx05WDqpTDyXl0Wnx_RAmAl1dbpg=
> 98c78 clang version 7.0.0
> (https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_git_clang.git=DwIBAg=5VD0RTtNlTh3ycd41b3MUw=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4=4d495SlcvobgBOFahId75gM-V2su4Qq2wiLOGkU-adI=_Qzsu689xEjjl9JvYCvJsIZLZZKDLB6rM-Uc0gqQvyg=
> 8c873daccce7ee5339b9fd82c81fe02b73543b65)
> (https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_git_llvm.git=DwIBAg=5VD0RTtNlTh3ycd41b3MUw=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4=4d495SlcvobgBOFahId75gM-V2su4Qq2wiLOGkU-adI=cFz6VP_YIYy_hubsx05WDqpTDyXl0Wnx_RAmAl1dbpg=
> 98c78e82f54be8fb0bb5f02e3ca674fbde10ef34); /* 0 4 */
> clang version 7.0.0
> (https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_git_clang.git=DwIBAg=5VD0RTtNlTh3ycd41b3MUw=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4=4d495SlcvobgBOFahId75gM-V2su4Qq2wiLOGkU-adI=_Qzsu689xEjjl9JvYCvJsIZLZZKDLB6rM-Uc0gqQvyg=
> 8c873daccce7ee5339b9fd82c81fe02b73543b65)
> (https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_git_llvm.git=DwIBAg=5VD0RTtNlTh3ycd41b3MUw=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4=4d495SlcvobgBOFahId75gM-V2su4Qq2wiLOGkU-adI=cFz6VP_YIYy_hubsx05WDqpTDyXl0Wnx_RAmAl1dbpg=
> 98c78 clang version 7.0.0
> (https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_git_clang.git=DwIBAg=5VD0RTtNlTh3ycd41b3MUw=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4=4d495SlcvobgBOFahId75gM-V2su4Qq2wiLOGkU-adI=_Qzsu689xEjjl9JvYCvJsIZLZZKDLB6rM-Uc0gqQvyg=
> 8c873daccce7ee5339b9fd82c81fe02b73543b65)
> (https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_git_llvm.git=DwIBAg=5VD0RTtNlTh3ycd41b3MUw=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4=4d495SlcvobgBOFahId75gM-V2su4Qq2wiLOGkU-adI=cFz6VP_YIYy_hubsx05WDqpTDyXl0Wnx_RAmAl1dbpg=
> 98c78e82f54be8fb0bb5f02e3ca674fbde10ef34); /* 4 4 */
> clang version 7.0.0
> (https://urldefense.proofpoint.com/v2/url?u=http-3A__llvm.org_git_clang.git=DwIBAg=5VD0RTtNlTh3ycd41b3MUw=i6WobKxbeG3slzHSIOxTVtYIJw7qjCE6S0spDTKL-J4=4d495SlcvobgBOFahId75gM-V2su4Qq2wiLOGkU-adI=_Qzsu689xEjjl9JvYCvJsIZLZZKDLB6rM-Uc0gqQvyg=
> 8c873daccce7ee5339b9fd82c81fe02b73543b65)
>
Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
Em Tue, Jun 12, 2018 at 05:31:24PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Thu, Jun 07, 2018 at 01:07:01PM -0700, Martin KaFai Lau escreveu:
> > On Thu, Jun 07, 2018 at 04:30:29PM -0300, Arnaldo Carvalho de Melo wrote:
> > > So this must be available in a newer llvm version? Which one?
>
> > I should have put in the details in my last email or
> > in the commit message, my bad.
>
> > 1. The tools/testing/selftests/bpf/Makefile has the CLANG_FLAGS and
> >LLC_FLAGS needed to compile the bpf prog. It requires a new
> >"-mattr=dwarf" llc option which was added to the future
> >llvm 7.0.
> [root@jouet bpf]# pahole hello.o
> struct clang version 5.0.1 (tags/RELEASE_501/final) {
> clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1
> (tags/RELEASE_501/final); /* 0 4 */
> clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1
> (tags/RELEASE_501/final); /* 4 4 */
> clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1
> (tags/RELEASE_501/final); /* 8 4 */
> clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1
> (tags/RELEASE_501/final); /*12 4 */
>
> /* size: 16, cachelines: 1, members: 4 */
> /* last cacheline: 16 bytes */
> };
> [root@jouet bpf]#
>
> Ok, I guess I saw this case in the llvm/clang git logs, so this one was
> generated with the older clang, will regenerate and add that "-mattr=dwarf"
> part.
[root@jouet bpf]# pahole hello.o
struct clang version 7.0.0 (http://llvm.org/git/clang.git
8c873daccce7ee5339b9fd82c81fe02b73543b65) (http://llvm.org/git/llvm.git
98c78e82f54be8fb0bb5f02e3ca674fbde10ef34) {
clang version 7.0.0 (http://llvm.org/git/clang.git
8c873daccce7ee5339b9fd82c81fe02b73543b65) (http://llvm.org/git/llvm.git 98c78
clang version 7.0.0 (http://llvm.org/git/clang.git
8c873daccce7ee5339b9fd82c81fe02b73543b65) (http://llvm.org/git/llvm.git
98c78e82f54be8fb0bb5f02e3ca674fbde10ef34); /* 0 4 */
clang version 7.0.0 (http://llvm.org/git/clang.git
8c873daccce7ee5339b9fd82c81fe02b73543b65) (http://llvm.org/git/llvm.git 98c78
clang version 7.0.0 (http://llvm.org/git/clang.git
8c873daccce7ee5339b9fd82c81fe02b73543b65) (http://llvm.org/git/llvm.git
98c78e82f54be8fb0bb5f02e3ca674fbde10ef34); /* 4 4 */
clang version 7.0.0 (http://llvm.org/git/clang.git
8c873daccce7ee5339b9fd82c81fe02b73543b65) (http://llvm.org/git/llvm.git 98c78
clang version 7.0.0 (http://llvm.org/git/clang.git
8c873daccce7ee5339b9fd82c81fe02b73543b65) (http://llvm.org/git/llvm.git
98c78e82f54be8fb0bb5f02e3ca674fbde10ef34); /* 8 4 */
clang version 7.0.0 (http://llvm.org/git/clang.git
8c873daccce7ee5339b9fd82c81fe02b73543b65) (http://llvm.org/git/llvm.git 98c78
clang version 7.0.0 (http://llvm.org/git/clang.git
8c873daccce7ee5339b9fd82c81fe02b73543b65) (http://llvm.org/git/llvm.git
98c78e82f54be8fb0bb5f02e3ca674fbde10ef34); /*12 4 */
/* size: 16, cachelines: 1, members: 4 */
/* last cacheline: 16 bytes */
};
[root@jouet bpf]#
Ideas?
[root@jouet bpf]# trace -e open*,hello.c
clang-6.0: error: unknown argument: '-mattr=dwarf'
ERROR: unable to compile hello.c
Hint: Check error message shown above.
Hint: You can also pre-compile it into .o using:
clang -target bpf -O2 -c hello.c
with proper -I and -D options.
event syntax error: 'hello.c'
\___ Failed to load hello.c from source: Error when
compiling BPF scriptlet
(add -v to see detail)
Run 'perf list' for a list of valid events
Usage: perf trace [] []
or: perf trace [] -- []
or: perf trace record [] []
or: perf trace record [] -- []
-e, --eventevent/syscall selector. use 'perf list' to list
available events
[root@jouet bpf]#
[root@jouet bpf]# trace -v -e open*,hello.c
bpf: builtin compilation failed: -95, try external compiler
Kernel build dir is set to /lib/modules/4.17.0-rc5/build
set env: KBUILD_DIR=/lib/modules/4.17.0-rc5/build
unset env: KBUILD_OPTS
include option is set to -nostdinc -isystem
/usr/lib/gcc/x86_64-redhat-linux/7/include
-I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated
-I/home/acme/git/linux/include -I./include
-I/home/acme/git/linux/arch/x86/include/uapi
-I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi
-I./include/generated/uapi -include
/home/acme/git/linux/include/linux/kconfig.h
set env: NR_CPUS=4
set env: LINUX_VERSION_CODE=0x41100
set env: CLANG_EXEC=/usr/local/bin/clang
set env: CLANG_OPTIONS=-g -mattr=dwarf
set env: KERNEL_INC_OPTIONS= -nostdinc -isystem
/usr/lib/gcc/x86_64-redhat-linux/7/include
-I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated
-I/home/acme/git/linux/include -I./include
-I/home/acme/git/linux/arch/x86/include/uapi
-I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi
-I./include/generated/uapi -include
Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
Em Thu, Jun 07, 2018 at 01:07:01PM -0700, Martin KaFai Lau escreveu:
> On Thu, Jun 07, 2018 at 04:30:29PM -0300, Arnaldo Carvalho de Melo wrote:
> > So this must be available in a newer llvm version? Which one?
> I should have put in the details in my last email or
> in the commit message, my bad.
> 1. The tools/testing/selftests/bpf/Makefile has the CLANG_FLAGS and
>LLC_FLAGS needed to compile the bpf prog. It requires a new
>"-mattr=dwarf" llc option which was added to the future
>llvm 7.0.
>Hence, I have been using the llvm's master in github which
>also has the llvm-objcopy.
> 2. The kernel's btf part only focus on the BPF map.
>Hence, the testing bpf program should have the map's key
>and map's value. e.g. tools/testing/selftests/bpf/test_btf_haskv.c
So, with llvm and clang HEAD I get:
[root@jouet bpf]# pahole -J hello.o
[root@jouet bpf]# file hello.o
hello.o: ELF 64-bit LSB relocatable, *unknown arch 0xf7* version 1 (SYSV), with
debug_info, not stripped
[root@jouet bpf]# llvm-readelf -s hello.o
There are 26 section headers, starting at offset 0xe30:
Section Headers:
[Nr] Name TypeAddress OffSize ES Flg
Lk Inf Al
[ 0] NULL 00 00 00
0 0 0
[ 1] .text PROGBITS 40 00 00 AX
0 0 4
[ 2] syscalls:sys_enter_openat PROGBITS 40 88 00 AX
0 0 8
[ 3] license PROGBITS c8 04 00 WA
0 0 1
[ 4] version PROGBITS cc 04 00 WA
0 0 4
[ 5] maps PROGBITS d0 10 00 WA
0 0 4
[ 6] .rodata.str1.1PROGBITS e0 0e 01 AMS
0 0 1
[ 7] .debug_strPROGBITS ee 00010e 01 MS
0 0 1
[ 8] .debug_locPROGBITS 0001fc 23 00
0 0 1
[ 9] .debug_abbrev PROGBITS 00021f e3 00
0 0 1
[10] .debug_info PROGBITS 000302 00015e 00
0 0 1
[11] .debug_ranges PROGBITS 000460 30 00
0 0 1
[12] .debug_macinfoPROGBITS 000490 01 00
0 0 1
[13] .debug_pubnames PROGBITS 000491 6e 00
0 0 1
[14] .debug_pubtypes PROGBITS 0004ff 5a 00
0 0 1
[15] .debug_frame PROGBITS 000560 28 00
0 0 8
[16] .debug_line PROGBITS 000588 6e 00
0 0 1
[17] .symtab SYMTAB 0005f8 000318 18
24 29 8
[18] .relsyscalls:sys_enter_openat REL 000910 10 10
17 2 8
[19] .rel.debug_info REL 000920 0001e0 10
17 10 8
[20] .rel.debug_pubnames REL 000b00 10 10
17 13 8
[21] .rel.debug_pubtypes REL 000b10 10 10
17 14 8
[22] .rel.debug_frame REL 000b20 20 10
17 15 8
[23] .rel.debug_line REL 000b40 10 10
17 16 8
[24] .strtab STRTAB 000b50 00018e 00
0 0 1
[25] .BTF PROGBITS 000cde 00014e 00
0 0 1
Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings), l (large)
I (info), L (link order), G (group), T (TLS), E (exclude), x (unknown)
O (extra OS processing required) o (OS specific), p (processor specific)
[root@jouet bpf]#
[root@jouet bpf]# pahole hello.o
struct clang version 5.0.1 (tags/RELEASE_501/final) {
clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1
(tags/RELEASE_501/final); /* 0 4 */
clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1
(tags/RELEASE_501/final); /* 4 4 */
clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1
(tags/RELEASE_501/final); /* 8 4 */
clang version 5.0.1 (tags/RELEASE_501/final) clang version 5.0.1
(tags/RELEASE_501/final); /*12 4 */
/* size: 16, cachelines: 1, members: 4 */
/* last cacheline: 16 bytes */
};
[root@jouet bpf]#
Ok, I guess I saw this case in the llvm/clang git logs, so this one was
generated with the older clang, will regenerate and add that "-mattr=dwarf"
part.
- Arnaldo
Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
Em Thu, Jun 07, 2018 at 01:07:01PM -0700, Martin KaFai Lau escreveu:
> On Thu, Jun 07, 2018 at 04:30:29PM -0300, Arnaldo Carvalho de Melo wrote:
> > So this must be available in a newer llvm version? Which one?
> I should have put in the details in my last email or
> in the commit message, my bad.
> 1. The tools/testing/selftests/bpf/Makefile has the CLANG_FLAGS and
>LLC_FLAGS needed to compile the bpf prog. It requires a new
>"-mattr=dwarf" llc option which was added to the future
>llvm 7.0.
>Hence, I have been using the llvm's master in github which
>also has the llvm-objcopy.
> 2. The kernel's btf part only focus on the BPF map.
>Hence, the testing bpf program should have the map's key
>and map's value. e.g. tools/testing/selftests/bpf/test_btf_haskv.c
Thanks for the version required to test this, but where is this
test_btf_haskv.c file? Which tree? net-next?
Ok, just pulled torvalds/master and there it is. Gotcha.
struct bpf_map_def SEC("maps") __bpf_stdout__ = {
.type = BPF_MAP_TYPE_PERF_EVENT_ARRAY,
.key_size = sizeof(int),
.value_size = sizeof(u32),
.max_entries = __NR_CPUS__,
};
This map is in the above hello.c example, but I guess its way too simple
:-)
Ok, I'll test this at home in another machine where I have the llvm's
git repo.
- Arnaldo
Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
On Thu, Jun 07, 2018 at 04:30:29PM -0300, Arnaldo Carvalho de Melo wrote:
> Em Thu, Jun 07, 2018 at 12:05:10PM -0700, Martin KaFai Lau escreveu:
> > On Thu, Jun 07, 2018 at 11:03:37AM -0300, Arnaldo Carvalho de Melo wrote:
> > > Em Thu, Jun 07, 2018 at 10:54:01AM -0300, Arnaldo Carvalho de Melo
> > > escreveu:
> > > > Em Tue, Jun 05, 2018 at 02:25:48PM -0700, Martin KaFai Lau escreveu:
> > > > > [ btw, the latest commit (1 commit) should be 94a11b59e592 ].
>
> > > So, the commit log message for the pahole patch is non-existent:
>
> > > https://github.com/iamkafai/pahole/commit/94a11b59e5920908085bfc8d24c92f95c8ffceaf
>
> > > we should do better in describing what is done and how, I'm staring
> > > with a message you sent to the kernel part:
> > > --
> > > This patch introduces BPF Type Format (BTF).
>
> > > BTF (BPF Type Format) is the meta data format which describes
> > > the data types of BPF program/map. Hence, it basically focus
> > > on the C programming language which the modern BPF is primary
> > > using. The first use case is to provide a generic pretty print
> > > capability for a BPF map.
>
> > I will add details in the next github respin/push.
>
> Ok, but I can do that if there is nothing else to do in the code at this
> stage :-)
>
> > > Now I'm going to do the step-by-step guide on testing the feature just
> > > introduced, and will try to convert from dwarf to BTF and back, compare
> > > the pahole output for types encoded in DWARF and BTF, etc.
>
> > > If you have something ressembling this already, please share.
>
> > The pahole only has the encoder part. I tested with the verbose output
> > from the "pahole -V -J". Loading the btf to the kernel is also tested.
>
> Ok, so here it goes my first stab at testing, using perf's BPF
> integration:
>
> [root@jouet bpf]# cat hello.c
> #include "stdio.h"
>
> int syscall_enter(openat)(void *ctx)
> {
> puts("Hello, world\n");
> return 0;
> }
> [root@jouet bpf]# cat ~/.perfconfig
> [llvm]
> dump-obj = true
> [root@jouet bpf]# perf trace -e open*,hello.c touch /tmp/hello.BTF
> LLVM: dumping hello.o
> 0.017 ( ): __bpf_stdout__:Hello, world
> 0.019 ( 0.011 ms): touch/28147 openat(dfd: CWD, filename:
> /etc/ld.so.cache, flags: CLOEXEC ) = 3
> 0.053 ( ): __bpf_stdout__:Hello, world
> 0.055 ( 0.011 ms): touch/28147 openat(dfd: CWD, filename:
> /lib64/libc.so.6, flags: CLOEXEC ) = 3
> 0.354 ( 0.012 ms): touch/28147 open(filename:
> /usr/lib/locale/locale-archive, flags: CLOEXEC ) = 3
> 0.411 ( ): __bpf_stdout__:Hello, world
> 0.412 ( 0.198 ms): touch/28147 openat(dfd: CWD, filename:
> /tmp/hello.BTF, flags: CREAT|NOCTTY|NONBLOCK|WRONLY, mode: IRUGO|IWUGO) = 3
> [root@jouet bpf]#
> [root@jouet bpf]# file hello.o
> hello.o: ELF 64-bit LSB relocatable, *unknown arch 0xf7* version 1 (SYSV),
> not stripped
> [root@jouet bpf]# pahole --btf_encode hello.o
> pahole: hello.o: No debugging information found
> [root@jouet bpf]#
>
> [root@jouet bpf]# readelf -s hello.o
>
> Symbol table '.symtab' contains 5 entries:
>Num:Value Size TypeBind Vis Ndx Name
> 0: 0 NOTYPE LOCAL DEFAULT UND
> 1: 0 NOTYPE GLOBAL DEFAULT7 __bpf_stdout__
> 2: 0 NOTYPE GLOBAL DEFAULT5 _license
> 3: 0 NOTYPE GLOBAL DEFAULT6 _version
> 4: 0 NOTYPE GLOBAL DEFAULT3
> syscall_enter_openat
> [root@jouet bpf]#
> [root@jouet bpf]# readelf -SW hello.o
> There are 10 section headers, starting at offset 0x1f8:
>
> Section Headers:
> [Nr] Name TypeAddress OffSize ES
> Flg Lk Inf Al
> [ 0] NULL 00 00 00
> 0 0 0
> [ 1] .strtab STRTAB 000178 7f 00
> 0 0 1
> [ 2] .text PROGBITS 40 00 00
> AX 0 0 4
> [ 3] syscalls:sys_enter_openat PROGBITS 40
> 88 00 AX 0 0 8
> [ 4] .relsyscalls:sys_enter_openat REL 000168
> 10 10 9 3 8
> [ 5] license PROGBITS c8 04 00
> WA 0 0 1
> [ 6] version PROGBITS cc 04 00
> WA 0 0 4
> [ 7] maps PROGBITS d0 10 00
> WA 0 0 4
> [ 8] .rodata.str1.1PROGBITS e0 0e 01
> AMS 0 0 1
> [ 9] .symtab SYMTAB f0 78 18
> 1 1 8
> Key to Flags:
> W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
> L (link order), O (extra OS processing required), G (group), T (TLS),
> C (compressed), x (unknown), o (OS specific), E (exclude),
> p (processor
Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
Em Thu, Jun 07, 2018 at 12:05:10PM -0700, Martin KaFai Lau escreveu:
> On Thu, Jun 07, 2018 at 11:03:37AM -0300, Arnaldo Carvalho de Melo wrote:
> > Em Thu, Jun 07, 2018 at 10:54:01AM -0300, Arnaldo Carvalho de Melo escreveu:
> > > Em Tue, Jun 05, 2018 at 02:25:48PM -0700, Martin KaFai Lau escreveu:
> > > > [ btw, the latest commit (1 commit) should be 94a11b59e592 ].
> > So, the commit log message for the pahole patch is non-existent:
> > https://github.com/iamkafai/pahole/commit/94a11b59e5920908085bfc8d24c92f95c8ffceaf
> > we should do better in describing what is done and how, I'm staring
> > with a message you sent to the kernel part:
> > --
> > This patch introduces BPF Type Format (BTF).
> > BTF (BPF Type Format) is the meta data format which describes
> > the data types of BPF program/map. Hence, it basically focus
> > on the C programming language which the modern BPF is primary
> > using. The first use case is to provide a generic pretty print
> > capability for a BPF map.
> I will add details in the next github respin/push.
Ok, but I can do that if there is nothing else to do in the code at this
stage :-)
> > Now I'm going to do the step-by-step guide on testing the feature just
> > introduced, and will try to convert from dwarf to BTF and back, compare
> > the pahole output for types encoded in DWARF and BTF, etc.
> > If you have something ressembling this already, please share.
> The pahole only has the encoder part. I tested with the verbose output
> from the "pahole -V -J". Loading the btf to the kernel is also tested.
Ok, so here it goes my first stab at testing, using perf's BPF
integration:
[root@jouet bpf]# cat hello.c
#include "stdio.h"
int syscall_enter(openat)(void *ctx)
{
puts("Hello, world\n");
return 0;
}
[root@jouet bpf]# cat ~/.perfconfig
[llvm]
dump-obj = true
[root@jouet bpf]# perf trace -e open*,hello.c touch /tmp/hello.BTF
LLVM: dumping hello.o
0.017 ( ): __bpf_stdout__:Hello, world
0.019 ( 0.011 ms): touch/28147 openat(dfd: CWD, filename:
/etc/ld.so.cache, flags: CLOEXEC ) = 3
0.053 ( ): __bpf_stdout__:Hello, world
0.055 ( 0.011 ms): touch/28147 openat(dfd: CWD, filename:
/lib64/libc.so.6, flags: CLOEXEC ) = 3
0.354 ( 0.012 ms): touch/28147 open(filename:
/usr/lib/locale/locale-archive, flags: CLOEXEC ) = 3
0.411 ( ): __bpf_stdout__:Hello, world
0.412 ( 0.198 ms): touch/28147 openat(dfd: CWD, filename: /tmp/hello.BTF,
flags: CREAT|NOCTTY|NONBLOCK|WRONLY, mode: IRUGO|IWUGO) = 3
[root@jouet bpf]#
[root@jouet bpf]# file hello.o
hello.o: ELF 64-bit LSB relocatable, *unknown arch 0xf7* version 1 (SYSV), not
stripped
[root@jouet bpf]# pahole --btf_encode hello.o
pahole: hello.o: No debugging information found
[root@jouet bpf]#
[root@jouet bpf]# readelf -s hello.o
Symbol table '.symtab' contains 5 entries:
Num:Value Size TypeBind Vis Ndx Name
0: 0 NOTYPE LOCAL DEFAULT UND
1: 0 NOTYPE GLOBAL DEFAULT7 __bpf_stdout__
2: 0 NOTYPE GLOBAL DEFAULT5 _license
3: 0 NOTYPE GLOBAL DEFAULT6 _version
4: 0 NOTYPE GLOBAL DEFAULT3 syscall_enter_openat
[root@jouet bpf]#
[root@jouet bpf]# readelf -SW hello.o
There are 10 section headers, starting at offset 0x1f8:
Section Headers:
[Nr] Name TypeAddress OffSize ES Flg
Lk Inf Al
[ 0] NULL 00 00 00
0 0 0
[ 1] .strtab STRTAB 000178 7f 00
0 0 1
[ 2] .text PROGBITS 40 00 00 AX
0 0 4
[ 3] syscalls:sys_enter_openat PROGBITS 40 88
00 AX 0 0 8
[ 4] .relsyscalls:sys_enter_openat REL 000168
10 10 9 3 8
[ 5] license PROGBITS c8 04 00 WA
0 0 1
[ 6] version PROGBITS cc 04 00 WA
0 0 4
[ 7] maps PROGBITS d0 10 00 WA
0 0 4
[ 8] .rodata.str1.1PROGBITS e0 0e 01 AMS
0 0 1
[ 9] .symtab SYMTAB f0 78 18
1 1 8
Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings), I (info),
L (link order), O (extra OS processing required), G (group), T (TLS),
C (compressed), x (unknown), o (OS specific), E (exclude),
p (processor specific)
[root@jouet bpf]#
Humm, lemme try something, add -g to clang-opt:
[root@jouet bpf]# cat ~/.perfconfig
[llvm]
dump-obj = true
clang-opt = -g
[root@jouet bpf]# perf trace -e open*,hello.c touch /tmp/hello.BTF
LLVM: dumping hello.o
0.185 ( ): __bpf_stdout__:Hello, world
0.188
Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
On Thu, Jun 07, 2018 at 11:03:37AM -0300, Arnaldo Carvalho de Melo wrote: > Em Thu, Jun 07, 2018 at 10:54:01AM -0300, Arnaldo Carvalho de Melo escreveu: > > Em Tue, Jun 05, 2018 at 02:25:48PM -0700, Martin KaFai Lau escreveu: > > > [ btw, the latest commit (1 commit) should be 94a11b59e592 ]. > > So, the commit log message for the pahole patch is non-existent: > > https://github.com/iamkafai/pahole/commit/94a11b59e5920908085bfc8d24c92f95c8ffceaf > > we should do better in describing what is done and how, I'm staring > with a message you sent to the kernel part: > > -- > This patch introduces BPF Type Format (BTF). > > BTF (BPF Type Format) is the meta data format which describes > the data types of BPF program/map. Hence, it basically focus > on the C programming language which the modern BPF is primary > using. The first use case is to provide a generic pretty print > capability for a BPF map. > -- I will add details in the next github respin/push. > > Now I'm going to do the step-by-step guide on testing the feature just > introduced, and will try to convert from dwarf to BTF and back, compare > the pahole output for types encoded in DWARF and BTF, etc. > > If you have something ressembling this already, please share. The pahole only has the encoder part. I tested with the verbose output from the "pahole -V -J". Loading the btf to the kernel is also tested.
Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
Em Thu, Jun 07, 2018 at 10:54:01AM -0300, Arnaldo Carvalho de Melo escreveu: > Em Tue, Jun 05, 2018 at 02:25:48PM -0700, Martin KaFai Lau escreveu: > > [ btw, the latest commit (1 commit) should be 94a11b59e592 ]. So, the commit log message for the pahole patch is non-existent: https://github.com/iamkafai/pahole/commit/94a11b59e5920908085bfc8d24c92f95c8ffceaf we should do better in describing what is done and how, I'm staring with a message you sent to the kernel part: -- This patch introduces BPF Type Format (BTF). BTF (BPF Type Format) is the meta data format which describes the data types of BPF program/map. Hence, it basically focus on the C programming language which the modern BPF is primary using. The first use case is to provide a generic pretty print capability for a BPF map. -- Now I'm going to do the step-by-step guide on testing the feature just introduced, and will try to convert from dwarf to BTF and back, compare the pahole output for types encoded in DWARF and BTF, etc. If you have something ressembling this already, please share. Thanks, - Arnaldo
Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
Em Tue, Jun 05, 2018 at 02:25:48PM -0700, Martin KaFai Lau escreveu: > On Thu, Apr 19, 2018 at 04:40:34PM -0300, Arnaldo Carvalho de Melo wrote: > > Em Wed, Apr 18, 2018 at 03:55:56PM -0700, Martin KaFai Lau escreveu: > > > This patch introduces BPF Type Format (BTF). > > > > > > BTF (BPF Type Format) is the meta data format which describes > > > the data types of BPF program/map. Hence, it basically focus > > > on the C programming language which the modern BPF is primary > > > using. The first use case is to provide a generic pretty print > > > capability for a BPF map. > > > > > > A modified pahole that can convert dwarf to BTF is here: > > > https://github.com/iamkafai/pahole/tree/btf > > > (Arnaldo, there is some BTF_KIND numbering changes on > > > Apr 18th, d61426c1571) > > > > Thanks for letting me know, I'm starting to look at this, > Hi Arnaldo, > > Do you have a chance to take a look and pull it? The kernel > changes will be in 4.18, so it will be handy if it is available in > the pahole repository. > > [ btw, the latest commit (1 commit) should be 94a11b59e592 ]. Yeah, the one I had before had: It also raises the number of types (and functions) limit from 0x7fff to 0x7fff. And on this last one I see that: /* Max # of type identifier */ -#define BTF_MAX_TYPE 0x7fff +#define BTF_MAX_TYPE 0x /* Max offset into the string section */ -#define BTF_MAX_NAME_OFFSET0x7fff +#define BTF_MAX_NAME_OFFSET0x So somehow (still reading) you'll be able to get more space, if we find necessary, to have more types and names, ok. Continuing... - Arnaldo > > > > - Arnaldo > > > > > Please see individual patch for details. > > > > > > v5: > > > - Remove BTF_KIND_FLOAT and BTF_KIND_FUNC which are not > > > currently used. They can be added in the future. > > > Some bpf_df_xxx() are removed together. > > > - Add comment in patch 7 to clarify that the new bpffs_map_fops > > > should not be extended further. > > > > > > v4: > > > - Fix warning (remove unneeded semicolon) > > > - Remove a redundant variable (nr_bytes) from btf_int_check_meta() in > > > patch 1. Caught by W=1. > > > > > > v3: > > > - Rebase to bpf-next > > > - Fix sparse warning (by adding static) > > > - Add BTF header logging: btf_verifier_log_hdr() > > > - Fix the alignment test on btf->type_off > > > - Add tests for the BTF header > > > - Lower the max BTF size to 16MB. It should be enough > > > for some time. We could raise it later if it would > > > be needed. > > > > > > v2: > > > - Use kvfree where needed in patch 1 and 2 > > > - Also consider BTF_INT_OFFSET() in the btf_int_check_meta() > > > in patch 1 > > > - Fix an incorrect goto target in map_create() during > > > the btf-error-path in patch 7 > > > - re-org some local vars to keep the rev xmas tree in btf.c > > > > > > Martin KaFai Lau (10): > > > bpf: btf: Introduce BPF Type Format (BTF) > > > bpf: btf: Validate type reference > > > bpf: btf: Check members of struct/union > > > bpf: btf: Add pretty print capability for data with BTF type info > > > bpf: btf: Add BPF_BTF_LOAD command > > > bpf: btf: Add BPF_OBJ_GET_INFO_BY_FD support to BTF fd > > > bpf: btf: Add pretty print support to the basic arraymap > > > bpf: btf: Sync bpf.h and btf.h to tools/ > > > bpf: btf: Add BTF support to libbpf > > > bpf: btf: Add BTF tests > > > > > > include/linux/bpf.h | 20 +- > > > include/linux/btf.h | 48 + > > > include/uapi/linux/bpf.h | 12 + > > > include/uapi/linux/btf.h | 130 ++ > > > kernel/bpf/Makefile |1 + > > > kernel/bpf/arraymap.c| 50 + > > > kernel/bpf/btf.c | 2064 > > > ++ > > > kernel/bpf/inode.c | 156 +- > > > kernel/bpf/syscall.c | 51 +- > > > tools/include/uapi/linux/bpf.h | 12 + > > > tools/include/uapi/linux/btf.h | 130 ++ > > > tools/lib/bpf/Build |2 +- > > > tools/lib/bpf/bpf.c | 92 +- > > > tools/lib/bpf/bpf.h | 16 + > > > tools/lib/bpf/btf.c | 374 + > > > tools/lib/bpf/btf.h | 22 + > > > tools/lib/bpf/libbpf.c | 148 +- > > > tools/lib/bpf/libbpf.h |3 + > > > tools/testing/selftests/bpf/Makefile | 26 +- > > > tools/testing/selftests/bpf/test_btf.c | 1669 + > > > tools/testing/selftests/bpf/test_btf_haskv.c | 48 + > > > tools/testing/selftests/bpf/test_btf_nokv.c | 43 + > > > 22 files changed, 5076 insertions(+), 41 deletions(-) > > > create mode 100644 include/linux/btf.h > > > create mode 100644 include/uapi/linux/btf.h > > >
Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
Em Tue, Jun 05, 2018 at 02:25:48PM -0700, Martin KaFai Lau escreveu: > On Thu, Apr 19, 2018 at 04:40:34PM -0300, Arnaldo Carvalho de Melo wrote: > > Em Wed, Apr 18, 2018 at 03:55:56PM -0700, Martin KaFai Lau escreveu: > > > This patch introduces BPF Type Format (BTF). > > > > > > BTF (BPF Type Format) is the meta data format which describes > > > the data types of BPF program/map. Hence, it basically focus > > > on the C programming language which the modern BPF is primary > > > using. The first use case is to provide a generic pretty print > > > capability for a BPF map. > > > > > > A modified pahole that can convert dwarf to BTF is here: > > > https://github.com/iamkafai/pahole/tree/btf > > > (Arnaldo, there is some BTF_KIND numbering changes on > > > Apr 18th, d61426c1571) > > > > Thanks for letting me know, I'm starting to look at this, > Hi Arnaldo, > > Do you have a chance to take a look and pull it? The kernel > changes will be in 4.18, so it will be handy if it is available in > the pahole repository. > > [ btw, the latest commit (1 commit) should be 94a11b59e592 ]. Got sidetracked, will get back to it later today. - Arnaldo
Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
On Thu, Apr 19, 2018 at 04:40:34PM -0300, Arnaldo Carvalho de Melo wrote: > Em Wed, Apr 18, 2018 at 03:55:56PM -0700, Martin KaFai Lau escreveu: > > This patch introduces BPF Type Format (BTF). > > > > BTF (BPF Type Format) is the meta data format which describes > > the data types of BPF program/map. Hence, it basically focus > > on the C programming language which the modern BPF is primary > > using. The first use case is to provide a generic pretty print > > capability for a BPF map. > > > > A modified pahole that can convert dwarf to BTF is here: > > https://github.com/iamkafai/pahole/tree/btf > > (Arnaldo, there is some BTF_KIND numbering changes on > > Apr 18th, d61426c1571) > > Thanks for letting me know, I'm starting to look at this, Hi Arnaldo, Do you have a chance to take a look and pull it? The kernel changes will be in 4.18, so it will be handy if it is available in the pahole repository. [ btw, the latest commit (1 commit) should be 94a11b59e592 ]. Thanks, Martin > > - Arnaldo > > > Please see individual patch for details. > > > > v5: > > - Remove BTF_KIND_FLOAT and BTF_KIND_FUNC which are not > > currently used. They can be added in the future. > > Some bpf_df_xxx() are removed together. > > - Add comment in patch 7 to clarify that the new bpffs_map_fops > > should not be extended further. > > > > v4: > > - Fix warning (remove unneeded semicolon) > > - Remove a redundant variable (nr_bytes) from btf_int_check_meta() in > > patch 1. Caught by W=1. > > > > v3: > > - Rebase to bpf-next > > - Fix sparse warning (by adding static) > > - Add BTF header logging: btf_verifier_log_hdr() > > - Fix the alignment test on btf->type_off > > - Add tests for the BTF header > > - Lower the max BTF size to 16MB. It should be enough > > for some time. We could raise it later if it would > > be needed. > > > > v2: > > - Use kvfree where needed in patch 1 and 2 > > - Also consider BTF_INT_OFFSET() in the btf_int_check_meta() > > in patch 1 > > - Fix an incorrect goto target in map_create() during > > the btf-error-path in patch 7 > > - re-org some local vars to keep the rev xmas tree in btf.c > > > > Martin KaFai Lau (10): > > bpf: btf: Introduce BPF Type Format (BTF) > > bpf: btf: Validate type reference > > bpf: btf: Check members of struct/union > > bpf: btf: Add pretty print capability for data with BTF type info > > bpf: btf: Add BPF_BTF_LOAD command > > bpf: btf: Add BPF_OBJ_GET_INFO_BY_FD support to BTF fd > > bpf: btf: Add pretty print support to the basic arraymap > > bpf: btf: Sync bpf.h and btf.h to tools/ > > bpf: btf: Add BTF support to libbpf > > bpf: btf: Add BTF tests > > > > include/linux/bpf.h | 20 +- > > include/linux/btf.h | 48 + > > include/uapi/linux/bpf.h | 12 + > > include/uapi/linux/btf.h | 130 ++ > > kernel/bpf/Makefile |1 + > > kernel/bpf/arraymap.c| 50 + > > kernel/bpf/btf.c | 2064 > > ++ > > kernel/bpf/inode.c | 156 +- > > kernel/bpf/syscall.c | 51 +- > > tools/include/uapi/linux/bpf.h | 12 + > > tools/include/uapi/linux/btf.h | 130 ++ > > tools/lib/bpf/Build |2 +- > > tools/lib/bpf/bpf.c | 92 +- > > tools/lib/bpf/bpf.h | 16 + > > tools/lib/bpf/btf.c | 374 + > > tools/lib/bpf/btf.h | 22 + > > tools/lib/bpf/libbpf.c | 148 +- > > tools/lib/bpf/libbpf.h |3 + > > tools/testing/selftests/bpf/Makefile | 26 +- > > tools/testing/selftests/bpf/test_btf.c | 1669 + > > tools/testing/selftests/bpf/test_btf_haskv.c | 48 + > > tools/testing/selftests/bpf/test_btf_nokv.c | 43 + > > 22 files changed, 5076 insertions(+), 41 deletions(-) > > create mode 100644 include/linux/btf.h > > create mode 100644 include/uapi/linux/btf.h > > create mode 100644 kernel/bpf/btf.c > > create mode 100644 tools/include/uapi/linux/btf.h > > create mode 100644 tools/lib/bpf/btf.c > > create mode 100644 tools/lib/bpf/btf.h > > create mode 100644 tools/testing/selftests/bpf/test_btf.c > > create mode 100644 tools/testing/selftests/bpf/test_btf_haskv.c > > create mode 100644 tools/testing/selftests/bpf/test_btf_nokv.c > > > > -- > > 2.9.5
Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
On 04/19/2018 12:55 AM, Martin KaFai Lau wrote: > This patch introduces BPF Type Format (BTF). > > BTF (BPF Type Format) is the meta data format which describes > the data types of BPF program/map. Hence, it basically focus > on the C programming language which the modern BPF is primary > using. The first use case is to provide a generic pretty print > capability for a BPF map. > > A modified pahole that can convert dwarf to BTF is here: > https://github.com/iamkafai/pahole/tree/btf > (Arnaldo, there is some BTF_KIND numbering changes on > Apr 18th, d61426c1571) > > Please see individual patch for details. > > v5: > - Remove BTF_KIND_FLOAT and BTF_KIND_FUNC which are not > currently used. They can be added in the future. > Some bpf_df_xxx() are removed together. > - Add comment in patch 7 to clarify that the new bpffs_map_fops > should not be extended further. > > v4: > - Fix warning (remove unneeded semicolon) > - Remove a redundant variable (nr_bytes) from btf_int_check_meta() in > patch 1. Caught by W=1. > > v3: > - Rebase to bpf-next > - Fix sparse warning (by adding static) > - Add BTF header logging: btf_verifier_log_hdr() > - Fix the alignment test on btf->type_off > - Add tests for the BTF header > - Lower the max BTF size to 16MB. It should be enough > for some time. We could raise it later if it would > be needed. > > v2: > - Use kvfree where needed in patch 1 and 2 > - Also consider BTF_INT_OFFSET() in the btf_int_check_meta() > in patch 1 > - Fix an incorrect goto target in map_create() during > the btf-error-path in patch 7 > - re-org some local vars to keep the rev xmas tree in btf.c Series applied to bpf-next, thanks Martin. As discussed please follow up with the bpftool patches. Thanks, Daniel
Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
On Thu, Apr 19, 2018 at 04:40:34PM -0300, Arnaldo Carvalho de Melo wrote: > Em Wed, Apr 18, 2018 at 03:55:56PM -0700, Martin KaFai Lau escreveu: > > This patch introduces BPF Type Format (BTF). > > > > BTF (BPF Type Format) is the meta data format which describes > > the data types of BPF program/map. Hence, it basically focus > > on the C programming language which the modern BPF is primary > > using. The first use case is to provide a generic pretty print > > capability for a BPF map. > > > > A modified pahole that can convert dwarf to BTF is here: > > https://github.com/iamkafai/pahole/tree/btf > > (Arnaldo, there is some BTF_KIND numbering changes on > > Apr 18th, d61426c1571) > > Thanks for letting me know, I'm starting to look at this, Thanks for reviewing. Feel free to comment directly on the github diff. Also, I think it may make sense to wait for the kernel pieces to land first. > > - Arnaldo > > > Please see individual patch for details. > > > > v5: > > - Remove BTF_KIND_FLOAT and BTF_KIND_FUNC which are not > > currently used. They can be added in the future. > > Some bpf_df_xxx() are removed together. > > - Add comment in patch 7 to clarify that the new bpffs_map_fops > > should not be extended further. > > > > v4: > > - Fix warning (remove unneeded semicolon) > > - Remove a redundant variable (nr_bytes) from btf_int_check_meta() in > > patch 1. Caught by W=1. > > > > v3: > > - Rebase to bpf-next > > - Fix sparse warning (by adding static) > > - Add BTF header logging: btf_verifier_log_hdr() > > - Fix the alignment test on btf->type_off > > - Add tests for the BTF header > > - Lower the max BTF size to 16MB. It should be enough > > for some time. We could raise it later if it would > > be needed. > > > > v2: > > - Use kvfree where needed in patch 1 and 2 > > - Also consider BTF_INT_OFFSET() in the btf_int_check_meta() > > in patch 1 > > - Fix an incorrect goto target in map_create() during > > the btf-error-path in patch 7 > > - re-org some local vars to keep the rev xmas tree in btf.c > > > > Martin KaFai Lau (10): > > bpf: btf: Introduce BPF Type Format (BTF) > > bpf: btf: Validate type reference > > bpf: btf: Check members of struct/union > > bpf: btf: Add pretty print capability for data with BTF type info > > bpf: btf: Add BPF_BTF_LOAD command > > bpf: btf: Add BPF_OBJ_GET_INFO_BY_FD support to BTF fd > > bpf: btf: Add pretty print support to the basic arraymap > > bpf: btf: Sync bpf.h and btf.h to tools/ > > bpf: btf: Add BTF support to libbpf > > bpf: btf: Add BTF tests > > > > include/linux/bpf.h | 20 +- > > include/linux/btf.h | 48 + > > include/uapi/linux/bpf.h | 12 + > > include/uapi/linux/btf.h | 130 ++ > > kernel/bpf/Makefile |1 + > > kernel/bpf/arraymap.c| 50 + > > kernel/bpf/btf.c | 2064 > > ++ > > kernel/bpf/inode.c | 156 +- > > kernel/bpf/syscall.c | 51 +- > > tools/include/uapi/linux/bpf.h | 12 + > > tools/include/uapi/linux/btf.h | 130 ++ > > tools/lib/bpf/Build |2 +- > > tools/lib/bpf/bpf.c | 92 +- > > tools/lib/bpf/bpf.h | 16 + > > tools/lib/bpf/btf.c | 374 + > > tools/lib/bpf/btf.h | 22 + > > tools/lib/bpf/libbpf.c | 148 +- > > tools/lib/bpf/libbpf.h |3 + > > tools/testing/selftests/bpf/Makefile | 26 +- > > tools/testing/selftests/bpf/test_btf.c | 1669 + > > tools/testing/selftests/bpf/test_btf_haskv.c | 48 + > > tools/testing/selftests/bpf/test_btf_nokv.c | 43 + > > 22 files changed, 5076 insertions(+), 41 deletions(-) > > create mode 100644 include/linux/btf.h > > create mode 100644 include/uapi/linux/btf.h > > create mode 100644 kernel/bpf/btf.c > > create mode 100644 tools/include/uapi/linux/btf.h > > create mode 100644 tools/lib/bpf/btf.c > > create mode 100644 tools/lib/bpf/btf.h > > create mode 100644 tools/testing/selftests/bpf/test_btf.c > > create mode 100644 tools/testing/selftests/bpf/test_btf_haskv.c > > create mode 100644 tools/testing/selftests/bpf/test_btf_nokv.c > > > > -- > > 2.9.5
Re: [PATCH bpf-next v5 00/10] BTF: BPF Type Format
Em Wed, Apr 18, 2018 at 03:55:56PM -0700, Martin KaFai Lau escreveu: > This patch introduces BPF Type Format (BTF). > > BTF (BPF Type Format) is the meta data format which describes > the data types of BPF program/map. Hence, it basically focus > on the C programming language which the modern BPF is primary > using. The first use case is to provide a generic pretty print > capability for a BPF map. > > A modified pahole that can convert dwarf to BTF is here: > https://github.com/iamkafai/pahole/tree/btf > (Arnaldo, there is some BTF_KIND numbering changes on > Apr 18th, d61426c1571) Thanks for letting me know, I'm starting to look at this, - Arnaldo > Please see individual patch for details. > > v5: > - Remove BTF_KIND_FLOAT and BTF_KIND_FUNC which are not > currently used. They can be added in the future. > Some bpf_df_xxx() are removed together. > - Add comment in patch 7 to clarify that the new bpffs_map_fops > should not be extended further. > > v4: > - Fix warning (remove unneeded semicolon) > - Remove a redundant variable (nr_bytes) from btf_int_check_meta() in > patch 1. Caught by W=1. > > v3: > - Rebase to bpf-next > - Fix sparse warning (by adding static) > - Add BTF header logging: btf_verifier_log_hdr() > - Fix the alignment test on btf->type_off > - Add tests for the BTF header > - Lower the max BTF size to 16MB. It should be enough > for some time. We could raise it later if it would > be needed. > > v2: > - Use kvfree where needed in patch 1 and 2 > - Also consider BTF_INT_OFFSET() in the btf_int_check_meta() > in patch 1 > - Fix an incorrect goto target in map_create() during > the btf-error-path in patch 7 > - re-org some local vars to keep the rev xmas tree in btf.c > > Martin KaFai Lau (10): > bpf: btf: Introduce BPF Type Format (BTF) > bpf: btf: Validate type reference > bpf: btf: Check members of struct/union > bpf: btf: Add pretty print capability for data with BTF type info > bpf: btf: Add BPF_BTF_LOAD command > bpf: btf: Add BPF_OBJ_GET_INFO_BY_FD support to BTF fd > bpf: btf: Add pretty print support to the basic arraymap > bpf: btf: Sync bpf.h and btf.h to tools/ > bpf: btf: Add BTF support to libbpf > bpf: btf: Add BTF tests > > include/linux/bpf.h | 20 +- > include/linux/btf.h | 48 + > include/uapi/linux/bpf.h | 12 + > include/uapi/linux/btf.h | 130 ++ > kernel/bpf/Makefile |1 + > kernel/bpf/arraymap.c| 50 + > kernel/bpf/btf.c | 2064 > ++ > kernel/bpf/inode.c | 156 +- > kernel/bpf/syscall.c | 51 +- > tools/include/uapi/linux/bpf.h | 12 + > tools/include/uapi/linux/btf.h | 130 ++ > tools/lib/bpf/Build |2 +- > tools/lib/bpf/bpf.c | 92 +- > tools/lib/bpf/bpf.h | 16 + > tools/lib/bpf/btf.c | 374 + > tools/lib/bpf/btf.h | 22 + > tools/lib/bpf/libbpf.c | 148 +- > tools/lib/bpf/libbpf.h |3 + > tools/testing/selftests/bpf/Makefile | 26 +- > tools/testing/selftests/bpf/test_btf.c | 1669 + > tools/testing/selftests/bpf/test_btf_haskv.c | 48 + > tools/testing/selftests/bpf/test_btf_nokv.c | 43 + > 22 files changed, 5076 insertions(+), 41 deletions(-) > create mode 100644 include/linux/btf.h > create mode 100644 include/uapi/linux/btf.h > create mode 100644 kernel/bpf/btf.c > create mode 100644 tools/include/uapi/linux/btf.h > create mode 100644 tools/lib/bpf/btf.c > create mode 100644 tools/lib/bpf/btf.h > create mode 100644 tools/testing/selftests/bpf/test_btf.c > create mode 100644 tools/testing/selftests/bpf/test_btf_haskv.c > create mode 100644 tools/testing/selftests/bpf/test_btf_nokv.c > > -- > 2.9.5
[PATCH bpf-next v5 00/10] BTF: BPF Type Format
This patch introduces BPF Type Format (BTF). BTF (BPF Type Format) is the meta data format which describes the data types of BPF program/map. Hence, it basically focus on the C programming language which the modern BPF is primary using. The first use case is to provide a generic pretty print capability for a BPF map. A modified pahole that can convert dwarf to BTF is here: https://github.com/iamkafai/pahole/tree/btf (Arnaldo, there is some BTF_KIND numbering changes on Apr 18th, d61426c1571) Please see individual patch for details. v5: - Remove BTF_KIND_FLOAT and BTF_KIND_FUNC which are not currently used. They can be added in the future. Some bpf_df_xxx() are removed together. - Add comment in patch 7 to clarify that the new bpffs_map_fops should not be extended further. v4: - Fix warning (remove unneeded semicolon) - Remove a redundant variable (nr_bytes) from btf_int_check_meta() in patch 1. Caught by W=1. v3: - Rebase to bpf-next - Fix sparse warning (by adding static) - Add BTF header logging: btf_verifier_log_hdr() - Fix the alignment test on btf->type_off - Add tests for the BTF header - Lower the max BTF size to 16MB. It should be enough for some time. We could raise it later if it would be needed. v2: - Use kvfree where needed in patch 1 and 2 - Also consider BTF_INT_OFFSET() in the btf_int_check_meta() in patch 1 - Fix an incorrect goto target in map_create() during the btf-error-path in patch 7 - re-org some local vars to keep the rev xmas tree in btf.c Martin KaFai Lau (10): bpf: btf: Introduce BPF Type Format (BTF) bpf: btf: Validate type reference bpf: btf: Check members of struct/union bpf: btf: Add pretty print capability for data with BTF type info bpf: btf: Add BPF_BTF_LOAD command bpf: btf: Add BPF_OBJ_GET_INFO_BY_FD support to BTF fd bpf: btf: Add pretty print support to the basic arraymap bpf: btf: Sync bpf.h and btf.h to tools/ bpf: btf: Add BTF support to libbpf bpf: btf: Add BTF tests include/linux/bpf.h | 20 +- include/linux/btf.h | 48 + include/uapi/linux/bpf.h | 12 + include/uapi/linux/btf.h | 130 ++ kernel/bpf/Makefile |1 + kernel/bpf/arraymap.c| 50 + kernel/bpf/btf.c | 2064 ++ kernel/bpf/inode.c | 156 +- kernel/bpf/syscall.c | 51 +- tools/include/uapi/linux/bpf.h | 12 + tools/include/uapi/linux/btf.h | 130 ++ tools/lib/bpf/Build |2 +- tools/lib/bpf/bpf.c | 92 +- tools/lib/bpf/bpf.h | 16 + tools/lib/bpf/btf.c | 374 + tools/lib/bpf/btf.h | 22 + tools/lib/bpf/libbpf.c | 148 +- tools/lib/bpf/libbpf.h |3 + tools/testing/selftests/bpf/Makefile | 26 +- tools/testing/selftests/bpf/test_btf.c | 1669 + tools/testing/selftests/bpf/test_btf_haskv.c | 48 + tools/testing/selftests/bpf/test_btf_nokv.c | 43 + 22 files changed, 5076 insertions(+), 41 deletions(-) create mode 100644 include/linux/btf.h create mode 100644 include/uapi/linux/btf.h create mode 100644 kernel/bpf/btf.c create mode 100644 tools/include/uapi/linux/btf.h create mode 100644 tools/lib/bpf/btf.c create mode 100644 tools/lib/bpf/btf.h create mode 100644 tools/testing/selftests/bpf/test_btf.c create mode 100644 tools/testing/selftests/bpf/test_btf_haskv.c create mode 100644 tools/testing/selftests/bpf/test_btf_nokv.c -- 2.9.5
