Re: pahole + BTF was: Re: [Question] bpf: about a new 'tools/bpf/bpf_dwarf2btf'
On Thu, Jul 26, 2018 at 04:21:31AM +0900, Taeung Song wrote: > > > On 07/26/2018 03:27 AM, Taeung Song wrote: > > Hi Arnaldo, > > > > On 07/26/2018 02:52 AM, Arnaldo Carvalho de Melo wrote: > > > Em Thu, Jul 26, 2018 at 02:23:32AM +0900, Taeung Song escreveu: > > > > Hi, > > > > > > > > Building bpf programs with .BTF section, > > > > I thought it'd be better to convert dwarf info to .BTF by > > > > a new tool such as 'tools/bpf/bpf_dwarf2btf' instead of pahole > > > > in the future. > > > > Currently for bpf binary that have .BTF section, > > > > we need to use pahole from https://github.com/iamkafai/pahole/tree/btf > > > > with the command line such as "pahole -J bpf_prog.o". > > > > I think it is great but if implementing new 'bpf_dwarf2btf' > > > > (dwarf parsing + btf encoder code written by Martin KaFai Lau on > > > > the pahole project i.e. btf.h, btf_encoder.c, btf_encoder.h, > > > > libbtf.c, libbtf.h), > > > > BPF developers would more easily use functionalities based on BTF. > > > > > > What would be easier exactly? Not having to install a package but build > > > it from the kernel sources? > > > > > > Many kernel developers already have pahole installed for other uses, so > > > no need to install anything. > > > > > > > Understood, but I think there are many non-kernel developers > > developing BPF programs and they mightn't have or use pahole. > > > > So, if providing the 'dwarf2btf' feature on tools/bpf or tools/bpf/bpftool, > > non-kernel developers can also more easily build bpf prog with .BPF, no ? Some quick thoughts, IMO, I suspect if it is in the distro's pahole package, it should be easy enough for kernel and non kernel developer to install. BTF usage is still evolving, we might re-evaluate going forward but at this point I think leveraging pahole's existing capability is a good option. > > > > Or, if tools/lib/bpf/ have the 'dwarf2btf' feature, > I think BPF developers can just use bpf programs that have dwarf info > after compiling with clang '-g' and llc '-mattr=dwarfris', even though not > using pahole. > Isn't it good way ? > > > > BTW, Daniel, I just pushed to pahole's main repository at: > > > > > > git://git.kernel.org/pub/scm/devel/pahole/pahole.git > > > > > > with the Martin's BTF patch, so no need to pull from the github one, > > > I'll tag v1.12 and announce the release so that distro package > > > maintainers can update their packages. Awesome! Thanks, Arnaldo! - Martin
Re: pahole + BTF was: Re: [Question] bpf: about a new 'tools/bpf/bpf_dwarf2btf'
On 07/26/2018 03:27 AM, Taeung Song wrote: Hi Arnaldo, On 07/26/2018 02:52 AM, Arnaldo Carvalho de Melo wrote: Em Thu, Jul 26, 2018 at 02:23:32AM +0900, Taeung Song escreveu: Hi, Building bpf programs with .BTF section, I thought it'd be better to convert dwarf info to .BTF by a new tool such as 'tools/bpf/bpf_dwarf2btf' instead of pahole in the future. Currently for bpf binary that have .BTF section, we need to use pahole from https://github.com/iamkafai/pahole/tree/btf with the command line such as "pahole -J bpf_prog.o". I think it is great but if implementing new 'bpf_dwarf2btf' (dwarf parsing + btf encoder code written by Martin KaFai Lau on the pahole project i.e. btf.h, btf_encoder.c, btf_encoder.h, libbtf.c, libbtf.h), BPF developers would more easily use functionalities based on BTF. What would be easier exactly? Not having to install a package but build it from the kernel sources? Many kernel developers already have pahole installed for other uses, so no need to install anything. Understood, but I think there are many non-kernel developers developing BPF programs and they mightn't have or use pahole. So, if providing the 'dwarf2btf' feature on tools/bpf or tools/bpf/bpftool, non-kernel developers can also more easily build bpf prog with .BPF, no ? Or, if tools/lib/bpf/ have the 'dwarf2btf' feature, I think BPF developers can just use bpf programs that have dwarf info after compiling with clang '-g' and llc '-mattr=dwarfris', even though not using pahole. Isn't it good way ? BTW, Daniel, I just pushed to pahole's main repository at: git://git.kernel.org/pub/scm/devel/pahole/pahole.git with the Martin's BTF patch, so no need to pull from the github one, I'll tag v1.12 and announce the release so that distro package maintainers can update their packages. What do you think about this ? Do you think this is needed ? Or, already implementing something like this ? If it is needed, I want to try to make 'tools/bpf/bpf_dwarf2btf' based on the pahole code. I'd appreciate it, if you reply to this The way Martin took advantage of the work done a long time ago to support CTF out of the same DWARF reading codebase was really cool, not that much work to do, just add a new format to pahole's codebase making it more useful. I got it ! Thanks, Taeung I was just so far overly picky with testing it and kept leaving for later to have a good documentation about testing it, vacation and perf maintainership duties kept making this take like forever, grumble :-\ - Arnaldo -- oh.. :'(
Re: pahole + BTF was: Re: [Question] bpf: about a new 'tools/bpf/bpf_dwarf2btf'
On 07/25/2018 08:27 PM, Taeung Song wrote: > On 07/26/2018 02:52 AM, Arnaldo Carvalho de Melo wrote: [...] >> BTW, Daniel, I just pushed to pahole's main repository at: >> >> git://git.kernel.org/pub/scm/devel/pahole/pahole.git >> >> with the Martin's BTF patch, so no need to pull from the github one, >> I'll tag v1.12 and announce the release so that distro package >> maintainers can update their packages. Awesome, thanks so much Arnaldo!
Re: pahole + BTF was: Re: [Question] bpf: about a new 'tools/bpf/bpf_dwarf2btf'
Hi Arnaldo, On 07/26/2018 02:52 AM, Arnaldo Carvalho de Melo wrote: Em Thu, Jul 26, 2018 at 02:23:32AM +0900, Taeung Song escreveu: Hi, Building bpf programs with .BTF section, I thought it'd be better to convert dwarf info to .BTF by a new tool such as 'tools/bpf/bpf_dwarf2btf' instead of pahole in the future. Currently for bpf binary that have .BTF section, we need to use pahole from https://github.com/iamkafai/pahole/tree/btf with the command line such as "pahole -J bpf_prog.o". I think it is great but if implementing new 'bpf_dwarf2btf' (dwarf parsing + btf encoder code written by Martin KaFai Lau on the pahole project i.e. btf.h, btf_encoder.c, btf_encoder.h, libbtf.c, libbtf.h), BPF developers would more easily use functionalities based on BTF. What would be easier exactly? Not having to install a package but build it from the kernel sources? Many kernel developers already have pahole installed for other uses, so no need to install anything. Understood, but I think there are many non-kernel developers developing BPF programs and they mightn't have or use pahole. So, if providing the 'dwarf2btf' feature on tools/bpf or tools/bpf/bpftool, non-kernel developers can also more easily build bpf prog with .BPF, no ? BTW, Daniel, I just pushed to pahole's main repository at: git://git.kernel.org/pub/scm/devel/pahole/pahole.git with the Martin's BTF patch, so no need to pull from the github one, I'll tag v1.12 and announce the release so that distro package maintainers can update their packages. What do you think about this ? Do you think this is needed ? Or, already implementing something like this ? If it is needed, I want to try to make 'tools/bpf/bpf_dwarf2btf' based on the pahole code. I'd appreciate it, if you reply to this The way Martin took advantage of the work done a long time ago to support CTF out of the same DWARF reading codebase was really cool, not that much work to do, just add a new format to pahole's codebase making it more useful. I got it ! Thanks, Taeung I was just so far overly picky with testing it and kept leaving for later to have a good documentation about testing it, vacation and perf maintainership duties kept making this take like forever, grumble :-\ - Arnaldo -- oh.. :'(