Re: [PATCH v2 bpf-next 2/2] tools/bpftool: fix bpftool build with bintutils >= 2.8

[Roman Gushchin]

On Tue, Dec 26, 2017 at 06:32:05PM -0800, Alexei Starovoitov wrote:
> On Fri, Dec 22, 2017 at 06:50:01PM +, Quentin Monnet wrote:
> > Hi Roman,
> > 
> > 2017-12-22 16:11 UTC+ ~ Roman Gushchin 
> > > Bpftool build is broken with binutils version 2.28 and later.
> > 
> > Could you check the binutils version? I believe it changed in 2.29
> > instead of 2.28. Could you update your commit log and subject
> > accordingly, please?

Yes, you're right. Thanks!

> > 
> > > The cause is commit 003ca0fd2286 ("Refactor disassembler selection")
> > > in the binutils repo, which changed the disassembler() function
> > > signature.
> > > 
> > > Fix this by adding a new "feature" to the tools/build/features
> > > infrastructure and make it responsible for decision which
> > > disassembler() function signature to use.
> > > 
> > > Signed-off-by: Roman Gushchin 
> > > Cc: Jakub Kicinski 
> > > Cc: Alexei Starovoitov 
> > > Cc: Daniel Borkmann 
> > > ---
> > >  tools/bpf/Makefile| 29 
> > > +++
> > >  tools/bpf/bpf_jit_disasm.c|  7 ++
> > >  tools/bpf/bpftool/Makefile| 24 
> > > +++
> > >  tools/bpf/bpftool/jit_disasm.c|  7 ++
> > >  tools/build/feature/Makefile  |  4 
> > >  tools/build/feature/test-disassembler-four-args.c | 15 
> > >  6 files changed, 86 insertions(+)
> > >  create mode 100644 tools/build/feature/test-disassembler-four-args.c
> > > 
> > > diff --git a/tools/bpf/Makefile b/tools/bpf/Makefile
> > > index 07a6697466ef..c8ec0ae16bf0 100644
> > > --- a/tools/bpf/Makefile
> > > +++ b/tools/bpf/Makefile
> > > @@ -9,6 +9,35 @@ MAKE = make
> > >  CFLAGS += -Wall -O2
> > >  CFLAGS += -D__EXPORTED_HEADERS__ -I../../include/uapi -I../../include
> > >  
> > > +ifeq ($(srctree),)
> > > +srctree := $(patsubst %/,%,$(dir $(CURDIR)))
> > > +srctree := $(patsubst %/,%,$(dir $(srctree)))
> > > +endif
> > > +
> > > +FEATURE_USER = .bpf
> > > +FEATURE_TESTS = libbfd disassembler-four-args
> > > +FEATURE_DISPLAY = libbfd disassembler-four-args
> > 
> > Thanks for adding libbfd as I requested. However, you do not use it in
> > the Makefile to prevent compilation if the feature is not detected (see
> > "bpfdep" or "elfdep" in tools/lib/bpf/Makefile. Sorry, I should have
> > pointed it in my previous review.
> > 
> > But actually, I have another issue related to the libbfd feature: since
> > commit 280e7c48c3b8 ("perf tools: fix BFD detection on opensuse") it
> > requires libiberty so that libbfd is correctly detected, but libiberty
> > is not needed on all distros (at least Ubuntu can have libbfd without
> > libiberty). Typically, detection fails on my setup, although I do have
> > libbfd installed. So forcing libbfd feature here may eventually force
> > users to install libraries they do not need to compile bpftool, which is
> > not what we want.
> > 
> > I do not have a clean work around to suggest. Maybe have one
> > "libbfd-something" feature that tries to compile without libiberty, then
> > another one that tries with it, and compile the tools if at least one of
> > them succeeds. But it's probably for another patch series. In the
> > meantime, would you please simply remove libbfd detection here and
> > accept my apologies for suggesting to add it in the previous review?
> 
> I think since libbfd is already used by bpftool it's a good thing
> to add feature detection. Even if it's not perfect on some setups.

Agree, we can enhance it later.

> 
> Roman,
> I think you still need to do one more respin to address commit log nit?
> 

Sure, will send soon-ish.

Thanks!

Re: [PATCH v2 bpf-next 2/2] tools/bpftool: fix bpftool build with bintutils >= 2.8

[Alexei Starovoitov]

On Fri, Dec 22, 2017 at 06:50:01PM +, Quentin Monnet wrote:
> Hi Roman,
> 
> 2017-12-22 16:11 UTC+ ~ Roman Gushchin 
> > Bpftool build is broken with binutils version 2.28 and later.
> 
> Could you check the binutils version? I believe it changed in 2.29
> instead of 2.28. Could you update your commit log and subject
> accordingly, please?
> 
> > The cause is commit 003ca0fd2286 ("Refactor disassembler selection")
> > in the binutils repo, which changed the disassembler() function
> > signature.
> > 
> > Fix this by adding a new "feature" to the tools/build/features
> > infrastructure and make it responsible for decision which
> > disassembler() function signature to use.
> > 
> > Signed-off-by: Roman Gushchin 
> > Cc: Jakub Kicinski 
> > Cc: Alexei Starovoitov 
> > Cc: Daniel Borkmann 
> > ---
> >  tools/bpf/Makefile| 29 
> > +++
> >  tools/bpf/bpf_jit_disasm.c|  7 ++
> >  tools/bpf/bpftool/Makefile| 24 +++
> >  tools/bpf/bpftool/jit_disasm.c|  7 ++
> >  tools/build/feature/Makefile  |  4 
> >  tools/build/feature/test-disassembler-four-args.c | 15 
> >  6 files changed, 86 insertions(+)
> >  create mode 100644 tools/build/feature/test-disassembler-four-args.c
> > 
> > diff --git a/tools/bpf/Makefile b/tools/bpf/Makefile
> > index 07a6697466ef..c8ec0ae16bf0 100644
> > --- a/tools/bpf/Makefile
> > +++ b/tools/bpf/Makefile
> > @@ -9,6 +9,35 @@ MAKE = make
> >  CFLAGS += -Wall -O2
> >  CFLAGS += -D__EXPORTED_HEADERS__ -I../../include/uapi -I../../include
> >  
> > +ifeq ($(srctree),)
> > +srctree := $(patsubst %/,%,$(dir $(CURDIR)))
> > +srctree := $(patsubst %/,%,$(dir $(srctree)))
> > +endif
> > +
> > +FEATURE_USER = .bpf
> > +FEATURE_TESTS = libbfd disassembler-four-args
> > +FEATURE_DISPLAY = libbfd disassembler-four-args
> 
> Thanks for adding libbfd as I requested. However, you do not use it in
> the Makefile to prevent compilation if the feature is not detected (see
> "bpfdep" or "elfdep" in tools/lib/bpf/Makefile. Sorry, I should have
> pointed it in my previous review.
> 
> But actually, I have another issue related to the libbfd feature: since
> commit 280e7c48c3b8 ("perf tools: fix BFD detection on opensuse") it
> requires libiberty so that libbfd is correctly detected, but libiberty
> is not needed on all distros (at least Ubuntu can have libbfd without
> libiberty). Typically, detection fails on my setup, although I do have
> libbfd installed. So forcing libbfd feature here may eventually force
> users to install libraries they do not need to compile bpftool, which is
> not what we want.
> 
> I do not have a clean work around to suggest. Maybe have one
> "libbfd-something" feature that tries to compile without libiberty, then
> another one that tries with it, and compile the tools if at least one of
> them succeeds. But it's probably for another patch series. In the
> meantime, would you please simply remove libbfd detection here and
> accept my apologies for suggesting to add it in the previous review?

I think since libbfd is already used by bpftool it's a good thing
to add feature detection. Even if it's not perfect on some setups.

Roman,
I think you still need to do one more respin to address commit log nit?

Re: [PATCH v2 bpf-next 2/2] tools/bpftool: fix bpftool build with bintutils >= 2.8

[Quentin Monnet]

Hi Roman,

2017-12-22 16:11 UTC+ ~ Roman Gushchin 
> Bpftool build is broken with binutils version 2.28 and later.

Could you check the binutils version? I believe it changed in 2.29
instead of 2.28. Could you update your commit log and subject
accordingly, please?

> The cause is commit 003ca0fd2286 ("Refactor disassembler selection")
> in the binutils repo, which changed the disassembler() function
> signature.
> 
> Fix this by adding a new "feature" to the tools/build/features
> infrastructure and make it responsible for decision which
> disassembler() function signature to use.
> 
> Signed-off-by: Roman Gushchin 
> Cc: Jakub Kicinski 
> Cc: Alexei Starovoitov 
> Cc: Daniel Borkmann 
> ---
>  tools/bpf/Makefile| 29 
> +++
>  tools/bpf/bpf_jit_disasm.c|  7 ++
>  tools/bpf/bpftool/Makefile| 24 +++
>  tools/bpf/bpftool/jit_disasm.c|  7 ++
>  tools/build/feature/Makefile  |  4 
>  tools/build/feature/test-disassembler-four-args.c | 15 
>  6 files changed, 86 insertions(+)
>  create mode 100644 tools/build/feature/test-disassembler-four-args.c
> 
> diff --git a/tools/bpf/Makefile b/tools/bpf/Makefile
> index 07a6697466ef..c8ec0ae16bf0 100644
> --- a/tools/bpf/Makefile
> +++ b/tools/bpf/Makefile
> @@ -9,6 +9,35 @@ MAKE = make
>  CFLAGS += -Wall -O2
>  CFLAGS += -D__EXPORTED_HEADERS__ -I../../include/uapi -I../../include
>  
> +ifeq ($(srctree),)
> +srctree := $(patsubst %/,%,$(dir $(CURDIR)))
> +srctree := $(patsubst %/,%,$(dir $(srctree)))
> +endif
> +
> +FEATURE_USER = .bpf
> +FEATURE_TESTS = libbfd disassembler-four-args
> +FEATURE_DISPLAY = libbfd disassembler-four-args

Thanks for adding libbfd as I requested. However, you do not use it in
the Makefile to prevent compilation if the feature is not detected (see
"bpfdep" or "elfdep" in tools/lib/bpf/Makefile. Sorry, I should have
pointed it in my previous review.

But actually, I have another issue related to the libbfd feature: since
commit 280e7c48c3b8 ("perf tools: fix BFD detection on opensuse") it
requires libiberty so that libbfd is correctly detected, but libiberty
is not needed on all distros (at least Ubuntu can have libbfd without
libiberty). Typically, detection fails on my setup, although I do have
libbfd installed. So forcing libbfd feature here may eventually force
users to install libraries they do not need to compile bpftool, which is
not what we want.

I do not have a clean work around to suggest. Maybe have one
"libbfd-something" feature that tries to compile without libiberty, then
another one that tries with it, and compile the tools if at least one of
them succeeds. But it's probably for another patch series. In the
meantime, would you please simply remove libbfd detection here and
accept my apologies for suggesting to add it in the previous review?

> +
> +check_feat := 1
> +NON_CHECK_FEAT_TARGETS := clean bpftool_clean
> +ifdef MAKECMDGOALS
> +ifeq ($(filter-out $(NON_CHECK_FEAT_TARGETS),$(MAKECMDGOALS)),)
> +  check_feat := 0
> +endif
> +endif
> +
> +ifeq ($(check_feat),1)
> +ifeq ($(FEATURES_DUMP),)
> +include $(srctree)/tools/build/Makefile.feature
> +else
> +include $(FEATURES_DUMP)
> +endif
> +endif
> +
> +ifeq ($(feature-disassembler-four-args), 1)
> +CFLAGS += -DDISASM_FOUR_ARGS_SIGNATURE
> +endif
> +
>  %.yacc.c: %.y
>   $(YACC) -o $@ -d $<
>  
> diff --git a/tools/bpf/bpf_jit_disasm.c b/tools/bpf/bpf_jit_disasm.c
> index 75bf526a0168..30044bc4f389 100644
> --- a/tools/bpf/bpf_jit_disasm.c
> +++ b/tools/bpf/bpf_jit_disasm.c
> @@ -72,7 +72,14 @@ static void get_asm_insns(uint8_t *image, size_t len, int 
> opcodes)
>  
>   disassemble_init_for_target();
>  
> +#ifdef DISASM_FOUR_ARGS_SIGNATURE
> + disassemble = disassembler(info.arch,
> +bfd_big_endian(bfdf),
> +info.mach,
> +bfdf);
> +#else
>   disassemble = disassembler(bfdf);
> +#endif
>   assert(disassemble);
>  
>   do {
> diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
> index f8f31a8d9269..2237bc43f71c 100644
> --- a/tools/bpf/bpftool/Makefile
> +++ b/tools/bpf/bpftool/Makefile
> @@ -46,6 +46,30 @@ LIBS = -lelf -lbfd -lopcodes $(LIBBPF)
>  INSTALL ?= install
>  RM ?= rm -f
>  
> +FEATURE_USER = .bpftool
> +FEATURE_TESTS = libbfd disassembler-four-args
> +FEATURE_DISPLAY = libbfd disassembler-four-args
> +
> +check_feat := 1
> +NON_CHECK_FEAT_TARGETS := clean uninstall doc doc-clean doc-install 
> doc-uninstall

Nit: exclude "install" as well? I know libbpf does not exclude it, but
if the user runs `make` then `make install` we do not need to check
again the features for binary installation.

> +ifdef MAKECMDGOALS
> +ifeq ($(filter-out 

[PATCH v2 bpf-next 2/2] tools/bpftool: fix bpftool build with bintutils >= 2.8

[Roman Gushchin]

Bpftool build is broken with binutils version 2.28 and later.
The cause is commit 003ca0fd2286 ("Refactor disassembler selection")
in the binutils repo, which changed the disassembler() function
signature.

Fix this by adding a new "feature" to the tools/build/features
infrastructure and make it responsible for decision which
disassembler() function signature to use.

Signed-off-by: Roman Gushchin 
Cc: Jakub Kicinski 
Cc: Alexei Starovoitov 
Cc: Daniel Borkmann 
---
 tools/bpf/Makefile| 29 +++
 tools/bpf/bpf_jit_disasm.c|  7 ++
 tools/bpf/bpftool/Makefile| 24 +++
 tools/bpf/bpftool/jit_disasm.c|  7 ++
 tools/build/feature/Makefile  |  4 
 tools/build/feature/test-disassembler-four-args.c | 15 
 6 files changed, 86 insertions(+)
 create mode 100644 tools/build/feature/test-disassembler-four-args.c

diff --git a/tools/bpf/Makefile b/tools/bpf/Makefile
index 07a6697466ef..c8ec0ae16bf0 100644
--- a/tools/bpf/Makefile
+++ b/tools/bpf/Makefile
@@ -9,6 +9,35 @@ MAKE = make
 CFLAGS += -Wall -O2
 CFLAGS += -D__EXPORTED_HEADERS__ -I../../include/uapi -I../../include
 
+ifeq ($(srctree),)
+srctree := $(patsubst %/,%,$(dir $(CURDIR)))
+srctree := $(patsubst %/,%,$(dir $(srctree)))
+endif
+
+FEATURE_USER = .bpf
+FEATURE_TESTS = libbfd disassembler-four-args
+FEATURE_DISPLAY = libbfd disassembler-four-args
+
+check_feat := 1
+NON_CHECK_FEAT_TARGETS := clean bpftool_clean
+ifdef MAKECMDGOALS
+ifeq ($(filter-out $(NON_CHECK_FEAT_TARGETS),$(MAKECMDGOALS)),)
+  check_feat := 0
+endif
+endif
+
+ifeq ($(check_feat),1)
+ifeq ($(FEATURES_DUMP),)
+include $(srctree)/tools/build/Makefile.feature
+else
+include $(FEATURES_DUMP)
+endif
+endif
+
+ifeq ($(feature-disassembler-four-args), 1)
+CFLAGS += -DDISASM_FOUR_ARGS_SIGNATURE
+endif
+
 %.yacc.c: %.y
$(YACC) -o $@ -d $<
 
diff --git a/tools/bpf/bpf_jit_disasm.c b/tools/bpf/bpf_jit_disasm.c
index 75bf526a0168..30044bc4f389 100644
--- a/tools/bpf/bpf_jit_disasm.c
+++ b/tools/bpf/bpf_jit_disasm.c
@@ -72,7 +72,14 @@ static void get_asm_insns(uint8_t *image, size_t len, int 
opcodes)
 
disassemble_init_for_target();
 
+#ifdef DISASM_FOUR_ARGS_SIGNATURE
+   disassemble = disassembler(info.arch,
+  bfd_big_endian(bfdf),
+  info.mach,
+  bfdf);
+#else
disassemble = disassembler(bfdf);
+#endif
assert(disassemble);
 
do {
diff --git a/tools/bpf/bpftool/Makefile b/tools/bpf/bpftool/Makefile
index f8f31a8d9269..2237bc43f71c 100644
--- a/tools/bpf/bpftool/Makefile
+++ b/tools/bpf/bpftool/Makefile
@@ -46,6 +46,30 @@ LIBS = -lelf -lbfd -lopcodes $(LIBBPF)
 INSTALL ?= install
 RM ?= rm -f
 
+FEATURE_USER = .bpftool
+FEATURE_TESTS = libbfd disassembler-four-args
+FEATURE_DISPLAY = libbfd disassembler-four-args
+
+check_feat := 1
+NON_CHECK_FEAT_TARGETS := clean uninstall doc doc-clean doc-install 
doc-uninstall
+ifdef MAKECMDGOALS
+ifeq ($(filter-out $(NON_CHECK_FEAT_TARGETS),$(MAKECMDGOALS)),)
+  check_feat := 0
+endif
+endif
+
+ifeq ($(check_feat),1)
+ifeq ($(FEATURES_DUMP),)
+include $(srctree)/tools/build/Makefile.feature
+else
+include $(FEATURES_DUMP)
+endif
+endif
+
+ifeq ($(feature-disassembler-four-args), 1)
+CFLAGS += -DDISASM_FOUR_ARGS_SIGNATURE
+endif
+
 include $(wildcard *.d)
 
 all: $(OUTPUT)bpftool
diff --git a/tools/bpf/bpftool/jit_disasm.c b/tools/bpf/bpftool/jit_disasm.c
index 1551d3918d4c..57d32e8a1391 100644
--- a/tools/bpf/bpftool/jit_disasm.c
+++ b/tools/bpf/bpftool/jit_disasm.c
@@ -107,7 +107,14 @@ void disasm_print_insn(unsigned char *image, ssize_t len, 
int opcodes)
 
disassemble_init_for_target();
 
+#ifdef DISASM_FOUR_ARGS_SIGNATURE
+   disassemble = disassembler(info.arch,
+  bfd_big_endian(bfdf),
+  info.mach,
+  bfdf);
+#else
disassemble = disassembler(bfdf);
+#endif
assert(disassemble);
 
if (json_output)
diff --git a/tools/build/feature/Makefile b/tools/build/feature/Makefile
index 96982640fbf8..17f2c73fff8b 100644
--- a/tools/build/feature/Makefile
+++ b/tools/build/feature/Makefile
@@ -13,6 +13,7 @@ FILES=  \
  test-hello.bin \
  test-libaudit.bin  \
  test-libbfd.bin\
+ test-disassembler-four-args.bin\
  test-liberty.bin   \
  test-liberty-z.bin \
  test-cplus-demangle.bin\
@@ -188,6 +189,9 @@ $(OUTPUT)test-libpython-version.bin:
 $(OUTPUT)test-libbfd.bin:
$(BUILD) -DPACKAGE='"perf"' -lbfd -lz -liberty