subject:"Re\: perf test LLVM \& clang 6 failing"

Re: perf test LLVM & clang 6 failing

2017-11-28 Thread Arnaldo Carvalho de Melo

Em Tue, Nov 28, 2017 at 02:55:53PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Mon, Nov 27, 2017 at 01:45:21PM -0800, Matthias Kaehlcke escreveu:
> > El Mon, Nov 27, 2017 at 01:57:56PM -0600 Josh Poimboeuf ha dit:
> > > On Mon, Nov 27, 2017 at 04:34:25PM -0300, Arnaldo Carvalho de Melo wrote:
> > > > Em Mon, Nov 27, 2017 at 11:11:56AM -0800, Yonghong Song escreveu:
> > > > > On 11/27/17 9:04 AM, Arnaldo Carvalho de Melo wrote:
> > > > > > Em Fri, Nov 24, 2017 at 04:16:52PM +0100, Daniel Borkmann escreveu:
> > > > > > > [ +Yonghong ]

> > > > > > + Josh
> > > > > > > On 11/24/2017 03:47 PM, Arnaldo Carvalho de Melo wrote:
> > > > > > > > FYI, just noticed, recently updated clang to version 6, from its
> > > > > > > > upstream git repo.

> > > > > > > Do you recall what was your LLVM version prior to this where it 
> > > > > > > was
> > > > > > > working fine? (Wild guess from below would be the BPF inline asm
> > > > > > > support that was added recently to LLVM (2865ab6996) vs asm() used
> > > > > > > in headers included in the stdin header causing trouble due to 
> > > > > > > arch
> > > > > > > mixup?)

> > > > > > So, if I go to the cset just before:

> > > > > > commit f5caf621ee357279e759c0911daf6d55c7d36f03
> > > > > > Author: Josh Poimboeuf 
> > > > > > Date:   Wed Sep 20 16:24:33 2017 -0500

> > > > > >  x86/asm: Fix inline asm call constraints for Clang
> > > > > > ---

> > > > > > 'perf test LLVM' works again:

> > > > > > [root@jouet ~]# perf test LLVM
> > > > > > 37: LLVM search and compile   :
> > > > > > 37.1: Basic BPF llvm compile  : Ok
> > > > > > 37.2: kbuild searching: Ok
> > > > > > 37.3: Compile source for BPF prologue generation  : Ok
> > > > > > 37.4: Compile source for BPF relocation   : Ok
> > > > > > [root@jouet ~]#

> > > > > > I.e. 'perf test LLVM' built from what is in my acme/perf/urgent 
> > > > > > branch,
> > > > > > targetted to v4.15, uses kernel headers and if I go to just before
> > > > > > f5caf621ee, it works again, both with clang from fedora26 (4.0.1) 
> > > > > > and
> > > > > > with 6.0, built from sources.

> > > > > This patch introduced a module level inline assembly.

> > > > > ...
> > > > > --- a/arch/x86/include/asm/asm.h
> > > > > +++ b/arch/x86/include/asm/asm.h
> > > > > @@ -132,4 +132,15 @@
> > > > >  /* For C file, we already have NOKPROBE_SYMBOL macro */
> > > > >  #endif

> > > > > +#ifndef __ASSEMBLY__
> > > > > +/*
> > > > > + * This output constraint should be used for any inline asm which 
> > > > > has a
> > > > > "call"
> > > > > + * instruction.  Otherwise the asm may be inserted before the frame 
> > > > > pointer
> > > > > + * gets set up by the containing function.  If you forget to do this,
> > > > > objtool
> > > > > + * may print a "call without frame pointer save/setup" warning.
> > > > > + */
> > > > > +register unsigned int __asm_call_sp asm("esp");
> > > > > +#define ASM_CALL_CONSTRAINT "+r" (__asm_call_sp)
> > > > > +#endif
> > > > > ...
> > > > > 
> > > > > This will cause "clang ... -target bpf ..." failure since 4.0 does 
> > > > > not have
> > > > > bpf inline asm support and 6.0 does not recognize the register 'esp'.

> > > > Ok, explains the problem then, Josh, ideas on how to proceed here?

> > > The original change to add the global inline asm:

> > >   5caf621ee35 ("x86/asm: Fix inline asm call constraints for Clang")

> > > was done to support clang in the first place.  Before that change, a
> > > clang-built kernel didn't even boot.  So I'm a bit perplexed by the fact
> > > that this change would be causing clang problems, since it was done to
> > > fix clang in the first place.

> > > Adding Andrey and Matthias, maybe they can help clarify things.

> > Indeed the change was needed to boot on x86.

> > I know next to nothing about BPF, if I understand correctly the error
> > is generated when compiling for the BPF "architecture" not for x86. In
> > this process x86 assembly headers are included, one of which contains
> > the declaration of the register variable, in an register that exists
> > on x86, but not BPS.

> > I guess the first questions is whether the x86 asm headers should/need
> > to be included when compiling for BPF. If this needed/can not be
> > easily avoided one option could be to put the declaration within an
> > ifdef __x86_64__ block.

> Right, unsure if this is the way to go, but fixes the problem here:

> +++ b/arch/x86/include/asm/asm.h
> @@ -136,6 +136,8 @@
>  #endif
>  
>  #ifndef __ASSEMBLY__
> +#if defined(__i386__) || defined(__x86_64__)
> +/* bpf target builds also include these headers */
>  /*
>   * This output constraint should be used for any inline asm which has a 
> "call"
>   * instruction.  Otherwise the asm may be inserted before the frame pointer
> @@ -145,5 +147,6 @@
>  register unsigned long current_stack_pointer asm(_ASM_SP);
>  #define

Re: perf test LLVM & clang 6 failing

2017-11-28 Thread Arnaldo Carvalho de Melo

Em Tue, Nov 28, 2017 at 02:55:53PM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Mon, Nov 27, 2017 at 01:45:21PM -0800, Matthias Kaehlcke escreveu:
> > El Mon, Nov 27, 2017 at 01:57:56PM -0600 Josh Poimboeuf ha dit:
> > > On Mon, Nov 27, 2017 at 04:34:25PM -0300, Arnaldo Carvalho de Melo wrote:
> > > > Em Mon, Nov 27, 2017 at 11:11:56AM -0800, Yonghong Song escreveu:
> > > > > On 11/27/17 9:04 AM, Arnaldo Carvalho de Melo wrote:
> > > > > > Em Fri, Nov 24, 2017 at 04:16:52PM +0100, Daniel Borkmann escreveu:
> > > > > > > [ +Yonghong ]

> > > > > > + Josh
> > > > > > > On 11/24/2017 03:47 PM, Arnaldo Carvalho de Melo wrote:
> > > > > > > > FYI, just noticed, recently updated clang to version 6, from its
> > > > > > > > upstream git repo.

> > > > > > > Do you recall what was your LLVM version prior to this where it 
> > > > > > > was
> > > > > > > working fine? (Wild guess from below would be the BPF inline asm
> > > > > > > support that was added recently to LLVM (2865ab6996) vs asm() used
> > > > > > > in headers included in the stdin header causing trouble due to 
> > > > > > > arch
> > > > > > > mixup?)

> > > > > > So, if I go to the cset just before:

> > > > > > commit f5caf621ee357279e759c0911daf6d55c7d36f03
> > > > > > Author: Josh Poimboeuf 
> > > > > > Date:   Wed Sep 20 16:24:33 2017 -0500

> > > > > >  x86/asm: Fix inline asm call constraints for Clang
> > > > > > ---

> > > > > > 'perf test LLVM' works again:

> > > > > > [root@jouet ~]# perf test LLVM
> > > > > > 37: LLVM search and compile   :
> > > > > > 37.1: Basic BPF llvm compile  : Ok
> > > > > > 37.2: kbuild searching: Ok
> > > > > > 37.3: Compile source for BPF prologue generation  : Ok
> > > > > > 37.4: Compile source for BPF relocation   : Ok
> > > > > > [root@jouet ~]#

> > > > > > I.e. 'perf test LLVM' built from what is in my acme/perf/urgent 
> > > > > > branch,
> > > > > > targetted to v4.15, uses kernel headers and if I go to just before
> > > > > > f5caf621ee, it works again, both with clang from fedora26 (4.0.1) 
> > > > > > and
> > > > > > with 6.0, built from sources.

> > > > > This patch introduced a module level inline assembly.

> > > > > ...
> > > > > --- a/arch/x86/include/asm/asm.h
> > > > > +++ b/arch/x86/include/asm/asm.h
> > > > > @@ -132,4 +132,15 @@
> > > > >  /* For C file, we already have NOKPROBE_SYMBOL macro */
> > > > >  #endif

> > > > > +#ifndef __ASSEMBLY__
> > > > > +/*
> > > > > + * This output constraint should be used for any inline asm which 
> > > > > has a
> > > > > "call"
> > > > > + * instruction.  Otherwise the asm may be inserted before the frame 
> > > > > pointer
> > > > > + * gets set up by the containing function.  If you forget to do this,
> > > > > objtool
> > > > > + * may print a "call without frame pointer save/setup" warning.
> > > > > + */
> > > > > +register unsigned int __asm_call_sp asm("esp");
> > > > > +#define ASM_CALL_CONSTRAINT "+r" (__asm_call_sp)
> > > > > +#endif
> > > > > ...
> > > > > 
> > > > > This will cause "clang ... -target bpf ..." failure since 4.0 does 
> > > > > not have
> > > > > bpf inline asm support and 6.0 does not recognize the register 'esp'.

> > > > Ok, explains the problem then, Josh, ideas on how to proceed here?

> > > The original change to add the global inline asm:

> > >   5caf621ee35 ("x86/asm: Fix inline asm call constraints for Clang")

> > > was done to support clang in the first place.  Before that change, a
> > > clang-built kernel didn't even boot.  So I'm a bit perplexed by the fact
> > > that this change would be causing clang problems, since it was done to
> > > fix clang in the first place.

> > > Adding Andrey and Matthias, maybe they can help clarify things.

> > Indeed the change was needed to boot on x86.

> > I know next to nothing about BPF, if I understand correctly the error
> > is generated when compiling for the BPF "architecture" not for x86. In
> > this process x86 assembly headers are included, one of which contains
> > the declaration of the register variable, in an register that exists
> > on x86, but not BPS.

> > I guess the first questions is whether the x86 asm headers should/need
> > to be included when compiling for BPF. If this needed/can not be
> > easily avoided one option could be to put the declaration within an
> > ifdef __x86_64__ block.

> Right, unsure if this is the way to go, but fixes the problem here:

> +++ b/arch/x86/include/asm/asm.h
> @@ -136,6 +136,8 @@
>  #endif
>  
>  #ifndef __ASSEMBLY__
> +#if defined(__i386__) || defined(__x86_64__)
> +/* bpf target builds also include these headers */
>  /*
>   * This output constraint should be used for any inline asm which has a 
> "call"
>   * instruction.  Otherwise the asm may be inserted before the frame pointer
> @@ -145,5 +147,6 @@
>  register unsigned long current_stack_pointer asm(_ASM_SP);
>  #define ASM_CALL_CONSTRAINT "+r"

Re: perf test LLVM & clang 6 failing

2017-11-28 Thread Arnaldo Carvalho de Melo

Em Mon, Nov 27, 2017 at 01:45:21PM -0800, Matthias Kaehlcke escreveu:
> El Mon, Nov 27, 2017 at 01:57:56PM -0600 Josh Poimboeuf ha dit:
> 
> > On Mon, Nov 27, 2017 at 04:34:25PM -0300, Arnaldo Carvalho de Melo wrote:
> > > Em Mon, Nov 27, 2017 at 11:11:56AM -0800, Yonghong Song escreveu:
> > > > On 11/27/17 9:04 AM, Arnaldo Carvalho de Melo wrote:
> > > > > Em Fri, Nov 24, 2017 at 04:16:52PM +0100, Daniel Borkmann escreveu:
> > > > > > [ +Yonghong ]
> > > > > 
> > > > > + Josh
> > > > > > On 11/24/2017 03:47 PM, Arnaldo Carvalho de Melo wrote:
> > > > > > > FYI, just noticed, recently updated clang to version 6, from its
> > > > > > > upstream git repo.
> > > > > > 
> > > > > > Do you recall what was your LLVM version prior to this where it was
> > > > > > working fine? (Wild guess from below would be the BPF inline asm
> > > > > > support that was added recently to LLVM (2865ab6996) vs asm() used
> > > > > > in headers included in the stdin header causing trouble due to arch
> > > > > > mixup?)
> > > > > 
> > > > > So, if I go to the cset just before:
> > > > > 
> > > > > commit f5caf621ee357279e759c0911daf6d55c7d36f03
> > > > > Author: Josh Poimboeuf 
> > > > > Date:   Wed Sep 20 16:24:33 2017 -0500
> > > > > 
> > > > >  x86/asm: Fix inline asm call constraints for Clang
> > > > > ---
> > > > > 
> > > > > 'perf test LLVM' works again:
> > > > > 
> > > > > [root@jouet ~]# perf test LLVM
> > > > > 37: LLVM search and compile   :
> > > > > 37.1: Basic BPF llvm compile  : Ok
> > > > > 37.2: kbuild searching: Ok
> > > > > 37.3: Compile source for BPF prologue generation  : Ok
> > > > > 37.4: Compile source for BPF relocation   : Ok
> > > > > [root@jouet ~]#
> > > > > 
> > > > > I.e. 'perf test LLVM' built from what is in my acme/perf/urgent 
> > > > > branch,
> > > > > targetted to v4.15, uses kernel headers and if I go to just before
> > > > > f5caf621ee, it works again, both with clang from fedora26 (4.0.1) and
> > > > > with 6.0, built from sources.
> > > > 
> > > > This patch introduced a module level inline assembly.
> > > > 
> > > > ...
> > > > --- a/arch/x86/include/asm/asm.h
> > > > +++ b/arch/x86/include/asm/asm.h
> > > > @@ -132,4 +132,15 @@
> > > >  /* For C file, we already have NOKPROBE_SYMBOL macro */
> > > >  #endif
> > > > 
> > > > +#ifndef __ASSEMBLY__
> > > > +/*
> > > > + * This output constraint should be used for any inline asm which has a
> > > > "call"
> > > > + * instruction.  Otherwise the asm may be inserted before the frame 
> > > > pointer
> > > > + * gets set up by the containing function.  If you forget to do this,
> > > > objtool
> > > > + * may print a "call without frame pointer save/setup" warning.
> > > > + */
> > > > +register unsigned int __asm_call_sp asm("esp");
> > > > +#define ASM_CALL_CONSTRAINT "+r" (__asm_call_sp)
> > > > +#endif
> > > > ...
> > > > 
> > > > This will cause "clang ... -target bpf ..." failure since 4.0 does not 
> > > > have
> > > > bpf inline asm support and 6.0 does not recognize the register 'esp'.
> > > 
> > > Ok, explains the problem then, Josh, ideas on how to proceed here?
> > 
> > The original change to add the global inline asm:
> > 
> >   5caf621ee35 ("x86/asm: Fix inline asm call constraints for Clang")
> > 
> > was done to support clang in the first place.  Before that change, a
> > clang-built kernel didn't even boot.  So I'm a bit perplexed by the fact
> > that this change would be causing clang problems, since it was done to
> > fix clang in the first place.
> > 
> > Adding Andrey and Matthias, maybe they can help clarify things.
> 
> Indeed the change was needed to boot on x86.
> 
> I know next to nothing about BPF, if I understand correctly the error
> is generated when compiling for the BPF "architecture" not for x86. In
> this process x86 assembly headers are included, one of which contains
> the declaration of the register variable, in an register that exists
> on x86, but not BPS.
> 
> I guess the first questions is whether the x86 asm headers should/need
> to be included when compiling for BPF. If this needed/can not be
> easily avoided one option could be to put the declaration within an
> ifdef __x86_64__ block.

Right, unsure if this is the way to go, but fixes the problem here:

diff --git a/arch/x86/include/asm/asm.h b/arch/x86/include/asm/asm.h
index 219faaec51df..d180d573fd38 100644
--- a/arch/x86/include/asm/asm.h
+++ b/arch/x86/include/asm/asm.h
@@ -136,6 +136,8 @@
 #endif
 
 #ifndef __ASSEMBLY__
+#if defined(__i386__) || defined(__x86_64__)
+/* bpf target builds also include these headers */
 /*
  * This output constraint should be used for any inline asm which has a "call"
  * instruction.  Otherwise the asm may be inserted before the frame pointer
@@ -145,5 +147,6 @@
 register unsigned long current_stack_pointer asm(_ASM_SP);
 #define ASM_CALL_CONSTRAINT "+r"

Re: perf test LLVM & clang 6 failing

2017-11-28 Thread Arnaldo Carvalho de Melo

Em Mon, Nov 27, 2017 at 01:45:21PM -0800, Matthias Kaehlcke escreveu:
> El Mon, Nov 27, 2017 at 01:57:56PM -0600 Josh Poimboeuf ha dit:
> 
> > On Mon, Nov 27, 2017 at 04:34:25PM -0300, Arnaldo Carvalho de Melo wrote:
> > > Em Mon, Nov 27, 2017 at 11:11:56AM -0800, Yonghong Song escreveu:
> > > > On 11/27/17 9:04 AM, Arnaldo Carvalho de Melo wrote:
> > > > > Em Fri, Nov 24, 2017 at 04:16:52PM +0100, Daniel Borkmann escreveu:
> > > > > > [ +Yonghong ]
> > > > > 
> > > > > + Josh
> > > > > > On 11/24/2017 03:47 PM, Arnaldo Carvalho de Melo wrote:
> > > > > > > FYI, just noticed, recently updated clang to version 6, from its
> > > > > > > upstream git repo.
> > > > > > 
> > > > > > Do you recall what was your LLVM version prior to this where it was
> > > > > > working fine? (Wild guess from below would be the BPF inline asm
> > > > > > support that was added recently to LLVM (2865ab6996) vs asm() used
> > > > > > in headers included in the stdin header causing trouble due to arch
> > > > > > mixup?)
> > > > > 
> > > > > So, if I go to the cset just before:
> > > > > 
> > > > > commit f5caf621ee357279e759c0911daf6d55c7d36f03
> > > > > Author: Josh Poimboeuf 
> > > > > Date:   Wed Sep 20 16:24:33 2017 -0500
> > > > > 
> > > > >  x86/asm: Fix inline asm call constraints for Clang
> > > > > ---
> > > > > 
> > > > > 'perf test LLVM' works again:
> > > > > 
> > > > > [root@jouet ~]# perf test LLVM
> > > > > 37: LLVM search and compile   :
> > > > > 37.1: Basic BPF llvm compile  : Ok
> > > > > 37.2: kbuild searching: Ok
> > > > > 37.3: Compile source for BPF prologue generation  : Ok
> > > > > 37.4: Compile source for BPF relocation   : Ok
> > > > > [root@jouet ~]#
> > > > > 
> > > > > I.e. 'perf test LLVM' built from what is in my acme/perf/urgent 
> > > > > branch,
> > > > > targetted to v4.15, uses kernel headers and if I go to just before
> > > > > f5caf621ee, it works again, both with clang from fedora26 (4.0.1) and
> > > > > with 6.0, built from sources.
> > > > 
> > > > This patch introduced a module level inline assembly.
> > > > 
> > > > ...
> > > > --- a/arch/x86/include/asm/asm.h
> > > > +++ b/arch/x86/include/asm/asm.h
> > > > @@ -132,4 +132,15 @@
> > > >  /* For C file, we already have NOKPROBE_SYMBOL macro */
> > > >  #endif
> > > > 
> > > > +#ifndef __ASSEMBLY__
> > > > +/*
> > > > + * This output constraint should be used for any inline asm which has a
> > > > "call"
> > > > + * instruction.  Otherwise the asm may be inserted before the frame 
> > > > pointer
> > > > + * gets set up by the containing function.  If you forget to do this,
> > > > objtool
> > > > + * may print a "call without frame pointer save/setup" warning.
> > > > + */
> > > > +register unsigned int __asm_call_sp asm("esp");
> > > > +#define ASM_CALL_CONSTRAINT "+r" (__asm_call_sp)
> > > > +#endif
> > > > ...
> > > > 
> > > > This will cause "clang ... -target bpf ..." failure since 4.0 does not 
> > > > have
> > > > bpf inline asm support and 6.0 does not recognize the register 'esp'.
> > > 
> > > Ok, explains the problem then, Josh, ideas on how to proceed here?
> > 
> > The original change to add the global inline asm:
> > 
> >   5caf621ee35 ("x86/asm: Fix inline asm call constraints for Clang")
> > 
> > was done to support clang in the first place.  Before that change, a
> > clang-built kernel didn't even boot.  So I'm a bit perplexed by the fact
> > that this change would be causing clang problems, since it was done to
> > fix clang in the first place.
> > 
> > Adding Andrey and Matthias, maybe they can help clarify things.
> 
> Indeed the change was needed to boot on x86.
> 
> I know next to nothing about BPF, if I understand correctly the error
> is generated when compiling for the BPF "architecture" not for x86. In
> this process x86 assembly headers are included, one of which contains
> the declaration of the register variable, in an register that exists
> on x86, but not BPS.
> 
> I guess the first questions is whether the x86 asm headers should/need
> to be included when compiling for BPF. If this needed/can not be
> easily avoided one option could be to put the declaration within an
> ifdef __x86_64__ block.

Right, unsure if this is the way to go, but fixes the problem here:

diff --git a/arch/x86/include/asm/asm.h b/arch/x86/include/asm/asm.h
index 219faaec51df..d180d573fd38 100644
--- a/arch/x86/include/asm/asm.h
+++ b/arch/x86/include/asm/asm.h
@@ -136,6 +136,8 @@
 #endif
 
 #ifndef __ASSEMBLY__
+#if defined(__i386__) || defined(__x86_64__)
+/* bpf target builds also include these headers */
 /*
  * This output constraint should be used for any inline asm which has a "call"
  * instruction.  Otherwise the asm may be inserted before the frame pointer
@@ -145,5 +147,6 @@
 register unsigned long current_stack_pointer asm(_ASM_SP);
 #define ASM_CALL_CONSTRAINT "+r" (current_stack_pointer)
 #endif

Re: perf test LLVM & clang 6 failing

2017-11-27 Thread Yonghong Song




On 11/27/17 1:45 PM, Matthias Kaehlcke wrote:

El Mon, Nov 27, 2017 at 01:57:56PM -0600 Josh Poimboeuf ha dit:


On Mon, Nov 27, 2017 at 04:34:25PM -0300, Arnaldo Carvalho de Melo wrote:

Em Mon, Nov 27, 2017 at 11:11:56AM -0800, Yonghong Song escreveu:

On 11/27/17 9:04 AM, Arnaldo Carvalho de Melo wrote:

Em Fri, Nov 24, 2017 at 04:16:52PM +0100, Daniel Borkmann escreveu:

[ +Yonghong ]


+ Josh

On 11/24/2017 03:47 PM, Arnaldo Carvalho de Melo wrote:

FYI, just noticed, recently updated clang to version 6, from its
upstream git repo.


Do you recall what was your LLVM version prior to this where it was
working fine? (Wild guess from below would be the BPF inline asm
support that was added recently to LLVM (2865ab6996) vs asm() used
in headers included in the stdin header causing trouble due to arch
mixup?)


So, if I go to the cset just before:

commit f5caf621ee357279e759c0911daf6d55c7d36f03
Author: Josh Poimboeuf 
Date:   Wed Sep 20 16:24:33 2017 -0500

  x86/asm: Fix inline asm call constraints for Clang
---

'perf test LLVM' works again:

[root@jouet ~]# perf test LLVM
37: LLVM search and compile   :
37.1: Basic BPF llvm compile  : Ok
37.2: kbuild searching: Ok
37.3: Compile source for BPF prologue generation  : Ok
37.4: Compile source for BPF relocation   : Ok
[root@jouet ~]#

I.e. 'perf test LLVM' built from what is in my acme/perf/urgent branch,
targetted to v4.15, uses kernel headers and if I go to just before
f5caf621ee, it works again, both with clang from fedora26 (4.0.1) and
with 6.0, built from sources.


This patch introduced a module level inline assembly.

...
--- a/arch/x86/include/asm/asm.h
+++ b/arch/x86/include/asm/asm.h
@@ -132,4 +132,15 @@
  /* For C file, we already have NOKPROBE_SYMBOL macro */
  #endif

+#ifndef __ASSEMBLY__
+/*
+ * This output constraint should be used for any inline asm which has a
"call"
+ * instruction.  Otherwise the asm may be inserted before the frame pointer
+ * gets set up by the containing function.  If you forget to do this,
objtool
+ * may print a "call without frame pointer save/setup" warning.
+ */
+register unsigned int __asm_call_sp asm("esp");
+#define ASM_CALL_CONSTRAINT "+r" (__asm_call_sp)
+#endif
...

This will cause "clang ... -target bpf ..." failure since 4.0 does not have
bpf inline asm support and 6.0 does not recognize the register 'esp'.


Ok, explains the problem then, Josh, ideas on how to proceed here?


The original change to add the global inline asm:

   5caf621ee35 ("x86/asm: Fix inline asm call constraints for Clang")

was done to support clang in the first place.  Before that change, a
clang-built kernel didn't even boot.  So I'm a bit perplexed by the fact
that this change would be causing clang problems, since it was done to
fix clang in the first place.

Adding Andrey and Matthias, maybe they can help clarify things.


Indeed the change was needed to boot on x86.

I know next to nothing about BPF, if I understand correctly the error
is generated when compiling for the BPF "architecture" not for x86. In
this process x86 assembly headers are included, one of which contains
the declaration of the register variable, in an register that exists
on x86, but not BPS.

I guess the first questions is whether the x86 asm headers should/need
to be included when compiling for BPF. If this needed/can not be
easily avoided one option could be to put the declaration within an
ifdef __x86_64__ block.


There is a way to do this. You can use the similar mechanism to
the one in linux:samples/bpf and linux:tools/testing/selftests/bpf.

Basically, you first do:
  clang ... -O2 -emit-llvm -S prog.c <=== this uses x86_64 as the 
default target

  llc -march=bpf -filetype=obj prog.ll <=== this uses bpf target
This should work.

Re: perf test LLVM & clang 6 failing

2017-11-27 Thread Yonghong Song




On 11/27/17 1:45 PM, Matthias Kaehlcke wrote:

El Mon, Nov 27, 2017 at 01:57:56PM -0600 Josh Poimboeuf ha dit:


On Mon, Nov 27, 2017 at 04:34:25PM -0300, Arnaldo Carvalho de Melo wrote:

Em Mon, Nov 27, 2017 at 11:11:56AM -0800, Yonghong Song escreveu:

On 11/27/17 9:04 AM, Arnaldo Carvalho de Melo wrote:

Em Fri, Nov 24, 2017 at 04:16:52PM +0100, Daniel Borkmann escreveu:

[ +Yonghong ]


+ Josh

On 11/24/2017 03:47 PM, Arnaldo Carvalho de Melo wrote:

FYI, just noticed, recently updated clang to version 6, from its
upstream git repo.


Do you recall what was your LLVM version prior to this where it was
working fine? (Wild guess from below would be the BPF inline asm
support that was added recently to LLVM (2865ab6996) vs asm() used
in headers included in the stdin header causing trouble due to arch
mixup?)


So, if I go to the cset just before:

commit f5caf621ee357279e759c0911daf6d55c7d36f03
Author: Josh Poimboeuf 
Date:   Wed Sep 20 16:24:33 2017 -0500

  x86/asm: Fix inline asm call constraints for Clang
---

'perf test LLVM' works again:

[root@jouet ~]# perf test LLVM
37: LLVM search and compile   :
37.1: Basic BPF llvm compile  : Ok
37.2: kbuild searching: Ok
37.3: Compile source for BPF prologue generation  : Ok
37.4: Compile source for BPF relocation   : Ok
[root@jouet ~]#

I.e. 'perf test LLVM' built from what is in my acme/perf/urgent branch,
targetted to v4.15, uses kernel headers and if I go to just before
f5caf621ee, it works again, both with clang from fedora26 (4.0.1) and
with 6.0, built from sources.


This patch introduced a module level inline assembly.

...
--- a/arch/x86/include/asm/asm.h
+++ b/arch/x86/include/asm/asm.h
@@ -132,4 +132,15 @@
  /* For C file, we already have NOKPROBE_SYMBOL macro */
  #endif

+#ifndef __ASSEMBLY__
+/*
+ * This output constraint should be used for any inline asm which has a
"call"
+ * instruction.  Otherwise the asm may be inserted before the frame pointer
+ * gets set up by the containing function.  If you forget to do this,
objtool
+ * may print a "call without frame pointer save/setup" warning.
+ */
+register unsigned int __asm_call_sp asm("esp");
+#define ASM_CALL_CONSTRAINT "+r" (__asm_call_sp)
+#endif
...

This will cause "clang ... -target bpf ..." failure since 4.0 does not have
bpf inline asm support and 6.0 does not recognize the register 'esp'.


Ok, explains the problem then, Josh, ideas on how to proceed here?


The original change to add the global inline asm:

   5caf621ee35 ("x86/asm: Fix inline asm call constraints for Clang")

was done to support clang in the first place.  Before that change, a
clang-built kernel didn't even boot.  So I'm a bit perplexed by the fact
that this change would be causing clang problems, since it was done to
fix clang in the first place.

Adding Andrey and Matthias, maybe they can help clarify things.


Indeed the change was needed to boot on x86.

I know next to nothing about BPF, if I understand correctly the error
is generated when compiling for the BPF "architecture" not for x86. In
this process x86 assembly headers are included, one of which contains
the declaration of the register variable, in an register that exists
on x86, but not BPS.

I guess the first questions is whether the x86 asm headers should/need
to be included when compiling for BPF. If this needed/can not be
easily avoided one option could be to put the declaration within an
ifdef __x86_64__ block.


There is a way to do this. You can use the similar mechanism to
the one in linux:samples/bpf and linux:tools/testing/selftests/bpf.

Basically, you first do:
  clang ... -O2 -emit-llvm -S prog.c <=== this uses x86_64 as the 
default target

  llc -march=bpf -filetype=obj prog.ll <=== this uses bpf target
This should work.

Re: perf test LLVM & clang 6 failing

2017-11-27 Thread Matthias Kaehlcke

El Mon, Nov 27, 2017 at 01:57:56PM -0600 Josh Poimboeuf ha dit:

> On Mon, Nov 27, 2017 at 04:34:25PM -0300, Arnaldo Carvalho de Melo wrote:
> > Em Mon, Nov 27, 2017 at 11:11:56AM -0800, Yonghong Song escreveu:
> > > On 11/27/17 9:04 AM, Arnaldo Carvalho de Melo wrote:
> > > > Em Fri, Nov 24, 2017 at 04:16:52PM +0100, Daniel Borkmann escreveu:
> > > > > [ +Yonghong ]
> > > > 
> > > > + Josh
> > > > > On 11/24/2017 03:47 PM, Arnaldo Carvalho de Melo wrote:
> > > > > > FYI, just noticed, recently updated clang to version 6, from its
> > > > > > upstream git repo.
> > > > > 
> > > > > Do you recall what was your LLVM version prior to this where it was
> > > > > working fine? (Wild guess from below would be the BPF inline asm
> > > > > support that was added recently to LLVM (2865ab6996) vs asm() used
> > > > > in headers included in the stdin header causing trouble due to arch
> > > > > mixup?)
> > > > 
> > > > So, if I go to the cset just before:
> > > > 
> > > > commit f5caf621ee357279e759c0911daf6d55c7d36f03
> > > > Author: Josh Poimboeuf 
> > > > Date:   Wed Sep 20 16:24:33 2017 -0500
> > > > 
> > > >  x86/asm: Fix inline asm call constraints for Clang
> > > > ---
> > > > 
> > > > 'perf test LLVM' works again:
> > > > 
> > > > [root@jouet ~]# perf test LLVM
> > > > 37: LLVM search and compile   :
> > > > 37.1: Basic BPF llvm compile  : Ok
> > > > 37.2: kbuild searching: Ok
> > > > 37.3: Compile source for BPF prologue generation  : Ok
> > > > 37.4: Compile source for BPF relocation   : Ok
> > > > [root@jouet ~]#
> > > > 
> > > > I.e. 'perf test LLVM' built from what is in my acme/perf/urgent branch,
> > > > targetted to v4.15, uses kernel headers and if I go to just before
> > > > f5caf621ee, it works again, both with clang from fedora26 (4.0.1) and
> > > > with 6.0, built from sources.
> > > 
> > > This patch introduced a module level inline assembly.
> > > 
> > > ...
> > > --- a/arch/x86/include/asm/asm.h
> > > +++ b/arch/x86/include/asm/asm.h
> > > @@ -132,4 +132,15 @@
> > >  /* For C file, we already have NOKPROBE_SYMBOL macro */
> > >  #endif
> > > 
> > > +#ifndef __ASSEMBLY__
> > > +/*
> > > + * This output constraint should be used for any inline asm which has a
> > > "call"
> > > + * instruction.  Otherwise the asm may be inserted before the frame 
> > > pointer
> > > + * gets set up by the containing function.  If you forget to do this,
> > > objtool
> > > + * may print a "call without frame pointer save/setup" warning.
> > > + */
> > > +register unsigned int __asm_call_sp asm("esp");
> > > +#define ASM_CALL_CONSTRAINT "+r" (__asm_call_sp)
> > > +#endif
> > > ...
> > > 
> > > This will cause "clang ... -target bpf ..." failure since 4.0 does not 
> > > have
> > > bpf inline asm support and 6.0 does not recognize the register 'esp'.
> > 
> > Ok, explains the problem then, Josh, ideas on how to proceed here?
> 
> The original change to add the global inline asm:
> 
>   5caf621ee35 ("x86/asm: Fix inline asm call constraints for Clang")
> 
> was done to support clang in the first place.  Before that change, a
> clang-built kernel didn't even boot.  So I'm a bit perplexed by the fact
> that this change would be causing clang problems, since it was done to
> fix clang in the first place.
> 
> Adding Andrey and Matthias, maybe they can help clarify things.

Indeed the change was needed to boot on x86.

I know next to nothing about BPF, if I understand correctly the error
is generated when compiling for the BPF "architecture" not for x86. In
this process x86 assembly headers are included, one of which contains
the declaration of the register variable, in an register that exists
on x86, but not BPS.

I guess the first questions is whether the x86 asm headers should/need
to be included when compiling for BPF. If this needed/can not be
easily avoided one option could be to put the declaration within an
ifdef __x86_64__ block.

Re: perf test LLVM & clang 6 failing

2017-11-27 Thread Matthias Kaehlcke

El Mon, Nov 27, 2017 at 01:57:56PM -0600 Josh Poimboeuf ha dit:

> On Mon, Nov 27, 2017 at 04:34:25PM -0300, Arnaldo Carvalho de Melo wrote:
> > Em Mon, Nov 27, 2017 at 11:11:56AM -0800, Yonghong Song escreveu:
> > > On 11/27/17 9:04 AM, Arnaldo Carvalho de Melo wrote:
> > > > Em Fri, Nov 24, 2017 at 04:16:52PM +0100, Daniel Borkmann escreveu:
> > > > > [ +Yonghong ]
> > > > 
> > > > + Josh
> > > > > On 11/24/2017 03:47 PM, Arnaldo Carvalho de Melo wrote:
> > > > > > FYI, just noticed, recently updated clang to version 6, from its
> > > > > > upstream git repo.
> > > > > 
> > > > > Do you recall what was your LLVM version prior to this where it was
> > > > > working fine? (Wild guess from below would be the BPF inline asm
> > > > > support that was added recently to LLVM (2865ab6996) vs asm() used
> > > > > in headers included in the stdin header causing trouble due to arch
> > > > > mixup?)
> > > > 
> > > > So, if I go to the cset just before:
> > > > 
> > > > commit f5caf621ee357279e759c0911daf6d55c7d36f03
> > > > Author: Josh Poimboeuf 
> > > > Date:   Wed Sep 20 16:24:33 2017 -0500
> > > > 
> > > >  x86/asm: Fix inline asm call constraints for Clang
> > > > ---
> > > > 
> > > > 'perf test LLVM' works again:
> > > > 
> > > > [root@jouet ~]# perf test LLVM
> > > > 37: LLVM search and compile   :
> > > > 37.1: Basic BPF llvm compile  : Ok
> > > > 37.2: kbuild searching: Ok
> > > > 37.3: Compile source for BPF prologue generation  : Ok
> > > > 37.4: Compile source for BPF relocation   : Ok
> > > > [root@jouet ~]#
> > > > 
> > > > I.e. 'perf test LLVM' built from what is in my acme/perf/urgent branch,
> > > > targetted to v4.15, uses kernel headers and if I go to just before
> > > > f5caf621ee, it works again, both with clang from fedora26 (4.0.1) and
> > > > with 6.0, built from sources.
> > > 
> > > This patch introduced a module level inline assembly.
> > > 
> > > ...
> > > --- a/arch/x86/include/asm/asm.h
> > > +++ b/arch/x86/include/asm/asm.h
> > > @@ -132,4 +132,15 @@
> > >  /* For C file, we already have NOKPROBE_SYMBOL macro */
> > >  #endif
> > > 
> > > +#ifndef __ASSEMBLY__
> > > +/*
> > > + * This output constraint should be used for any inline asm which has a
> > > "call"
> > > + * instruction.  Otherwise the asm may be inserted before the frame 
> > > pointer
> > > + * gets set up by the containing function.  If you forget to do this,
> > > objtool
> > > + * may print a "call without frame pointer save/setup" warning.
> > > + */
> > > +register unsigned int __asm_call_sp asm("esp");
> > > +#define ASM_CALL_CONSTRAINT "+r" (__asm_call_sp)
> > > +#endif
> > > ...
> > > 
> > > This will cause "clang ... -target bpf ..." failure since 4.0 does not 
> > > have
> > > bpf inline asm support and 6.0 does not recognize the register 'esp'.
> > 
> > Ok, explains the problem then, Josh, ideas on how to proceed here?
> 
> The original change to add the global inline asm:
> 
>   5caf621ee35 ("x86/asm: Fix inline asm call constraints for Clang")
> 
> was done to support clang in the first place.  Before that change, a
> clang-built kernel didn't even boot.  So I'm a bit perplexed by the fact
> that this change would be causing clang problems, since it was done to
> fix clang in the first place.
> 
> Adding Andrey and Matthias, maybe they can help clarify things.

Indeed the change was needed to boot on x86.

I know next to nothing about BPF, if I understand correctly the error
is generated when compiling for the BPF "architecture" not for x86. In
this process x86 assembly headers are included, one of which contains
the declaration of the register variable, in an register that exists
on x86, but not BPS.

I guess the first questions is whether the x86 asm headers should/need
to be included when compiling for BPF. If this needed/can not be
easily avoided one option could be to put the declaration within an
ifdef __x86_64__ block.

Re: perf test LLVM & clang 6 failing

2017-11-27 Thread Arnaldo Carvalho de Melo

Em Mon, Nov 27, 2017 at 04:34:25PM -0300, Arnaldo Carvalho de Melo escreveu:
> So, I noticed that any maps are failing, I'll go dig, but may be some
> new security tightening, even running as root, this was working
> recently, was even part of our discussion on the bpf_probe_read_str()
> trouble with clang's optimizer:
> 
> [root@jouet bpf]# cat open.c
> #include "bpf.h"
> 
> SEC("prog=do_sys_open filename")
> int prog(void *ctx, int err, char *filename_ptr)
> {
>   char filename[128];
>   int len = bpf_probe_read_str(filename, sizeof(filename), filename_ptr); 
>   if (len > 0) {
>   if (len == 1)
>   perf_event_output(ctx, &__bpf_stdout__, 
> BPF_F_CURRENT_CPU, filename, len);
>   else if (len < 128)
>   perf_event_output(ctx, &__bpf_stdout__, 
> BPF_F_CURRENT_CPU, filename, len);
> }
>   return 1;
> }
> [root@jouet bpf]#
> 
> Found 1 probe_trace_events.
> Opening /sys/kernel/debug/tracing//kprobe_events write=1
> Writing event: p:perf_bpf_probe/prog _text+2493856 filename=%si:x64
> In map_prologue, ntevs=1
> mapping[0]=0
> libbpf: failed to create map (name: '__bpf_stdout__'): Invalid argument
> libbpf: failed to load object 'open.c'
> bpf: load objects failed
> event syntax error: 'open.c'
>  \___ Operation not permitted
> 
> 
> Using 'perf ftrace' to trace just 'perf trace':
> 
> [root@jouet bpf]# perf ftrace -G SyS_bpf perf trace -v -e open.c,open cat 
> /tmp/somefile 2> /dev/null
>  0)   |  SyS_bpf() {
>  0)   |capable() {
>  0)   |  ns_capable_common() {
>  0)   |security_capable() {
>  0)   0.045 us|  cap_capable();
>  0)   |  selinux_capable() {
>  0)   0.274 us|cred_has_capability();
>  0)   0.518 us|  }
>  0)   1.464 us|}
>  0)   1.783 us|  }
>  0)   2.130 us|}
>  0)   0.458 us|check_uarg_tail_zero();
>  0)   |__check_object_size() {
>  0)   0.046 us|  __virt_addr_valid();
>  0)   0.040 us|  check_stack_object();
>  0)   0.510 us|}
>  0)   4.161 us|  }
> [root@jouet bpf]#
> 
> /me goes to look at SyS_bpf() in this kernel... (4.14.0+).

Tracing 'perf trace' with 'perf trace' we see:

# perf trace -e bpf perf trace -e open.c,open cat /tmp/somefile

  0.000 ( 0.015 ms): perf/16767 bpf(cmd: MAP_CREATE, uattr: 0x7ffc3c8c7ac0, 
size: 72) = -1 EINVAL Invalid argument
#

Humm,

# perf probe check_stack_object%return 'ret=$retval'
Added new event:
  probe:check_stack_object (on check_stack_object%return with ret=$retval)

You can now use it in all perf tools, such as:

perf record -e probe:check_stack_object -aR sleep 1

#

# perf trace -e bpf,probe:check* perf trace -e open.c,open cat /tmp/somefile

Re: perf test LLVM & clang 6 failing

2017-11-27 Thread Arnaldo Carvalho de Melo

Em Mon, Nov 27, 2017 at 04:34:25PM -0300, Arnaldo Carvalho de Melo escreveu:
> So, I noticed that any maps are failing, I'll go dig, but may be some
> new security tightening, even running as root, this was working
> recently, was even part of our discussion on the bpf_probe_read_str()
> trouble with clang's optimizer:
> 
> [root@jouet bpf]# cat open.c
> #include "bpf.h"
> 
> SEC("prog=do_sys_open filename")
> int prog(void *ctx, int err, char *filename_ptr)
> {
>   char filename[128];
>   int len = bpf_probe_read_str(filename, sizeof(filename), filename_ptr); 
>   if (len > 0) {
>   if (len == 1)
>   perf_event_output(ctx, &__bpf_stdout__, 
> BPF_F_CURRENT_CPU, filename, len);
>   else if (len < 128)
>   perf_event_output(ctx, &__bpf_stdout__, 
> BPF_F_CURRENT_CPU, filename, len);
> }
>   return 1;
> }
> [root@jouet bpf]#
> 
> Found 1 probe_trace_events.
> Opening /sys/kernel/debug/tracing//kprobe_events write=1
> Writing event: p:perf_bpf_probe/prog _text+2493856 filename=%si:x64
> In map_prologue, ntevs=1
> mapping[0]=0
> libbpf: failed to create map (name: '__bpf_stdout__'): Invalid argument
> libbpf: failed to load object 'open.c'
> bpf: load objects failed
> event syntax error: 'open.c'
>  \___ Operation not permitted
> 
> 
> Using 'perf ftrace' to trace just 'perf trace':
> 
> [root@jouet bpf]# perf ftrace -G SyS_bpf perf trace -v -e open.c,open cat 
> /tmp/somefile 2> /dev/null
>  0)   |  SyS_bpf() {
>  0)   |capable() {
>  0)   |  ns_capable_common() {
>  0)   |security_capable() {
>  0)   0.045 us|  cap_capable();
>  0)   |  selinux_capable() {
>  0)   0.274 us|cred_has_capability();
>  0)   0.518 us|  }
>  0)   1.464 us|}
>  0)   1.783 us|  }
>  0)   2.130 us|}
>  0)   0.458 us|check_uarg_tail_zero();
>  0)   |__check_object_size() {
>  0)   0.046 us|  __virt_addr_valid();
>  0)   0.040 us|  check_stack_object();
>  0)   0.510 us|}
>  0)   4.161 us|  }
> [root@jouet bpf]#
> 
> /me goes to look at SyS_bpf() in this kernel... (4.14.0+).

Tracing 'perf trace' with 'perf trace' we see:

# perf trace -e bpf perf trace -e open.c,open cat /tmp/somefile

  0.000 ( 0.015 ms): perf/16767 bpf(cmd: MAP_CREATE, uattr: 0x7ffc3c8c7ac0, 
size: 72) = -1 EINVAL Invalid argument
#

Humm,

# perf probe check_stack_object%return 'ret=$retval'
Added new event:
  probe:check_stack_object (on check_stack_object%return with ret=$retval)

You can now use it in all perf tools, such as:

perf record -e probe:check_stack_object -aR sleep 1

#

# perf trace -e bpf,probe:check* perf trace -e open.c,open cat /tmp/somefile

Re: perf test LLVM & clang 6 failing

2017-11-27 Thread Josh Poimboeuf

On Mon, Nov 27, 2017 at 04:34:25PM -0300, Arnaldo Carvalho de Melo wrote:
> Em Mon, Nov 27, 2017 at 11:11:56AM -0800, Yonghong Song escreveu:
> > On 11/27/17 9:04 AM, Arnaldo Carvalho de Melo wrote:
> > > Em Fri, Nov 24, 2017 at 04:16:52PM +0100, Daniel Borkmann escreveu:
> > > > [ +Yonghong ]
> > > 
> > > + Josh
> > > > On 11/24/2017 03:47 PM, Arnaldo Carvalho de Melo wrote:
> > > > > FYI, just noticed, recently updated clang to version 6, from its
> > > > > upstream git repo.
> > > > 
> > > > Do you recall what was your LLVM version prior to this where it was
> > > > working fine? (Wild guess from below would be the BPF inline asm
> > > > support that was added recently to LLVM (2865ab6996) vs asm() used
> > > > in headers included in the stdin header causing trouble due to arch
> > > > mixup?)
> > > 
> > > So, if I go to the cset just before:
> > > 
> > > commit f5caf621ee357279e759c0911daf6d55c7d36f03
> > > Author: Josh Poimboeuf 
> > > Date:   Wed Sep 20 16:24:33 2017 -0500
> > > 
> > >  x86/asm: Fix inline asm call constraints for Clang
> > > ---
> > > 
> > > 'perf test LLVM' works again:
> > > 
> > > [root@jouet ~]# perf test LLVM
> > > 37: LLVM search and compile   :
> > > 37.1: Basic BPF llvm compile  : Ok
> > > 37.2: kbuild searching: Ok
> > > 37.3: Compile source for BPF prologue generation  : Ok
> > > 37.4: Compile source for BPF relocation   : Ok
> > > [root@jouet ~]#
> > > 
> > > I.e. 'perf test LLVM' built from what is in my acme/perf/urgent branch,
> > > targetted to v4.15, uses kernel headers and if I go to just before
> > > f5caf621ee, it works again, both with clang from fedora26 (4.0.1) and
> > > with 6.0, built from sources.
> > 
> > This patch introduced a module level inline assembly.
> > 
> > ...
> > --- a/arch/x86/include/asm/asm.h
> > +++ b/arch/x86/include/asm/asm.h
> > @@ -132,4 +132,15 @@
> >  /* For C file, we already have NOKPROBE_SYMBOL macro */
> >  #endif
> > 
> > +#ifndef __ASSEMBLY__
> > +/*
> > + * This output constraint should be used for any inline asm which has a
> > "call"
> > + * instruction.  Otherwise the asm may be inserted before the frame pointer
> > + * gets set up by the containing function.  If you forget to do this,
> > objtool
> > + * may print a "call without frame pointer save/setup" warning.
> > + */
> > +register unsigned int __asm_call_sp asm("esp");
> > +#define ASM_CALL_CONSTRAINT "+r" (__asm_call_sp)
> > +#endif
> > ...
> > 
> > This will cause "clang ... -target bpf ..." failure since 4.0 does not have
> > bpf inline asm support and 6.0 does not recognize the register 'esp'.
> 
> Ok, explains the problem then, Josh, ideas on how to proceed here?

The original change to add the global inline asm:

  5caf621ee35 ("x86/asm: Fix inline asm call constraints for Clang")

was done to support clang in the first place.  Before that change, a
clang-built kernel didn't even boot.  So I'm a bit perplexed by the fact
that this change would be causing clang problems, since it was done to
fix clang in the first place.

Adding Andrey and Matthias, maybe they can help clarify things.

-- 
Josh

Re: perf test LLVM & clang 6 failing

2017-11-27 Thread Josh Poimboeuf

On Mon, Nov 27, 2017 at 04:34:25PM -0300, Arnaldo Carvalho de Melo wrote:
> Em Mon, Nov 27, 2017 at 11:11:56AM -0800, Yonghong Song escreveu:
> > On 11/27/17 9:04 AM, Arnaldo Carvalho de Melo wrote:
> > > Em Fri, Nov 24, 2017 at 04:16:52PM +0100, Daniel Borkmann escreveu:
> > > > [ +Yonghong ]
> > > 
> > > + Josh
> > > > On 11/24/2017 03:47 PM, Arnaldo Carvalho de Melo wrote:
> > > > > FYI, just noticed, recently updated clang to version 6, from its
> > > > > upstream git repo.
> > > > 
> > > > Do you recall what was your LLVM version prior to this where it was
> > > > working fine? (Wild guess from below would be the BPF inline asm
> > > > support that was added recently to LLVM (2865ab6996) vs asm() used
> > > > in headers included in the stdin header causing trouble due to arch
> > > > mixup?)
> > > 
> > > So, if I go to the cset just before:
> > > 
> > > commit f5caf621ee357279e759c0911daf6d55c7d36f03
> > > Author: Josh Poimboeuf 
> > > Date:   Wed Sep 20 16:24:33 2017 -0500
> > > 
> > >  x86/asm: Fix inline asm call constraints for Clang
> > > ---
> > > 
> > > 'perf test LLVM' works again:
> > > 
> > > [root@jouet ~]# perf test LLVM
> > > 37: LLVM search and compile   :
> > > 37.1: Basic BPF llvm compile  : Ok
> > > 37.2: kbuild searching: Ok
> > > 37.3: Compile source for BPF prologue generation  : Ok
> > > 37.4: Compile source for BPF relocation   : Ok
> > > [root@jouet ~]#
> > > 
> > > I.e. 'perf test LLVM' built from what is in my acme/perf/urgent branch,
> > > targetted to v4.15, uses kernel headers and if I go to just before
> > > f5caf621ee, it works again, both with clang from fedora26 (4.0.1) and
> > > with 6.0, built from sources.
> > 
> > This patch introduced a module level inline assembly.
> > 
> > ...
> > --- a/arch/x86/include/asm/asm.h
> > +++ b/arch/x86/include/asm/asm.h
> > @@ -132,4 +132,15 @@
> >  /* For C file, we already have NOKPROBE_SYMBOL macro */
> >  #endif
> > 
> > +#ifndef __ASSEMBLY__
> > +/*
> > + * This output constraint should be used for any inline asm which has a
> > "call"
> > + * instruction.  Otherwise the asm may be inserted before the frame pointer
> > + * gets set up by the containing function.  If you forget to do this,
> > objtool
> > + * may print a "call without frame pointer save/setup" warning.
> > + */
> > +register unsigned int __asm_call_sp asm("esp");
> > +#define ASM_CALL_CONSTRAINT "+r" (__asm_call_sp)
> > +#endif
> > ...
> > 
> > This will cause "clang ... -target bpf ..." failure since 4.0 does not have
> > bpf inline asm support and 6.0 does not recognize the register 'esp'.
> 
> Ok, explains the problem then, Josh, ideas on how to proceed here?

The original change to add the global inline asm:

  5caf621ee35 ("x86/asm: Fix inline asm call constraints for Clang")

was done to support clang in the first place.  Before that change, a
clang-built kernel didn't even boot.  So I'm a bit perplexed by the fact
that this change would be causing clang problems, since it was done to
fix clang in the first place.

Adding Andrey and Matthias, maybe they can help clarify things.

-- 
Josh

Re: perf test LLVM & clang 6 failing

2017-11-27 Thread Arnaldo Carvalho de Melo

Em Mon, Nov 27, 2017 at 11:11:56AM -0800, Yonghong Song escreveu:
> On 11/27/17 9:04 AM, Arnaldo Carvalho de Melo wrote:
> > Em Fri, Nov 24, 2017 at 04:16:52PM +0100, Daniel Borkmann escreveu:
> > > [ +Yonghong ]
> > 
> > + Josh
> > > On 11/24/2017 03:47 PM, Arnaldo Carvalho de Melo wrote:
> > > > FYI, just noticed, recently updated clang to version 6, from its
> > > > upstream git repo.
> > > 
> > > Do you recall what was your LLVM version prior to this where it was
> > > working fine? (Wild guess from below would be the BPF inline asm
> > > support that was added recently to LLVM (2865ab6996) vs asm() used
> > > in headers included in the stdin header causing trouble due to arch
> > > mixup?)
> > 
> > So, if I go to the cset just before:
> > 
> > commit f5caf621ee357279e759c0911daf6d55c7d36f03
> > Author: Josh Poimboeuf 
> > Date:   Wed Sep 20 16:24:33 2017 -0500
> > 
> >  x86/asm: Fix inline asm call constraints for Clang
> > ---
> > 
> > 'perf test LLVM' works again:
> > 
> > [root@jouet ~]# perf test LLVM
> > 37: LLVM search and compile   :
> > 37.1: Basic BPF llvm compile  : Ok
> > 37.2: kbuild searching: Ok
> > 37.3: Compile source for BPF prologue generation  : Ok
> > 37.4: Compile source for BPF relocation   : Ok
> > [root@jouet ~]#
> > 
> > I.e. 'perf test LLVM' built from what is in my acme/perf/urgent branch,
> > targetted to v4.15, uses kernel headers and if I go to just before
> > f5caf621ee, it works again, both with clang from fedora26 (4.0.1) and
> > with 6.0, built from sources.
> 
> This patch introduced a module level inline assembly.
> 
> ...
> --- a/arch/x86/include/asm/asm.h
> +++ b/arch/x86/include/asm/asm.h
> @@ -132,4 +132,15 @@
>  /* For C file, we already have NOKPROBE_SYMBOL macro */
>  #endif
> 
> +#ifndef __ASSEMBLY__
> +/*
> + * This output constraint should be used for any inline asm which has a
> "call"
> + * instruction.  Otherwise the asm may be inserted before the frame pointer
> + * gets set up by the containing function.  If you forget to do this,
> objtool
> + * may print a "call without frame pointer save/setup" warning.
> + */
> +register unsigned int __asm_call_sp asm("esp");
> +#define ASM_CALL_CONSTRAINT "+r" (__asm_call_sp)
> +#endif
> ...
> 
> This will cause "clang ... -target bpf ..." failure since 4.0 does not have
> bpf inline asm support and 6.0 does not recognize the register 'esp'.

Ok, explains the problem then, Josh, ideas on how to proceed here?
 
> > 'perf test BPF' breaks for another reason, needs another investigation,
> > maybe you guys can spot something changed recently in that area:
> > 
> > libbpf: failed to create map (name: 'flip_table'): Invalid argument
> > libbpf: failed to load object '[basic_bpf_test]'


So, I noticed that any maps are failing, I'll go dig, but may be some
new security tightening, even running as root, this was working
recently, was even part of our discussion on the bpf_probe_read_str()
trouble with clang's optimizer:

[root@jouet bpf]# cat open.c
#include "bpf.h"

SEC("prog=do_sys_open filename")
int prog(void *ctx, int err, char *filename_ptr)
{
char filename[128];
int len = bpf_probe_read_str(filename, sizeof(filename), filename_ptr); 
if (len > 0) {
if (len == 1)
perf_event_output(ctx, &__bpf_stdout__, 
BPF_F_CURRENT_CPU, filename, len);
else if (len < 128)
perf_event_output(ctx, &__bpf_stdout__, 
BPF_F_CURRENT_CPU, filename, len);
}
return 1;
}
[root@jouet bpf]#

Found 1 probe_trace_events.
Opening /sys/kernel/debug/tracing//kprobe_events write=1
Writing event: p:perf_bpf_probe/prog _text+2493856 filename=%si:x64
In map_prologue, ntevs=1
mapping[0]=0
libbpf: failed to create map (name: '__bpf_stdout__'): Invalid argument
libbpf: failed to load object 'open.c'
bpf: load objects failed
event syntax error: 'open.c'
 \___ Operation not permitted


Using 'perf ftrace' to trace just 'perf trace':

[root@jouet bpf]# perf ftrace -G SyS_bpf perf trace -v -e open.c,open cat 
/tmp/somefile 2> /dev/null
 0)   |  SyS_bpf() {
 0)   |capable() {
 0)   |  ns_capable_common() {
 0)   |security_capable() {
 0)   0.045 us|  cap_capable();
 0)   |  selinux_capable() {
 0)   0.274 us|cred_has_capability();
 0)   0.518 us|  }
 0)   1.464 us|}
 0)   1.783 us|  }
 0)   2.130 us|}
 0)   0.458 us|check_uarg_tail_zero();
 0)   |__check_object_size() {
 0)   0.046 us|  __virt_addr_valid();
 0)   0.040 us|  check_stack_object();
 0)   0.510 us|}
 0)   4.161 us|  }
[root@jouet bpf]#

/me goes to look at SyS_bpf() in this kernel... (4.14.0+).

- Arnaldo

 
> Not sure

Re: perf test LLVM & clang 6 failing

2017-11-27 Thread Arnaldo Carvalho de Melo

Em Mon, Nov 27, 2017 at 11:11:56AM -0800, Yonghong Song escreveu:
> On 11/27/17 9:04 AM, Arnaldo Carvalho de Melo wrote:
> > Em Fri, Nov 24, 2017 at 04:16:52PM +0100, Daniel Borkmann escreveu:
> > > [ +Yonghong ]
> > 
> > + Josh
> > > On 11/24/2017 03:47 PM, Arnaldo Carvalho de Melo wrote:
> > > > FYI, just noticed, recently updated clang to version 6, from its
> > > > upstream git repo.
> > > 
> > > Do you recall what was your LLVM version prior to this where it was
> > > working fine? (Wild guess from below would be the BPF inline asm
> > > support that was added recently to LLVM (2865ab6996) vs asm() used
> > > in headers included in the stdin header causing trouble due to arch
> > > mixup?)
> > 
> > So, if I go to the cset just before:
> > 
> > commit f5caf621ee357279e759c0911daf6d55c7d36f03
> > Author: Josh Poimboeuf 
> > Date:   Wed Sep 20 16:24:33 2017 -0500
> > 
> >  x86/asm: Fix inline asm call constraints for Clang
> > ---
> > 
> > 'perf test LLVM' works again:
> > 
> > [root@jouet ~]# perf test LLVM
> > 37: LLVM search and compile   :
> > 37.1: Basic BPF llvm compile  : Ok
> > 37.2: kbuild searching: Ok
> > 37.3: Compile source for BPF prologue generation  : Ok
> > 37.4: Compile source for BPF relocation   : Ok
> > [root@jouet ~]#
> > 
> > I.e. 'perf test LLVM' built from what is in my acme/perf/urgent branch,
> > targetted to v4.15, uses kernel headers and if I go to just before
> > f5caf621ee, it works again, both with clang from fedora26 (4.0.1) and
> > with 6.0, built from sources.
> 
> This patch introduced a module level inline assembly.
> 
> ...
> --- a/arch/x86/include/asm/asm.h
> +++ b/arch/x86/include/asm/asm.h
> @@ -132,4 +132,15 @@
>  /* For C file, we already have NOKPROBE_SYMBOL macro */
>  #endif
> 
> +#ifndef __ASSEMBLY__
> +/*
> + * This output constraint should be used for any inline asm which has a
> "call"
> + * instruction.  Otherwise the asm may be inserted before the frame pointer
> + * gets set up by the containing function.  If you forget to do this,
> objtool
> + * may print a "call without frame pointer save/setup" warning.
> + */
> +register unsigned int __asm_call_sp asm("esp");
> +#define ASM_CALL_CONSTRAINT "+r" (__asm_call_sp)
> +#endif
> ...
> 
> This will cause "clang ... -target bpf ..." failure since 4.0 does not have
> bpf inline asm support and 6.0 does not recognize the register 'esp'.

Ok, explains the problem then, Josh, ideas on how to proceed here?
 
> > 'perf test BPF' breaks for another reason, needs another investigation,
> > maybe you guys can spot something changed recently in that area:
> > 
> > libbpf: failed to create map (name: 'flip_table'): Invalid argument
> > libbpf: failed to load object '[basic_bpf_test]'


So, I noticed that any maps are failing, I'll go dig, but may be some
new security tightening, even running as root, this was working
recently, was even part of our discussion on the bpf_probe_read_str()
trouble with clang's optimizer:

[root@jouet bpf]# cat open.c
#include "bpf.h"

SEC("prog=do_sys_open filename")
int prog(void *ctx, int err, char *filename_ptr)
{
char filename[128];
int len = bpf_probe_read_str(filename, sizeof(filename), filename_ptr); 
if (len > 0) {
if (len == 1)
perf_event_output(ctx, &__bpf_stdout__, 
BPF_F_CURRENT_CPU, filename, len);
else if (len < 128)
perf_event_output(ctx, &__bpf_stdout__, 
BPF_F_CURRENT_CPU, filename, len);
}
return 1;
}
[root@jouet bpf]#

Found 1 probe_trace_events.
Opening /sys/kernel/debug/tracing//kprobe_events write=1
Writing event: p:perf_bpf_probe/prog _text+2493856 filename=%si:x64
In map_prologue, ntevs=1
mapping[0]=0
libbpf: failed to create map (name: '__bpf_stdout__'): Invalid argument
libbpf: failed to load object 'open.c'
bpf: load objects failed
event syntax error: 'open.c'
 \___ Operation not permitted


Using 'perf ftrace' to trace just 'perf trace':

[root@jouet bpf]# perf ftrace -G SyS_bpf perf trace -v -e open.c,open cat 
/tmp/somefile 2> /dev/null
 0)   |  SyS_bpf() {
 0)   |capable() {
 0)   |  ns_capable_common() {
 0)   |security_capable() {
 0)   0.045 us|  cap_capable();
 0)   |  selinux_capable() {
 0)   0.274 us|cred_has_capability();
 0)   0.518 us|  }
 0)   1.464 us|}
 0)   1.783 us|  }
 0)   2.130 us|}
 0)   0.458 us|check_uarg_tail_zero();
 0)   |__check_object_size() {
 0)   0.046 us|  __virt_addr_valid();
 0)   0.040 us|  check_stack_object();
 0)   0.510 us|}
 0)   4.161 us|  }
[root@jouet bpf]#

/me goes to look at SyS_bpf() in this kernel... (4.14.0+).

- Arnaldo

 
> Not sure whether this is the

Re: perf test LLVM & clang 6 failing

2017-11-27 Thread Yonghong Song




On 11/27/17 9:04 AM, Arnaldo Carvalho de Melo wrote:

Em Fri, Nov 24, 2017 at 04:16:52PM +0100, Daniel Borkmann escreveu:

[ +Yonghong ]


+ Josh
  

On 11/24/2017 03:47 PM, Arnaldo Carvalho de Melo wrote:

FYI, just noticed, recently updated clang to version 6, from its
upstream git repo.


Do you recall what was your LLVM version prior to this where it was
working fine? (Wild guess from below would be the BPF inline asm
support that was added recently to LLVM (2865ab6996) vs asm() used
in headers included in the stdin header causing trouble due to arch
mixup?)


So, if I go to the cset just before:

commit f5caf621ee357279e759c0911daf6d55c7d36f03
Author: Josh Poimboeuf 
Date:   Wed Sep 20 16:24:33 2017 -0500

 x86/asm: Fix inline asm call constraints for Clang
---

'perf test LLVM' works again:

[root@jouet ~]# perf test LLVM
37: LLVM search and compile   :
37.1: Basic BPF llvm compile  : Ok
37.2: kbuild searching: Ok
37.3: Compile source for BPF prologue generation  : Ok
37.4: Compile source for BPF relocation   : Ok
[root@jouet ~]#

I.e. 'perf test LLVM' built from what is in my acme/perf/urgent branch,
targetted to v4.15, uses kernel headers and if I go to just before
f5caf621ee, it works again, both with clang from fedora26 (4.0.1) and
with 6.0, built from sources.


This patch introduced a module level inline assembly.

...
--- a/arch/x86/include/asm/asm.h
+++ b/arch/x86/include/asm/asm.h
@@ -132,4 +132,15 @@
 /* For C file, we already have NOKPROBE_SYMBOL macro */
 #endif

+#ifndef __ASSEMBLY__
+/*
+ * This output constraint should be used for any inline asm which has a 
"call"

+ * instruction.  Otherwise the asm may be inserted before the frame pointer
+ * gets set up by the containing function.  If you forget to do this, 
objtool

+ * may print a "call without frame pointer save/setup" warning.
+ */
+register unsigned int __asm_call_sp asm("esp");
+#define ASM_CALL_CONSTRAINT "+r" (__asm_call_sp)
+#endif
...

This will cause "clang ... -target bpf ..." failure since 4.0 does not 
have bpf inline asm support and 6.0 does not recognize the register 'esp'.




'perf test BPF' breaks for another reason, needs another investigation,
maybe you guys can spot something changed recently in that area:

libbpf: failed to create map (name: 'flip_table'): Invalid argument
libbpf: failed to load object '[basic_bpf_test]'


Not sure whether this is the root cause or not, but the below section name
   SEC("func=SyS_epoll_wait")
is not a good one.
If it is used to represent the program name in the kernel, the kernel 
will reject it with invalid argument since it contains "=".




Complete logs up to this error are below, I'll go read Yonghong message
to see if I can get the 'perf test LLVM' case fixed.

- Arnaldo

[root@jouet ~]# perf test BPF
39: BPF filter:
39.1: Basic BPF filtering : FAILED!
39.2: BPF pinning : Skip
39.3: BPF prologue generation : Skip
39.4: BPF relocation checker  : Skip
[root@jouet ~]# perf test -v BPF
39: BPF filter:
39.1: Basic BPF filtering :
--- start ---
test child forked, pid 3541
Kernel build dir is set to /lib/modules/4.14.0+/build
set env: KBUILD_DIR=/lib/modules/4.14.0+/build
unset env: KBUILD_OPTS
include option is set to  -nostdinc -isystem 
/usr/lib/gcc/x86_64-redhat-linux/7/include 
-I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  
-I/home/acme/git/linux/include -I./include 
-I/home/acme/git/linux/arch/x86/include/uapi 
-I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi 
-I./include/generated/uapi -include /home/acme/git/linux/include/linux/kconfig.h
set env: NR_CPUS=4
set env: LINUX_VERSION_CODE=0x40e00
set env: CLANG_EXEC=/usr/lib64/ccache/clang
set env: CLANG_OPTIONS=-xc
set env: KERNEL_INC_OPTIONS= -nostdinc -isystem 
/usr/lib/gcc/x86_64-redhat-linux/7/include 
-I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  
-I/home/acme/git/linux/include -I./include 
-I/home/acme/git/linux/arch/x86/include/uapi 
-I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi 
-I./include/generated/uapi -include /home/acme/git/linux/include/linux/kconfig.h
set env: WORKING_DIR=/lib/modules/4.14.0+/build
set env: CLANG_SOURCE=-
llvm compiling command template: echo '/*
  * bpf-script-example.c
  * Test basic LLVM building
  */
#ifndef LINUX_VERSION_CODE
# error Need LINUX_VERSION_CODE
# error Example: for 4.2 kernel, put 'clang-opt="-DLINUX_VERSION_CODE=0x40200" 
into llvm section of ~/.perfconfig'
#endif
#define BPF_ANY 0
#define BPF_MAP_TYPE_ARRAY 2
#define BPF_FUNC_map_lookup_elem 1
#define BPF_FUNC_map_update_elem 2

static

Re: perf test LLVM & clang 6 failing

2017-11-27 Thread Yonghong Song




On 11/27/17 9:04 AM, Arnaldo Carvalho de Melo wrote:

Em Fri, Nov 24, 2017 at 04:16:52PM +0100, Daniel Borkmann escreveu:

[ +Yonghong ]


+ Josh
  

On 11/24/2017 03:47 PM, Arnaldo Carvalho de Melo wrote:

FYI, just noticed, recently updated clang to version 6, from its
upstream git repo.


Do you recall what was your LLVM version prior to this where it was
working fine? (Wild guess from below would be the BPF inline asm
support that was added recently to LLVM (2865ab6996) vs asm() used
in headers included in the stdin header causing trouble due to arch
mixup?)


So, if I go to the cset just before:

commit f5caf621ee357279e759c0911daf6d55c7d36f03
Author: Josh Poimboeuf 
Date:   Wed Sep 20 16:24:33 2017 -0500

 x86/asm: Fix inline asm call constraints for Clang
---

'perf test LLVM' works again:

[root@jouet ~]# perf test LLVM
37: LLVM search and compile   :
37.1: Basic BPF llvm compile  : Ok
37.2: kbuild searching: Ok
37.3: Compile source for BPF prologue generation  : Ok
37.4: Compile source for BPF relocation   : Ok
[root@jouet ~]#

I.e. 'perf test LLVM' built from what is in my acme/perf/urgent branch,
targetted to v4.15, uses kernel headers and if I go to just before
f5caf621ee, it works again, both with clang from fedora26 (4.0.1) and
with 6.0, built from sources.


This patch introduced a module level inline assembly.

...
--- a/arch/x86/include/asm/asm.h
+++ b/arch/x86/include/asm/asm.h
@@ -132,4 +132,15 @@
 /* For C file, we already have NOKPROBE_SYMBOL macro */
 #endif

+#ifndef __ASSEMBLY__
+/*
+ * This output constraint should be used for any inline asm which has a 
"call"

+ * instruction.  Otherwise the asm may be inserted before the frame pointer
+ * gets set up by the containing function.  If you forget to do this, 
objtool

+ * may print a "call without frame pointer save/setup" warning.
+ */
+register unsigned int __asm_call_sp asm("esp");
+#define ASM_CALL_CONSTRAINT "+r" (__asm_call_sp)
+#endif
...

This will cause "clang ... -target bpf ..." failure since 4.0 does not 
have bpf inline asm support and 6.0 does not recognize the register 'esp'.




'perf test BPF' breaks for another reason, needs another investigation,
maybe you guys can spot something changed recently in that area:

libbpf: failed to create map (name: 'flip_table'): Invalid argument
libbpf: failed to load object '[basic_bpf_test]'


Not sure whether this is the root cause or not, but the below section name
   SEC("func=SyS_epoll_wait")
is not a good one.
If it is used to represent the program name in the kernel, the kernel 
will reject it with invalid argument since it contains "=".




Complete logs up to this error are below, I'll go read Yonghong message
to see if I can get the 'perf test LLVM' case fixed.

- Arnaldo

[root@jouet ~]# perf test BPF
39: BPF filter:
39.1: Basic BPF filtering : FAILED!
39.2: BPF pinning : Skip
39.3: BPF prologue generation : Skip
39.4: BPF relocation checker  : Skip
[root@jouet ~]# perf test -v BPF
39: BPF filter:
39.1: Basic BPF filtering :
--- start ---
test child forked, pid 3541
Kernel build dir is set to /lib/modules/4.14.0+/build
set env: KBUILD_DIR=/lib/modules/4.14.0+/build
unset env: KBUILD_OPTS
include option is set to  -nostdinc -isystem 
/usr/lib/gcc/x86_64-redhat-linux/7/include 
-I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  
-I/home/acme/git/linux/include -I./include 
-I/home/acme/git/linux/arch/x86/include/uapi 
-I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi 
-I./include/generated/uapi -include /home/acme/git/linux/include/linux/kconfig.h
set env: NR_CPUS=4
set env: LINUX_VERSION_CODE=0x40e00
set env: CLANG_EXEC=/usr/lib64/ccache/clang
set env: CLANG_OPTIONS=-xc
set env: KERNEL_INC_OPTIONS= -nostdinc -isystem 
/usr/lib/gcc/x86_64-redhat-linux/7/include 
-I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  
-I/home/acme/git/linux/include -I./include 
-I/home/acme/git/linux/arch/x86/include/uapi 
-I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi 
-I./include/generated/uapi -include /home/acme/git/linux/include/linux/kconfig.h
set env: WORKING_DIR=/lib/modules/4.14.0+/build
set env: CLANG_SOURCE=-
llvm compiling command template: echo '/*
  * bpf-script-example.c
  * Test basic LLVM building
  */
#ifndef LINUX_VERSION_CODE
# error Need LINUX_VERSION_CODE
# error Example: for 4.2 kernel, put 'clang-opt="-DLINUX_VERSION_CODE=0x40200" 
into llvm section of ~/.perfconfig'
#endif
#define BPF_ANY 0
#define BPF_MAP_TYPE_ARRAY 2
#define BPF_FUNC_map_lookup_elem 1
#define BPF_FUNC_map_update_elem 2

static void

Re: perf test LLVM & clang 6 failing

2017-11-27 Thread Arnaldo Carvalho de Melo

Em Fri, Nov 24, 2017 at 04:16:52PM +0100, Daniel Borkmann escreveu:
> [ +Yonghong ]

+ Josh
 
> On 11/24/2017 03:47 PM, Arnaldo Carvalho de Melo wrote:
> > FYI, just noticed, recently updated clang to version 6, from its
> > upstream git repo.
> 
> Do you recall what was your LLVM version prior to this where it was
> working fine? (Wild guess from below would be the BPF inline asm
> support that was added recently to LLVM (2865ab6996) vs asm() used
> in headers included in the stdin header causing trouble due to arch
> mixup?)

So, if I go to the cset just before:

commit f5caf621ee357279e759c0911daf6d55c7d36f03
Author: Josh Poimboeuf 
Date:   Wed Sep 20 16:24:33 2017 -0500

x86/asm: Fix inline asm call constraints for Clang
---

'perf test LLVM' works again:

[root@jouet ~]# perf test LLVM
37: LLVM search and compile   :
37.1: Basic BPF llvm compile  : Ok
37.2: kbuild searching: Ok
37.3: Compile source for BPF prologue generation  : Ok
37.4: Compile source for BPF relocation   : Ok
[root@jouet ~]#

I.e. 'perf test LLVM' built from what is in my acme/perf/urgent branch,
targetted to v4.15, uses kernel headers and if I go to just before
f5caf621ee, it works again, both with clang from fedora26 (4.0.1) and
with 6.0, built from sources.

'perf test BPF' breaks for another reason, needs another investigation,
maybe you guys can spot something changed recently in that area:

libbpf: failed to create map (name: 'flip_table'): Invalid argument
libbpf: failed to load object '[basic_bpf_test]'

Complete logs up to this error are below, I'll go read Yonghong message
to see if I can get the 'perf test LLVM' case fixed.

- Arnaldo

[root@jouet ~]# perf test BPF
39: BPF filter:
39.1: Basic BPF filtering : FAILED!
39.2: BPF pinning : Skip
39.3: BPF prologue generation : Skip
39.4: BPF relocation checker  : Skip
[root@jouet ~]# perf test -v BPF
39: BPF filter:
39.1: Basic BPF filtering :
--- start ---
test child forked, pid 3541
Kernel build dir is set to /lib/modules/4.14.0+/build
set env: KBUILD_DIR=/lib/modules/4.14.0+/build
unset env: KBUILD_OPTS
include option is set to  -nostdinc -isystem 
/usr/lib/gcc/x86_64-redhat-linux/7/include 
-I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  
-I/home/acme/git/linux/include -I./include 
-I/home/acme/git/linux/arch/x86/include/uapi 
-I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi 
-I./include/generated/uapi -include 
/home/acme/git/linux/include/linux/kconfig.h 
set env: NR_CPUS=4
set env: LINUX_VERSION_CODE=0x40e00
set env: CLANG_EXEC=/usr/lib64/ccache/clang
set env: CLANG_OPTIONS=-xc 
set env: KERNEL_INC_OPTIONS= -nostdinc -isystem 
/usr/lib/gcc/x86_64-redhat-linux/7/include 
-I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  
-I/home/acme/git/linux/include -I./include 
-I/home/acme/git/linux/arch/x86/include/uapi 
-I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi 
-I./include/generated/uapi -include 
/home/acme/git/linux/include/linux/kconfig.h 
set env: WORKING_DIR=/lib/modules/4.14.0+/build
set env: CLANG_SOURCE=-
llvm compiling command template: echo '/*
 * bpf-script-example.c
 * Test basic LLVM building
 */
#ifndef LINUX_VERSION_CODE
# error Need LINUX_VERSION_CODE
# error Example: for 4.2 kernel, put 'clang-opt="-DLINUX_VERSION_CODE=0x40200" 
into llvm section of ~/.perfconfig'
#endif
#define BPF_ANY 0
#define BPF_MAP_TYPE_ARRAY 2
#define BPF_FUNC_map_lookup_elem 1
#define BPF_FUNC_map_update_elem 2

static void *(*bpf_map_lookup_elem)(void *map, void *key) =
(void *) BPF_FUNC_map_lookup_elem;
static void *(*bpf_map_update_elem)(void *map, void *key, void *value, int 
flags) =
(void *) BPF_FUNC_map_update_elem;

struct bpf_map_def {
unsigned int type;
unsigned int key_size;
unsigned int value_size;
unsigned int max_entries;
};

#define SEC(NAME) __attribute__((section(NAME), used))
struct bpf_map_def SEC("maps") flip_table = {
.type = BPF_MAP_TYPE_ARRAY,
.key_size = sizeof(int),
.value_size = sizeof(int),
.max_entries = 1,
};

SEC("func=SyS_epoll_wait")
int bpf_func__SyS_epoll_wait(void *ctx)
{
int ind =0;
int *flag = bpf_map_lookup_elem(_table, );
int new_flag;
if (!flag)
return 0;
/* flip flag and store back */
new_flag = !*flag;
bpf_map_update_elem(_table, , _flag, BPF_ANY);
return new_flag;
}
char _license[] SEC("license") = "GPL";
int _version SEC("version") = LINUX_VERSION_CODE;
' | $CLANG_EXEC -D__KERNEL__ -D__NR_CPUS__=$NR_CPUS

Re: perf test LLVM & clang 6 failing

2017-11-27 Thread Arnaldo Carvalho de Melo

Em Fri, Nov 24, 2017 at 04:16:52PM +0100, Daniel Borkmann escreveu:
> [ +Yonghong ]

+ Josh
 
> On 11/24/2017 03:47 PM, Arnaldo Carvalho de Melo wrote:
> > FYI, just noticed, recently updated clang to version 6, from its
> > upstream git repo.
> 
> Do you recall what was your LLVM version prior to this where it was
> working fine? (Wild guess from below would be the BPF inline asm
> support that was added recently to LLVM (2865ab6996) vs asm() used
> in headers included in the stdin header causing trouble due to arch
> mixup?)

So, if I go to the cset just before:

commit f5caf621ee357279e759c0911daf6d55c7d36f03
Author: Josh Poimboeuf 
Date:   Wed Sep 20 16:24:33 2017 -0500

x86/asm: Fix inline asm call constraints for Clang
---

'perf test LLVM' works again:

[root@jouet ~]# perf test LLVM
37: LLVM search and compile   :
37.1: Basic BPF llvm compile  : Ok
37.2: kbuild searching: Ok
37.3: Compile source for BPF prologue generation  : Ok
37.4: Compile source for BPF relocation   : Ok
[root@jouet ~]#

I.e. 'perf test LLVM' built from what is in my acme/perf/urgent branch,
targetted to v4.15, uses kernel headers and if I go to just before
f5caf621ee, it works again, both with clang from fedora26 (4.0.1) and
with 6.0, built from sources.

'perf test BPF' breaks for another reason, needs another investigation,
maybe you guys can spot something changed recently in that area:

libbpf: failed to create map (name: 'flip_table'): Invalid argument
libbpf: failed to load object '[basic_bpf_test]'

Complete logs up to this error are below, I'll go read Yonghong message
to see if I can get the 'perf test LLVM' case fixed.

- Arnaldo

[root@jouet ~]# perf test BPF
39: BPF filter:
39.1: Basic BPF filtering : FAILED!
39.2: BPF pinning : Skip
39.3: BPF prologue generation : Skip
39.4: BPF relocation checker  : Skip
[root@jouet ~]# perf test -v BPF
39: BPF filter:
39.1: Basic BPF filtering :
--- start ---
test child forked, pid 3541
Kernel build dir is set to /lib/modules/4.14.0+/build
set env: KBUILD_DIR=/lib/modules/4.14.0+/build
unset env: KBUILD_OPTS
include option is set to  -nostdinc -isystem 
/usr/lib/gcc/x86_64-redhat-linux/7/include 
-I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  
-I/home/acme/git/linux/include -I./include 
-I/home/acme/git/linux/arch/x86/include/uapi 
-I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi 
-I./include/generated/uapi -include 
/home/acme/git/linux/include/linux/kconfig.h 
set env: NR_CPUS=4
set env: LINUX_VERSION_CODE=0x40e00
set env: CLANG_EXEC=/usr/lib64/ccache/clang
set env: CLANG_OPTIONS=-xc 
set env: KERNEL_INC_OPTIONS= -nostdinc -isystem 
/usr/lib/gcc/x86_64-redhat-linux/7/include 
-I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  
-I/home/acme/git/linux/include -I./include 
-I/home/acme/git/linux/arch/x86/include/uapi 
-I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi 
-I./include/generated/uapi -include 
/home/acme/git/linux/include/linux/kconfig.h 
set env: WORKING_DIR=/lib/modules/4.14.0+/build
set env: CLANG_SOURCE=-
llvm compiling command template: echo '/*
 * bpf-script-example.c
 * Test basic LLVM building
 */
#ifndef LINUX_VERSION_CODE
# error Need LINUX_VERSION_CODE
# error Example: for 4.2 kernel, put 'clang-opt="-DLINUX_VERSION_CODE=0x40200" 
into llvm section of ~/.perfconfig'
#endif
#define BPF_ANY 0
#define BPF_MAP_TYPE_ARRAY 2
#define BPF_FUNC_map_lookup_elem 1
#define BPF_FUNC_map_update_elem 2

static void *(*bpf_map_lookup_elem)(void *map, void *key) =
(void *) BPF_FUNC_map_lookup_elem;
static void *(*bpf_map_update_elem)(void *map, void *key, void *value, int 
flags) =
(void *) BPF_FUNC_map_update_elem;

struct bpf_map_def {
unsigned int type;
unsigned int key_size;
unsigned int value_size;
unsigned int max_entries;
};

#define SEC(NAME) __attribute__((section(NAME), used))
struct bpf_map_def SEC("maps") flip_table = {
.type = BPF_MAP_TYPE_ARRAY,
.key_size = sizeof(int),
.value_size = sizeof(int),
.max_entries = 1,
};

SEC("func=SyS_epoll_wait")
int bpf_func__SyS_epoll_wait(void *ctx)
{
int ind =0;
int *flag = bpf_map_lookup_elem(_table, );
int new_flag;
if (!flag)
return 0;
/* flip flag and store back */
new_flag = !*flag;
bpf_map_update_elem(_table, , _flag, BPF_ANY);
return new_flag;
}
char _license[] SEC("license") = "GPL";
int _version SEC("version") = LINUX_VERSION_CODE;
' | $CLANG_EXEC -D__KERNEL__ -D__NR_CPUS__=$NR_CPUS

Re: perf test LLVM & clang 6 failing

2017-11-25 Thread Yonghong Song




On 11/24/17 11:09 AM, Arnaldo Carvalho de Melo wrote:

Em Fri, Nov 24, 2017 at 04:16:52PM +0100, Daniel Borkmann escreveu:

[ +Yonghong ]

On 11/24/2017 03:47 PM, Arnaldo Carvalho de Melo wrote:

FYI, just noticed, recently updated clang to version 6, from its
upstream git repo.


Do you recall what was your LLVM version prior to this where it was
working fine? (Wild guess from below would be the BPF inline asm


IIRC it was 4.0


I tried the following example (the almost the same one as the failed
compilation one before except a few SEC(...) marking which should not
impact the result):

$ cat bpf_prog.c
#include 
#include 

int bpf_func__vfs_llseek(void *ctx)
{
return 0;
}
$ cat run.sh
KERNEL_INC_OPTIONS="-nostdinc -isystem 
/usr/lib/gcc/x86_64-redhat-linux/4.8.5/include 
-I/home/yhs/work/net-next/arch/x86/include 
-I/home/yhs/work/linux-bld/arch/x86/include 
-I/home/yhs/work/net-next/include -I/home/yhs/work/linux-bld/include 
-I/home/yhs/work/net-next/arch/x86/include/uapi 
-I/home/yhs/work/linux-bld/arch/x86/include/generated/uapi 
-I/home/yhs/work/net-next/include/uapi 
-I/home/yhs/work/linux-bld/include/generated/uapi -include 
/home/yhs/work/net-next/include/linux/kconfig.h"


clang -D__KERNEL__ -D__NR_CPUS__=4 -xc \
  $KERNEL_INC_OPTIONS \
  -Wno-unused-value -Wno-pointer-sign \
  -c bpf_prog.c -O2 -target bpf
$

You can actually get KERNEL_INC_OPTIONS by compiling kernel samples/bpf.

This program failed at both llvm 4.0 and latest trunk (6.0).

The same symptom:
In file included from bpf_prog.c:2:
In file included from 
/home/yhs/work/net-next/arch/x86/include/uapi/asm/ptrace.h:5:

In file included from /home/yhs/work/net-next/include/linux/compiler.h:237:
In file included from 
/home/yhs/work/net-next/arch/x86/include/asm/barrier.h:5:
In file included from 
/home/yhs/work/net-next/arch/x86/include/asm/alternative.h:10:
/home/yhs/work/net-next/arch/x86/include/asm/asm.h:145:50: error: 
unknown register name 'esp' in asm

register unsigned long current_stack_pointer asm(_ASM_SP);
 ^
/home/yhs/work/net-next/arch/x86/include/asm/asm.h:44:18: note: expanded 
from macro '_ASM_SP'

#define _ASM_SP __ASM_REG(sp)

I suggest that you use the same compilation process as
samples/bpf and tools/testing/selftests/bpf:

$ cat run1.sh
KERNEL_INC_OPTIONS="-nostdinc -isystem 
/usr/lib/gcc/x86_64-redhat-linux/4.8.5/include 
-I/home/yhs/work/net-next/arch/x86/include 
-I/home/yhs/work/linux-bld/arch/x86/include 
-I/home/yhs/work/net-next/include -I/home/yhs/work/linux-bld/include 
-I/home/yhs/work/net-next/arch/x86/include/uapi 
-I/home/yhs/work/linux-bld/arch/x86/include/generated/uapi 
-I/home/yhs/work/net-next/include/uapi 
-I/home/yhs/work/linux-bld/include/generated/uapi -include 
/home/yhs/work/net-next/include/linux/kconfig.h"


clang -D__KERNEL__ -D__NR_CPUS__=4 -xc \
  $KERNEL_INC_OPTIONS \
  -Wno-unused-value -Wno-pointer-sign \
  -c bpf_prog.c -O2 -emit-llvm -S
llc -filetype=obj -march=bpf bpf_prog.ll
$

This is especially true if you are doing tracing there ptrace.h may be 
included in your header files.


The "clang" compiler will be able to handle host inline asm properly and
filter out functions which uses host inline asm's. If any file-level
inline asms are still left (e.g., arm64), the llvm 6.0 should be able
to handle this but not llvm 4.0.




support that was added recently to LLVM (2865ab6996) vs asm() used
in headers included in the stdin header causing trouble due to arch
mixup?)


root@jouet ~]# perf test -v LLVM
37: LLVM search and compile   :
37.1: Basic BPF llvm compile  :
--- start ---
test child forked, pid 5255
Kernel build dir is set to /lib/modules/4.14.0+/build
set env: KBUILD_DIR=/lib/modules/4.14.0+/build
unset env: KBUILD_OPTS
include option is set to  -nostdinc -isystem 
/usr/lib/gcc/x86_64-redhat-linux/7/include 
-I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  
-I/home/acme/git/linux/include -I./include 
-I/home/acme/git/linux/arch/x86/include/uapi 
-I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi 
-I./include/generated/uapi -include /home/acme/git/linux/include/linux/kconfig.h
set env: NR_CPUS=4
set env: LINUX_VERSION_CODE=0x40e00
set env: CLANG_EXEC=/usr/local/bin/clang
set env: CLANG_OPTIONS=-xc
set env: KERNEL_INC_OPTIONS= -nostdinc -isystem 
/usr/lib/gcc/x86_64-redhat-linux/7/include 
-I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  
-I/home/acme/git/linux/include -I./include 
-I/home/acme/git/linux/arch/x86/include/uapi 
-I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi 
-I./include/generated/uapi -include /home/acme/git/linux/include/linux/kconfig.h
set env: WORKING_DIR=/lib/modules/4.14.0+/build
set env: CLANG_SOURCE=-
llvm compiling command template: echo '/*
  * bpf-script-example.c
  * Test basic LLVM building
  */
#ifndef

Re: perf test LLVM & clang 6 failing

2017-11-25 Thread Yonghong Song




On 11/24/17 11:09 AM, Arnaldo Carvalho de Melo wrote:

Em Fri, Nov 24, 2017 at 04:16:52PM +0100, Daniel Borkmann escreveu:

[ +Yonghong ]

On 11/24/2017 03:47 PM, Arnaldo Carvalho de Melo wrote:

FYI, just noticed, recently updated clang to version 6, from its
upstream git repo.


Do you recall what was your LLVM version prior to this where it was
working fine? (Wild guess from below would be the BPF inline asm


IIRC it was 4.0


I tried the following example (the almost the same one as the failed
compilation one before except a few SEC(...) marking which should not
impact the result):

$ cat bpf_prog.c
#include 
#include 

int bpf_func__vfs_llseek(void *ctx)
{
return 0;
}
$ cat run.sh
KERNEL_INC_OPTIONS="-nostdinc -isystem 
/usr/lib/gcc/x86_64-redhat-linux/4.8.5/include 
-I/home/yhs/work/net-next/arch/x86/include 
-I/home/yhs/work/linux-bld/arch/x86/include 
-I/home/yhs/work/net-next/include -I/home/yhs/work/linux-bld/include 
-I/home/yhs/work/net-next/arch/x86/include/uapi 
-I/home/yhs/work/linux-bld/arch/x86/include/generated/uapi 
-I/home/yhs/work/net-next/include/uapi 
-I/home/yhs/work/linux-bld/include/generated/uapi -include 
/home/yhs/work/net-next/include/linux/kconfig.h"


clang -D__KERNEL__ -D__NR_CPUS__=4 -xc \
  $KERNEL_INC_OPTIONS \
  -Wno-unused-value -Wno-pointer-sign \
  -c bpf_prog.c -O2 -target bpf
$

You can actually get KERNEL_INC_OPTIONS by compiling kernel samples/bpf.

This program failed at both llvm 4.0 and latest trunk (6.0).

The same symptom:
In file included from bpf_prog.c:2:
In file included from 
/home/yhs/work/net-next/arch/x86/include/uapi/asm/ptrace.h:5:

In file included from /home/yhs/work/net-next/include/linux/compiler.h:237:
In file included from 
/home/yhs/work/net-next/arch/x86/include/asm/barrier.h:5:
In file included from 
/home/yhs/work/net-next/arch/x86/include/asm/alternative.h:10:
/home/yhs/work/net-next/arch/x86/include/asm/asm.h:145:50: error: 
unknown register name 'esp' in asm

register unsigned long current_stack_pointer asm(_ASM_SP);
 ^
/home/yhs/work/net-next/arch/x86/include/asm/asm.h:44:18: note: expanded 
from macro '_ASM_SP'

#define _ASM_SP __ASM_REG(sp)

I suggest that you use the same compilation process as
samples/bpf and tools/testing/selftests/bpf:

$ cat run1.sh
KERNEL_INC_OPTIONS="-nostdinc -isystem 
/usr/lib/gcc/x86_64-redhat-linux/4.8.5/include 
-I/home/yhs/work/net-next/arch/x86/include 
-I/home/yhs/work/linux-bld/arch/x86/include 
-I/home/yhs/work/net-next/include -I/home/yhs/work/linux-bld/include 
-I/home/yhs/work/net-next/arch/x86/include/uapi 
-I/home/yhs/work/linux-bld/arch/x86/include/generated/uapi 
-I/home/yhs/work/net-next/include/uapi 
-I/home/yhs/work/linux-bld/include/generated/uapi -include 
/home/yhs/work/net-next/include/linux/kconfig.h"


clang -D__KERNEL__ -D__NR_CPUS__=4 -xc \
  $KERNEL_INC_OPTIONS \
  -Wno-unused-value -Wno-pointer-sign \
  -c bpf_prog.c -O2 -emit-llvm -S
llc -filetype=obj -march=bpf bpf_prog.ll
$

This is especially true if you are doing tracing there ptrace.h may be 
included in your header files.


The "clang" compiler will be able to handle host inline asm properly and
filter out functions which uses host inline asm's. If any file-level
inline asms are still left (e.g., arm64), the llvm 6.0 should be able
to handle this but not llvm 4.0.




support that was added recently to LLVM (2865ab6996) vs asm() used
in headers included in the stdin header causing trouble due to arch
mixup?)


root@jouet ~]# perf test -v LLVM
37: LLVM search and compile   :
37.1: Basic BPF llvm compile  :
--- start ---
test child forked, pid 5255
Kernel build dir is set to /lib/modules/4.14.0+/build
set env: KBUILD_DIR=/lib/modules/4.14.0+/build
unset env: KBUILD_OPTS
include option is set to  -nostdinc -isystem 
/usr/lib/gcc/x86_64-redhat-linux/7/include 
-I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  
-I/home/acme/git/linux/include -I./include 
-I/home/acme/git/linux/arch/x86/include/uapi 
-I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi 
-I./include/generated/uapi -include /home/acme/git/linux/include/linux/kconfig.h
set env: NR_CPUS=4
set env: LINUX_VERSION_CODE=0x40e00
set env: CLANG_EXEC=/usr/local/bin/clang
set env: CLANG_OPTIONS=-xc
set env: KERNEL_INC_OPTIONS= -nostdinc -isystem 
/usr/lib/gcc/x86_64-redhat-linux/7/include 
-I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  
-I/home/acme/git/linux/include -I./include 
-I/home/acme/git/linux/arch/x86/include/uapi 
-I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi 
-I./include/generated/uapi -include /home/acme/git/linux/include/linux/kconfig.h
set env: WORKING_DIR=/lib/modules/4.14.0+/build
set env: CLANG_SOURCE=-
llvm compiling command template: echo '/*
  * bpf-script-example.c
  * Test basic LLVM building
  */
#ifndef

Re: perf test LLVM & clang 6 failing

2017-11-24 Thread Arnaldo Carvalho de Melo

Em Fri, Nov 24, 2017 at 04:16:52PM +0100, Daniel Borkmann escreveu:
> [ +Yonghong ]
> 
> On 11/24/2017 03:47 PM, Arnaldo Carvalho de Melo wrote:
> > FYI, just noticed, recently updated clang to version 6, from its
> > upstream git repo.
> 
> Do you recall what was your LLVM version prior to this where it was
> working fine? (Wild guess from below would be the BPF inline asm

IIRC it was 4.0

> support that was added recently to LLVM (2865ab6996) vs asm() used
> in headers included in the stdin header causing trouble due to arch
> mixup?)
> 
> > root@jouet ~]# perf test -v LLVM
> > 37: LLVM search and compile   :
> > 37.1: Basic BPF llvm compile  :
> > --- start ---
> > test child forked, pid 5255
> > Kernel build dir is set to /lib/modules/4.14.0+/build
> > set env: KBUILD_DIR=/lib/modules/4.14.0+/build
> > unset env: KBUILD_OPTS
> > include option is set to  -nostdinc -isystem 
> > /usr/lib/gcc/x86_64-redhat-linux/7/include 
> > -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  
> > -I/home/acme/git/linux/include -I./include 
> > -I/home/acme/git/linux/arch/x86/include/uapi 
> > -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi 
> > -I./include/generated/uapi -include 
> > /home/acme/git/linux/include/linux/kconfig.h 
> > set env: NR_CPUS=4
> > set env: LINUX_VERSION_CODE=0x40e00
> > set env: CLANG_EXEC=/usr/local/bin/clang
> > set env: CLANG_OPTIONS=-xc 
> > set env: KERNEL_INC_OPTIONS= -nostdinc -isystem 
> > /usr/lib/gcc/x86_64-redhat-linux/7/include 
> > -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  
> > -I/home/acme/git/linux/include -I./include 
> > -I/home/acme/git/linux/arch/x86/include/uapi 
> > -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi 
> > -I./include/generated/uapi -include 
> > /home/acme/git/linux/include/linux/kconfig.h 
> > set env: WORKING_DIR=/lib/modules/4.14.0+/build
> > set env: CLANG_SOURCE=-
> > llvm compiling command template: echo '/*
> >  * bpf-script-example.c
> >  * Test basic LLVM building
> >  */
> > #ifndef LINUX_VERSION_CODE
> > # error Need LINUX_VERSION_CODE
> > # error Example: for 4.2 kernel, put 
> > 'clang-opt="-DLINUX_VERSION_CODE=0x40200" into llvm section of 
> > ~/.perfconfig'
> > #endif
> > #define BPF_ANY 0
> > #define BPF_MAP_TYPE_ARRAY 2
> > #define BPF_FUNC_map_lookup_elem 1
> > #define BPF_FUNC_map_update_elem 2
> > 
> > static void *(*bpf_map_lookup_elem)(void *map, void *key) =
> > (void *) BPF_FUNC_map_lookup_elem;
> > static void *(*bpf_map_update_elem)(void *map, void *key, void *value, int 
> > flags) =
> > (void *) BPF_FUNC_map_update_elem;
> > 
> > struct bpf_map_def {
> > unsigned int type;
> > unsigned int key_size;
> > unsigned int value_size;
> > unsigned int max_entries;
> > };
> > 
> > #define SEC(NAME) __attribute__((section(NAME), used))
> > struct bpf_map_def SEC("maps") flip_table = {
> > .type = BPF_MAP_TYPE_ARRAY,
> > .key_size = sizeof(int),
> > .value_size = sizeof(int),
> > .max_entries = 1,
> > };
> > 
> > SEC("func=SyS_epoll_wait")
> > int bpf_func__SyS_epoll_wait(void *ctx)
> > {
> > int ind =0;
> > int *flag = bpf_map_lookup_elem(_table, );
> > int new_flag;
> > if (!flag)
> > return 0;
> > /* flip flag and store back */
> > new_flag = !*flag;
> > bpf_map_update_elem(_table, , _flag, BPF_ANY);
> > return new_flag;
> > }
> > char _license[] SEC("license") = "GPL";
> > int _version SEC("version") = LINUX_VERSION_CODE;
> > ' | $CLANG_EXEC -D__KERNEL__ -D__NR_CPUS__=$NR_CPUS 
> > -DLINUX_VERSION_CODE=$LINUX_VERSION_CODE $CLANG_OPTIONS $KERNEL_INC_OPTIONS 
> > -Wno-unused-value -Wno-pointer-sign -working-directory $WORKING_DIR -c 
> > "$CLANG_SOURCE" -target bpf -O2 -o -
> > test child finished with 0
> >  end 
> > LLVM search and compile subtest 0: Ok
> > 37.2: kbuild searching:
> > --- start ---
> > test child forked, pid 5728
> > Kernel build dir is set to /lib/modules/4.14.0+/build
> > set env: KBUILD_DIR=/lib/modules/4.14.0+/build
> > unset env: KBUILD_OPTS
> > include option is set to  -nostdinc -isystem 
> > /usr/lib/gcc/x86_64-redhat-linux/7/include 
> > -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  
> > -I/home/acme/git/linux/include -I./include 
> > -I/home/acme/git/linux/arch/x86/include/uapi 
> > -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi 
> > -I./include/generated/uapi -include 
> > /home/acme/git/linux/include/linux/kconfig.h 
> > set env: NR_CPUS=4
> > set env: LINUX_VERSION_CODE=0x40e00
> > set env: CLANG_EXEC=/usr/local/bin/clang
> > set env: CLANG_OPTIONS=-xc 
> > set env: KERNEL_INC_OPTIONS= -nostdinc -isystem 
> > /usr/lib/gcc/x86_64-redhat-linux/7/include 
> > -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  
> > -I/home/acme/git/linux/include -I./include 
> >

Re: perf test LLVM & clang 6 failing

2017-11-24 Thread Arnaldo Carvalho de Melo

Em Fri, Nov 24, 2017 at 04:16:52PM +0100, Daniel Borkmann escreveu:
> [ +Yonghong ]
> 
> On 11/24/2017 03:47 PM, Arnaldo Carvalho de Melo wrote:
> > FYI, just noticed, recently updated clang to version 6, from its
> > upstream git repo.
> 
> Do you recall what was your LLVM version prior to this where it was
> working fine? (Wild guess from below would be the BPF inline asm

IIRC it was 4.0

> support that was added recently to LLVM (2865ab6996) vs asm() used
> in headers included in the stdin header causing trouble due to arch
> mixup?)
> 
> > root@jouet ~]# perf test -v LLVM
> > 37: LLVM search and compile   :
> > 37.1: Basic BPF llvm compile  :
> > --- start ---
> > test child forked, pid 5255
> > Kernel build dir is set to /lib/modules/4.14.0+/build
> > set env: KBUILD_DIR=/lib/modules/4.14.0+/build
> > unset env: KBUILD_OPTS
> > include option is set to  -nostdinc -isystem 
> > /usr/lib/gcc/x86_64-redhat-linux/7/include 
> > -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  
> > -I/home/acme/git/linux/include -I./include 
> > -I/home/acme/git/linux/arch/x86/include/uapi 
> > -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi 
> > -I./include/generated/uapi -include 
> > /home/acme/git/linux/include/linux/kconfig.h 
> > set env: NR_CPUS=4
> > set env: LINUX_VERSION_CODE=0x40e00
> > set env: CLANG_EXEC=/usr/local/bin/clang
> > set env: CLANG_OPTIONS=-xc 
> > set env: KERNEL_INC_OPTIONS= -nostdinc -isystem 
> > /usr/lib/gcc/x86_64-redhat-linux/7/include 
> > -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  
> > -I/home/acme/git/linux/include -I./include 
> > -I/home/acme/git/linux/arch/x86/include/uapi 
> > -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi 
> > -I./include/generated/uapi -include 
> > /home/acme/git/linux/include/linux/kconfig.h 
> > set env: WORKING_DIR=/lib/modules/4.14.0+/build
> > set env: CLANG_SOURCE=-
> > llvm compiling command template: echo '/*
> >  * bpf-script-example.c
> >  * Test basic LLVM building
> >  */
> > #ifndef LINUX_VERSION_CODE
> > # error Need LINUX_VERSION_CODE
> > # error Example: for 4.2 kernel, put 
> > 'clang-opt="-DLINUX_VERSION_CODE=0x40200" into llvm section of 
> > ~/.perfconfig'
> > #endif
> > #define BPF_ANY 0
> > #define BPF_MAP_TYPE_ARRAY 2
> > #define BPF_FUNC_map_lookup_elem 1
> > #define BPF_FUNC_map_update_elem 2
> > 
> > static void *(*bpf_map_lookup_elem)(void *map, void *key) =
> > (void *) BPF_FUNC_map_lookup_elem;
> > static void *(*bpf_map_update_elem)(void *map, void *key, void *value, int 
> > flags) =
> > (void *) BPF_FUNC_map_update_elem;
> > 
> > struct bpf_map_def {
> > unsigned int type;
> > unsigned int key_size;
> > unsigned int value_size;
> > unsigned int max_entries;
> > };
> > 
> > #define SEC(NAME) __attribute__((section(NAME), used))
> > struct bpf_map_def SEC("maps") flip_table = {
> > .type = BPF_MAP_TYPE_ARRAY,
> > .key_size = sizeof(int),
> > .value_size = sizeof(int),
> > .max_entries = 1,
> > };
> > 
> > SEC("func=SyS_epoll_wait")
> > int bpf_func__SyS_epoll_wait(void *ctx)
> > {
> > int ind =0;
> > int *flag = bpf_map_lookup_elem(_table, );
> > int new_flag;
> > if (!flag)
> > return 0;
> > /* flip flag and store back */
> > new_flag = !*flag;
> > bpf_map_update_elem(_table, , _flag, BPF_ANY);
> > return new_flag;
> > }
> > char _license[] SEC("license") = "GPL";
> > int _version SEC("version") = LINUX_VERSION_CODE;
> > ' | $CLANG_EXEC -D__KERNEL__ -D__NR_CPUS__=$NR_CPUS 
> > -DLINUX_VERSION_CODE=$LINUX_VERSION_CODE $CLANG_OPTIONS $KERNEL_INC_OPTIONS 
> > -Wno-unused-value -Wno-pointer-sign -working-directory $WORKING_DIR -c 
> > "$CLANG_SOURCE" -target bpf -O2 -o -
> > test child finished with 0
> >  end 
> > LLVM search and compile subtest 0: Ok
> > 37.2: kbuild searching:
> > --- start ---
> > test child forked, pid 5728
> > Kernel build dir is set to /lib/modules/4.14.0+/build
> > set env: KBUILD_DIR=/lib/modules/4.14.0+/build
> > unset env: KBUILD_OPTS
> > include option is set to  -nostdinc -isystem 
> > /usr/lib/gcc/x86_64-redhat-linux/7/include 
> > -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  
> > -I/home/acme/git/linux/include -I./include 
> > -I/home/acme/git/linux/arch/x86/include/uapi 
> > -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi 
> > -I./include/generated/uapi -include 
> > /home/acme/git/linux/include/linux/kconfig.h 
> > set env: NR_CPUS=4
> > set env: LINUX_VERSION_CODE=0x40e00
> > set env: CLANG_EXEC=/usr/local/bin/clang
> > set env: CLANG_OPTIONS=-xc 
> > set env: KERNEL_INC_OPTIONS= -nostdinc -isystem 
> > /usr/lib/gcc/x86_64-redhat-linux/7/include 
> > -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  
> > -I/home/acme/git/linux/include -I./include 
> >

Re: perf test LLVM & clang 6 failing

2017-11-24 Thread Daniel Borkmann

[ +Yonghong ]

On 11/24/2017 03:47 PM, Arnaldo Carvalho de Melo wrote:
> FYI, just noticed, recently updated clang to version 6, from its
> upstream git repo.

Do you recall what was your LLVM version prior to this where it was
working fine? (Wild guess from below would be the BPF inline asm
support that was added recently to LLVM (2865ab6996) vs asm() used
in headers included in the stdin header causing trouble due to arch
mixup?)

> root@jouet ~]# perf test -v LLVM
> 37: LLVM search and compile   :
> 37.1: Basic BPF llvm compile  :
> --- start ---
> test child forked, pid 5255
> Kernel build dir is set to /lib/modules/4.14.0+/build
> set env: KBUILD_DIR=/lib/modules/4.14.0+/build
> unset env: KBUILD_OPTS
> include option is set to  -nostdinc -isystem 
> /usr/lib/gcc/x86_64-redhat-linux/7/include 
> -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  
> -I/home/acme/git/linux/include -I./include 
> -I/home/acme/git/linux/arch/x86/include/uapi 
> -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi 
> -I./include/generated/uapi -include 
> /home/acme/git/linux/include/linux/kconfig.h 
> set env: NR_CPUS=4
> set env: LINUX_VERSION_CODE=0x40e00
> set env: CLANG_EXEC=/usr/local/bin/clang
> set env: CLANG_OPTIONS=-xc 
> set env: KERNEL_INC_OPTIONS= -nostdinc -isystem 
> /usr/lib/gcc/x86_64-redhat-linux/7/include 
> -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  
> -I/home/acme/git/linux/include -I./include 
> -I/home/acme/git/linux/arch/x86/include/uapi 
> -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi 
> -I./include/generated/uapi -include 
> /home/acme/git/linux/include/linux/kconfig.h 
> set env: WORKING_DIR=/lib/modules/4.14.0+/build
> set env: CLANG_SOURCE=-
> llvm compiling command template: echo '/*
>  * bpf-script-example.c
>  * Test basic LLVM building
>  */
> #ifndef LINUX_VERSION_CODE
> # error Need LINUX_VERSION_CODE
> # error Example: for 4.2 kernel, put 
> 'clang-opt="-DLINUX_VERSION_CODE=0x40200" into llvm section of ~/.perfconfig'
> #endif
> #define BPF_ANY 0
> #define BPF_MAP_TYPE_ARRAY 2
> #define BPF_FUNC_map_lookup_elem 1
> #define BPF_FUNC_map_update_elem 2
> 
> static void *(*bpf_map_lookup_elem)(void *map, void *key) =
>   (void *) BPF_FUNC_map_lookup_elem;
> static void *(*bpf_map_update_elem)(void *map, void *key, void *value, int 
> flags) =
>   (void *) BPF_FUNC_map_update_elem;
> 
> struct bpf_map_def {
>   unsigned int type;
>   unsigned int key_size;
>   unsigned int value_size;
>   unsigned int max_entries;
> };
> 
> #define SEC(NAME) __attribute__((section(NAME), used))
> struct bpf_map_def SEC("maps") flip_table = {
>   .type = BPF_MAP_TYPE_ARRAY,
>   .key_size = sizeof(int),
>   .value_size = sizeof(int),
>   .max_entries = 1,
> };
> 
> SEC("func=SyS_epoll_wait")
> int bpf_func__SyS_epoll_wait(void *ctx)
> {
>   int ind =0;
>   int *flag = bpf_map_lookup_elem(_table, );
>   int new_flag;
>   if (!flag)
>   return 0;
>   /* flip flag and store back */
>   new_flag = !*flag;
>   bpf_map_update_elem(_table, , _flag, BPF_ANY);
>   return new_flag;
> }
> char _license[] SEC("license") = "GPL";
> int _version SEC("version") = LINUX_VERSION_CODE;
> ' | $CLANG_EXEC -D__KERNEL__ -D__NR_CPUS__=$NR_CPUS 
> -DLINUX_VERSION_CODE=$LINUX_VERSION_CODE $CLANG_OPTIONS $KERNEL_INC_OPTIONS 
> -Wno-unused-value -Wno-pointer-sign -working-directory $WORKING_DIR -c 
> "$CLANG_SOURCE" -target bpf -O2 -o -
> test child finished with 0
>  end 
> LLVM search and compile subtest 0: Ok
> 37.2: kbuild searching:
> --- start ---
> test child forked, pid 5728
> Kernel build dir is set to /lib/modules/4.14.0+/build
> set env: KBUILD_DIR=/lib/modules/4.14.0+/build
> unset env: KBUILD_OPTS
> include option is set to  -nostdinc -isystem 
> /usr/lib/gcc/x86_64-redhat-linux/7/include 
> -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  
> -I/home/acme/git/linux/include -I./include 
> -I/home/acme/git/linux/arch/x86/include/uapi 
> -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi 
> -I./include/generated/uapi -include 
> /home/acme/git/linux/include/linux/kconfig.h 
> set env: NR_CPUS=4
> set env: LINUX_VERSION_CODE=0x40e00
> set env: CLANG_EXEC=/usr/local/bin/clang
> set env: CLANG_OPTIONS=-xc 
> set env: KERNEL_INC_OPTIONS= -nostdinc -isystem 
> /usr/lib/gcc/x86_64-redhat-linux/7/include 
> -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  
> -I/home/acme/git/linux/include -I./include 
> -I/home/acme/git/linux/arch/x86/include/uapi 
> -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi 
> -I./include/generated/uapi -include 
> /home/acme/git/linux/include/linux/kconfig.h 
> set env: WORKING_DIR=/lib/modules/4.14.0+/build
> set env: CLANG_SOURCE=-
> llvm

Re: perf test LLVM & clang 6 failing

2017-11-24 Thread Daniel Borkmann

[ +Yonghong ]

On 11/24/2017 03:47 PM, Arnaldo Carvalho de Melo wrote:
> FYI, just noticed, recently updated clang to version 6, from its
> upstream git repo.

Do you recall what was your LLVM version prior to this where it was
working fine? (Wild guess from below would be the BPF inline asm
support that was added recently to LLVM (2865ab6996) vs asm() used
in headers included in the stdin header causing trouble due to arch
mixup?)

> root@jouet ~]# perf test -v LLVM
> 37: LLVM search and compile   :
> 37.1: Basic BPF llvm compile  :
> --- start ---
> test child forked, pid 5255
> Kernel build dir is set to /lib/modules/4.14.0+/build
> set env: KBUILD_DIR=/lib/modules/4.14.0+/build
> unset env: KBUILD_OPTS
> include option is set to  -nostdinc -isystem 
> /usr/lib/gcc/x86_64-redhat-linux/7/include 
> -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  
> -I/home/acme/git/linux/include -I./include 
> -I/home/acme/git/linux/arch/x86/include/uapi 
> -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi 
> -I./include/generated/uapi -include 
> /home/acme/git/linux/include/linux/kconfig.h 
> set env: NR_CPUS=4
> set env: LINUX_VERSION_CODE=0x40e00
> set env: CLANG_EXEC=/usr/local/bin/clang
> set env: CLANG_OPTIONS=-xc 
> set env: KERNEL_INC_OPTIONS= -nostdinc -isystem 
> /usr/lib/gcc/x86_64-redhat-linux/7/include 
> -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  
> -I/home/acme/git/linux/include -I./include 
> -I/home/acme/git/linux/arch/x86/include/uapi 
> -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi 
> -I./include/generated/uapi -include 
> /home/acme/git/linux/include/linux/kconfig.h 
> set env: WORKING_DIR=/lib/modules/4.14.0+/build
> set env: CLANG_SOURCE=-
> llvm compiling command template: echo '/*
>  * bpf-script-example.c
>  * Test basic LLVM building
>  */
> #ifndef LINUX_VERSION_CODE
> # error Need LINUX_VERSION_CODE
> # error Example: for 4.2 kernel, put 
> 'clang-opt="-DLINUX_VERSION_CODE=0x40200" into llvm section of ~/.perfconfig'
> #endif
> #define BPF_ANY 0
> #define BPF_MAP_TYPE_ARRAY 2
> #define BPF_FUNC_map_lookup_elem 1
> #define BPF_FUNC_map_update_elem 2
> 
> static void *(*bpf_map_lookup_elem)(void *map, void *key) =
>   (void *) BPF_FUNC_map_lookup_elem;
> static void *(*bpf_map_update_elem)(void *map, void *key, void *value, int 
> flags) =
>   (void *) BPF_FUNC_map_update_elem;
> 
> struct bpf_map_def {
>   unsigned int type;
>   unsigned int key_size;
>   unsigned int value_size;
>   unsigned int max_entries;
> };
> 
> #define SEC(NAME) __attribute__((section(NAME), used))
> struct bpf_map_def SEC("maps") flip_table = {
>   .type = BPF_MAP_TYPE_ARRAY,
>   .key_size = sizeof(int),
>   .value_size = sizeof(int),
>   .max_entries = 1,
> };
> 
> SEC("func=SyS_epoll_wait")
> int bpf_func__SyS_epoll_wait(void *ctx)
> {
>   int ind =0;
>   int *flag = bpf_map_lookup_elem(_table, );
>   int new_flag;
>   if (!flag)
>   return 0;
>   /* flip flag and store back */
>   new_flag = !*flag;
>   bpf_map_update_elem(_table, , _flag, BPF_ANY);
>   return new_flag;
> }
> char _license[] SEC("license") = "GPL";
> int _version SEC("version") = LINUX_VERSION_CODE;
> ' | $CLANG_EXEC -D__KERNEL__ -D__NR_CPUS__=$NR_CPUS 
> -DLINUX_VERSION_CODE=$LINUX_VERSION_CODE $CLANG_OPTIONS $KERNEL_INC_OPTIONS 
> -Wno-unused-value -Wno-pointer-sign -working-directory $WORKING_DIR -c 
> "$CLANG_SOURCE" -target bpf -O2 -o -
> test child finished with 0
>  end 
> LLVM search and compile subtest 0: Ok
> 37.2: kbuild searching:
> --- start ---
> test child forked, pid 5728
> Kernel build dir is set to /lib/modules/4.14.0+/build
> set env: KBUILD_DIR=/lib/modules/4.14.0+/build
> unset env: KBUILD_OPTS
> include option is set to  -nostdinc -isystem 
> /usr/lib/gcc/x86_64-redhat-linux/7/include 
> -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  
> -I/home/acme/git/linux/include -I./include 
> -I/home/acme/git/linux/arch/x86/include/uapi 
> -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi 
> -I./include/generated/uapi -include 
> /home/acme/git/linux/include/linux/kconfig.h 
> set env: NR_CPUS=4
> set env: LINUX_VERSION_CODE=0x40e00
> set env: CLANG_EXEC=/usr/local/bin/clang
> set env: CLANG_OPTIONS=-xc 
> set env: KERNEL_INC_OPTIONS= -nostdinc -isystem 
> /usr/lib/gcc/x86_64-redhat-linux/7/include 
> -I/home/acme/git/linux/arch/x86/include -I./arch/x86/include/generated  
> -I/home/acme/git/linux/include -I./include 
> -I/home/acme/git/linux/arch/x86/include/uapi 
> -I./arch/x86/include/generated/uapi -I/home/acme/git/linux/include/uapi 
> -I./include/generated/uapi -include 
> /home/acme/git/linux/include/linux/kconfig.h 
> set env: WORKING_DIR=/lib/modules/4.14.0+/build
> set env: CLANG_SOURCE=-
> llvm

Re: perf test LLVM & clang 6 failing

Re: perf test LLVM & clang 6 failing

Re: perf test LLVM & clang 6 failing

Re: perf test LLVM & clang 6 failing

Re: perf test LLVM & clang 6 failing

Re: perf test LLVM & clang 6 failing

Re: perf test LLVM & clang 6 failing

Re: perf test LLVM & clang 6 failing

Re: perf test LLVM & clang 6 failing

Re: perf test LLVM & clang 6 failing

Re: perf test LLVM & clang 6 failing

Re: perf test LLVM & clang 6 failing

Re: perf test LLVM & clang 6 failing

Re: perf test LLVM & clang 6 failing

Re: perf test LLVM & clang 6 failing

Re: perf test LLVM & clang 6 failing

Re: perf test LLVM & clang 6 failing

Re: perf test LLVM & clang 6 failing

Re: perf test LLVM & clang 6 failing

Re: perf test LLVM & clang 6 failing

Re: perf test LLVM & clang 6 failing

Re: perf test LLVM & clang 6 failing

Re: perf test LLVM & clang 6 failing

Re: perf test LLVM & clang 6 failing

24 matches

Site Navigation

Mail list logo

Footer information