Bug#856474: stap: include runtime_defines.h not found

2017-03-06 Thread Jack Henschel 
On 03/04/2017 10:24 AM, Ritesh Raj Sarraf wrote:
> Thank you very much, Ben. I tested with your fix and it resolves the issue.

Can confirm, the patch fixes it.

Thank you Ben and Ritesh for your super quick actions! :-)

Greetings
Jack



signature.asc
Description: OpenPGP digital signature

Bug#856474: stap: include runtime_defines.h not found

2017-03-04 Thread Ritesh Raj Sarraf 
On Sat, 2017-03-04 at 02:12 +, Ben Hutchings wrote:
> On Sat, 2017-03-04 at 01:39 +, Ben Hutchings wrote:
> [...]
> > I investigated this and found that it occurs when the kernel source and
> > object trees are separate (an "out-of-tree" build, not to be confused
> > with out-of-tree modules).  We separate them in Debian kernel header
> > packages to avoid duplicating source files for each flavour.
> > 
> > When this is the case, the compiler is called in the root of the
> > object tree, and the kernel build system adjusts -I options in the
> > compiler flags to refer to subdirectories of the source tree if
> > necessary.  Any directory name beginning with /, ./ or ../ is excluded
> > from this adjustment.
> > 
> > systemtap uses -I"/usr/share/systemtap/runtime", which ought to be
> > excluded... but make has no understanding of shell quoting, so it is
> > wrongly adjusted to something like
> > -I/usr/src/linux-headers-4.9.0-2-common/"/usr/share/systemtap/runtime"
> > and runtime_defines.h cannot be found.
> 
> By the way, kbuild has been doing this ever since it started supporting
> out-of-tree builds (around Linux 2.6.0).  However, both the adjusted
> and original -I options were used.  Since Linux 4.8 only the adjusted
> option is used.
> 

Thank you very much, Ben. I tested with your fix and it resolves the issue.


-- 
Ritesh Raj Sarraf | http://people.debian.org/~rrs
Debian - The Universal Operating System

signature.asc
Description: This is a digitally signed message part

Bug#856474: stap: include runtime_defines.h not found

2017-03-03 Thread Jack Henschel 
On Sat, 04 Mar 2017 01:39:58 + Ben Hutchings  wrote:
> So my fix is to delete quotes before checking for the exclusions:
Wow Ben, that was a really quick fix!

I'll test it on Monday and report back.




signature.asc
Description: OpenPGP digital signature

Bug#856474: stap: include runtime_defines.h not found

2017-03-03 Thread Ben Hutchings 
On Sat, 2017-03-04 at 01:39 +, Ben Hutchings wrote:
[...]
> I investigated this and found that it occurs when the kernel source and
> object trees are separate (an "out-of-tree" build, not to be confused
> with out-of-tree modules).  We separate them in Debian kernel header
> packages to avoid duplicating source files for each flavour.
> 
> When this is the case, the compiler is called in the root of the
> object tree, and the kernel build system adjusts -I options in the
> compiler flags to refer to subdirectories of the source tree if
> necessary.  Any directory name beginning with /, ./ or ../ is excluded
> from this adjustment.
> 
> systemtap uses -I"/usr/share/systemtap/runtime", which ought to be
> excluded... but make has no understanding of shell quoting, so it is
> wrongly adjusted to something like
> -I/usr/src/linux-headers-4.9.0-2-common/"/usr/share/systemtap/runtime"
> and runtime_defines.h cannot be found.

By the way, kbuild has been doing this ever since it started supporting
out-of-tree builds (around Linux 2.6.0).  However, both the adjusted
and original -I options were used.  Since Linux 4.8 only the adjusted
option is used.

Ben.

-- 
Ben Hutchings
All the simple programs have been written, and all the good names
taken.


signature.asc
Description: This is a digitally signed message part

Bug#856474: stap: include runtime_defines.h not found

2017-03-03 Thread Ben Hutchings 
Control: tag -1 upstream patch

On Fri, 03 Mar 2017 23:10:56 +0530 Ritesh Raj Sarraf  wrote:
[...]
> Dear (Linux) Kernel Team:
> 
> We currently have systemtap 3.0 in Debian testing  where this problem was
> reported.
> 
> In Experimental, I have also pushed the 3.1 release. The 3.1 release is 
> reported
> to work perfectly in our tests with the custom built kernel. The problem is 
> only
> seen with the Debian built Linux kernels.
[...]

I investigated this and found that it occurs when the kernel source and
object trees are separate (an "out-of-tree" build, not to be confused
with out-of-tree modules).  We separate them in Debian kernel header
packages to avoid duplicating source files for each flavour.

When this is the case, the compiler is called in the root of the
object tree, and the kernel build system adjusts -I options in the
compiler flags to refer to subdirectories of the source tree if
necessary.  Any directory name beginning with /, ./ or ../ is excluded
from this adjustment.

systemtap uses -I"/usr/share/systemtap/runtime", which ought to be
excluded... but make has no understanding of shell quoting, so it is
wrongly adjusted to something like
-I/usr/src/linux-headers-4.9.0-2-common/"/usr/share/systemtap/runtime"
and runtime_defines.h cannot be found.

So my fix is to delete quotes before checking for the exclusions:

--- a/scripts/Kbuild.include
+++ b/scripts/Kbuild.include
@@ -206,7 +206,7 @@ hdr-inst := -f $(srctree)/scripts/Makefile.headersinst obj
 # Prefix -I with $(srctree) if it is not an absolute path.
 # skip if -I has no parameter
 addtree = $(if $(patsubst -I%,%,$(1)), \
-$(if $(filter-out -I/% -I./% -I../%,$(1)),$(patsubst 
-I%,-I$(srctree)/%,$(1)),$(1)))
+$(if $(filter-out -I/% -I./% -I../%,$(subst $(quote),,$(subst 
$(squote),,$(1,$(patsubst -I%,-I$(srctree)/%,$(1)),$(1)))
 
 # Find all -I options and call addtree
 flags = $(foreach o,$($(1)),$(if $(filter -I%,$(o)),$(call addtree,$(o)),$(o)))
--- END ---

Ben.

-- 
Ben Hutchings
All the simple programs have been written, and all the good names
taken.


signature.asc
Description: This is a digitally signed message part

Bug#856474: stap: include runtime_defines.h not found

2017-03-03 Thread Gerald Turner 
I modified systemtap buildrun.cxx so that the Makefile it produces
during Pass 3 doesn't have quotes around the include path, i.e.:

Before:

  EXTRA_CFLAGS += -I"/usr/share/systemtap/runtime"

After:

  EXTRA_CFLAGS += -I/usr/share/systemtap/runtime

Then stap works fine against the Debian kernel, running 4.10-1~exp1 at
the moment.

I cannot fathom how Debian's linux-headers packages would affect this
argument, but for the record, here's systemtap running gcc in Pass 3,
elided:

  gcc-6 ... 
-I/usr/src/linux-headers-4.10.0-trunk-common/"/usr/share/systemtap/runtime"   
...

And with buildrun.cxx modified so that quotations are removed from the
Makefile:

  gcc-6 ... -I/usr/share/systemtap/runtime ...

How is /usr/src/linux-headers-4.10.0-trunk-common/ getting interpolated
into that path argument?  Attached a Makefile produced by stap - maybe
somebody could guess why that interpolation occurs at line 125?

Apologies if I'm adding more noise than signal to the bug report.  Frank
Ch. Eigler already addressed the quotation issue earlier in this bug
report.

-- 
Gerald Turner Encrypted mail preferred!
OpenPGP: 4096R / CA89 B27A 30FA 66C5 1B80  3858 EC94 2276 FDB8 716D
--- systemtap-3.1.orig/buildrun.cxx
+++ systemtap-3.1/buildrun.cxx
@@ -495,7 +495,7 @@ compile_pass (systemtap_session& s)
   #if CHECK_POINTER_ARITH_PR5947
   o << "EXTRA_CFLAGS += -Wpointer-arith" << endl;
   #endif
-  o << "EXTRA_CFLAGS += -I\"" << s.runtime_path << "\"" << endl;
+  o << "EXTRA_CFLAGS += -I" << s.runtime_path << endl;
   // XXX: this may help ppc toc overflow
   // o << "CFLAGS := $(subst -Os,-O2,$(CFLAGS)) -fminimal-toc" << endl;
   o << "obj-m := " << s.module_name << ".o" << endl;
_KBUILD_CFLAGS := $(call flags,KBUILD_CFLAGS)
stap_check_gcc = $(shell set -x; if $(CC) $(1) -S -o /dev/null -xc /dev/null > 
/dev/null 2>&1; then echo "$(1)"; else echo "$(2)"; fi)
CHECK_BUILD := $(CC) $(NOSTDINC_FLAGS) $(KBUILD_CPPFLAGS) $(CPPFLAGS) 
$(LINUXINCLUDE) $(_KBUILD_CFLAGS) $(CFLAGS_KERNEL) $(EXTRA_CFLAGS) $(CFLAGS) 
-DKBUILD_BASENAME=\"stap_1008\" -Werror -S -o /dev/null -xc 
stap_check_build = $(shell set -x; if $(CHECK_BUILD) $(1) > /dev/null 2>&1 ; 
then echo "$(2)"; else echo "$(3)"; fi)
SYSTEMTAP_RUNTIME = "/usr/share/systemtap/runtime"
CONFIG_MODULE_SIG := n
EXTRA_CFLAGS :=
EXTRA_CFLAGS += -Iinclude2/asm/mach-default
EXTRA_CFLAGS += -I/lib/modules/4.10.0-trunk-amd64/build
STAPCONF_HEADER := 
/tmp/stapmZGYID/stapconf_0c48b85f89be3e4d6b5ccca7b814cb38_755.h
$(STAPCONF_HEADER):
@> $@
if $(CHECK_BUILD) $(SYSTEMTAP_RUNTIME)/linux/autoconf-hrtimer-rel.c > 
/dev/null 2>&1; then echo "#define STAPCONF_HRTIMER_REL 1"; fi >> $@
>> $@
if $(CHECK_BUILD) 
$(SYSTEMTAP_RUNTIME)/linux/autoconf-generated-compile.c > /dev/null 2>&1; then 
echo "#define STAPCONF_GENERATED_COMPILE 1"; fi >> $@
if $(CHECK_BUILD) 
$(SYSTEMTAP_RUNTIME)/linux/autoconf-hrtimer-getset-expires.c > /dev/null 2>&1; 
then echo "#define STAPCONF_HRTIMER_GETSET_EXPIRES 1"; fi >> $@
if $(CHECK_BUILD) $(SYSTEMTAP_RUNTIME)/linux/autoconf-inode-private.c > 
/dev/null 2>&1; then echo "#define STAPCONF_INODE_PRIVATE 1"; fi >> $@
if $(CHECK_BUILD) $(SYSTEMTAP_RUNTIME)/linux/autoconf-inode-rwsem.c > 
/dev/null 2>&1; then echo "#define STAPCONF_INODE_RWSEM 1"; fi >> $@
if $(CHECK_BUILD) $(SYSTEMTAP_RUNTIME)/linux/autoconf-constant-tsc.c > 
/dev/null 2>&1; then echo "#define STAPCONF_CONSTANT_TSC 1"; fi >> $@
if $(CHECK_BUILD) $(SYSTEMTAP_RUNTIME)/linux/autoconf-ktime-get-real.c 
> /dev/null 2>&1; then echo "#define STAPCONF_KTIME_GET_REAL 1"; fi >> $@
if $(CHECK_BUILD) $(SYSTEMTAP_RUNTIME)/linux/autoconf-x86-uniregs.c > 
/dev/null 2>&1; then echo "#define STAPCONF_X86_UNIREGS 1"; fi >> $@
if $(CHECK_BUILD) $(SYSTEMTAP_RUNTIME)/linux/autoconf-nameidata.c > 
/dev/null 2>&1; then echo "#define STAPCONF_NAMEIDATA_CLEANUP 1"; fi >> $@
echo "#define STAPCONF_UNREGISTER_KPROBES 1">> $@
if $(CHECK_BUILD) 
$(SYSTEMTAP_RUNTIME)/linux/autoconf-kprobe-symbol-name.c > /dev/null 2>&1; then 
echo "#define STAPCONF_KPROBE_SYMBOL_NAME 1"; fi >> $@
if $(CHECK_BUILD) $(SYSTEMTAP_RUNTIME)/linux/autoconf-real-parent.c > 
/dev/null 2>&1; then echo "#define STAPCONF_REAL_PARENT 1"; fi >> $@
if $(CHECK_BUILD) $(SYSTEMTAP_RUNTIME)/linux/autoconf-uaccess.c > 
/dev/null 2>&1; then echo "#define STAPCONF_LINUX_UACCESS_H 1"; fi >> $@
if $(CHECK_BUILD) $(SYSTEMTAP_RUNTIME)/linux/autoconf-oneachcpu-retry.c 
> /dev/null 2>&1; then echo "#define STAPCONF_ONEACHCPU_RETRY 1"; fi >> $@
if $(CHECK_BUILD) $(SYSTEMTAP_RUNTIME)/linux/autoconf-dpath-path.c > 
/dev/null 2>&1; then echo "#define STAPCONF_DPATH_PATH 1"; fi >> $@
>> $@
>> $@
echo "#define STAPCONF_SYNCHRONIZE_SCHED 1">> $@
if $(CHECK_BUILD) $(SYSTEMTAP_RUNTIME)/linux/autoconf-task-uid.c > 
/dev/null 2>&1; then echo "#define STAPCONF_TASK_UID 1"; fi >> $@
if

Bug#856474: stap: include runtime_defines.h not found

2017-03-03 Thread Ritesh Raj Sarraf 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Control: reassign -1 linux


Assigning it to the kernel team.

In this bug report, we have 2 confirmations of systemtap working fine with non-
Debian build Linux kernel (upstream kernel from Linus's tree built with the
bindeb-pkg make target)

systemtap relies on the running Linux kernel's header files to convert its
systemtap script into kernel object. With the Debian kernels, systemtap fails to
build the stap scripts.


Dear (Linux) Kernel Team:

We currently have systemtap 3.0 in Debian testing  where this problem was
reported.

In Experimental, I have also pushed the 3.1 release. The 3.1 release is reported
to work perfectly in our tests with the custom built kernel. The problem is only
seen with the Debian built Linux kernels.

At this time, we do not know what difference in the Debian kernel is causing
this failure. Since this bug makes systemtap of no use at all, the bug is marked
"grave".


All other details (and many verbose logs) are present in the full bug report.


On Thu, 2017-03-02 at 23:10 +0530, Ritesh Raj Sarraf wrote:
> Control: severity -1 grave
> Control: tag -1 +help
> 
> On Thu, 2017-03-02 at 14:37 +0530, Ritesh Raj Sarraf wrote:
> > 
> > > Also I can confirm this not an issue with the _packaging_ of systemtap in
> > > Debian,
> > > I downloaded and compiled the latest release tarball (3.1) directly from
> > > upstream and still got the same error.
> > 
> > Thanks for the information.
> > 
> 
> I just pushed 3.1 to Experimental, the reason being 3.0 is going to be useless
> for Debian Stretch. We need 3.1 to get it working proper with 4.9 kernel too.
> 
> This upload (3.1) still doesn't solve the runtime_defines.h issue with Debian
> kernels. But, at least, we know now that the issue is something specific to
> Debian kernels only. As is shown below, and also mentioned by Frank.
> 
> Given that this bug makes systemtap useless, I have marked the bug accordingly
> to grave. I have also marked this for help because I don't really have that
> much
> "free" time to investigate it right now.
> 
> @Jack: I would like to reassign this bug to the kernel team. But it'd be nice
> if
> I could get more users confirm that the bug doesn't appear with non-Debian
> kernels. Would you be in a position to pick systemtap from Experimental, and
> build a pristine upstream kernel image (you need to clone Linus's tree and
> just
> run the bindeb-pkg make target) and test the combination together ?
> 
> 
> Here are my results.
> 
> rrs@learner:~$ stap -v -e 'probe vfs.read {printf("read performed\n");
> exit()}'
> Pass 1: parsed user script and 470 library scripts using
> 123812virt/48564res/6640shr/42124data kb, in 110usr/10sys/129real ms.
> Pass 2: analyzed script: 1 probe, 1 function, 7 embeds, 0 globals using
> 280416virt/206592res/7948shr/198728data kb, in 1400usr/190sys/1594real ms.
> Pass 3: using cached
> /home/rrs/.systemtap/cache/f8/stap_f83081804fadc6e30161178b5574195d_2502.c
> Pass 4: using cached
> /home/rrs/.systemtap/cache/f8/stap_f83081804fadc6e30161178b5574195d_2502.ko
> Pass 5: starting run.
> read performed
> Pass 5: run completed in 10usr/30sys/530real ms.
> 2017-03-02 / 22:58:35 ♒♒♒  ☺  
> 
> rrs@learner:~$ uname -a
> Linux learner 4.10.1+ #20 SMP Sun Feb 26 21:22:45 IST 2017 x86_64 GNU/Linux
> 2017-03-02 / 22:58:39 ♒♒♒  ☺  
> 
> vs
> 
> rrs@chutzpah:~/Community/Packaging/systemtap (master)$ stap -ve 'probe begin {
> log("hello world") exit () }'
> Pass 1: parsed user script and 471 library scripts using
> 123528virt/48648res/6764shr/41840data kb, in 140usr/0sys/149real ms.
> Pass 2: analyzed script: 1 probe, 2 functions, 0 embeds, 0 globals using
> 124452virt/49620res/6956shr/42764data kb, in 10usr/0sys/5real ms.
> Pass 3: translated to C into
> "/tmp/stapJNAO0g/stap_eed6170377f19ad002c0d2b53e5e7956_1067_src.c" using
> 124452virt/49620res/6956shr/42764data kb, in 0usr/0sys/1real ms.
> /tmp/stapJNAO0g/stap_eed6170377f19ad002c0d2b53e5e7956_1067_src.c:10:29: fatal
> error: runtime_defines.h: No such file or directory
>  #include "runtime_defines.h"
>  ^
> compilation terminated.
> /usr/src/linux-headers-4.9.0-2-common/scripts/Makefile.build:298: recipe for
> target '/tmp/stapJNAO0g/stap_eed6170377f19ad002c0d2b53e5e7956_1067_src.o'
> failed
> make[3]: ***
> [/tmp/stapJNAO0g/stap_eed6170377f19ad002c0d2b53e5e7956_1067_src.o] Error 1
> make[3]: *** Waiting for unfinished jobs
> /usr/src/linux-headers-4.9.0-2-common/Makefile:1507: recipe for target
> '_module_/tmp/stapJNAO0g' failed
> make[2]: *** [_module_/tmp/stapJNAO0g] Error 2
> Makefile:150: recipe for target 'sub-make' failed
> make[1]: *** [sub-make] Error 2
> Makefile:8: recipe for target 'all' failed
> make: *** [all] Error 2
> WARNING: kbuild exited with status: 2
> Pass 4: compiled C into "stap_eed6170377f19ad002c0d2b53e5e7956_1067.ko" in
> 7810usr/760sys/9195real ms.
> Pass 4: compilation failed.  [man error::pass4]
> Tip:

Bug#856474: stap: include runtime_defines.h not found

2017-03-03 Thread Jack Henschel 
Can confirm, the stable kernel (compiled directly from kernel.org) works just 
fine (along with the included packaging) with SystemTap 3.1 from Debian 
experimental:

$ stap -V
Systemtap translator/driver (version 3.1/0.168, Debian version 3.1-1 
(experimental))
Copyright (C) 2005-2017 Red Hat, Inc. and others
This is free software; see the source for copying conditions.
tested kernel versions: 2.6.18 ... 4.10-rc8
enabled features: AVAHI LIBSQLITE3 NLS NSS
$ uname -a
Linux vm000949 4.10.1 #2 SMP Fri Mar 3 15:00:38 CET 2017 x86_64 GNU/Linux
$ stap -v -e 'probe vfs.read { printf("read performed\n"); exit() }'
Pass 1: parsed user script and 465 library scripts using 
122332virt/46968res/6636shr/40644data kb, in 170usr/20sys/191real ms.
Pass 2: analyzed script: 1 probe, 1 function, 7 embeds, 0 globals using 
281772virt/207800res/7924shr/200084data kb, in 1860usr/330sys/2217real ms.
Pass 3: using cached 
/home/jack/.systemtap/cache/28/stap_284d447b34881d240eeab3f29fc1d9dc_2511.c
Pass 4: using cached 
/home/jack/.systemtap/cache/28/stap_284d447b34881d240eeab3f29fc1d9dc_2511.ko
Pass 5: starting run.
read performed
Pass 5: run completed in 10usr/0sys/430real ms.

Bug#856474: stap: include runtime_defines.h not found

2017-03-03 Thread Frank Ch. Eigler 
Hi -

On Thu, Mar 02, 2017 at 01:25:03PM -0800, Gerald Turner wrote:
> FWIW, I attempted to kludge around the mangled include argument by
> running stap with "-B 'EXTRA_CFLAGS += -I/usr/share/systemtap/runtime'".
> This seems to work around the compilation failure of missing
> runtime_defines.h in include path, however compiliation then fails due
> to what appears to be API changes in Linux kernel 4.9.

Those errors all have systemtap-side porting logic associated with
them, a combination of compile-time (c #ifdefs) and build-time (kbuild
flags) conditionals.  I suspect your workaround hid one problem and
created these others.  We need to keep looking for the cause.

- FChE

Bug#856474: stap: include runtime_defines.h not found

2017-03-03 Thread Ritesh Raj Sarraf 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Hello Gerald,


On Thu, 2017-03-02 at 13:25 -0800, Gerald Turner wrote:
> FWIW, I attempted to kludge around the mangled include argument by
> running stap with "-B 'EXTRA_CFLAGS += -I/usr/share/systemtap/runtime'".
> This seems to work around the compilation failure of missing
> runtime_defines.h in include path, however compiliation then fails due
> to what appears to be API changes in Linux kernel 4.9.
> 
> For example:
> 
>   In file included from /usr/share/systemtap/runtime/linux/runtime.h:209:0,
>    from /usr/share/systemtap/runtime/runtime.h:26,
>    from
> /tmp/stapqySw7o/stap_6ed5af627765aaab098e97da6b2931f0_2608_src.c:25:
>   /usr/share/systemtap/runtime/linux/access_process_vm.h:50:13: error: too
> many arguments to function ‘get_user_pages’
>  ret = get_user_pages (tsk, mm, addr, 1, write, 1, , );
>    ^~
> 
> The prototype changed in Linux 4.9:
> 
>    long get_user_pages(unsigned long start, unsigned long nr_pages,
>    int write, int force, struct page **pages,
>    struct vm_area_struct **vmas);
> 
> It looks like systemtap 3.1 is expecting the prototype in Linux versions
> prior to 4.6:
> 

This is very weird. I was so hoping that 3.1 would fix the interface change
related build failures. The 3.1 stap release worked fine for me locally on
custom build 4.10 kernel.

The 3.1 release notes also claim successful tests with 4.8 and 4.10 kernels.


>    long get_user_pages(struct task_struct *tsk, struct mm_struct *mm,
>    unsigned long start, unsigned long nr_pages,
>    int write, int force, struct page **pages,
>    struct vm_area_struct **vmas);
> 
> Log of systemtap 3.1 attached.
- -- 
Ritesh Raj Sarraf | http://people.debian.org/~rrs
Debian - The Universal Operating System
-BEGIN PGP SIGNATURE-

iQIzBAEBCgAdFiEEQCVDstmIVAB/Yn02pjpYo/LhdWkFAli5VkMACgkQpjpYo/Lh
dWlSOBAAkjOB4cNsHuaoK2qo0SOv67/XEsuxeN9jLHo3rL/Q0AwdiLZAWAzLUxLZ
CXsrj0nosYin+c/MVBE4BrNv/UsbKl8telKr4rofcCO8AVftuI5Mn2sJsZWm2+AH
VELAQSKCms/9xY94lvaWV+YPIjVWh2oSTZBbBuosVHzq8g8z1SAFo9d6EkQP301z
CA9ZUgYdiBCgujjY6/FjxLhWrbuPz7eq8Xy7zoU8MyAi1igtm8bO/Qykwq01m5A8
X+1gOP0BIjx59i48WhX1z91EZk25oBDP9cuU4qdp/Cs7hhusV29sFP0jrJC3mLCZ
+cN3pijxuzxBZxv7kzYdmp7Q1YfAq/wkwB8V2DEG71jdh4y4CWMSrRNm+NmNftYW
xYTeJ3+01R6hDqmdjXCC3mtpu4ZAQ2Z2411WCceDiiYCswdF1bXTq3A6A3Mr6itw
aPROmV1yz5AMbAWiY4KPsOEWB3TyaVfzbgBjSOh11NiFWU6tEGhM8vgrOrAzAjek
WvmowAC5g308nSJBgATc/XxDcjBVfC9YoaRvvw74yKsP/y8GK2g4pvc35A7UowrQ
FFVyeQxSRuGHMSU3WMfxfK3ii/zQRJG0m9C+OvqOioQ7/ygWD/AbDsPLAyqoBz2F
L62cliiJ14zpF9vO5XuUt5/juILwR4++3G9qu5Xlt5kJgQHLCOY=
=0q9Z
-END PGP SIGNATURE-

Bug#856474: stap: include runtime_defines.h not found

2017-03-02 Thread Gerald Turner 
FWIW, I attempted to kludge around the mangled include argument by
running stap with "-B 'EXTRA_CFLAGS += -I/usr/share/systemtap/runtime'".
This seems to work around the compilation failure of missing
runtime_defines.h in include path, however compiliation then fails due
to what appears to be API changes in Linux kernel 4.9.

For example:

  In file included from /usr/share/systemtap/runtime/linux/runtime.h:209:0,
   from /usr/share/systemtap/runtime/runtime.h:26,
   from 
/tmp/stapqySw7o/stap_6ed5af627765aaab098e97da6b2931f0_2608_src.c:25:
  /usr/share/systemtap/runtime/linux/access_process_vm.h:50:13: error: too many 
arguments to function ‘get_user_pages’
 ret = get_user_pages (tsk, mm, addr, 1, write, 1, , );
   ^~

The prototype changed in Linux 4.9:

   long get_user_pages(unsigned long start, unsigned long nr_pages,
   int write, int force, struct page **pages,
   struct vm_area_struct **vmas);

It looks like systemtap 3.1 is expecting the prototype in Linux versions
prior to 4.6:

   long get_user_pages(struct task_struct *tsk, struct mm_struct *mm,
   unsigned long start, unsigned long nr_pages,
   int write, int force, struct page **pages,
   struct vm_area_struct **vmas);

Log of systemtap 3.1 attached.

-- 
Gerald Turner Encrypted mail preferred!
OpenPGP: 4096R / CA89 B27A 30FA 66C5 1B80  3858 EC94 2276 FDB8 716D
# stap -v -B 'EXTRA_CFLAGS += -I/usr/share/systemtap/runtime' -e 'probe 
vfs.read {printf("read performed\n"); exit()}'

Pass 1: parsed user script and 465 library scripts using 
113944virt/46692res/6452shr/40600data kb, in 110usr/20sys/132real ms.
WARNING: cannot find module 
/root/.systemtap/cache/a5/typequery_a5e864b17cd7d295a3825fb1ec09b900_785.ko 
debuginfo: No DWARF information found [man warning::debuginfo]
WARNING: cannot find module 
/root/.systemtap/cache/a5/typequery_a5e864b17cd7d295a3825fb1ec09b900_785.ko 
debuginfo: No DWARF information found [man warning::debuginfo]
WARNING: cannot find module 
/root/.systemtap/cache/a5/typequery_a5e864b17cd7d295a3825fb1ec09b900_785.ko 
debuginfo: No DWARF information found [man warning::debuginfo]
WARNING: cannot find module 
/root/.systemtap/cache/a5/typequery_a5e864b17cd7d295a3825fb1ec09b900_785.ko 
debuginfo: No DWARF information found [man warning::debuginfo]
WARNING: cannot find module 
/root/.systemtap/cache/a5/typequery_a5e864b17cd7d295a3825fb1ec09b900_785.ko 
debuginfo: No DWARF information found [man warning::debuginfo]
WARNING: cannot find module 
/root/.systemtap/cache/a5/typequery_a5e864b17cd7d295a3825fb1ec09b900_785.ko 
debuginfo: No DWARF information found [man warning::debuginfo]
WARNING: cannot find module 
/root/.systemtap/cache/a5/typequery_a5e864b17cd7d295a3825fb1ec09b900_785.ko 
debuginfo: No DWARF information found [man warning::debuginfo]
WARNING: cannot find module 
/root/.systemtap/cache/a5/typequery_a5e864b17cd7d295a3825fb1ec09b900_785.ko 
debuginfo: No DWARF information found [man warning::debuginfo]
WARNING: cannot find module 
/root/.systemtap/cache/a5/typequery_a5e864b17cd7d295a3825fb1ec09b900_785.ko 
debuginfo: No DWARF information found [man warning::debuginfo]
WARNING: cannot find module 
/root/.systemtap/cache/a5/typequery_a5e864b17cd7d295a3825fb1ec09b900_785.ko 
debuginfo: No DWARF information found [man warning::debuginfo]
WARNING: cannot find module 
/root/.systemtap/cache/a5/typequery_a5e864b17cd7d295a3825fb1ec09b900_785.ko 
debuginfo: No DWARF information found [man warning::debuginfo]
WARNING: cannot find module 
/root/.systemtap/cache/a5/typequery_a5e864b17cd7d295a3825fb1ec09b900_785.ko 
debuginfo: No DWARF information found [man warning::debuginfo]
WARNING: cannot find module 
/root/.systemtap/cache/a5/typequery_a5e864b17cd7d295a3825fb1ec09b900_785.ko 
debuginfo: No DWARF information found [man warning::debuginfo]
WARNING: cannot find module 
/root/.systemtap/cache/a5/typequery_a5e864b17cd7d295a3825fb1ec09b900_785.ko 
debuginfo: No DWARF information found [man warning::debuginfo]
WARNING: cannot find module 
/root/.systemtap/cache/a5/typequery_a5e864b17cd7d295a3825fb1ec09b900_785.ko 
debuginfo: No DWARF information found [man warning::debuginfo]
WARNING: cannot find module 
/root/.systemtap/cache/a5/typequery_a5e864b17cd7d295a3825fb1ec09b900_785.ko 
debuginfo: No DWARF information found [man warning::debuginfo]
WARNING: cannot find module 
/root/.systemtap/cache/a5/typequery_a5e864b17cd7d295a3825fb1ec09b900_785.ko 
debuginfo: No DWARF information found [man warning::debuginfo]
WARNING: cannot find module 
/root/.systemtap/cache/a5/typequery_a5e864b17cd7d295a3825fb1ec09b900_785.ko 
debuginfo: No DWARF information found [man warning::debuginfo]
WARNING: cannot find module 
/root/.systemtap/cache/a5/typequery_a5e864b17cd7d295a3825fb1ec09b900_785.ko 
debuginfo: No DWARF information found [man

Bug#856474: stap: include runtime_defines.h not found

2017-03-02 Thread Jack Henschel 


On 03/02/2017 06:40 PM, Ritesh Raj Sarraf wrote:
> I just pushed 3.1 to Experimental, the reason being 3.0 is going to be useless
> for Debian Stretch. We need 3.1 to get it working proper with 4.9 kernel too.
Did I get that right: you are planning to ship SystemTap 3.1 with Stretch?
(If so: great, I'm all for it!)

> @Jack: I would like to reassign this bug to the kernel team. But it'd be nice 
> if
> I could get more users confirm that the bug doesn't appear with non-Debian
> kernels. Would you be in a position to pick systemtap from Experimental, and
> build a pristine upstream kernel image (you need to clone Linus's tree and 
> just
> run the bindeb-pkg make target) and test the combination together ?
Sure, I always wanted to build my own kernel, I guess I finally have a good 
reason to do so :-)



signature.asc
Description: OpenPGP digital signature

Bug#856474: stap: include runtime_defines.h not found

2017-03-02 Thread Ritesh Raj Sarraf 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Control: severity -1 grave
Control: tag -1 +help

On Thu, 2017-03-02 at 14:37 +0530, Ritesh Raj Sarraf wrote:
> 
> > Also I can confirm this not an issue with the _packaging_ of systemtap in
> > Debian,
> > I downloaded and compiled the latest release tarball (3.1) directly from
> > upstream and still got the same error.
> 
> Thanks for the information.
> 

I just pushed 3.1 to Experimental, the reason being 3.0 is going to be useless
for Debian Stretch. We need 3.1 to get it working proper with 4.9 kernel too.

This upload (3.1) still doesn't solve the runtime_defines.h issue with Debian
kernels. But, at least, we know now that the issue is something specific to
Debian kernels only. As is shown below, and also mentioned by Frank.

Given that this bug makes systemtap useless, I have marked the bug accordingly
to grave. I have also marked this for help because I don't really have that much
"free" time to investigate it right now.

@Jack: I would like to reassign this bug to the kernel team. But it'd be nice if
I could get more users confirm that the bug doesn't appear with non-Debian
kernels. Would you be in a position to pick systemtap from Experimental, and
build a pristine upstream kernel image (you need to clone Linus's tree and just
run the bindeb-pkg make target) and test the combination together ?


Here are my results.

rrs@learner:~$ stap -v -e 'probe vfs.read {printf("read performed\n"); exit()}'
Pass 1: parsed user script and 470 library scripts using
123812virt/48564res/6640shr/42124data kb, in 110usr/10sys/129real ms.
Pass 2: analyzed script: 1 probe, 1 function, 7 embeds, 0 globals using
280416virt/206592res/7948shr/198728data kb, in 1400usr/190sys/1594real ms.
Pass 3: using cached
/home/rrs/.systemtap/cache/f8/stap_f83081804fadc6e30161178b5574195d_2502.c
Pass 4: using cached
/home/rrs/.systemtap/cache/f8/stap_f83081804fadc6e30161178b5574195d_2502.ko
Pass 5: starting run.
read performed
Pass 5: run completed in 10usr/30sys/530real ms.
2017-03-02 / 22:58:35 ♒♒♒  ☺  

rrs@learner:~$ uname -a
Linux learner 4.10.1+ #20 SMP Sun Feb 26 21:22:45 IST 2017 x86_64 GNU/Linux
2017-03-02 / 22:58:39 ♒♒♒  ☺  

vs

rrs@chutzpah:~/Community/Packaging/systemtap (master)$ stap -ve 'probe begin { 
log("hello world") exit () }'
Pass 1: parsed user script and 471 library scripts using 
123528virt/48648res/6764shr/41840data kb, in 140usr/0sys/149real ms.
Pass 2: analyzed script: 1 probe, 2 functions, 0 embeds, 0 globals using 
124452virt/49620res/6956shr/42764data kb, in 10usr/0sys/5real ms.
Pass 3: translated to C into 
"/tmp/stapJNAO0g/stap_eed6170377f19ad002c0d2b53e5e7956_1067_src.c" using 
124452virt/49620res/6956shr/42764data kb, in 0usr/0sys/1real ms.
/tmp/stapJNAO0g/stap_eed6170377f19ad002c0d2b53e5e7956_1067_src.c:10:29: fatal 
error: runtime_defines.h: No such file or directory
 #include "runtime_defines.h"
 ^
compilation terminated.
/usr/src/linux-headers-4.9.0-2-common/scripts/Makefile.build:298: recipe for 
target '/tmp/stapJNAO0g/stap_eed6170377f19ad002c0d2b53e5e7956_1067_src.o' failed
make[3]: *** [/tmp/stapJNAO0g/stap_eed6170377f19ad002c0d2b53e5e7956_1067_src.o] 
Error 1
make[3]: *** Waiting for unfinished jobs
/usr/src/linux-headers-4.9.0-2-common/Makefile:1507: recipe for target 
'_module_/tmp/stapJNAO0g' failed
make[2]: *** [_module_/tmp/stapJNAO0g] Error 2
Makefile:150: recipe for target 'sub-make' failed
make[1]: *** [sub-make] Error 2
Makefile:8: recipe for target 'all' failed
make: *** [all] Error 2
WARNING: kbuild exited with status: 2
Pass 4: compiled C into "stap_eed6170377f19ad002c0d2b53e5e7956_1067.ko" in 
7810usr/760sys/9195real ms.
Pass 4: compilation failed.  [man error::pass4]
Tip: /usr/share/doc/systemtap/README.Debian should help you get started.
23:00 ♒♒♒☹  => 1  

rrs@chutzpah:~/Community/Packaging/systemtap (master)$ uname -a
Linux chutzpah 4.9.0-2-amd64 #1 SMP Debian 4.9.13-1 (2017-02-27) x86_64 
GNU/Linux
23:00 ♒♒♒   ☺

- -- 
Ritesh Raj Sarraf | http://people.debian.org/~rrs
Debian - The Universal Operating System
-BEGIN PGP SIGNATURE-

iQIzBAEBCgAdFiEEQCVDstmIVAB/Yn02pjpYo/LhdWkFAli4WRoACgkQpjpYo/Lh
dWmNDw/+N08GXR4COXzZh+PU57mp1uHuK/8x4Atj3xGOrfvpbSZvo+uH0fVi9sRx
Nu7TK5z5EJ45RrdMwI2evrP9lx+wpUUDGlFUTLzh+J1GoN3Ea/T+TsWc/3CvXB8L
BKSfxI5FNQNM3Goq4ouTQUsSnb1YRoBN6vMSDGKS+5d6/n/0N6GoEk4ZLY4xOsrv
p8XnIB6RxiS3oT2hmjTewdbxd1l8gpyQv5Nhgg8y7tL64oo5Cd2+VSAO+YKXI2JN
LsNwgPpllMXW7LkhKDrYpmyStFmlNViLCrSiWiKZyMajRuCieV+xsdl8oVpxybOd
X9ZkGTzT6/4dLQ6P+2yfEj5hJatrbWXLQiysu6Npej+durQX+n5FBuIAU0/fdOzY
3eCh2nT63inwr3GLfZPccylggRBnC5lZV0p4WAL733kiIfaRfA2biN7c92hm1PdA
zQOazT2i6atfA/wjtob/whVa/R/CDnuiztG3Le7NG3qv5oYKStTqF2S4CJPzszta
ftpzw3PxzrFrOoiWCvpPY5rVvCx5hnsOfce64uVe0+tjK9ikepnW0pe28HihsDAT
Eh0g3fLXfnS8NkpV/ZWptVZ5iT3rYR4lfhUmbrEgqtk78AKzi8FTXdtwhaSFE9mH
WbhK+v2om8TfJJUCE8U2cU4B55vXSv4nR+bhDW+6qrVojwcDdwg=
=lo3e
-END PGP SIGNATURE-

Bug#856474: stap: include runtime_defines.h not found

2017-03-02 Thread Ritesh Raj Sarraf 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On Wed, 2017-03-01 at 19:48 +0100, Jack Henschel wrote:
> I mentioned that in one of my earlier mails (#22).

Sorry about that. I completely missed that one.

There's also another odd thing I noticed.

```
- -I/usr/src/linux-headers-4.9.0-1-common//tmp/stapcukYUi -I/tmp/stapcukYUi
```


> Also I can confirm this not an issue with the _packaging_ of systemtap in
> Debian,
> I downloaded and compiled the latest release tarball (3.1) directly from
> upstream and still got the same error.

Thanks for the information.

- -- 
Ritesh Raj Sarraf | http://people.debian.org/~rrs
Debian - The Universal Operating System
-BEGIN PGP SIGNATURE-

iQIzBAEBCgAdFiEEQCVDstmIVAB/Yn02pjpYo/LhdWkFAli34MYACgkQpjpYo/Lh
dWkwqg/+K+uzl+0XK+NBHtoPDXFtz1yjxz6Nc0IrrqP62ObeikX9EX5uTWcKnn/L
vx6a/jWBlTMnY6yzNGyF1UXPlzYs1NFo6X8seGN1FGs1KylWTOLyAl1z/ZUQdY87
og3TAtWZzLgG8G5wxmhqsrvY98FTAyWOZahZW3CGPg4DwNXjcTpAZZgf6+X4jcnv
d72evuQKbo1NlqWWClWaRsFDxfT7rQdlKxjgZpcyKtgtxtEaJMh6AsQNY2NRFVDw
zgtztkQeurOnVf3JknIflXrNrw4S4jRC/Rxt6cb6a4UWT9FWv+NmOKNLktsv7cm/
Qqr/B1FiW1WTLW9yMNNRsJ73MG9WcESTqRdskzfSVyfwdfrfkq0PlugROPsGQs68
aSVHUF4f4haQxtmzHZg5MYTfnK/fPNFhGf335LFjJdYUyT35yfwJkCuM1EOp6Sif
4AnxhK81p3p5wU8JwhseOgV7dO4OoQ4DCB9XJA9InQkg/WiVct+B9hAusBUGqSV0
s3juvdO095qyW6X8/SofhG8MfCNsHHXI9gUmHXMwmrowFXmL+NmBoiIVagAtlKIO
iSg6YnSBJElbg+9/37+WZjpGLhS+Bg4/mt7w5l0k6yzfS8RbJXXtLChkikZGxZc7
YS5jl4VDaXQnDOt3cTj74GJxJ0mx77HjX+RtaEba+rpdihmMH1M=
=9pKd
-END PGP SIGNATURE-

Bug#856474: stap: include runtime_defines.h not found

2017-03-01 Thread Jack Henschel 
On 03/01/2017 06:44 PM, Ritesh Raj Sarraf wrote:
> Thanks to hints from Frank, I can see the culprit now.
> 
> -Werror  
> -I/usr/src/linux-headers-4.9.0-2-common/"/usr/share/systemtap/runtime"  
> -DMODULE
> 
> 
> The full compile command is:
> 
>gcc-6 
> -Wp,-MD,/tmp/staptZYmtX/.stap_7ef754ea40ea0990a9bfaf74753e8a9d_1538_aux_0.o.d 
>  -nostdinc -isystem /usr/lib/gcc/x86_64-linux-gnu/6/include 
> -I/usr/src/linux-headers-4.9.0-2-common/arch/x86/include 
> -I./arch/x86/include/generated/uapi -I./arch/x86/include/generated  
> -I/usr/src/linux-headers-4.9.0-2-common/include -I./include 
> -I/usr/src/linux-headers-4.9.0-2-common/arch/x86/include/uapi 
> -I/usr/src/linux-headers-4.9.0-2-common/include/uapi 
> -I./include/generated/uapi -include 
> /usr/src/linux-headers-4.9.0-2-common/include/linux/kconfig.h 
> -I/usr/src/linux-headers-4.9.0-2-common//tmp/staptZYmtX -I/tmp/staptZYmtX 
> -D__KERNEL__ -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs 
> -fno-strict-aliasing -fno-common -Werror-implicit-function-declaration 
> -Wno-format-security -std=gnu89 -fno-PIE -mno-sse -mno-mmx -mno-sse2 
> -mno-3dnow -mno-avx -m64 -falign-jumps=1 -falign-loops=1 -mno-80387 
> -mno-fp-ret-in-387 -mpreferred-stack-boundary=3 -mskip-rax-setup 
> -mtune=generic -mno-red-zone -mcmodel=kernel -funit-at-a-time 
> -maccumulate-outgoing-args -DCONFIG_X86_X32_ABI -DCONFIG_AS_CFI=1 
> -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -DCONFIG_AS_CFI_SECTIONS=1 
> -DCONFIG_AS_FXSAVEQ=1 -DCONFIG_AS_SSSE3=1 -DCONFIG_AS_CRC32=1 
> -DCONFIG_AS_AVX=1 -DCONFIG_AS_AVX2=1 -DCONFIG_AS_AVX512=1 
> -DCONFIG_AS_SHA1_NI=1 -DCONFIG_AS_SHA256_NI=1 -pipe -Wno-sign-compare 
> -fno-asynchronous-unwind-tables -fno-delete-null-pointer-checks 
> -Wno-frame-address -O2 --param=allow-store-data-races=0 
> -Wframe-larger-than=2048 -fstack-protector-strong 
> -Wno-unused-but-set-variable -Wno-unused-const-variable 
> -fno-var-tracking-assignments -pg -mfentry -DCC_USING_FENTRY 
> -Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow 
> -fconserve-stack -Werror=implicit-int -Werror=strict-prototypes 
> -Werror=date-time -Werror=incompatible-pointer-types -DCC_HAVE_ASM_GOTO  
> -I/usr/src/linux-headers-4.9.0-2-common/include2/asm/mach-default  
> -I/lib/modules/4.9.0-2-amd64/build -include 
> /tmp/staptZYmtX/stapconf_d8a215ec355839da11c41ac20d4e7e1e_702.h 
> -freorder-blocks -fasynchronous-unwind-tables -Wframe-larger-than=512 
> -fno-ipa-icf -Wno-unused -Werror  
> -I/usr/src/linux-headers-4.9.0-2-common/"/usr/share/systemtap/runtime"  
> -DMODULE  
> -DKBUILD_BASENAME='"stap_7ef754ea40ea0990a9bfaf74753e8a9d_1538_aux_0"'  
> -DKBUILD_MODNAME='"stap_7ef754ea40ea0990a9bfaf74753e8a9d_1538"' -c -o 
> /tmp/staptZYmtX/.tmp_stap_7ef754ea40ea0990a9bfaf74753e8a9d_1538_aux_0.o 
> /tmp/staptZYmtX/stap_7ef754ea40ea0990a9bfaf74753e8a9d_1538_aux_0.c
> /tmp/staptZYmtX/stap_7ef754ea40ea0990a9bfaf74753e8a9d_1538_src.c:10:29: fatal 
> error: runtime_defines.h: No such file or directory
> 
I mentioned that in one of my earlier mails (#22).
Also I can confirm this not an issue with the _packaging_ of systemtap in 
Debian,
I downloaded and compiled the latest release tarball (3.1) directly from 
upstream and still got the same error.



signature.asc
Description: OpenPGP digital signature

Bug#856474: stap: include runtime_defines.h not found

2017-03-01 Thread Ritesh Raj Sarraf 
On Wed, 2017-03-01 at 12:30 -0500, Frank Ch. Eigler wrote:
> Hi -
> 
> > [...]  Running in verbose mode, it shows: [...]  Those paths are
> > provided by the linux-headers-xxx-amd64 (or equivalent arch)
> > package. In Debian, build/ and source/ paths are pointing to
> > separate locations, provided by separate package. [...]
> 
> Could you provide a make V=1 transcript of any nontrivial out-of-tree
> module (plus its Makefile) that builds & works on your kernel?

Here's a log attached. It was generated with V=1 through dkms on out-of-tree
module for broadcom-sta driver.


-- 
Ritesh Raj Sarraf | http://people.debian.org/~rrs
Debian - The Universal Operating SystemDKMS make.log for broadcom-sta-6.30.223.271 for kernel 4.9.0-2-amd64 (x86_64)
Wed Mar  1 23:38:34 IST 2017
/bin/sh: 1: [: Illegal number: 
/bin/sh: 1: [: Illegal number: 
Wireless Extension is the only possible API for this kernel version
Using Wireless Extension API
KBUILD_NOPEDANTIC=1 make -C /lib/modules/4.9.0-2-amd64/build M=`pwd`
make[1]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.
make[1]: Entering directory '/usr/src/linux-headers-4.9.0-2-amd64'
make -C /usr/src/linux-headers-4.9.0-2-amd64 KBUILD_SRC=/usr/src/linux-headers-4.9.0-2-common \
-f /usr/src/linux-headers-4.9.0-2-common/Makefile 
test -e include/generated/autoconf.h -a -e include/config/auto.conf || (		\
echo >&2;			\
echo >&2 "  ERROR: Kernel configuration is invalid.";		\
echo >&2 " include/generated/autoconf.h or include/config/auto.conf are missing.";\
echo >&2 " Run 'make oldconfig && make prepare' on kernel src to fix it.";	\
echo >&2 ;			\
/bin/false)
mkdir -p /var/lib/dkms/broadcom-sta/6.30.223.271/build/.tmp_versions ; rm -f /var/lib/dkms/broadcom-sta/6.30.223.271/build/.tmp_versions/*
make -f /usr/src/linux-headers-4.9.0-2-common/scripts/Makefile.build obj=/var/lib/dkms/broadcom-sta/6.30.223.271/build
CFG80211 API is prefered for this kernel version
Using CFG80211 API
Kernel architecture is X86_64
   rm -f /var/lib/dkms/broadcom-sta/6.30.223.271/build/built-in.o; ar rcsD /var/lib/dkms/broadcom-sta/6.30.223.271/build/built-in.o
   gcc-6 -Wp,-MD,/var/lib/dkms/broadcom-sta/6.30.223.271/build/src/shared/.linux_osl.o.d  -nostdinc -isystem /usr/lib/gcc/x86_64-linux-gnu/6/include -I/usr/src/linux-headers-4.9.0-2-common/arch/x86/include -I./arch/x86/include/generated/uapi -I./arch/x86/include/generated  -I/usr/src/linux-headers-4.9.0-2-common/include -I./include -I/usr/src/linux-headers-4.9.0-2-common/arch/x86/include/uapi -I/usr/src/linux-headers-4.9.0-2-common/include/uapi -I./include/generated/uapi -include /usr/src/linux-headers-4.9.0-2-common/include/linux/kconfig.h -I/usr/src/linux-headers-4.9.0-2-common//var/lib/dkms/broadcom-sta/6.30.223.271/build -I/var/lib/dkms/broadcom-sta/6.30.223.271/build -D__KERNEL__ -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Werror-implicit-function-declaration -Wno-format-security -std=gnu89 -fno-PIE -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -m64 -falign-jumps=1 -falign-loops=1 -mno-80387 -mno-fp-ret-in-387 -mpreferred-stack-boundary=3 -mskip-rax-setup -mtune=generic -mno-red-zone -mcmodel=kernel -funit-at-a-time -maccumulate-outgoing-args -DCONFIG_X86_X32_ABI -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -DCONFIG_AS_CFI_SECTIONS=1 -DCONFIG_AS_FXSAVEQ=1 -DCONFIG_AS_SSSE3=1 -DCONFIG_AS_CRC32=1 -DCONFIG_AS_AVX=1 -DCONFIG_AS_AVX2=1 -DCONFIG_AS_AVX512=1 -DCONFIG_AS_SHA1_NI=1 -DCONFIG_AS_SHA256_NI=1 -pipe -Wno-sign-compare -fno-asynchronous-unwind-tables -fno-delete-null-pointer-checks -Wno-frame-address -O2 --param=allow-store-data-races=0 -Wframe-larger-than=2048 -fstack-protector-strong -Wno-unused-but-set-variable -Wno-unused-const-variable -fno-var-tracking-assignments -g -pg -mfentry -DCC_USING_FENTRY -Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow -fconserve-stack -Werror=implicit-int -Werror=strict-prototypes -Werror=date-time -Werror=incompatible-pointer-types -DCC_HAVE_ASM_GOTO -DUSE_CFG80211  -I/var/lib/dkms/broadcom-sta/6.30.223.271/build/src/include  -I/var/lib/dkms/broadcom-sta/6.30.223.271/build/src/common/include  -I/var/lib/dkms/broadcom-sta/6.30.223.271/build/src/wl/sys  -I/var/lib/dkms/broadcom-sta/6.30.223.271/build/src/wl/phy  -I/var/lib/dkms/broadcom-sta/6.30.223.271/build/src/wl/ppr/include  -I/var/lib/dkms/broadcom-sta/6.30.223.271/build/src/shared/bcmwifi/include -Wno-date-time  -DMODULE  -DKBUILD_BASENAME='"linux_osl"'  -DKBUILD_MODNAME='"wl"' -c -o /var/lib/dkms/broadcom-sta/6.30.223.271/build/src/shared/.tmp_linux_osl.o /var/lib/dkms/broadcom-sta/6.30.223.271/build/src/shared/linux_osl.c
  if [ "-pg" = "-pg" ]; then if [ /var/lib/dkms/broadcom-sta/6.30.223.271/build/src/shared/linux_osl.o != "scripts/mod/empty.o" ]; then ./scripts/recordmcount  "/var/lib/dkms/broadcom-sta/6.30.223.271/build/src/shared/linux_osl.o"; fi; fi;
   gcc-6

Bug#856474: stap: include runtime_defines.h not found

2017-03-01 Thread Ritesh Raj Sarraf 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Thanks to hints from Frank, I can see the culprit now.

- -Werror  
-I/usr/src/linux-headers-4.9.0-2-common/"/usr/share/systemtap/runtime"  -DMODULE


The full compile command is:

   gcc-6 
-Wp,-MD,/tmp/staptZYmtX/.stap_7ef754ea40ea0990a9bfaf74753e8a9d_1538_aux_0.o.d  
-nostdinc -isystem /usr/lib/gcc/x86_64-linux-gnu/6/include 
-I/usr/src/linux-headers-4.9.0-2-common/arch/x86/include 
-I./arch/x86/include/generated/uapi -I./arch/x86/include/generated  
-I/usr/src/linux-headers-4.9.0-2-common/include -I./include 
-I/usr/src/linux-headers-4.9.0-2-common/arch/x86/include/uapi 
-I/usr/src/linux-headers-4.9.0-2-common/include/uapi -I./include/generated/uapi 
-include /usr/src/linux-headers-4.9.0-2-common/include/linux/kconfig.h 
-I/usr/src/linux-headers-4.9.0-2-common//tmp/staptZYmtX -I/tmp/staptZYmtX 
-D__KERNEL__ -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs 
-fno-strict-aliasing -fno-common -Werror-implicit-function-declaration 
-Wno-format-security -std=gnu89 -fno-PIE -mno-sse -mno-mmx -mno-sse2 -mno-3dnow 
-mno-avx -m64 -falign-jumps=1 -falign-loops=1 -mno-80387 -mno-fp-ret-in-387 
-mpreferred-stack-boundary=3 -mskip-rax-setup -mtune=generic -mno-red-zone 
-mcmodel=kernel -funit-at-a-time -maccumulate-outgoing-args 
-DCONFIG_X86_X32_ABI -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 
-DCONFIG_AS_CFI_SECTIONS=1 -DCONFIG_AS_FXSAVEQ=1 -DCONFIG_AS_SSSE3=1 
-DCONFIG_AS_CRC32=1 -DCONFIG_AS_AVX=1 -DCONFIG_AS_AVX2=1 -DCONFIG_AS_AVX512=1 
-DCONFIG_AS_SHA1_NI=1 -DCONFIG_AS_SHA256_NI=1 -pipe -Wno-sign-compare 
-fno-asynchronous-unwind-tables -fno-delete-null-pointer-checks 
-Wno-frame-address -O2 --param=allow-store-data-races=0 
-Wframe-larger-than=2048 -fstack-protector-strong -Wno-unused-but-set-variable 
-Wno-unused-const-variable -fno-var-tracking-assignments -pg -mfentry 
-DCC_USING_FENTRY -Wdeclaration-after-statement -Wno-pointer-sign 
-fno-strict-overflow -fconserve-stack -Werror=implicit-int 
-Werror=strict-prototypes -Werror=date-time -Werror=incompatible-pointer-types 
-DCC_HAVE_ASM_GOTO  
-I/usr/src/linux-headers-4.9.0-2-common/include2/asm/mach-default  
-I/lib/modules/4.9.0-2-amd64/build -include 
/tmp/staptZYmtX/stapconf_d8a215ec355839da11c41ac20d4e7e1e_702.h 
-freorder-blocks -fasynchronous-unwind-tables -Wframe-larger-than=512 
-fno-ipa-icf -Wno-unused -Werror  
-I/usr/src/linux-headers-4.9.0-2-common/"/usr/share/systemtap/runtime"  
-DMODULE  
-DKBUILD_BASENAME='"stap_7ef754ea40ea0990a9bfaf74753e8a9d_1538_aux_0"'  
-DKBUILD_MODNAME='"stap_7ef754ea40ea0990a9bfaf74753e8a9d_1538"' -c -o 
/tmp/staptZYmtX/.tmp_stap_7ef754ea40ea0990a9bfaf74753e8a9d_1538_aux_0.o 
/tmp/staptZYmtX/stap_7ef754ea40ea0990a9bfaf74753e8a9d_1538_aux_0.c
/tmp/staptZYmtX/stap_7ef754ea40ea0990a9bfaf74753e8a9d_1538_src.c:10:29: fatal 
error: runtime_defines.h: No such file or directory


On Wed, 2017-03-01 at 23:03 +0530, Ritesh Raj Sarraf wrote:
> On Wed, 2017-03-01 at 12:07 -0500, Frank Ch. Eigler wrote:
> > We've received intermittent reports of this sort of thing happening on
> > a few non-RH platforms.  A common element seemed to be some
> > nonstandard kernel build tree environment - some sort of customized
> > out-of-tree module builder/processing script or makefile, which was
> > confused by quotations in the stap-synthesized Makefile.  We have not
> > gotten to the bottom of it yet, but it didn't seem like a stap bug.
> > 
> > https://sourceware.org/ml/systemtap/2017-q1/msg00025.html
> 
> Thank you for replying quick. I realized your email after I sent another one.
> That email has some details about how the Debian kernel is organized,
> especially
> the linux-headers package split into 2 and installing files into 2 different
> paths.
> 
> rrs@chutzpah:~$ ls /usr/src/
> broadcom-sta-6.30.223.271/linux-headers-4.9.0-2-common/
> linux-headers-4.9.0-2-amd64/  linux-kbuild-4.9@
> 23:00 ♒♒♒   ☺
> 
> 
> IIRC, on the RH side, it used to be a single path. Maybe still is the same.
> 
> Similarly, I did not see this error on my custom kernel too.
> 
> rrs@learner:/var/tmp$ ls /usr/src/
> linux-headers-4.10.1+/linux-headers-4.9.0-2-common/  linux-kbuild-4.9@
> linux-headers-4.9.0-2-amd64/  linux-headers-4.9.9+/
> 2017-03-01 / 23:00:52 ♒♒♒  ☺  
> rrs@learner:/var/tmp$ uname -a
> Linux learner 4.10.1+ #20 SMP Sun Feb 26 21:22:45 IST 2017 x86_64 GNU/Linux
> 2017-03-01 / 23:00:58 ♒♒♒  ☺  
> 
> 
> I suspect the same, that this may be specific to Debian Linux kernel.
> I have also had eBPF compiler failing on something similar (Debian Bug:
> 849841).
> 
- -- 
Ritesh Raj Sarraf | http://people.debian.org/~rrs
Debian - The Universal Operating System
-BEGIN PGP SIGNATURE-

iQIzBAEBCgAdFiEEQCVDstmIVAB/Yn02pjpYo/LhdWkFAli3CJEACgkQpjpYo/Lh
dWmjwxAAmK4QUX/kzfTbfbjbUf+id4hGwJMyusbhYeaIS29ntEv4v/MqkpqzPiVt
rBNkImwm3VG8BQITzP8j0zbQhH2nVT2xE6eK3YCJ6CghtV/IVaZDU8QOodgZ2SP7
nsK3Z68q4/dfTql3NGo1YCJIxEcmQcEj2I0g8k6qBCjNezDlYfwfkNCK0u4UmcLb

Bug#856474: stap: include runtime_defines.h not found

2017-03-01 Thread Ritesh Raj Sarraf 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

This is specific to the linux-headers package and has nothing to do with
systemtap. Please ignore this one.

On Wed, 2017-03-01 at 22:56 +0530, Ritesh Raj Sarraf wrote:
> Running in verbose mode, it shows:
> 
> test -e include/generated/autoconf.h -a -e include/config/auto.conf ||
> (\
> echo >&2;-   \
> echo >&2 "  ERROR: Kernel configuration is invalid.";   \
> echo >&2 " include/generated/autoconf.h or include/config/auto.conf
> are
> missing.";\
> echo >&2 " Run 'make oldconfig && make prepare' on kernel src to fix
> it.";  \
> echo >&2 ;  \
> /bin/false)
- -- 
Ritesh Raj Sarraf | http://people.debian.org/~rrs
Debian - The Universal Operating System
-BEGIN PGP SIGNATURE-

iQIzBAEBCgAdFiEEQCVDstmIVAB/Yn02pjpYo/LhdWkFAli3B44ACgkQpjpYo/Lh
dWm2Ng/8Dd1mJUiEFClEYVb2nrmqh7HvE9SvZ7fFT1wx7CHEA9bcwrTeprBhmFj7
i4WxNaREVraPPUtxply6nQvUP+Y3IFFouD6qfCyFCpm6Vt9VUVLsV9z1L4y2VMx1
zKbqKMqq4ogyOMFfXHe8R264/uZ+3fiOnXfL6Z5NkcgRaB3mbllECxXKccMx5V/D
4kWuvn340Nt5tbJIY8zF2o4GMvDhYJ3sWrlqC37/4agWBozxQgyqEnRCO7LldZev
T47DKn7GcTsFJAdlUWL88pPzgr5Qaz7BCvgcMK5Thc91Pm1CM5y9nZIMv+bfdpnG
FuhuClRFxac0rGDcyvx5tSY15fnw/rVbEWUdl5/ZGhAZZnL85G4Bn/dydosHu4Ss
imgMN46UzsI0OvmXgHB6wvHyusOeDYzVzDiQPv0gSA2HURk0MDq4BbdXMSRP+Ge7
DPqE6PVMKKVdRPD3nCQnEUuUiQuIEZdc94B/AYSQSbNKRdJK1mu8fJtLh2KOrZnn
tkdrZn2Q8erNMw+Cn51MWnCWT8pUGTu++1zpdKPdwApOdDNU3Xi8IZNuStDvLy2E
14KheXfO4R72pqu2kGyQLyh11gjctemBfGcXC+Rn5bnNGXJB8bb3UUsGrkQdSBM7
TEwwjBrcN9ehp3huJ9n3zn9YdcUCm8QFTX2cnig2Rv1JFA4EOng=
=m61F
-END PGP SIGNATURE-

Bug#856474: stap: include runtime_defines.h not found

2017-03-01 Thread Ritesh Raj Sarraf 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On Wed, 2017-03-01 at 12:07 -0500, Frank Ch. Eigler wrote:
> We've received intermittent reports of this sort of thing happening on
> a few non-RH platforms.  A common element seemed to be some
> nonstandard kernel build tree environment - some sort of customized
> out-of-tree module builder/processing script or makefile, which was
> confused by quotations in the stap-synthesized Makefile.  We have not
> gotten to the bottom of it yet, but it didn't seem like a stap bug.
> 
> https://sourceware.org/ml/systemtap/2017-q1/msg00025.html

Thank you for replying quick. I realized your email after I sent another one.
That email has some details about how the Debian kernel is organized, especially
the linux-headers package split into 2 and installing files into 2 different
paths.

rrs@chutzpah:~$ ls /usr/src/
broadcom-sta-6.30.223.271/linux-headers-4.9.0-2-common/
linux-headers-4.9.0-2-amd64/  linux-kbuild-4.9@
23:00 ♒♒♒   ☺


IIRC, on the RH side, it used to be a single path. Maybe still is the same.

Similarly, I did not see this error on my custom kernel too.

rrs@learner:/var/tmp$ ls /usr/src/
linux-headers-4.10.1+/linux-headers-4.9.0-2-common/  linux-kbuild-4.9@
linux-headers-4.9.0-2-amd64/  linux-headers-4.9.9+/
2017-03-01 / 23:00:52 ♒♒♒  ☺  
rrs@learner:/var/tmp$ uname -a
Linux learner 4.10.1+ #20 SMP Sun Feb 26 21:22:45 IST 2017 x86_64 GNU/Linux
2017-03-01 / 23:00:58 ♒♒♒  ☺  


I suspect the same, that this may be specific to Debian Linux kernel.
I have also had eBPF compiler failing on something similar (Debian Bug: 849841).

- -- 
Ritesh Raj Sarraf | http://people.debian.org/~rrs
Debian - The Universal Operating System
-BEGIN PGP SIGNATURE-

iQIzBAEBCgAdFiEEQCVDstmIVAB/Yn02pjpYo/LhdWkFAli3Be8ACgkQpjpYo/Lh
dWl1ehAAiYole5d4WCRL/LOKcvzNlJw+KWQVecMRgU44K1cZsm/UqWXmIxMwkU35
csFsbjS8XK5fpz64lzpQkMqmtJ3hJwuX61iI+SHLbiHk1iN2UOQZ6ZkGsKUBgr6c
HjXvpKKM3JaQCeMtkAoFanH1a8S0u6bemPD3inLk69YtVpG/1fNQT08FYZUGBwZP
yg/AOqfX5/h+aCA8y0P0oF2jEjYtAvxAT4iog7QnYCu/KpNbdD4RzvOIM5vAb7if
r45oId5k9OwJKn/DHKZj6rYj2v7+oSkEcPtTJdLOUkIsnqE88MAcYqY498HRsiHv
tCVAln5DmaZNJzE6DTB5efSXk86nFlVhFYHXj3+j3z7IJ6RHCW5aZ2rVniMpu4/E
73MT55louz/AGwSrib+MEfiodpu3GI3Dk7uhDs36QxAObGb94DjNzjb5j+ZwPZRO
KuFa1jVWv9puIyXhX3J6Q98oU8rJPdZuA0crtDJOueSdHh1GLs6jFIMmpsN4Vup0
DPcLt6xq0rcakiWUKCZ+hfYbRcjRPJsVcN2nfmR/+vBX7s+eElcXgRrc7re/A979
3sE3uSE7fgmA3U33g7gHCieRKedgyaNigWhA8LVxT0WbNh6YBwKh6M6JluJ7azHH
bRYmsTBv2cyoOTTz7ZejaVU+XayANuOfBDp0MRrYAO6aDcUEPtc=
=Qg77
-END PGP SIGNATURE-

Bug#856474: stap: include runtime_defines.h not found

2017-03-01 Thread Frank Ch. Eigler 
Hi -

> [...]  Running in verbose mode, it shows: [...]  Those paths are
> provided by the linux-headers-xxx-amd64 (or equivalent arch)
> package. In Debian, build/ and source/ paths are pointing to
> separate locations, provided by separate package. [...]

Could you provide a make V=1 transcript of any nontrivial out-of-tree
module (plus its Makefile) that builds & works on your kernel?

- FChE

Bug#856474: stap: include runtime_defines.h not found

2017-03-01 Thread Ritesh Raj Sarraf 
Hello Frank,

We have this bug report in Debian, where systemtap is looking for a local header
file rather than using the system provided one (provided by the 
systemtap-common 
package).

The translate.cxx file writes the code. This looks like deliberately done, under
certain conditions. I am not sure what condition triggers this code.

Can you please provide any help ?

Running in verbose mode, it shows:

test -e include/generated/autoconf.h -a -e include/config/auto.conf ||
(\
echo >&2;-   \
echo >&2 "  ERROR: Kernel configuration is invalid.";   \
echo >&2 " include/generated/autoconf.h or include/config/auto.conf are
missing.";\
echo >&2 " Run 'make oldconfig && make prepare' on kernel src to fix
it.";  \
echo >&2 ;  \
/bin/false)


Those paths are provided by the linux-headers-xxx-amd64 (or equivalent arch)
package. In Debian, build/ and source/ paths are pointing to separate locations,
provided by separate package.

rrs@chutzpah:/lib/modules/4.9.0-2-amd64$ ls -l
total 4080
lrwxrwxrwx  1 root root  36 Feb 17 18:48 build -> /usr/src/linux-headers-
4.9.0-2-amd64/
drwxr-xr-x 12 root root4096 Feb 20 22:57 kernel/
-rw-r--r--  1 root root 1005836 Mar  1 19:55 modules.alias
-rw-r--r--  1 root root  965121 Mar  1 19:55 modules.alias.bin
-rw-r--r--  1 root root4027 Feb 27 21:28 modules.builtin
-rw-r--r--  1 root root5485 Mar  1 19:55 modules.builtin.bin
-rw-r--r--  1 root root  393918 Mar  1 19:55 modules.dep
-rw-r--r--  1 root root  543473 Mar  1 19:55 modules.dep.bin
-rw-r--r--  1 root root 402 Mar  1 19:55 modules.devname
-rw-r--r--  1 root root  131711 Feb 27 21:28 modules.order
-rw-r--r--  1 root root 523 Mar  1 19:55 modules.softdep
-rw-r--r--  1 root root  489848 Mar  1 19:55 modules.symbols
-rw-r--r--  1 root root  605149 Mar  1 19:55 modules.symbols.bin
lrwxrwxrwx  1 root root  37 Mar  1 22:52 source -> /usr/src/linux-headers-
4.9.0-2-common/
drwxr-xr-x  3 root root4096 Mar  1 19:55 updates/
22:52 ♒♒♒   ☺


The full verbose log from stap is attached with this email.

I suspect it to be a Debian kernel problem but wanted to check with you first.


Thanks,
Ritesh

On Wed, 2017-03-01 at 22:28 +0530, Ritesh Raj Sarraf wrote:
> Control: tag -1 +confirmed -moreinfo
> 
> On Wed, 2017-03-01 at 18:58 +0530, Ritesh Raj Sarraf wrote:
> > But in my logs, runtime_defines.h is properly detected. I'll try to run the
> > command on a Debian kernel to verify if it is broken or not.
> 
> I have been able to reproduce this on a Debian kernel.
> 
> rrs@chutzpah:~$ stap-
> prep  
> 22:25 ♒♒♒   ☺
> rrs@chutzpah:~$ stap -v -e 'probe vfs.read {printf("read performed\n");
> exit()}'   
> Pass 1: parsed user script and 119 library scripts using
> 115056virt/39720res/6372shr/33400data kb, in 130usr/0sys/340real ms.
> Pass 2: analyzed script: 1 probe, 1 function, 4 embeds, 0 globals using
> 256132virt/182128res/7752shr/174476data kb, in 1770usr/160sys/6312real ms.
> Pass 3: translated to C into
> "/tmp/stapUHT1si/stap_7ef754ea40ea0990a9bfaf74753e8a9d_1538_src.c" using
> 256132virt/182376res/8000shr/174476data kb, in 0usr/0sys/7real ms.
> /tmp/stapUHT1si/stap_7ef754ea40ea0990a9bfaf74753e8a9d_1538_src.c:10:29: fatal
> error: runtime_defines.h: No such file or directory
>  #include "runtime_defines.h"
>  ^
> compilation terminated.
> /usr/src/linux-headers-4.9.0-2-common/scripts/Makefile.build:298: recipe for
> target '/tmp/stapUHT1si/stap_7ef754ea40ea0990a9bfaf74753e8a9d_1538_src.o'
> failed
> make[3]: ***
> [/tmp/stapUHT1si/stap_7ef754ea40ea0990a9bfaf74753e8a9d_1538_src.o]
> Error 1
> make[3]: *** Waiting for unfinished jobs
> /usr/src/linux-headers-4.9.0-2-common/Makefile:1507: recipe for target
> '_module_/tmp/stapUHT1si' failed
> make[2]: *** [_module_/tmp/stapUHT1si] Error 2
> Makefile:150: recipe for target 'sub-make' failed
> make[1]: *** [sub-make] Error 2
> Makefile:8: recipe for target 'all' failed
> make: *** [all] Error 2
> WARNING: kbuild exited with status: 2
> Pass 4: compiled C into "stap_7ef754ea40ea0990a9bfaf74753e8a9d_1538.ko" in
> 7560usr/630sys/10583real ms.
> Pass 4: compilation failed.  [man error::pass4]
> Tip: /usr/share/doc/systemtap/README.Debian should help you get started.
> 22:26 ♒♒♒☹  => 1  
> 
-- 
Ritesh Raj Sarraf | http://people.debian.org/~rrs
Debian - The Universal Operating SystemSystemtap translator/driver (version 3.0/0.168, Debian version 3.0-8 (UNRELEASED))
Copyright (C) 2005-2015 Red Hat, Inc. and others
This is free software; see the source for copying conditions.
enabled features: AVAHI LIBSQLITE3 NLS NSS TR1_UNORDERED_MAP
Created temporary directory "/tmp/staptZYmtX"
Session arch: x86_64 release: 4.9.0-2-amd64
Parsed kernel "/lib/modules/4.9.0-2-amd64/build/.config", containing 5049 tuples
Parsed kernel

Bug#856474: stap: include runtime_defines.h not found

2017-03-01 Thread Frank Ch. Eigler 
Hi -

> I have been able to reproduce this on a Debian kernel.
> [...]
> /tmp/stapUHT1si/stap_7ef754ea40ea0990a9bfaf74753e8a9d_1538_src.c:10:29: fatal
> error: runtime_defines.h: No such file or directory
>  #include "runtime_defines.h"
>  ^
> compilation terminated.
> [...]

We've received intermittent reports of this sort of thing happening on
a few non-RH platforms.  A common element seemed to be some
nonstandard kernel build tree environment - some sort of customized
out-of-tree module builder/processing script or makefile, which was
confused by quotations in the stap-synthesized Makefile.  We have not
gotten to the bottom of it yet, but it didn't seem like a stap bug.

https://sourceware.org/ml/systemtap/2017-q1/msg00025.html

- FChE

Bug#856474: stap: include runtime_defines.h not found

2017-03-01 Thread Ritesh Raj Sarraf 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Control: tag -1 +confirmed -moreinfo

On Wed, 2017-03-01 at 18:58 +0530, Ritesh Raj Sarraf wrote:
> But in my logs, runtime_defines.h is properly detected. I'll try to run the
> command on a Debian kernel to verify if it is broken or not.

I have been able to reproduce this on a Debian kernel.

rrs@chutzpah:~$ stap-
prep  
22:25 ♒♒♒   ☺
rrs@chutzpah:~$ stap -v -e 'probe vfs.read {printf("read performed\n");
exit()}'   
Pass 1: parsed user script and 119 library scripts using
115056virt/39720res/6372shr/33400data kb, in 130usr/0sys/340real ms.
Pass 2: analyzed script: 1 probe, 1 function, 4 embeds, 0 globals using
256132virt/182128res/7752shr/174476data kb, in 1770usr/160sys/6312real ms.
Pass 3: translated to C into
"/tmp/stapUHT1si/stap_7ef754ea40ea0990a9bfaf74753e8a9d_1538_src.c" using
256132virt/182376res/8000shr/174476data kb, in 0usr/0sys/7real ms.
/tmp/stapUHT1si/stap_7ef754ea40ea0990a9bfaf74753e8a9d_1538_src.c:10:29: fatal
error: runtime_defines.h: No such file or directory
 #include "runtime_defines.h"
 ^
compilation terminated.
/usr/src/linux-headers-4.9.0-2-common/scripts/Makefile.build:298: recipe for
target '/tmp/stapUHT1si/stap_7ef754ea40ea0990a9bfaf74753e8a9d_1538_src.o' failed
make[3]: *** [/tmp/stapUHT1si/stap_7ef754ea40ea0990a9bfaf74753e8a9d_1538_src.o]
Error 1
make[3]: *** Waiting for unfinished jobs
/usr/src/linux-headers-4.9.0-2-common/Makefile:1507: recipe for target
'_module_/tmp/stapUHT1si' failed
make[2]: *** [_module_/tmp/stapUHT1si] Error 2
Makefile:150: recipe for target 'sub-make' failed
make[1]: *** [sub-make] Error 2
Makefile:8: recipe for target 'all' failed
make: *** [all] Error 2
WARNING: kbuild exited with status: 2
Pass 4: compiled C into "stap_7ef754ea40ea0990a9bfaf74753e8a9d_1538.ko" in
7560usr/630sys/10583real ms.
Pass 4: compilation failed.  [man error::pass4]
Tip: /usr/share/doc/systemtap/README.Debian should help you get started.
22:26 ♒♒♒☹  => 1  

- -- 
Ritesh Raj Sarraf | http://people.debian.org/~rrs
Debian - The Universal Operating System
-BEGIN PGP SIGNATURE-

iQIzBAEBCgAdFiEEQCVDstmIVAB/Yn02pjpYo/LhdWkFAli2/c8ACgkQpjpYo/Lh
dWkUbA/+PAOJ6+j07mVcMbv/ums5LWSQSb4SBLoRC9BRunUOOozA8yfWpflHRXrS
raLamnkE26xpd6HFZ0VVT3dFUH12malwn2GH2uEzVQMjiShxAqXaEJfuaLbuUIjk
T7Wg/TusRUhRBjAgoIAMIa5CiKjW/mxn0JcBZHiiNoppaOEpJ8jl+FuEvhtN3HLK
SbFT+1VcCxMK27yhEjuTDhoeSoCqmXhyIxvK9sSyGDCFG/90OKB3Og5VkPcg
KxskOy0dIClRxUO2nvmxfsMeDImtZedUgsWZrq0/1+bCkeoHpCArqWqVfeAG86fl
STWW/04sMXG2cFhSEmV0igH/UbTuOYdzVz2XpAjZLMo+awNYs+6RhRiy4Hj2JV+Z
0EXseGdtHTojYKM1wZ6PI9nLtYbYIYgxiGezAQdaI9SeTiMCKHe5Llpa8OsRHo0c
7ts5FvEWL+7EpKdp+zO3UN4dQiG8nLwmKk9OGLyBmZq1FGyHg5qPWrWcyLLFMJc5
zf97Jpt082tZ2s2WVPLJdMH8zWSjb3c98EhS4qRE5F53bQdxVNEBbh2wx/Aokv7l
htBguJ4D/czjh/Qd4RbMybcKStvKPxu2Elkq1dLqF0RypJoAXlIGrrJDqtY3xZX8
DCmQCxvA0tkAb67uvpSbISBX9mEPZ7HvetSX175S2YZg2CVnYdY=
=wssJ
-END PGP SIGNATURE-

Bug#856474: stap: include runtime_defines.h not found

2017-03-01 Thread Jack Henschel 
> Ritesh Raj Sarraf  hat am 1. März 2017 um 14:28 geschrieben:
> That file is provided by the systemtap-common package, which is properly 
> marked
> as a Depends for systemtap.
I know, the package is installed and the files are present:
$ find /usr/share/systemtap/ -iname *runtime_defines.h*
/usr/share/systemtap/runtime/runtime_defines.h
/usr/share/systemtap/runtime/linux/runtime_defines.h

I'm kind of guessing there is something wrong the include statements the 
compiler gets:
> gcc-6 
> -Wp,-MD,/tmp/stapcukYUi/.stap_a31ee9d9da16ac912b90eb18969508b5_1547_aux_0.o.d 
>  -nostdinc -isystem /usr/lib/gcc/x86_64-linux-gnu/6/include 
> -I/usr/src/linux-headers-4.9.0-1-common/arch/x86/include 
> -I./arch/x86/include/generated/uapi -I./arch/x86/include/generated  
> -I/usr/src/linux-headers-4.9.0-1-common/include -I./include 
> -I/usr/src/linux-headers-4.9.0-1-common/arch/x86/include/uapi 
> -I/usr/src/linux-headers-4.9.0-1-common/include/uapi 
> -I./include/generated/uapi -include 
> /usr/src/linux-headers-4.9.0-1-common/include/linux/kconfig.h 
> -I/usr/src/linux-headers-4.9.0-1-common//tmp/stapcukYUi -I/tmp/stapcukYUi 
> -D__KERNEL__ -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs 
> -fno-strict-aliasing -fno-common -Werror-implicit-function-declaration 
> -Wno-format-security -std=gnu89 -fno-PIE -mno-sse -mno-mmx -mno-sse2 
> -mno-3dnow -mno-avx -m64 -falign-jumps=1 -falign-loops=1 -mno-80387 
> -mno-fp-ret-in-387 -mpreferred-stack-boundary=3 -mskip-rax-setup 
> -mtune=generic -mno-red-zone -mcmodel=kernel -funit-at-a-time 
> -maccumulate-outgoing-args -DCONFIG_X86_X32_ABI -DCONFIG_AS_CFI=1 
> -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -DCONFIG_AS_CFI_SECTIONS=1 
> -DCONFIG_AS_FXSAVEQ=1 -DCONFIG_AS_SSSE3=1 -DCONFIG_AS_CRC32=1 
> -DCONFIG_AS_AVX=1 -DCONFIG_AS_AVX2=1 -DCONFIG_AS_AVX512=1 
> -DCONFIG_AS_SHA1_NI=1 -DCONFIG_AS_SHA256_NI=1 -pipe -Wno-sign-compare 
> -fno-asynchronous-unwind-tables -fno-delete-null-pointer-checks 
> -Wno-frame-address -O2 --param=allow-store-data-races=0 
> -Wframe-larger-than=2048 -fstack-protector-strong 
> -Wno-unused-but-set-variable -Wno-unused-const-variable 
> -fno-var-tracking-assignments -pg -mfentry -DCC_USING_FENTRY 
> -Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow 
> -fconserve-stack -Werror=implicit-int -Werror=strict-prototypes 
> -Werror=date-time -Werror=incompatible-pointer-types -DCC_HAVE_ASM_GOTO  
> -I/usr/src/linux-headers-4.9.0-1-common/include2/asm/mach-default  
> -I/lib/modules/4.9.0-1-amd64/build -include 
> /tmp/stapcukYUi/stapconf_0763f53e6c686e80bdfa9e79a1380e9b_711.h 
> -freorder-blocks -fasynchronous-unwind-tables -Wframe-larger-than=512 
> -fno-ipa-icf -Wno-unused -Werror  
> -I/usr/src/linux-headers-4.9.0-1-common/"/usr/share/systemtap/runtime"  
> -DMODULE  
> -DKBUILD_BASENAME='"stap_a31ee9d9da16ac912b90eb18969508b5_1547_aux_0"'  
> -DKBUILD_MODNAME='"stap_a31ee9d9da16ac912b90eb18969508b5_1547"' -c -o 
> /tmp/stapcukYUi/.tmp_stap_a31ee9d9da16ac912b90eb18969508b5_1547_aux_0.o 
> /tmp/stapcukYUi/stap_a31ee9d9da16ac912b90eb18969508b5_1547_aux_0.c
> /tmp/stapcukYUi/stap_a31ee9d9da16ac912b90eb18969508b5_1547_src.c:10:29: fatal 
> error: runtime_defines.h: No such file or directory
> #include "runtime_defines.h"

Is the argument 
`-I/usr/src/linux-headers-4.9.0-1-common/"/usr/share/systemtap/runtime"` valid?

> Over here, with a custom built kernel (Linux 4.10), I am getting compilation
> errors, which are not relevant to the issue you've reported.
Indeed, those errors do not have anything in common.

Bug#856474: stap: include runtime_defines.h not found

2017-03-01 Thread Ritesh Raj Sarraf 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Control: tag -1 +moreinfo

On Wed, 2017-03-01 at 13:53 +0100, Jack Henschel wrote:
> after I set up systemtap, I ran the example command provided in the
> documentation:
> https://sourceware.org/systemtap/SystemTap_Beginners_Guide/using-systemtap.htm
> l#testing
> > stap -v -e 'probe vfs.read {printf("read performed\n"); exit()}'
> 
> GCC (6) complains upon compiling the C source code into the kernel module that
> it is missing C Headers:
> > /tmp/stapHaiI5M/stap_a31ee9d9da16ac912b90eb18969508b5_1547_src.c:10:29:
> fatal error: runtime_defines.h: No such file or directory
> > #include "runtime_defines.h"

That file is provided by the systemtap-common package, which is properly marked
as a Depends for systemtap.

Over here, with a custom built kernel (Linux 4.10), I am getting compilation
errors, which are not relevant to the issue you've reported.

But in my logs, runtime_defines.h is properly detected. I'll try to run the
command on a Debian kernel to verify if it is broken or not.


- -- 
Ritesh Raj Sarraf | http://people.debian.org/~rrs
Debian - The Universal Operating System
-BEGIN PGP SIGNATURE-

iQIzBAEBCgAdFiEEQCVDstmIVAB/Yn02pjpYo/LhdWkFAli2zIcACgkQpjpYo/Lh
dWl7Eg//U23jiXQGAbybxlpgfn6GJA9OGuyAKHEqehvE/NBWrvHS6sSE/bU7nLBc
YENuEyR/u5dtQCzrYleDZL+pLrWVw7RZVbrwXzcXMPPwEPP1MIEFrCw9aGfk+j1j
OtDqAaUNkEfCLO4+Y7QLKKVbCWecjExHkY2SOF+p//Om2xOv1caWswytj7nqWspT
tKfTWj6FREMDyg2uivQkyyWqgWhmCtZm8ngQfK+zh9lpkBSJBn9cxGGKOsQgG99c
hthkWROMT+LxQJCV9JgBgfQcXeQvTgWgkaWKXSYFXUPTF4SJjORWEbhbQJh0ER8A
wxVoFlY3LG2OTVb2Z2Qp9WoY/cAjEw+muQuUoWDKoE2E9zcVhFeA2b15h04lzugE
la83E0Eqd9VwlluDRTajl+58PUgyARkiz6Sh3eE3CJqbnhAPZv6FX2qsOdWdMQ0q
6c+3OBo9uf26Q4is79o0mraqb/KQ0uULyjR2ItHlUVs+cwSl/HylcWYt9QbVjcep
zLJzjXN03P8meKCp0TpHgjanLqyD/3ntSCXR2rAxBCHF8Qzxsimw2RXcv1DEAXhF
vkfoOqLEnej5yVxtRhSYsDth+QJALpBfqgNEgvCmJqw1L0UOagPAQ7uoqcOQ/w9s
ZhyhYAuVAdXZ2c0rP0PcadTDMaUsvxk5Pd1v7+ij5Vv9JyJL7d4=
=2SEl
-END PGP SIGNATURE-

Bug#856474: stap: include runtime_defines.h not found

2017-03-01 Thread Ritesh Raj Sarraf 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On Wed, 2017-03-01 at 18:58 +0530, Ritesh Raj Sarraf wrote:
> But in my logs, runtime_defines.h is properly detected. I'll try to run the
> command on a Debian kernel to verify if it is broken or not.

Forgot to add the log.

```
rrs@learner:/tmp$ sudo stap -v -e 'probe vfs.read {printf("read performed\n");
exit()}'
[sudo] password for rrs: 
Pass 1: parsed user script and 118 library scripts using
106660virt/39568res/6268shr/33552data kb, in 200usr/20sys/258real ms.
Pass 2: analyzed script: 1 probe, 1 function, 4 embeds, 0 globals using
246904virt/181448res/7688shr/173796data kb, in 1810usr/660sys/4018real ms.
Pass 3: translated to C into
"/tmp/stapJQUBQF/stap_8ef46d39a073456b4762098be7ff765b_1501_src.c" using
246904virt/181640res/7880shr/173796data kb, in 10usr/50sys/60real ms.
In file included from /usr/share/systemtap/runtime/linux/runtime.h:209:0,
 from /usr/share/systemtap/runtime/runtime.h:24,
 from
/tmp/stapJQUBQF/stap_8ef46d39a073456b4762098be7ff765b_1501_src.c:25:
/usr/share/systemtap/runtime/linux/access_process_vm.h: In function
‘__access_process_vm_’:
/usr/share/systemtap/runtime/linux/access_process_vm.h:36:61: error: passing
argument 6 of ‘get_user_pages_remote’ makes pointer from integer without a cast
[-Werror=int-conversion]
   ret = get_user_pages_remote (tsk, mm, addr, 1, write, 1, , );
 ^
In file included from ./include/linux/pid_namespace.h:6:0,
 from ./include/linux/ptrace.h:8,
 from ./include/linux/ftrace.h:13,
 from ./include/linux/kprobes.h:42,
 from /usr/share/systemtap/runtime/linux/runtime.h:21,
 from /usr/share/systemtap/runtime/runtime.h:24,
 from
/tmp/stapJQUBQF/stap_8ef46d39a073456b4762098be7ff765b_1501_src.c:25:
./include/linux/mm.h:1267:6: note: expected ‘struct page **’ but argument is of
type ‘int’
 long get_user_pages_remote(struct task_struct *tsk, struct mm_struct *mm,
  ^
In file included from /usr/share/systemtap/runtime/linux/runtime.h:209:0,
 from /usr/share/systemtap/runtime/runtime.h:24,
 from
/tmp/stapJQUBQF/stap_8ef46d39a073456b4762098be7ff765b_1501_src.c:25:
/usr/share/systemtap/runtime/linux/access_process_vm.h:36:64: error: passing
argument 7 of ‘get_user_pages_remote’ from incompatible pointer type [-
Werror=incompatible-pointer-types]
   ret = get_user_pages_remote (tsk, mm, addr, 1, write, 1, , );
^
In file included from ./include/linux/pid_namespace.h:6:0,
 from ./include/linux/ptrace.h:8,
 from ./include/linux/ftrace.h:13,
 from ./include/linux/kprobes.h:42,
 from /usr/share/systemtap/runtime/linux/runtime.h:21,
 from /usr/share/systemtap/runtime/runtime.h:24,
 from
/tmp/stapJQUBQF/stap_8ef46d39a073456b4762098be7ff765b_1501_src.c:25:
./include/linux/mm.h:1267:6: note: expected ‘struct vm_area_struct **’ but
argument is of type ‘struct page **’
 long get_user_pages_remote(struct task_struct *tsk, struct mm_struct *mm,
  ^
In file included from /usr/share/systemtap/runtime/linux/runtime.h:209:0,
 from /usr/share/systemtap/runtime/runtime.h:24,
 from
/tmp/stapJQUBQF/stap_8ef46d39a073456b4762098be7ff765b_1501_src.c:25:
/usr/share/systemtap/runtime/linux/access_process_vm.h:36:71: error: passing
argument 8 of ‘get_user_pages_remote’ from incompatible pointer type [-
Werror=incompatible-pointer-types]
   ret = get_user_pages_remote (tsk, mm, addr, 1, write, 1, , );
   ^
In file included from ./include/linux/pid_namespace.h:6:0,
 from ./include/linux/ptrace.h:8,
 from ./include/linux/ftrace.h:13,
 from ./include/linux/kprobes.h:42,
 from /usr/share/systemtap/runtime/linux/runtime.h:21,
 from /usr/share/systemtap/runtime/runtime.h:24,
 from
/tmp/stapJQUBQF/stap_8ef46d39a073456b4762098be7ff765b_1501_src.c:25:
./include/linux/mm.h:1267:6: note: expected ‘int *’ but argument is of type
‘struct vm_area_struct **’
 long get_user_pages_remote(struct task_struct *tsk, struct mm_struct *mm,
  ^
cc1: all warnings being treated as errors
scripts/Makefile.build:294: recipe for target
'/tmp/stapJQUBQF/stap_8ef46d39a073456b4762098be7ff765b_1501_src.o' failed
make[1]: *** [/tmp/stapJQUBQF/stap_8ef46d39a073456b4762098be7ff765b_1501_src.o]
Error 1
Makefile:1490: recipe for target '_module_/tmp/stapJQUBQF' failed
make: *** [_module_/tmp/stapJQUBQF] Error 2
WARNING: kbuild exited with status: 2
Pass 4: compiled C into "stap_8ef46d39a073456b4762098be7ff765b_1501.ko" in

Bug#856474: stap: include runtime_defines.h not found

2017-03-01 Thread Jack Henschel 
Package: systemtap
Version: 3.0-7
Severity: normal
Tags: stretch

Hi,

after I set up systemtap, I ran the example command provided in the 
documentation:
https://sourceware.org/systemtap/SystemTap_Beginners_Guide/using-systemtap.html#testing
> stap -v -e 'probe vfs.read {printf("read performed\n"); exit()}'

GCC (6) complains upon compiling the C source code into the kernel module that 
it is missing C Headers:
> /tmp/stapHaiI5M/stap_a31ee9d9da16ac912b90eb18969508b5_1547_src.c:10:29: fatal 
> error: runtime_defines.h: No such file or directory
> #include "runtime_defines.h"

For a more verbose log, please see the attachment.

-- System Information:
Debian Release: 9.0
  APT prefers testing-debug
  APT policy: (500, 'testing-debug'), (500, 'testing')
Architecture: amd64 (x86_64)

Kernel: Linux 4.9.0-1-amd64 (SMP w/2 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages systemtap depends on:
ii  libavahi-client3   0.6.32-2
ii  libavahi-common3   0.6.32-2
ii  libc6  2.24-9
ii  libdw1 0.168-0.2
ii  libelf10.168-0.2
ii  libgcc11:6.3.0-6
ii  libnspr4   2:4.12-6
ii  libnss32:3.26.2-1
ii  libsqlite3-0   3.16.2-2
ii  libstdc++6 6.3.0-6
ii  make   4.1-9.1
ii  systemtap-common   3.0-7
ii  systemtap-runtime  3.0-7

$ dpkg -l | grep linux-
ii  linux-base   4.5  
all  Linux image base package
ii  linux-compiler-gcc-6-x86 4.9.6-3  
amd64Compiler for Linux on x86 (meta-package)
ii  linux-headers-4.9.0-1-amd64  4.9.6-3  
amd64Header files for Linux 4.9.0-1-amd64
ii  linux-headers-4.9.0-1-common 4.9.6-3  
all  Common header files for Linux 4.9.0-1
ii  linux-image-4.9.0-1-amd644.9.6-3  
amd64Linux 4.9 for 64-bit PCs (signed)
ii  linux-image-4.9.0-1-amd64-dbgsym 4.9.6-3  
amd64Debug symbols for linux-image-4.9.0-1-amd64
ii  linux-image-amd644.9+78   
amd64Linux for 64-bit PCs (meta-package)
ii  linux-kbuild-4.9 4.9.6-3  
amd64Kbuild infrastructure for Linux 4.9
ii  linux-libc-dev:amd64 4.9.6-3  
amd64Linux support headers for userspace development
ii  util-linux-locales   2.29.1-1 
all  locales files for util-linux


$ dpkg -l | grep gcc
ii  gcc  4:6.3.0-1
amd64GNU C compiler
ii  gcc-66.3.0-6  
amd64GNU C compiler
ii  gcc-6-base:amd64 6.3.0-6  
amd64GCC, the GNU Compiler Collection (base package)
ii  libgcc-6-dev:amd64   6.3.0-6  
amd64GCC support library (development files)
ii  libgcc1:amd641:6.3.0-6
amd64GCC support library
ii  linux-compiler-gcc-6-x86 4.9.6-3  
amd64Compiler for Linux on x86 (meta-package)# Output of: stap -v --vp 00020 -e 'probe vfs.read {printf("read performed\n"); 
exit()}'
#
Pass 1: parsed user script and 113 library scripts using 
105244virt/38144res/6204shr/32136data kb, in 110usr/10sys/125real ms.
Pass 2: analyzed script: 1 probe, 1 function, 4 embeds, 0 globals using 
246212virt/180548res/7572shr/173104data kb, in 1780usr/100sys/1878real ms.
Pass 3: translated to C into 
"/tmp/staphL9QAU/stap_a31ee9d9da16ac912b90eb18969508b5_1547_src.c" using 
246212virt/180736res/7760shr/173104data kb, in 0usr/0sys/6real ms.
Running env -uARCH -uKBUILD_EXTMOD -uCROSS_COMPILE -uKBUILD_IMAGE 
-uKCONFIG_CONFIG -uINSTALL_PATH 
PATH=/usr/bin:/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 
make -C /lib/modules/4.9.0-1-amd64/build M=/tmp/staphL9QAU modules ARCH=x86_64 
CONFIG_DEBUG_INFO= V=1 -j3
make: Entering directory '/usr/src/linux-headers-4.9.0-1-amd64'
make -C /usr/src/linux-headers-4.9.0-1-amd64 
KBUILD_SRC=/usr/src/linux-headers-4.9.0-1-common \
-f /usr/src/linux-headers-4.9.0-1-common/Makefile modules
test -e include/generated/autoconf.h -a -e include/config/auto.conf || (
\
echo >&2;   \
echo >&2 "  ERROR: Kernel configuration is invalid.";   \
echo >&2 " include/generated/autoconf.h or include/config/auto.conf are 
missing.";\
echo >&2 " Run 'make oldconfig && make prepare' on kernel src to fix 
it.";  \
echo >&2 ;