Re: [lttng-dev] LTTng 32 bits on 64 bits systems - kernel events... but no user-event

2021-06-07 Thread Jonathan Rajotte-Julien via lttng-dev
Hi Julien,

A fix is on the way for libfd-tracker lib problem: 

https://review.lttng.org/c/lttng-tools/+/6045 

> Thank you ! The following modification is working :
> lttng-sessiond --consumerd32-libdir=/usr/local/lib32
> --consumerd32-path=/usr/local/lib32/lttng/libexec/lttng-consumerd --daemonize

Glad we could help.

Cheers
___
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev


Re: [lttng-dev] LTTng 32 bits on 64 bits systems - kernel events... but no user-event

2021-06-07 Thread MONTET Julien via lttng-dev
Hi,

I installed KeKriek version because it is the one recommanded by the apt-get of 
Debian Buster 10.9 (LTTng 2.10).
Debian -- Détails du paquet lttng-tools dans 
buster
I tried multiple times to install LTTng 2.12 without success, so I stayed in 
LTTng 2.10.

Find below the output of :  lttng-sessiond -vvv --verbose-consumer
https://paste.ubuntu.com/p/NrtyCqhHZc/

(I have got the same with 
LTTNG_CONSUMERD32_BIN=/usr/local/lib32/lttng/libexec/lttng-consumerd 
LTTNG_CONSUMERD32_LIBDIR=/usr/local/lib32 lttng-sessiond -vvv 
--verbose-consumer)

Yes, I had the issue below few days ago with LTTng 2.12
"For 2.12 there is some problems when compiling only lttng-consumerd 
(fd-tracker lib and libcommon problem)."

Thank you ! The following modification is working :
lttng-sessiond --consumerd32-libdir=/usr/local/lib32 
--consumerd32-path=/usr/local/lib32/lttng/libexec/lttng-consumerd --daemonize

Regards,

Julien


De : Jonathan Rajotte-Julien 
Envoyé : lundi 7 juin 2021 17:24
À : MONTET Julien 
Cc : lttng-dev 
Objet : Re: [lttng-dev] LTTng 32 bits on 64 bits systems - kernel events... but 
no user-event

Hi,

> I am currently trying to add lttng 32bits on a 64bits system with a custom
> kernel 4.19.177+ x86_64 (Debian 10.9)
> My former attempt only with only a 100% arm32 bits was successful (raspberry).
> I am facing now a strange issue : I can record kernel event but not userspace
> event... but I can see these user-event (see below)!

Ok.

> (tuto : [ 
> https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flttng.org%2Fdocs%2F%23doc-instrumenting-32-bit-app-on-64-bit-systemdata=04%7C01%7Cjulien.montet%40reseau.eseo.fr%7C9b85f601067749845a2408d929c85983%7C4d7ad1591265437ab9f62946247d5bf9%7C0%7C0%7C637586762735252731%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=5NB2nNrI17uwSE1Xpn1Uf6sbrRXbEePidPNXUh4ps7k%3Dreserved=0
> | 
> https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flttng.org%2Fdocs%2F%23doc-instrumenting-32-bit-app-on-64-bit-systemdata=04%7C01%7Cjulien.montet%40reseau.eseo.fr%7C9b85f601067749845a2408d929c85983%7C4d7ad1591265437ab9f62946247d5bf9%7C0%7C0%7C637586762735252731%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=5NB2nNrI17uwSE1Xpn1Uf6sbrRXbEePidPNXUh4ps7k%3Dreserved=0
>  ] )

> I followed two times the tutorial : on lttng 2.12 and lttng 2.10.
> I am on lttng (LTTng Trace Control) 2.10.11 - KeKriek now

Any particular reasons?

> All installation seemed to install without any error.
> The only thing I added is at the very end where I did a : ln -s
> /usr/local/bin/lttng /usr/bin/lttng (to directly call commands like "lttng
> start")

> Here some interesting outputs, I am using this projet as an example : [
> https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flttng.org%2Fdocs%2F%23doc-tracing-your-own-user-applicationdata=04%7C01%7Cjulien.montet%40reseau.eseo.fr%7C9b85f601067749845a2408d929c85983%7C4d7ad1591265437ab9f62946247d5bf9%7C0%7C0%7C637586762735262680%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=6RTEjlZmpq2CZoIUD%2Fs5ySx60Zqd4JMByVejF8nQ7Vk%3Dreserved=0
>  |
> https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Flttng.org%2Fdocs%2F%23doc-tracing-your-own-user-applicationdata=04%7C01%7Cjulien.montet%40reseau.eseo.fr%7C9b85f601067749845a2408d929c85983%7C4d7ad1591265437ab9f62946247d5bf9%7C0%7C0%7C637586762735262680%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=6RTEjlZmpq2CZoIUD%2Fs5ySx60Zqd4JMByVejF8nQ7Vk%3Dreserved=0
>  ]
> lttng list -u : OK [ 
> https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpaste.ubuntu.com%2Fp%2FhZptnNzySw%2Fdata=04%7C01%7Cjulien.montet%40reseau.eseo.fr%7C9b85f601067749845a2408d929c85983%7C4d7ad1591265437ab9f62946247d5bf9%7C0%7C0%7C637586762735262680%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=o71InJDxz1UgYSztgj5ENRr48MmLhsvQvPCO4GeOJ40%3Dreserved=0
>  | Ubuntu Pastebin ]
> LTTNG_UST_DEBUG=1 ./hello : seems OK : [ 
> https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpaste.ubuntu.com%2Fp%2Fw6xHrJsWJ9%2Fdata=04%7C01%7Cjulien.montet%40reseau.eseo.fr%7C9b85f601067749845a2408d929c85983%7C4d7ad1591265437ab9f62946247d5bf9%7C0%7C0%7C637586762735262680%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000sdata=%2FWcFiJDvDaGP0BxZaHVsk7MH91zwvcgmw7vOnDkkHGg%3Dreserved=0
> | Ubuntu Pastebin ]
> kernel-event output : [ 
> 

Re: [lttng-dev] LTTng 32 bits on 64 bits systems - kernel events... but no user-event

2021-06-07 Thread Jonathan Rajotte-Julien via lttng-dev
Hi, 

> I am currently trying to add lttng 32bits on a 64bits system with a custom
> kernel 4.19.177+ x86_64 (Debian 10.9)
> My former attempt only with only a 100% arm32 bits was successful (raspberry).
> I am facing now a strange issue : I can record kernel event but not userspace
> event... but I can see these user-event (see below)!

Ok. 

> (tuto : [ 
> https://lttng.org/docs/#doc-instrumenting-32-bit-app-on-64-bit-system
> | https://lttng.org/docs/#doc-instrumenting-32-bit-app-on-64-bit-system ] )

> I followed two times the tutorial : on lttng 2.12 and lttng 2.10.
> I am on lttng (LTTng Trace Control) 2.10.11 - KeKriek now

Any particular reasons?

> All installation seemed to install without any error.
> The only thing I added is at the very end where I did a : ln -s
> /usr/local/bin/lttng /usr/bin/lttng (to directly call commands like "lttng
> start")

> Here some interesting outputs, I am using this projet as an example : [
> https://lttng.org/docs/#doc-tracing-your-own-user-application |
> https://lttng.org/docs/#doc-tracing-your-own-user-application ]
> lttng list -u : OK [ https://paste.ubuntu.com/p/hZptnNzySw/ | Ubuntu Pastebin 
> ]
> LTTNG_UST_DEBUG=1 ./hello : seems OK : [ 
> https://paste.ubuntu.com/p/w6xHrJsWJ9/
> | Ubuntu Pastebin ]
> kernel-event output : [ https://paste.ubuntu.com/p/5KfyS8wVdb/ | Ubuntu 
> Pastebin
> ]
> user-event ouput : none - the output folder stated with lttng create
> my-user-session --output=/tmp/my-user-session doesn't even appear
> apt-file search lttng : [ https://paste.ubuntu.com/p/xMcCc7bqwk/ | Ubuntu
> Pastebin ]
> lsmod | grep lttng : [ https://paste.ubuntu.com/p/7YNkNsh9Fr/ | Ubuntu 
> Pastebin
> ]

Can you provide the config.log from the configure steps 

We are missing the most important logs: lttng-sessiond. 

$ lttng-sessiond -vvv --verbose-consumer 

> Note : file /usr/local/bin/lttng (the exe I am using) gives :
> /usr/local/bin/lttng: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV),
> dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux
> 3.2.0, BuildID[sha1]=542a108c40e386c17027c2c8f8fcb30e278b7748, with 
> debug_info,
> not stripped

This is normal. Concretely to support tracing of 32 bit applications only a 32 
bits
lttng-consumerd executable is needed which is launched by the lttng-sessiond 
executable. 

> I am wondering if the installation didn't pick a 64bits RCU and analyses
> programms only on this.

Hmmm, what indicate this? I seriously doubt that this can happen. 

> However, I have currently absolutely no idea how to debug this.

I decided to do the process on a clean ubuntu 20.04 machine.
For 2.12 there is some problems when compiling only lttng-consumerd (fd-tracker 
lib and libcommon problem). I plan on providing a patch shortly.

But the major problem is that the path passed at configure time for the 
lttng-consumerd32 bit daemon are simply not used at runtime.
See this bug I just opened [1]. This is valid for 2.10 all the way to master 
AFAIK.

[1] https://bugs.lttng.org/issues/1318 

In the meantime you will have to pass the lttng-consumerd 32 bit path as 
argument or env variables to lttng-sessiond as such: 

$ lttng-sessiond -vvv --verbose-consumer --consumerd32-libdir=/usr/local/lib32 
--consumerd32-path=/usr/local/lib32/lttng/libexec/lttng-consumerd

or 

$ LTTNG_CONSUMERD32_BIN=/usr/local/lib32/lttng/libexec/lttng-consumerd 
LTTNG_CONSUMERD32_LIBDIR=/usr/local/lib32 lttng-sessiond -vvv --verbose-consumer


>From there the 32bits application is traced correctly and data is gathered 
>without any problem (at least on 2.12).


```
[15:22:24.500655502] (+?.?) ubuntu2004.localdomain 
lttng_ust_statedump:start: { cpu_id = 3 }, { }
[15:22:24.500661538] (+0.06036) ubuntu2004.localdomain 
lttng_ust_statedump:procname: { cpu_id = 3 }, { procname = "hello" }
[15:22:24.502611224] (+0.001949686) ubuntu2004.localdomain 
lttng_ust_statedump:bin_info: { cpu_id = 3 }, { baddr = 0xF7EC9000, memsz = 
20596, path = "/usr/lib32/libdl-2.31.so", is_pic = 1, has_build_id = 1, 
has_debug_link = 1 }
[15:22:24.502614501] (+0.03277) ubuntu2004.localdomain 
lttng_ust_statedump:build_id: { cpu_id = 3 }, { baddr = 0xF7EC9000, 
_build_id_length = 20, build_id = [ [0] = 0x75, [1] = 0x1E, [2] = 0xDD, [3] = 
0x47, [4] = 0x72, [5] = 0x3, [6] 
```

Cheers
___
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev


Re: [lttng-dev] LTTng 32 bits on 64 bits systems - kernel events... but no user-event

2021-06-07 Thread Mathieu Desnoyers via lttng-dev



- On Jun 4, 2021, at 9:09 AM, lttng-dev lttng-dev@lists.lttng.org wrote:

> Hi LTTng team,

> I am currently trying to add lttng 32bits on a 64bits system with a custom
> kernel 4.19.177+ x86_64 (Debian 10.9)
> My former attempt only with only a 100% arm32 bits was successful (raspberry).
> I am facing now a strange issue : I can record kernel event but not userspace
> event... but I can see these user-event (see below)!
> (tuto : [ 
> https://lttng.org/docs/#doc-instrumenting-32-bit-app-on-64-bit-system
> | https://lttng.org/docs/#doc-instrumenting-32-bit-app-on-64-bit-system ] )

> I followed two times the tutorial : on lttng 2.12 and lttng 2.10.
> I am on lttng (LTTng Trace Control) 2.10.11 - KeKriek now
> All installation seemed to install without any error.
> The only thing I added is at the very end where I did a : ln -s
> /usr/local/bin/lttng /usr/bin/lttng (to directly call commands like "lttng
> start")

> Here some interesting outputs, I am using this projet as an example : [
> https://lttng.org/docs/#doc-tracing-your-own-user-application |
> https://lttng.org/docs/#doc-tracing-your-own-user-application ]
> lttng list -u : OK [ https://paste.ubuntu.com/p/hZptnNzySw/ | Ubuntu Pastebin 
> ]
> LTTNG_UST_DEBUG=1 ./hello : seems OK : [ 
> https://paste.ubuntu.com/p/w6xHrJsWJ9/
> | Ubuntu Pastebin ]
> kernel-event output : [ https://paste.ubuntu.com/p/5KfyS8wVdb/ | Ubuntu 
> Pastebin
> ]
> user-event ouput : none - the output folder stated with lttng create
> my-user-session --output=/tmp/my-user-session doesn't even appear
> apt-file search lttng : [ https://paste.ubuntu.com/p/xMcCc7bqwk/ | Ubuntu
> Pastebin ]
> lsmod | grep lttng : [ https://paste.ubuntu.com/p/7YNkNsh9Fr/ | Ubuntu 
> Pastebin
> ]

> Note : file /usr/local/bin/lttng (the exe I am using) gives :
> /usr/local/bin/lttng: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV),
> dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux
> 3.2.0, BuildID[sha1]=542a108c40e386c17027c2c8f8fcb30e278b7748, with 
> debug_info,
> not stripped

> I am wondering if the installation didn't pick a 64bits RCU and analyses
> programms only on this.
> However, I have currently absolutely no idea how to debug this.

> Could you advise me some methods ?

Did you follow this section of the documentation ? 

https://lttng.org/docs/#doc-instrumenting-32-bit-app-on-64-bit-system 

Thanks, 

Mathieu 

> Regards,

> ___
> lttng-dev mailing list
> lttng-dev@lists.lttng.org
> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
-- 
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
___
lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev