Re: kprobe: __blkdev_put probe is missed

2020-06-22 Thread Masami Hiramatsu
On Tue, 23 Jun 2020 09:38:01 +0900 Masami Hiramatsu wrote: > On Tue, 23 Jun 2020 08:47:06 +0900 > Masami Hiramatsu wrote: > > > On Mon, 22 Jun 2020 09:01:48 -0400 > > Steven Rostedt wrote: > > > > > On Mon, 22 Jun 2020 08:27:53 +0800 > > > Ming Lei wrote: > > > > > > > Can you kprobe guys

Re: kprobe: __blkdev_put probe is missed

2020-06-22 Thread Masami Hiramatsu
On Tue, 23 Jun 2020 08:47:06 +0900 Masami Hiramatsu wrote: > On Mon, 22 Jun 2020 09:01:48 -0400 > Steven Rostedt wrote: > > > On Mon, 22 Jun 2020 08:27:53 +0800 > > Ming Lei wrote: > > > > > Can you kprobe guys improve the implementation for covering this case? > > > For example, put probe

Re: kprobe: __blkdev_put probe is missed

2020-06-22 Thread Masami Hiramatsu
On Mon, 22 Jun 2020 09:01:48 -0400 Steven Rostedt wrote: > On Mon, 22 Jun 2020 08:27:53 +0800 > Ming Lei wrote: > > > Can you kprobe guys improve the implementation for covering this case? > > For example, put probe on 3) in case the above situation is recognized. > > To do so would require

Re: kprobe: __blkdev_put probe is missed

2020-06-22 Thread Steven Rostedt
On Mon, 22 Jun 2020 08:27:53 +0800 Ming Lei wrote: > Can you kprobe guys improve the implementation for covering this case? > For example, put probe on 3) in case the above situation is recognized. To do so would require solving the halting problem.

Re: kprobe: __blkdev_put probe is missed

2020-06-21 Thread Masami Hiramatsu
On Mon, 22 Jun 2020 08:27:53 +0800 Ming Lei wrote: > I mean it isn't from user's viewpoint, and the binary code is usually a > black box for final kprobe user. > > IMO, all your and Steven's input are just from kprobe/trace developer's > viewpoint. > Can you think about the issue from kprobe

Re: kprobe: __blkdev_put probe is missed

2020-06-21 Thread Ming Lei
Hi Masami, On Sat, Jun 20, 2020 at 10:37:47AM +0900, Masami Hiramatsu wrote: > Hi Ming, > > On Sat, 20 Jun 2020 07:28:20 +0800 > Ming Lei wrote: > > > > > > > Ah, after all it is as expected. With your kconfig, the kernel is > > > very agressively optimized. > > > > > > $ objdump -dS vmlinux

Re: kprobe: __blkdev_put probe is missed

2020-06-19 Thread Masami Hiramatsu
Hi Ming, On Sat, 20 Jun 2020 07:28:20 +0800 Ming Lei wrote: > > > > Ah, after all it is as expected. With your kconfig, the kernel is > > very agressively optimized. > > > > $ objdump -dS vmlinux | less > > ... > > 81256dc3 <__blkdev_put>: > > { > > 81256dc3: e8 98 85 df

Re: kprobe: __blkdev_put probe is missed

2020-06-19 Thread Steven Rostedt
On Sat, 20 Jun 2020 07:28:20 +0800 Ming Lei wrote: > Thanks for your investigation. > > Some trace tools can just trace on function entry, such as bcc, and some > user script always trace on function entry. > > I guess the issue should belong to kprobe implementation: The issue is that

Re: kprobe: __blkdev_put probe is missed

2020-06-19 Thread Ming Lei
Hi Masami, On Sat, Jun 20, 2020 at 12:35:09AM +0900, Masami Hiramatsu wrote: > Hi Ming, > > On Fri, 19 Jun 2020 21:32:40 +0800 > Ming Lei wrote: > > > On Fri, Jun 19, 2020 at 08:19:54AM -0400, Steven Rostedt wrote: > > > On Fri, 19 Jun 2020 15:28:59 +0800 > > > Ming Lei wrote: > > > > > > >

Re: kprobe: __blkdev_put probe is missed

2020-06-19 Thread Masami Hiramatsu
Hi Ming, On Fri, 19 Jun 2020 21:32:40 +0800 Ming Lei wrote: > On Fri, Jun 19, 2020 at 08:19:54AM -0400, Steven Rostedt wrote: > > On Fri, 19 Jun 2020 15:28:59 +0800 > > Ming Lei wrote: > > > > > > > > > > OK, then let's make events (for sure) > > > > > > > >

Re: kprobe: __blkdev_put probe is missed

2020-06-19 Thread Ming Lei
On Fri, Jun 19, 2020 at 08:19:54AM -0400, Steven Rostedt wrote: > On Fri, 19 Jun 2020 15:28:59 +0800 > Ming Lei wrote: > > > > > > > OK, then let's make events (for sure) > > > > > > root@devnote2:/sys/kernel/debug/tracing# echo p __blkdev_put >> > > > kprobe_events > > >

Re: kprobe: __blkdev_put probe is missed

2020-06-19 Thread Steven Rostedt
On Fri, 19 Jun 2020 15:28:59 +0800 Ming Lei wrote: > > > > OK, then let's make events (for sure) > > > > root@devnote2:/sys/kernel/debug/tracing# echo p __blkdev_put >> > > kprobe_events > > root@devnote2:/sys/kernel/debug/tracing# echo r __blkdev_put >> > > kprobe_events > >

Re: kprobe: __blkdev_put probe is missed

2020-06-19 Thread Ming Lei
Hi Masami, On Fri, Jun 19, 2020 at 02:12:39PM +0900, Masami Hiramatsu wrote: > Hi Ming, > > On Fri, 19 Jun 2020 07:19:01 +0800 > Ming Lei wrote: > > > > I'm using 5.4 on ubuntu and can not reproduce it with kprobe_event. > > > > > > root@devnote2:/sys/kernel/tracing# uname -a > > > Linux

Re: kprobe: __blkdev_put probe is missed

2020-06-18 Thread Masami Hiramatsu
Hi Ming, On Fri, 19 Jun 2020 07:19:01 +0800 Ming Lei wrote: > > I'm using 5.4 on ubuntu and can not reproduce it with kprobe_event. > > > > root@devnote2:/sys/kernel/tracing# uname -a > > Linux devnote2 5.4.0-37-generic #41-Ubuntu SMP Wed Jun 3 18:57:02 UTC 2020 > > x86_64 x86_64 x86_64

Re: kprobe: __blkdev_put probe is missed

2020-06-18 Thread Ming Lei
On Thu, Jun 18, 2020 at 10:56:02PM +0900, Masami Hiramatsu wrote: > Hi Ming, > > On Thu, 18 Jun 2020 20:54:38 +0800 > Ming Lei wrote: > > > On Wed, Jun 17, 2020 at 06:30:39PM +0800, Ming Lei wrote: > > > Hello Guys, > > > > > > I found probe on __blkdev_put is missed, which can be observed > >

Re: kprobe: __blkdev_put probe is missed

2020-06-18 Thread Masami Hiramatsu
Hi Ming, On Thu, 18 Jun 2020 20:54:38 +0800 Ming Lei wrote: > On Wed, Jun 17, 2020 at 06:30:39PM +0800, Ming Lei wrote: > > Hello Guys, > > > > I found probe on __blkdev_put is missed, which can be observed > > via bcc/perf reliably: > > > > 1) start trace > > - perf probe __blkdev_put > > -

Re: kprobe: __blkdev_put probe is missed

2020-06-18 Thread Ming Lei
On Wed, Jun 17, 2020 at 06:30:39PM +0800, Ming Lei wrote: > Hello Guys, > > I found probe on __blkdev_put is missed, which can be observed > via bcc/perf reliably: > > 1) start trace > - perf probe __blkdev_put > - perf trace -a -e probe:__blkdev_put > > or > > /usr/share/bcc/tools/stackcount