Re: [PATCH -tip v2 0/3] ftrace, kprobes: Introduce IPMODIFY flag for ftrace_ops to detect conflicts
(2014/06/19 23:18), Josh Poimboeuf wrote:
> On Thu, Jun 19, 2014 at 02:03:31PM +0900, Masami Hiramatsu wrote:
>> (2014/06/19 11:08), Josh Poimboeuf wrote:
>>> On Tue, Jun 17, 2014 at 11:04:36AM +, Masami Hiramatsu wrote:
Hi,
Here is the version 2 of the series of patches which introduces
IPMODIFY flag for ftrace_ops to detect conflicts of ftrace users
who can modify regs->ip in their handler.
In this version, I fixed some bugs in previous version and
added a patch which made kprobe itself free from IPMODIFY
except for jprobe.
>>>
>>> Hi Masami,
>>>
>>> This seems better, but I still saw a few issues. I'm not sure if the
>>> issues are specific to stap or kprobes. For the following issues I used
>>> this command to set a kprobe:
>>>
>>> stap -v -e 'probe kernel.function("meminfo_proc_show")
>>> {printf("meminfo_proc_show called\n");}'
>>>
>>> With patches 1-2, when I used stap to kprobe the function after it was
>>> already kpatched, stap didn't return an error and instead acted like it
>>> succeeded (though the probe didn't work):
>>>
>>> $ sudo stap -v -e 'probe kernel.function("meminfo_proc_show")
>>> {printf("meminfo_proc_show called\n");}'
>>> Pass 1: parsed user script and 112 library script(s) using
>>> 221516virt/41612res/6028shr/36228data kb, in 130usr/0sys/132real ms.
>>> Pass 2: analyzed script: 1 probe(s), 0 function(s), 0 embed(s), 0
>>> global(s) using 255840virt/77132res/7132shr/70552data kb, in
>>> 510usr/20sys/577real ms.
>>> Pass 3: translated to C into
>>> "/tmp/stap3Qunba/stap_2690192fea570fb7bba78c7ed7fa1e0d_898_src.c" using
>>> 255840virt/77392res/7392shr/70552data kb, in 10usr/0sys/4real ms.
>>> Pass 4: compiled C into "stap_2690192fea570fb7bba78c7ed7fa1e0d_898.ko" in
>>> 5020usr/640sys/7105real ms.
>>> Pass 5: starting run.
>>> (no error)
>>
>> Yeah, I guess you can see some warning messages in dmesg (by
>> arm_kprobe) at this point.
>
> Ah, you're right:
>
> Jun 19 08:03:10 treble kernel: [ cut here ]
> Jun 19 08:03:10 treble kernel: WARNING: CPU: 1 PID: 17991 at
> kernel/kprobes.c:953 arm_kprobe+0xa7/0xe0()
> Jun 19 08:03:10 treble kernel: Failed to init kprobe-ftrace (-16)
> Jun 19 08:03:10 treble kernel: Modules linked in:
> stap_1faf9cc0ccf85c0d203c74ab6f604b_17991(OE) ...defra
> Jun 19 08:03:10 treble kernel: videobuf2_vmalloc serio_raw microcode
> sdhci_pci bluetooth videobuf2_m...
> Jun 19 08:03:10 treble kernel: CPU: 1 PID: 17991 Comm: stapio Tainted: G
> U W OE 3.15.0+ #1
> Jun 19 08:03:10 treble kernel: Hardware name: LENOVO 2356BH8/2356BH8, BIOS
> G7ET63WW (2.05 ) 11/12/2012
> Jun 19 08:03:10 treble kernel: 9023f19e
> 8803dcce7d80 816f31fd
> Jun 19 08:03:10 treble kernel: 8803dcce7dc8 8803dcce7db8
> 8108914d a08248e0
> Jun 19 08:03:10 treble kernel: a08248f0
>
> Jun 19 08:03:10 treble kernel: Call Trace:
> Jun 19 08:03:10 treble kernel: [] dump_stack+0x45/0x56
> Jun 19 08:03:10 treble kernel: []
> warn_slowpath_common+0x7d/0xa0
> Jun 19 08:03:10 treble kernel: []
> warn_slowpath_fmt+0x5c/0x80
> Jun 19 08:03:10 treble kernel: [] arm_kprobe+0xa7/0xe0
> Jun 19 08:03:10 treble kernel: []
> register_kprobe+0x557/0x5d0
> Jun 19 08:03:10 treble kernel: [] ?
> meminfo_proc_open+0x30/0x30
> Jun 19 08:03:10 treble kernel: []
> _stp_ctl_write_cmd+0x8d5/0x930 [stap_1faf9c...7991]
> Jun 19 08:03:10 treble kernel: [] vfs_write+0xba/0x1e0
> Jun 19 08:03:10 treble kernel: [] SyS_write+0x55/0xd0
> Jun 19 08:03:10 treble kernel: []
> system_call_fastpath+0x16/0x1b
> Jun 19 08:03:10 treble kernel: ---[ end trace 19615ed55413a30d ]---
>
> Why not change arm_kprobe() to return an error?
Actually, arm_kprobe() is widely used at deeper point.
And the 3rd patch will solve the problem. So I decided just adding
IPMODIFY flag at the 2nd patch.
>>> With all 3 patches, I expected kprobes and kpatch to be able to ftrace
>>> the same function. But when I tried to kpatch the function after it was
>>> kprobed, I got the following oops in stap:
>>>
>>> [ 455.842797] BUG: unable to handle kernel NULL pointer dereference at
>>> 0020
>>> [ 455.843388] IP: [] _stp_module_notifier+0x1e/0x320
>>> [stap_2690192fea570fb7bba78c7ed7fa1e_20189]
>>
>> Hmm, since this happens in _stp_module_notifier() which is a code in
>> systemtap,
>> I guess it's a systemtap problem.
>>
>> Could you test it with kprobe-tracer as below?
>>
>> # (do something kpatch related activation)
>> # echo p meminfo_proc_show > /sys/kernel/debug/tracing/kprobe_events
>> # echo 1 > /sys/kernel/debug/tracing/events/kprobe/enable
>
> That worked, thanks.
Moreover, I couldn't reproduced the stap case on my fedora20.
Perhaps, Maybe a different version of systemtap I used.
Thank you,
--
Masami HIRAMATSU
Software Platform Research
Re: [PATCH -tip v2 0/3] ftrace, kprobes: Introduce IPMODIFY flag for ftrace_ops to detect conflicts
On Thu, Jun 19, 2014 at 02:03:31PM +0900, Masami Hiramatsu wrote:
> (2014/06/19 11:08), Josh Poimboeuf wrote:
> > On Tue, Jun 17, 2014 at 11:04:36AM +, Masami Hiramatsu wrote:
> >> Hi,
> >>
> >> Here is the version 2 of the series of patches which introduces
> >> IPMODIFY flag for ftrace_ops to detect conflicts of ftrace users
> >> who can modify regs->ip in their handler.
> >> In this version, I fixed some bugs in previous version and
> >> added a patch which made kprobe itself free from IPMODIFY
> >> except for jprobe.
> >
> > Hi Masami,
> >
> > This seems better, but I still saw a few issues. I'm not sure if the
> > issues are specific to stap or kprobes. For the following issues I used
> > this command to set a kprobe:
> >
> > stap -v -e 'probe kernel.function("meminfo_proc_show")
> > {printf("meminfo_proc_show called\n");}'
> >
> > With patches 1-2, when I used stap to kprobe the function after it was
> > already kpatched, stap didn't return an error and instead acted like it
> > succeeded (though the probe didn't work):
> >
> > $ sudo stap -v -e 'probe kernel.function("meminfo_proc_show")
> > {printf("meminfo_proc_show called\n");}'
> > Pass 1: parsed user script and 112 library script(s) using
> > 221516virt/41612res/6028shr/36228data kb, in 130usr/0sys/132real ms.
> > Pass 2: analyzed script: 1 probe(s), 0 function(s), 0 embed(s), 0
> > global(s) using 255840virt/77132res/7132shr/70552data kb, in
> > 510usr/20sys/577real ms.
> > Pass 3: translated to C into
> > "/tmp/stap3Qunba/stap_2690192fea570fb7bba78c7ed7fa1e0d_898_src.c" using
> > 255840virt/77392res/7392shr/70552data kb, in 10usr/0sys/4real ms.
> > Pass 4: compiled C into "stap_2690192fea570fb7bba78c7ed7fa1e0d_898.ko" in
> > 5020usr/640sys/7105real ms.
> > Pass 5: starting run.
> > (no error)
>
> Yeah, I guess you can see some warning messages in dmesg (by
> arm_kprobe) at this point.
Ah, you're right:
Jun 19 08:03:10 treble kernel: [ cut here ]
Jun 19 08:03:10 treble kernel: WARNING: CPU: 1 PID: 17991 at
kernel/kprobes.c:953 arm_kprobe+0xa7/0xe0()
Jun 19 08:03:10 treble kernel: Failed to init kprobe-ftrace (-16)
Jun 19 08:03:10 treble kernel: Modules linked in:
stap_1faf9cc0ccf85c0d203c74ab6f604b_17991(OE) ...defra
Jun 19 08:03:10 treble kernel: videobuf2_vmalloc serio_raw microcode
sdhci_pci bluetooth videobuf2_m...
Jun 19 08:03:10 treble kernel: CPU: 1 PID: 17991 Comm: stapio Tainted: G
U W OE 3.15.0+ #1
Jun 19 08:03:10 treble kernel: Hardware name: LENOVO 2356BH8/2356BH8, BIOS
G7ET63WW (2.05 ) 11/12/2012
Jun 19 08:03:10 treble kernel: 9023f19e
8803dcce7d80 816f31fd
Jun 19 08:03:10 treble kernel: 8803dcce7dc8 8803dcce7db8
8108914d a08248e0
Jun 19 08:03:10 treble kernel: a08248f0
Jun 19 08:03:10 treble kernel: Call Trace:
Jun 19 08:03:10 treble kernel: [] dump_stack+0x45/0x56
Jun 19 08:03:10 treble kernel: []
warn_slowpath_common+0x7d/0xa0
Jun 19 08:03:10 treble kernel: []
warn_slowpath_fmt+0x5c/0x80
Jun 19 08:03:10 treble kernel: [] arm_kprobe+0xa7/0xe0
Jun 19 08:03:10 treble kernel: []
register_kprobe+0x557/0x5d0
Jun 19 08:03:10 treble kernel: [] ?
meminfo_proc_open+0x30/0x30
Jun 19 08:03:10 treble kernel: []
_stp_ctl_write_cmd+0x8d5/0x930 [stap_1faf9c...7991]
Jun 19 08:03:10 treble kernel: [] vfs_write+0xba/0x1e0
Jun 19 08:03:10 treble kernel: [] SyS_write+0x55/0xd0
Jun 19 08:03:10 treble kernel: []
system_call_fastpath+0x16/0x1b
Jun 19 08:03:10 treble kernel: ---[ end trace 19615ed55413a30d ]---
Why not change arm_kprobe() to return an error?
>
> >
> > With all 3 patches, I expected kprobes and kpatch to be able to ftrace
> > the same function. But when I tried to kpatch the function after it was
> > kprobed, I got the following oops in stap:
> >
> > [ 455.842797] BUG: unable to handle kernel NULL pointer dereference at
> > 0020
> > [ 455.843388] IP: [] _stp_module_notifier+0x1e/0x320
> > [stap_2690192fea570fb7bba78c7ed7fa1e_20189]
>
> Hmm, since this happens in _stp_module_notifier() which is a code in
> systemtap,
> I guess it's a systemtap problem.
>
> Could you test it with kprobe-tracer as below?
>
> # (do something kpatch related activation)
> # echo p meminfo_proc_show > /sys/kernel/debug/tracing/kprobe_events
> # echo 1 > /sys/kernel/debug/tracing/events/kprobe/enable
That worked, thanks.
>
> Thank you,
>
> > [ 455.844011] PGD 33f898067 PUD 422083067 PMD 0
> > [ 455.844638] Oops: [#1] SMP
> > [ 455.845255] Modules linked in: kpatch(OE+)
> > stap_2690192fea570fb7bba78c7ed7fa1e_20189(OE) rfcomm ipt_MASQUERADE fuse
> > ccm xt_CHECKSUM tun ip6t_rpfilter ip6t_REJECT xt_conntrack ebtable_nat
> > ebtable_broute bridge stp llc ebtable_filter ebtables ip6table_nat
> > nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle
Re: [PATCH -tip v2 0/3] ftrace, kprobes: Introduce IPMODIFY flag for ftrace_ops to detect conflicts
On Thu, Jun 19, 2014 at 02:03:31PM +0900, Masami Hiramatsu wrote:
(2014/06/19 11:08), Josh Poimboeuf wrote:
On Tue, Jun 17, 2014 at 11:04:36AM +, Masami Hiramatsu wrote:
Hi,
Here is the version 2 of the series of patches which introduces
IPMODIFY flag for ftrace_ops to detect conflicts of ftrace users
who can modify regs-ip in their handler.
In this version, I fixed some bugs in previous version and
added a patch which made kprobe itself free from IPMODIFY
except for jprobe.
Hi Masami,
This seems better, but I still saw a few issues. I'm not sure if the
issues are specific to stap or kprobes. For the following issues I used
this command to set a kprobe:
stap -v -e 'probe kernel.function(meminfo_proc_show)
{printf(meminfo_proc_show called\n);}'
With patches 1-2, when I used stap to kprobe the function after it was
already kpatched, stap didn't return an error and instead acted like it
succeeded (though the probe didn't work):
$ sudo stap -v -e 'probe kernel.function(meminfo_proc_show)
{printf(meminfo_proc_show called\n);}'
Pass 1: parsed user script and 112 library script(s) using
221516virt/41612res/6028shr/36228data kb, in 130usr/0sys/132real ms.
Pass 2: analyzed script: 1 probe(s), 0 function(s), 0 embed(s), 0
global(s) using 255840virt/77132res/7132shr/70552data kb, in
510usr/20sys/577real ms.
Pass 3: translated to C into
/tmp/stap3Qunba/stap_2690192fea570fb7bba78c7ed7fa1e0d_898_src.c using
255840virt/77392res/7392shr/70552data kb, in 10usr/0sys/4real ms.
Pass 4: compiled C into stap_2690192fea570fb7bba78c7ed7fa1e0d_898.ko in
5020usr/640sys/7105real ms.
Pass 5: starting run.
(no error)
Yeah, I guess you can see some warning messages in dmesg (by
arm_kprobe) at this point.
Ah, you're right:
Jun 19 08:03:10 treble kernel: [ cut here ]
Jun 19 08:03:10 treble kernel: WARNING: CPU: 1 PID: 17991 at
kernel/kprobes.c:953 arm_kprobe+0xa7/0xe0()
Jun 19 08:03:10 treble kernel: Failed to init kprobe-ftrace (-16)
Jun 19 08:03:10 treble kernel: Modules linked in:
stap_1faf9cc0ccf85c0d203c74ab6f604b_17991(OE) ...defra
Jun 19 08:03:10 treble kernel: videobuf2_vmalloc serio_raw microcode
sdhci_pci bluetooth videobuf2_m...
Jun 19 08:03:10 treble kernel: CPU: 1 PID: 17991 Comm: stapio Tainted: G
U W OE 3.15.0+ #1
Jun 19 08:03:10 treble kernel: Hardware name: LENOVO 2356BH8/2356BH8, BIOS
G7ET63WW (2.05 ) 11/12/2012
Jun 19 08:03:10 treble kernel: 9023f19e
8803dcce7d80 816f31fd
Jun 19 08:03:10 treble kernel: 8803dcce7dc8 8803dcce7db8
8108914d a08248e0
Jun 19 08:03:10 treble kernel: a08248f0
Jun 19 08:03:10 treble kernel: Call Trace:
Jun 19 08:03:10 treble kernel: [816f31fd] dump_stack+0x45/0x56
Jun 19 08:03:10 treble kernel: [8108914d]
warn_slowpath_common+0x7d/0xa0
Jun 19 08:03:10 treble kernel: [810891cc]
warn_slowpath_fmt+0x5c/0x80
Jun 19 08:03:10 treble kernel: [816ff9d7] arm_kprobe+0xa7/0xe0
Jun 19 08:03:10 treble kernel: [817007f7]
register_kprobe+0x557/0x5d0
Jun 19 08:03:10 treble kernel: [81254db0] ?
meminfo_proc_open+0x30/0x30
Jun 19 08:03:10 treble kernel: [a081fc95]
_stp_ctl_write_cmd+0x8d5/0x930 [stap_1faf9c...7991]
Jun 19 08:03:10 treble kernel: [811e5dba] vfs_write+0xba/0x1e0
Jun 19 08:03:10 treble kernel: [811e6975] SyS_write+0x55/0xd0
Jun 19 08:03:10 treble kernel: [81703179]
system_call_fastpath+0x16/0x1b
Jun 19 08:03:10 treble kernel: ---[ end trace 19615ed55413a30d ]---
Why not change arm_kprobe() to return an error?
With all 3 patches, I expected kprobes and kpatch to be able to ftrace
the same function. But when I tried to kpatch the function after it was
kprobed, I got the following oops in stap:
[ 455.842797] BUG: unable to handle kernel NULL pointer dereference at
0020
[ 455.843388] IP: [a0833d1e] _stp_module_notifier+0x1e/0x320
[stap_2690192fea570fb7bba78c7ed7fa1e_20189]
Hmm, since this happens in _stp_module_notifier() which is a code in
systemtap,
I guess it's a systemtap problem.
Could you test it with kprobe-tracer as below?
# (do something kpatch related activation)
# echo p meminfo_proc_show /sys/kernel/debug/tracing/kprobe_events
# echo 1 /sys/kernel/debug/tracing/events/kprobe/enable
That worked, thanks.
Thank you,
[ 455.844011] PGD 33f898067 PUD 422083067 PMD 0
[ 455.844638] Oops: [#1] SMP
[ 455.845255] Modules linked in: kpatch(OE+)
stap_2690192fea570fb7bba78c7ed7fa1e_20189(OE) rfcomm ipt_MASQUERADE fuse
ccm xt_CHECKSUM tun ip6t_rpfilter ip6t_REJECT xt_conntrack ebtable_nat
ebtable_broute bridge stp llc ebtable_filter ebtables ip6table_nat
nf_conntrack_ipv6
Re: [PATCH -tip v2 0/3] ftrace, kprobes: Introduce IPMODIFY flag for ftrace_ops to detect conflicts
(2014/06/19 23:18), Josh Poimboeuf wrote:
On Thu, Jun 19, 2014 at 02:03:31PM +0900, Masami Hiramatsu wrote:
(2014/06/19 11:08), Josh Poimboeuf wrote:
On Tue, Jun 17, 2014 at 11:04:36AM +, Masami Hiramatsu wrote:
Hi,
Here is the version 2 of the series of patches which introduces
IPMODIFY flag for ftrace_ops to detect conflicts of ftrace users
who can modify regs-ip in their handler.
In this version, I fixed some bugs in previous version and
added a patch which made kprobe itself free from IPMODIFY
except for jprobe.
Hi Masami,
This seems better, but I still saw a few issues. I'm not sure if the
issues are specific to stap or kprobes. For the following issues I used
this command to set a kprobe:
stap -v -e 'probe kernel.function(meminfo_proc_show)
{printf(meminfo_proc_show called\n);}'
With patches 1-2, when I used stap to kprobe the function after it was
already kpatched, stap didn't return an error and instead acted like it
succeeded (though the probe didn't work):
$ sudo stap -v -e 'probe kernel.function(meminfo_proc_show)
{printf(meminfo_proc_show called\n);}'
Pass 1: parsed user script and 112 library script(s) using
221516virt/41612res/6028shr/36228data kb, in 130usr/0sys/132real ms.
Pass 2: analyzed script: 1 probe(s), 0 function(s), 0 embed(s), 0
global(s) using 255840virt/77132res/7132shr/70552data kb, in
510usr/20sys/577real ms.
Pass 3: translated to C into
/tmp/stap3Qunba/stap_2690192fea570fb7bba78c7ed7fa1e0d_898_src.c using
255840virt/77392res/7392shr/70552data kb, in 10usr/0sys/4real ms.
Pass 4: compiled C into stap_2690192fea570fb7bba78c7ed7fa1e0d_898.ko in
5020usr/640sys/7105real ms.
Pass 5: starting run.
(no error)
Yeah, I guess you can see some warning messages in dmesg (by
arm_kprobe) at this point.
Ah, you're right:
Jun 19 08:03:10 treble kernel: [ cut here ]
Jun 19 08:03:10 treble kernel: WARNING: CPU: 1 PID: 17991 at
kernel/kprobes.c:953 arm_kprobe+0xa7/0xe0()
Jun 19 08:03:10 treble kernel: Failed to init kprobe-ftrace (-16)
Jun 19 08:03:10 treble kernel: Modules linked in:
stap_1faf9cc0ccf85c0d203c74ab6f604b_17991(OE) ...defra
Jun 19 08:03:10 treble kernel: videobuf2_vmalloc serio_raw microcode
sdhci_pci bluetooth videobuf2_m...
Jun 19 08:03:10 treble kernel: CPU: 1 PID: 17991 Comm: stapio Tainted: G
U W OE 3.15.0+ #1
Jun 19 08:03:10 treble kernel: Hardware name: LENOVO 2356BH8/2356BH8, BIOS
G7ET63WW (2.05 ) 11/12/2012
Jun 19 08:03:10 treble kernel: 9023f19e
8803dcce7d80 816f31fd
Jun 19 08:03:10 treble kernel: 8803dcce7dc8 8803dcce7db8
8108914d a08248e0
Jun 19 08:03:10 treble kernel: a08248f0
Jun 19 08:03:10 treble kernel: Call Trace:
Jun 19 08:03:10 treble kernel: [816f31fd] dump_stack+0x45/0x56
Jun 19 08:03:10 treble kernel: [8108914d]
warn_slowpath_common+0x7d/0xa0
Jun 19 08:03:10 treble kernel: [810891cc]
warn_slowpath_fmt+0x5c/0x80
Jun 19 08:03:10 treble kernel: [816ff9d7] arm_kprobe+0xa7/0xe0
Jun 19 08:03:10 treble kernel: [817007f7]
register_kprobe+0x557/0x5d0
Jun 19 08:03:10 treble kernel: [81254db0] ?
meminfo_proc_open+0x30/0x30
Jun 19 08:03:10 treble kernel: [a081fc95]
_stp_ctl_write_cmd+0x8d5/0x930 [stap_1faf9c...7991]
Jun 19 08:03:10 treble kernel: [811e5dba] vfs_write+0xba/0x1e0
Jun 19 08:03:10 treble kernel: [811e6975] SyS_write+0x55/0xd0
Jun 19 08:03:10 treble kernel: [81703179]
system_call_fastpath+0x16/0x1b
Jun 19 08:03:10 treble kernel: ---[ end trace 19615ed55413a30d ]---
Why not change arm_kprobe() to return an error?
Actually, arm_kprobe() is widely used at deeper point.
And the 3rd patch will solve the problem. So I decided just adding
IPMODIFY flag at the 2nd patch.
With all 3 patches, I expected kprobes and kpatch to be able to ftrace
the same function. But when I tried to kpatch the function after it was
kprobed, I got the following oops in stap:
[ 455.842797] BUG: unable to handle kernel NULL pointer dereference at
0020
[ 455.843388] IP: [a0833d1e] _stp_module_notifier+0x1e/0x320
[stap_2690192fea570fb7bba78c7ed7fa1e_20189]
Hmm, since this happens in _stp_module_notifier() which is a code in
systemtap,
I guess it's a systemtap problem.
Could you test it with kprobe-tracer as below?
# (do something kpatch related activation)
# echo p meminfo_proc_show /sys/kernel/debug/tracing/kprobe_events
# echo 1 /sys/kernel/debug/tracing/events/kprobe/enable
That worked, thanks.
Moreover, I couldn't reproduced the stap case on my fedora20.
Perhaps, Maybe a different version of systemtap I used.
Thank you,
--
Masami HIRAMATSU
Software Platform Research Dept. Linux Technology Research Center
Hitachi, Ltd.,
Re: Re: [PATCH -tip v2 0/3] ftrace, kprobes: Introduce IPMODIFY flag for ftrace_ops to detect conflicts
(2014/06/19 11:08), Josh Poimboeuf wrote:
> On Tue, Jun 17, 2014 at 11:04:36AM +, Masami Hiramatsu wrote:
>> Hi,
>>
>> Here is the version 2 of the series of patches which introduces
>> IPMODIFY flag for ftrace_ops to detect conflicts of ftrace users
>> who can modify regs->ip in their handler.
>> In this version, I fixed some bugs in previous version and
>> added a patch which made kprobe itself free from IPMODIFY
>> except for jprobe.
>
> Hi Masami,
>
> This seems better, but I still saw a few issues. I'm not sure if the
> issues are specific to stap or kprobes. For the following issues I used
> this command to set a kprobe:
>
> stap -v -e 'probe kernel.function("meminfo_proc_show")
> {printf("meminfo_proc_show called\n");}'
>
> With patches 1-2, when I used stap to kprobe the function after it was
> already kpatched, stap didn't return an error and instead acted like it
> succeeded (though the probe didn't work):
>
> $ sudo stap -v -e 'probe kernel.function("meminfo_proc_show")
> {printf("meminfo_proc_show called\n");}'
> Pass 1: parsed user script and 112 library script(s) using
> 221516virt/41612res/6028shr/36228data kb, in 130usr/0sys/132real ms.
> Pass 2: analyzed script: 1 probe(s), 0 function(s), 0 embed(s), 0 global(s)
> using 255840virt/77132res/7132shr/70552data kb, in 510usr/20sys/577real ms.
> Pass 3: translated to C into
> "/tmp/stap3Qunba/stap_2690192fea570fb7bba78c7ed7fa1e0d_898_src.c" using
> 255840virt/77392res/7392shr/70552data kb, in 10usr/0sys/4real ms.
> Pass 4: compiled C into "stap_2690192fea570fb7bba78c7ed7fa1e0d_898.ko" in
> 5020usr/640sys/7105real ms.
> Pass 5: starting run.
> (no error)
Yeah, I guess you can see some warning messages in dmesg (by
arm_kprobe) at this point.
>
> With all 3 patches, I expected kprobes and kpatch to be able to ftrace
> the same function. But when I tried to kpatch the function after it was
> kprobed, I got the following oops in stap:
>
> [ 455.842797] BUG: unable to handle kernel NULL pointer dereference at
> 0020
> [ 455.843388] IP: [] _stp_module_notifier+0x1e/0x320
> [stap_2690192fea570fb7bba78c7ed7fa1e_20189]
Hmm, since this happens in _stp_module_notifier() which is a code in systemtap,
I guess it's a systemtap problem.
Could you test it with kprobe-tracer as below?
# (do something kpatch related activation)
# echo p meminfo_proc_show > /sys/kernel/debug/tracing/kprobe_events
# echo 1 > /sys/kernel/debug/tracing/events/kprobe/enable
Thank you,
> [ 455.844011] PGD 33f898067 PUD 422083067 PMD 0
> [ 455.844638] Oops: [#1] SMP
> [ 455.845255] Modules linked in: kpatch(OE+)
> stap_2690192fea570fb7bba78c7ed7fa1e_20189(OE) rfcomm ipt_MASQUERADE fuse ccm
> xt_CHECKSUM tun ip6t_rpfilter ip6t_REJECT xt_conntrack ebtable_nat
> ebtable_broute bridge stp llc ebtable_filter ebtables ip6table_nat
> nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle
> ip6table_security ip6table_raw ip6table_filter ip6_tables iptable_nat
> nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack
> iptable_mangle iptable_security iptable_raw bnep arc4 iwldvm mac80211
> iTCO_wdt snd_hda_codec_hdmi iTCO_vendor_support x86_pkg_temp_thermal
> snd_hda_codec_realtek coretemp iwlwifi snd_hda_codec_generic kvm_intel
> snd_hda_intel kvm uvcvideo snd_hda_controller cfg80211 snd_hda_codec btusb
> videobuf2_vmalloc bluetooth videobuf2_memops snd_hwdep snd_seq nfsd
> videobuf2_core
> [ 455.848272] v4l2_common videodev snd_seq_device e1000e microcode
> snd_pcm sdhci_pci media joydev sdhci serio_raw i2c_i801 pcspkr mmc_core
> thinkpad_acpi mei_me snd_timer auth_rpcgss mei snd lpc_ich ptp mfd_core
> shpchp nfs_acl lockd pps_core wmi tpm_tis soundcore tpm rfkill sunrpc
> dm_crypt i915 i2c_algo_bit drm_kms_helper drm crct10dif_pclmul crc32_pclmul
> crc32c_intel ghash_clmulni_intel i2c_core video
> [ 455.850887] CPU: 3 PID: 19857 Comm: insmod Tainted: GW OE
> 3.16.0-rc1+ #2
> [ 455.851768] Hardware name: LENOVO 2356BH8/2356BH8, BIOS G7ET63WW (2.05 )
> 11/12/2012
> [ 455.852638] task: 880095d65460 ti: 88039d1d4000 task.ti:
> 88039d1d4000
> [ 455.853456] RIP: 0010:[] []
> _stp_module_notifier+0x1e/0x320 [stap_2690192fea570fb7bba78c7ed7fa1e_20189]
> [ 455.854335] RSP: 0018:88039d1d7ce0 EFLAGS: 00010246
> [ 455.855212] RAX: a0837f50 RBX: RCX:
>
> [ 455.856109] RDX: a08400e0 RSI: 0001 RDI:
> a0837f50
> [ 455.856986] RBP: 88039d1d7d00 R08: R09:
>
> [ 455.857880] R10: 0001 R11: c9001aed2d8f R12:
> 81c593e0
> [ 455.858761] R13: 0001 R14: a08400e0 R15:
>
> [ 455.859640] FS: 7feac5f10740() GS:88043e2c()
> knlGS:
> [ 455.860523] CS: 0010 DS: ES: CR0: 80050033
> [ 455.861403] CR2:
Re: [PATCH -tip v2 0/3] ftrace, kprobes: Introduce IPMODIFY flag for ftrace_ops to detect conflicts
On Tue, Jun 17, 2014 at 11:04:36AM +, Masami Hiramatsu wrote:
> Hi,
>
> Here is the version 2 of the series of patches which introduces
> IPMODIFY flag for ftrace_ops to detect conflicts of ftrace users
> who can modify regs->ip in their handler.
> In this version, I fixed some bugs in previous version and
> added a patch which made kprobe itself free from IPMODIFY
> except for jprobe.
Hi Masami,
This seems better, but I still saw a few issues. I'm not sure if the
issues are specific to stap or kprobes. For the following issues I used
this command to set a kprobe:
stap -v -e 'probe kernel.function("meminfo_proc_show")
{printf("meminfo_proc_show called\n");}'
With patches 1-2, when I used stap to kprobe the function after it was
already kpatched, stap didn't return an error and instead acted like it
succeeded (though the probe didn't work):
$ sudo stap -v -e 'probe kernel.function("meminfo_proc_show")
{printf("meminfo_proc_show called\n");}'
Pass 1: parsed user script and 112 library script(s) using
221516virt/41612res/6028shr/36228data kb, in 130usr/0sys/132real ms.
Pass 2: analyzed script: 1 probe(s), 0 function(s), 0 embed(s), 0 global(s)
using 255840virt/77132res/7132shr/70552data kb, in 510usr/20sys/577real ms.
Pass 3: translated to C into
"/tmp/stap3Qunba/stap_2690192fea570fb7bba78c7ed7fa1e0d_898_src.c" using
255840virt/77392res/7392shr/70552data kb, in 10usr/0sys/4real ms.
Pass 4: compiled C into "stap_2690192fea570fb7bba78c7ed7fa1e0d_898.ko" in
5020usr/640sys/7105real ms.
Pass 5: starting run.
(no error)
With all 3 patches, I expected kprobes and kpatch to be able to ftrace
the same function. But when I tried to kpatch the function after it was
kprobed, I got the following oops in stap:
[ 455.842797] BUG: unable to handle kernel NULL pointer dereference at
0020
[ 455.843388] IP: [] _stp_module_notifier+0x1e/0x320
[stap_2690192fea570fb7bba78c7ed7fa1e_20189]
[ 455.844011] PGD 33f898067 PUD 422083067 PMD 0
[ 455.844638] Oops: [#1] SMP
[ 455.845255] Modules linked in: kpatch(OE+)
stap_2690192fea570fb7bba78c7ed7fa1e_20189(OE) rfcomm ipt_MASQUERADE fuse ccm
xt_CHECKSUM tun ip6t_rpfilter ip6t_REJECT xt_conntrack ebtable_nat
ebtable_broute bridge stp llc ebtable_filter ebtables ip6table_nat
nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security
ip6table_raw ip6table_filter ip6_tables iptable_nat nf_conntrack_ipv4
nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security
iptable_raw bnep arc4 iwldvm mac80211 iTCO_wdt snd_hda_codec_hdmi
iTCO_vendor_support x86_pkg_temp_thermal snd_hda_codec_realtek coretemp iwlwifi
snd_hda_codec_generic kvm_intel snd_hda_intel kvm uvcvideo snd_hda_controller
cfg80211 snd_hda_codec btusb videobuf2_vmalloc bluetooth videobuf2_memops
snd_hwdep snd_seq nfsd videobuf2_core
[ 455.848272] v4l2_common videodev snd_seq_device e1000e microcode snd_pcm
sdhci_pci media joydev sdhci serio_raw i2c_i801 pcspkr mmc_core thinkpad_acpi
mei_me snd_timer auth_rpcgss mei snd lpc_ich ptp mfd_core shpchp nfs_acl lockd
pps_core wmi tpm_tis soundcore tpm rfkill sunrpc dm_crypt i915 i2c_algo_bit
drm_kms_helper drm crct10dif_pclmul crc32_pclmul crc32c_intel
ghash_clmulni_intel i2c_core video
[ 455.850887] CPU: 3 PID: 19857 Comm: insmod Tainted: GW OE
3.16.0-rc1+ #2
[ 455.851768] Hardware name: LENOVO 2356BH8/2356BH8, BIOS G7ET63WW (2.05 )
11/12/2012
[ 455.852638] task: 880095d65460 ti: 88039d1d4000 task.ti:
88039d1d4000
[ 455.853456] RIP: 0010:[] []
_stp_module_notifier+0x1e/0x320 [stap_2690192fea570fb7bba78c7ed7fa1e_20189]
[ 455.854335] RSP: 0018:88039d1d7ce0 EFLAGS: 00010246
[ 455.855212] RAX: a0837f50 RBX: RCX:
[ 455.856109] RDX: a08400e0 RSI: 0001 RDI:
a0837f50
[ 455.856986] RBP: 88039d1d7d00 R08: R09:
[ 455.857880] R10: 0001 R11: c9001aed2d8f R12:
81c593e0
[ 455.858761] R13: 0001 R14: a08400e0 R15:
[ 455.859640] FS: 7feac5f10740() GS:88043e2c()
knlGS:
[ 455.860523] CS: 0010 DS: ES: CR0: 80050033
[ 455.861403] CR2: 0020 CR3: 0004224e7000 CR4:
001407e0
[ 455.862309] Stack:
[ 455.863236] fffd 81c593e0 0001
a08400e0
[ 455.864163] 88039d1d7d38 810b45bc 81c557c0
[ 455.865103] 0001 a08400e0
88039d1d7d78
[ 455.866067] Call Trace:
[ 455.867100] [] notifier_call_chain+0x4c/0x70
[ 455.868202] [] __blocking_notifier_call_chain+0x4d/0x70
[ 455.869155] [] blocking_notifier_call_chain+0x16/0x20
[ 455.870111] [] load_module+0x1aac/0x25f0
[ 455.871067] [] ? kernel_read+0x50/0x80
[ 455.872022] []
Re: [PATCH -tip v2 0/3] ftrace, kprobes: Introduce IPMODIFY flag for ftrace_ops to detect conflicts
On Tue, Jun 17, 2014 at 11:04:36AM +, Masami Hiramatsu wrote:
Hi,
Here is the version 2 of the series of patches which introduces
IPMODIFY flag for ftrace_ops to detect conflicts of ftrace users
who can modify regs-ip in their handler.
In this version, I fixed some bugs in previous version and
added a patch which made kprobe itself free from IPMODIFY
except for jprobe.
Hi Masami,
This seems better, but I still saw a few issues. I'm not sure if the
issues are specific to stap or kprobes. For the following issues I used
this command to set a kprobe:
stap -v -e 'probe kernel.function(meminfo_proc_show)
{printf(meminfo_proc_show called\n);}'
With patches 1-2, when I used stap to kprobe the function after it was
already kpatched, stap didn't return an error and instead acted like it
succeeded (though the probe didn't work):
$ sudo stap -v -e 'probe kernel.function(meminfo_proc_show)
{printf(meminfo_proc_show called\n);}'
Pass 1: parsed user script and 112 library script(s) using
221516virt/41612res/6028shr/36228data kb, in 130usr/0sys/132real ms.
Pass 2: analyzed script: 1 probe(s), 0 function(s), 0 embed(s), 0 global(s)
using 255840virt/77132res/7132shr/70552data kb, in 510usr/20sys/577real ms.
Pass 3: translated to C into
/tmp/stap3Qunba/stap_2690192fea570fb7bba78c7ed7fa1e0d_898_src.c using
255840virt/77392res/7392shr/70552data kb, in 10usr/0sys/4real ms.
Pass 4: compiled C into stap_2690192fea570fb7bba78c7ed7fa1e0d_898.ko in
5020usr/640sys/7105real ms.
Pass 5: starting run.
(no error)
With all 3 patches, I expected kprobes and kpatch to be able to ftrace
the same function. But when I tried to kpatch the function after it was
kprobed, I got the following oops in stap:
[ 455.842797] BUG: unable to handle kernel NULL pointer dereference at
0020
[ 455.843388] IP: [a0833d1e] _stp_module_notifier+0x1e/0x320
[stap_2690192fea570fb7bba78c7ed7fa1e_20189]
[ 455.844011] PGD 33f898067 PUD 422083067 PMD 0
[ 455.844638] Oops: [#1] SMP
[ 455.845255] Modules linked in: kpatch(OE+)
stap_2690192fea570fb7bba78c7ed7fa1e_20189(OE) rfcomm ipt_MASQUERADE fuse ccm
xt_CHECKSUM tun ip6t_rpfilter ip6t_REJECT xt_conntrack ebtable_nat
ebtable_broute bridge stp llc ebtable_filter ebtables ip6table_nat
nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security
ip6table_raw ip6table_filter ip6_tables iptable_nat nf_conntrack_ipv4
nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security
iptable_raw bnep arc4 iwldvm mac80211 iTCO_wdt snd_hda_codec_hdmi
iTCO_vendor_support x86_pkg_temp_thermal snd_hda_codec_realtek coretemp iwlwifi
snd_hda_codec_generic kvm_intel snd_hda_intel kvm uvcvideo snd_hda_controller
cfg80211 snd_hda_codec btusb videobuf2_vmalloc bluetooth videobuf2_memops
snd_hwdep snd_seq nfsd videobuf2_core
[ 455.848272] v4l2_common videodev snd_seq_device e1000e microcode snd_pcm
sdhci_pci media joydev sdhci serio_raw i2c_i801 pcspkr mmc_core thinkpad_acpi
mei_me snd_timer auth_rpcgss mei snd lpc_ich ptp mfd_core shpchp nfs_acl lockd
pps_core wmi tpm_tis soundcore tpm rfkill sunrpc dm_crypt i915 i2c_algo_bit
drm_kms_helper drm crct10dif_pclmul crc32_pclmul crc32c_intel
ghash_clmulni_intel i2c_core video
[ 455.850887] CPU: 3 PID: 19857 Comm: insmod Tainted: GW OE
3.16.0-rc1+ #2
[ 455.851768] Hardware name: LENOVO 2356BH8/2356BH8, BIOS G7ET63WW (2.05 )
11/12/2012
[ 455.852638] task: 880095d65460 ti: 88039d1d4000 task.ti:
88039d1d4000
[ 455.853456] RIP: 0010:[a0833d1e] [a0833d1e]
_stp_module_notifier+0x1e/0x320 [stap_2690192fea570fb7bba78c7ed7fa1e_20189]
[ 455.854335] RSP: 0018:88039d1d7ce0 EFLAGS: 00010246
[ 455.855212] RAX: a0837f50 RBX: RCX:
[ 455.856109] RDX: a08400e0 RSI: 0001 RDI:
a0837f50
[ 455.856986] RBP: 88039d1d7d00 R08: R09:
[ 455.857880] R10: 0001 R11: c9001aed2d8f R12:
81c593e0
[ 455.858761] R13: 0001 R14: a08400e0 R15:
[ 455.859640] FS: 7feac5f10740() GS:88043e2c()
knlGS:
[ 455.860523] CS: 0010 DS: ES: CR0: 80050033
[ 455.861403] CR2: 0020 CR3: 0004224e7000 CR4:
001407e0
[ 455.862309] Stack:
[ 455.863236] fffd 81c593e0 0001
a08400e0
[ 455.864163] 88039d1d7d38 810b45bc 81c557c0
[ 455.865103] 0001 a08400e0
88039d1d7d78
[ 455.866067] Call Trace:
[ 455.867100] [810b45bc] notifier_call_chain+0x4c/0x70
[ 455.868202] [810b491d] __blocking_notifier_call_chain+0x4d/0x70
[ 455.869155] [810b4956] blocking_notifier_call_chain+0x16/0x20
[ 455.870111] [8110749c]
Re: Re: [PATCH -tip v2 0/3] ftrace, kprobes: Introduce IPMODIFY flag for ftrace_ops to detect conflicts
(2014/06/19 11:08), Josh Poimboeuf wrote:
On Tue, Jun 17, 2014 at 11:04:36AM +, Masami Hiramatsu wrote:
Hi,
Here is the version 2 of the series of patches which introduces
IPMODIFY flag for ftrace_ops to detect conflicts of ftrace users
who can modify regs-ip in their handler.
In this version, I fixed some bugs in previous version and
added a patch which made kprobe itself free from IPMODIFY
except for jprobe.
Hi Masami,
This seems better, but I still saw a few issues. I'm not sure if the
issues are specific to stap or kprobes. For the following issues I used
this command to set a kprobe:
stap -v -e 'probe kernel.function(meminfo_proc_show)
{printf(meminfo_proc_show called\n);}'
With patches 1-2, when I used stap to kprobe the function after it was
already kpatched, stap didn't return an error and instead acted like it
succeeded (though the probe didn't work):
$ sudo stap -v -e 'probe kernel.function(meminfo_proc_show)
{printf(meminfo_proc_show called\n);}'
Pass 1: parsed user script and 112 library script(s) using
221516virt/41612res/6028shr/36228data kb, in 130usr/0sys/132real ms.
Pass 2: analyzed script: 1 probe(s), 0 function(s), 0 embed(s), 0 global(s)
using 255840virt/77132res/7132shr/70552data kb, in 510usr/20sys/577real ms.
Pass 3: translated to C into
/tmp/stap3Qunba/stap_2690192fea570fb7bba78c7ed7fa1e0d_898_src.c using
255840virt/77392res/7392shr/70552data kb, in 10usr/0sys/4real ms.
Pass 4: compiled C into stap_2690192fea570fb7bba78c7ed7fa1e0d_898.ko in
5020usr/640sys/7105real ms.
Pass 5: starting run.
(no error)
Yeah, I guess you can see some warning messages in dmesg (by
arm_kprobe) at this point.
With all 3 patches, I expected kprobes and kpatch to be able to ftrace
the same function. But when I tried to kpatch the function after it was
kprobed, I got the following oops in stap:
[ 455.842797] BUG: unable to handle kernel NULL pointer dereference at
0020
[ 455.843388] IP: [a0833d1e] _stp_module_notifier+0x1e/0x320
[stap_2690192fea570fb7bba78c7ed7fa1e_20189]
Hmm, since this happens in _stp_module_notifier() which is a code in systemtap,
I guess it's a systemtap problem.
Could you test it with kprobe-tracer as below?
# (do something kpatch related activation)
# echo p meminfo_proc_show /sys/kernel/debug/tracing/kprobe_events
# echo 1 /sys/kernel/debug/tracing/events/kprobe/enable
Thank you,
[ 455.844011] PGD 33f898067 PUD 422083067 PMD 0
[ 455.844638] Oops: [#1] SMP
[ 455.845255] Modules linked in: kpatch(OE+)
stap_2690192fea570fb7bba78c7ed7fa1e_20189(OE) rfcomm ipt_MASQUERADE fuse ccm
xt_CHECKSUM tun ip6t_rpfilter ip6t_REJECT xt_conntrack ebtable_nat
ebtable_broute bridge stp llc ebtable_filter ebtables ip6table_nat
nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle
ip6table_security ip6table_raw ip6table_filter ip6_tables iptable_nat
nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack
iptable_mangle iptable_security iptable_raw bnep arc4 iwldvm mac80211
iTCO_wdt snd_hda_codec_hdmi iTCO_vendor_support x86_pkg_temp_thermal
snd_hda_codec_realtek coretemp iwlwifi snd_hda_codec_generic kvm_intel
snd_hda_intel kvm uvcvideo snd_hda_controller cfg80211 snd_hda_codec btusb
videobuf2_vmalloc bluetooth videobuf2_memops snd_hwdep snd_seq nfsd
videobuf2_core
[ 455.848272] v4l2_common videodev snd_seq_device e1000e microcode
snd_pcm sdhci_pci media joydev sdhci serio_raw i2c_i801 pcspkr mmc_core
thinkpad_acpi mei_me snd_timer auth_rpcgss mei snd lpc_ich ptp mfd_core
shpchp nfs_acl lockd pps_core wmi tpm_tis soundcore tpm rfkill sunrpc
dm_crypt i915 i2c_algo_bit drm_kms_helper drm crct10dif_pclmul crc32_pclmul
crc32c_intel ghash_clmulni_intel i2c_core video
[ 455.850887] CPU: 3 PID: 19857 Comm: insmod Tainted: GW OE
3.16.0-rc1+ #2
[ 455.851768] Hardware name: LENOVO 2356BH8/2356BH8, BIOS G7ET63WW (2.05 )
11/12/2012
[ 455.852638] task: 880095d65460 ti: 88039d1d4000 task.ti:
88039d1d4000
[ 455.853456] RIP: 0010:[a0833d1e] [a0833d1e]
_stp_module_notifier+0x1e/0x320 [stap_2690192fea570fb7bba78c7ed7fa1e_20189]
[ 455.854335] RSP: 0018:88039d1d7ce0 EFLAGS: 00010246
[ 455.855212] RAX: a0837f50 RBX: RCX:
[ 455.856109] RDX: a08400e0 RSI: 0001 RDI:
a0837f50
[ 455.856986] RBP: 88039d1d7d00 R08: R09:
[ 455.857880] R10: 0001 R11: c9001aed2d8f R12:
81c593e0
[ 455.858761] R13: 0001 R14: a08400e0 R15:
[ 455.859640] FS: 7feac5f10740() GS:88043e2c()
knlGS:
[ 455.860523] CS: 0010 DS: ES: CR0: 80050033
[ 455.861403] CR2: 0020 CR3: 0004224e7000 CR4:
001407e0
[
Re: [PATCH -tip v2 0/3] ftrace, kprobes: Introduce IPMODIFY flag for ftrace_ops to detect conflicts
(2014/06/17 20:04), Masami Hiramatsu wrote:
> Hi,
>
> Here is the version 2 of the series of patches which introduces
> IPMODIFY flag for ftrace_ops to detect conflicts of ftrace users
> who can modify regs->ip in their handler.
> In this version, I fixed some bugs in previous version and
> added a patch which made kprobe itself free from IPMODIFY
> except for jprobe.
>
> Currently, only kprobes can change the regs->ip in the handler,
> but recently kpatch is also want to change it. Moreover, since
> the ftrace itself exported to modules, it might be considerable
> senario.
>
> Here we talked on github.
> https://github.com/dynup/kpatch/issues/47
>
> To protect modified regs-ip from each other, this series
> introduces FTRACE_OPS_FL_IPMODIFY flag and ftrace now ensures
> the flag can be set on each function entry location. If there
> is someone who already reserve regs->ip on target function
> entry, ftrace_set_filter_ip or register_ftrace_function will
> return -EBUSY. Users must handle that.
>
> The 3rd patch adds a special reservation of IPMODIFY on the
> jprobed address, since it is the only user who will change
> the regs->ip. Other kprobes do not change it anymore.
>
> Testing:
> BTW, I've tested it with samples/kprobes/{k,j}probe_example.ko
> and a small test module which I added to the last of this mail.
>
> Here is the test script. I think I should put this under
> tools/testing, maybe kprobes? or selftest/kprobes?
>
>
> #!/bin/bash
>
> echo "IPMODIFY test"
> ADDR=0x`grep do_fork /proc/kallsyms | cut -f 1 -d" "`
>
> echo "Case1: kprobe->jprobe->ipmodify(fail)"
> modprobe kprobe_example
> modprobe jprobe_example
> insmod ./ipmodify.ko && echo "Case1: [FAIL]" || echo "Case1: [OK]"
Oops, I missed the target=$ADDR for these insmod...
Anyway, without passing target=$ADDR option, ipmodify.ko tries to
register ftrace_ops with an empty filter hash which means
trace all functions. So anyway jprobe and that ftrace_ops collide
with each other on the do_fork and the 2nd one should be failed.
Thus, of course this test doesn't fail with or without target=
param.
Thank you,
--
Masami HIRAMATSU
IT Management Research Dept. Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hiramatsu...@hitachi.com
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH -tip v2 0/3] ftrace, kprobes: Introduce IPMODIFY flag for ftrace_ops to detect conflicts
Hi,
Here is the version 2 of the series of patches which introduces
IPMODIFY flag for ftrace_ops to detect conflicts of ftrace users
who can modify regs->ip in their handler.
In this version, I fixed some bugs in previous version and
added a patch which made kprobe itself free from IPMODIFY
except for jprobe.
Currently, only kprobes can change the regs->ip in the handler,
but recently kpatch is also want to change it. Moreover, since
the ftrace itself exported to modules, it might be considerable
senario.
Here we talked on github.
https://github.com/dynup/kpatch/issues/47
To protect modified regs-ip from each other, this series
introduces FTRACE_OPS_FL_IPMODIFY flag and ftrace now ensures
the flag can be set on each function entry location. If there
is someone who already reserve regs->ip on target function
entry, ftrace_set_filter_ip or register_ftrace_function will
return -EBUSY. Users must handle that.
The 3rd patch adds a special reservation of IPMODIFY on the
jprobed address, since it is the only user who will change
the regs->ip. Other kprobes do not change it anymore.
Testing:
BTW, I've tested it with samples/kprobes/{k,j}probe_example.ko
and a small test module which I added to the last of this mail.
Here is the test script. I think I should put this under
tools/testing, maybe kprobes? or selftest/kprobes?
#!/bin/bash
echo "IPMODIFY test"
ADDR=0x`grep do_fork /proc/kallsyms | cut -f 1 -d" "`
echo "Case1: kprobe->jprobe->ipmodify(fail)"
modprobe kprobe_example
modprobe jprobe_example
insmod ./ipmodify.ko && echo "Case1: [FAIL]" || echo "Case1: [OK]"
rmmod kprobe_example jprobe_example
echo "Case2: jprobe->kprobe->ipmodify(fail)"
modprobe jprobe_example
modprobe kprobe_example
insmod ./ipmodify.ko && echo "Case2: [FAIL]" || echo "Case2: [OK]"
rmmod kprobe_example jprobe_example
echo "Case3: jprobe->ipmodify(fail)"
modprobe jprobe_example
insmod ./ipmodify.ko && echo "Case3: [FAIL]" || echo "Case3: [OK]"
rmmod jprobe_example
echo "Case4: ipmodify->jprobe(fail)"
insmod ./ipmodify.ko
modprobe jprobe_example && echo "Case4: [FAIL]" || echo "Case4: [OK]"
rmmod ipmodify
echo "Case5: ipmodify->kprobe->jprobe(fail)"
insmod ./ipmodify.ko
modprobe kprobe_example
modprobe jprobe_example && echo "Case5: [FAIL]" || echo "Case5: [OK]"
rmmod ipmodify kprobe_example
-
Thank you,
---
Masami Hiramatsu (3):
ftrace: Simplify ftrace_hash_disable/enable path in ftrace_hash_move
ftrace, kprobes: Support IPMODIFY flag to find IP modify conflict
kprobes: Set IPMODIFY flag only if the probe can change regs->ip
Documentation/kprobes.txt| 12 +--
Documentation/trace/ftrace.txt |5 +
arch/x86/kernel/kprobes/ftrace.c |9 +-
include/linux/ftrace.h | 10 ++
kernel/kprobes.c | 115 +++
kernel/trace/ftrace.c| 164 +-
6 files changed, 265 insertions(+), 50 deletions(-)
--
-
#include
#include
#include
static void ftrace_ipmodify_handler(unsigned long a0, unsigned long a1,
struct ftrace_ops *op, struct pt_regs *regs)
{
return;
}
static struct ftrace_ops test_ops __read_mostly = {
.func = ftrace_ipmodify_handler,
.flags = FTRACE_OPS_FL_SAVE_REGS | FTRACE_OPS_FL_IPMODIFY,
};
static unsigned long target;
module_param(target, ulong, 0444);
static int __init ipmodify_init(void)
{
int ret;
unsigned long ip = target;
ret = ftrace_set_filter_ip(_ops, ip, 0, 0);
pr_err("ipmodify_test: set filter ip 0x%lx, ret = %d\n", ip, ret);
if (ret >= 0) {
ret = register_ftrace_function(_ops);
pr_err("ipmodify_test: register ftrace on 0x%lx, ret = %d\n",
ip, ret);
}
return ret;
}
static void __exit ipmodify_exit(void)
{
int ret;
unsigned long ip = target;
ret = unregister_ftrace_function(_ops);
pr_err("ipmodify_test: unregister ftrace on 0x%lx, ret = %d\n", ip,
ret);
if (ret >= 0) {
ret = ftrace_set_filter_ip(_ops, ip, 1, 0);
pr_err("ipmodify_test: clear filter ip 0x%lx, ret = %d\n",
ip, ret);
}
}
module_init(ipmodify_init)
module_exit(ipmodify_exit)
MODULE_LICENSE("GPL");
-
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[PATCH -tip v2 0/3] ftrace, kprobes: Introduce IPMODIFY flag for ftrace_ops to detect conflicts
Hi,
Here is the version 2 of the series of patches which introduces
IPMODIFY flag for ftrace_ops to detect conflicts of ftrace users
who can modify regs-ip in their handler.
In this version, I fixed some bugs in previous version and
added a patch which made kprobe itself free from IPMODIFY
except for jprobe.
Currently, only kprobes can change the regs-ip in the handler,
but recently kpatch is also want to change it. Moreover, since
the ftrace itself exported to modules, it might be considerable
senario.
Here we talked on github.
https://github.com/dynup/kpatch/issues/47
To protect modified regs-ip from each other, this series
introduces FTRACE_OPS_FL_IPMODIFY flag and ftrace now ensures
the flag can be set on each function entry location. If there
is someone who already reserve regs-ip on target function
entry, ftrace_set_filter_ip or register_ftrace_function will
return -EBUSY. Users must handle that.
The 3rd patch adds a special reservation of IPMODIFY on the
jprobed address, since it is the only user who will change
the regs-ip. Other kprobes do not change it anymore.
Testing:
BTW, I've tested it with samples/kprobes/{k,j}probe_example.ko
and a small test module which I added to the last of this mail.
Here is the test script. I think I should put this under
tools/testing, maybe kprobes? or selftest/kprobes?
#!/bin/bash
echo IPMODIFY test
ADDR=0x`grep do_fork /proc/kallsyms | cut -f 1 -d `
echo Case1: kprobe-jprobe-ipmodify(fail)
modprobe kprobe_example
modprobe jprobe_example
insmod ./ipmodify.ko echo Case1: [FAIL] || echo Case1: [OK]
rmmod kprobe_example jprobe_example
echo Case2: jprobe-kprobe-ipmodify(fail)
modprobe jprobe_example
modprobe kprobe_example
insmod ./ipmodify.ko echo Case2: [FAIL] || echo Case2: [OK]
rmmod kprobe_example jprobe_example
echo Case3: jprobe-ipmodify(fail)
modprobe jprobe_example
insmod ./ipmodify.ko echo Case3: [FAIL] || echo Case3: [OK]
rmmod jprobe_example
echo Case4: ipmodify-jprobe(fail)
insmod ./ipmodify.ko
modprobe jprobe_example echo Case4: [FAIL] || echo Case4: [OK]
rmmod ipmodify
echo Case5: ipmodify-kprobe-jprobe(fail)
insmod ./ipmodify.ko
modprobe kprobe_example
modprobe jprobe_example echo Case5: [FAIL] || echo Case5: [OK]
rmmod ipmodify kprobe_example
-
Thank you,
---
Masami Hiramatsu (3):
ftrace: Simplify ftrace_hash_disable/enable path in ftrace_hash_move
ftrace, kprobes: Support IPMODIFY flag to find IP modify conflict
kprobes: Set IPMODIFY flag only if the probe can change regs-ip
Documentation/kprobes.txt| 12 +--
Documentation/trace/ftrace.txt |5 +
arch/x86/kernel/kprobes/ftrace.c |9 +-
include/linux/ftrace.h | 10 ++
kernel/kprobes.c | 115 +++
kernel/trace/ftrace.c| 164 +-
6 files changed, 265 insertions(+), 50 deletions(-)
--
-ipmodify.c
#include linux/kernel.h
#include linux/module.h
#include linux/ftrace.h
static void ftrace_ipmodify_handler(unsigned long a0, unsigned long a1,
struct ftrace_ops *op, struct pt_regs *regs)
{
return;
}
static struct ftrace_ops test_ops __read_mostly = {
.func = ftrace_ipmodify_handler,
.flags = FTRACE_OPS_FL_SAVE_REGS | FTRACE_OPS_FL_IPMODIFY,
};
static unsigned long target;
module_param(target, ulong, 0444);
static int __init ipmodify_init(void)
{
int ret;
unsigned long ip = target;
ret = ftrace_set_filter_ip(test_ops, ip, 0, 0);
pr_err(ipmodify_test: set filter ip 0x%lx, ret = %d\n, ip, ret);
if (ret = 0) {
ret = register_ftrace_function(test_ops);
pr_err(ipmodify_test: register ftrace on 0x%lx, ret = %d\n,
ip, ret);
}
return ret;
}
static void __exit ipmodify_exit(void)
{
int ret;
unsigned long ip = target;
ret = unregister_ftrace_function(test_ops);
pr_err(ipmodify_test: unregister ftrace on 0x%lx, ret = %d\n, ip,
ret);
if (ret = 0) {
ret = ftrace_set_filter_ip(test_ops, ip, 1, 0);
pr_err(ipmodify_test: clear filter ip 0x%lx, ret = %d\n,
ip, ret);
}
}
module_init(ipmodify_init)
module_exit(ipmodify_exit)
MODULE_LICENSE(GPL);
-
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH -tip v2 0/3] ftrace, kprobes: Introduce IPMODIFY flag for ftrace_ops to detect conflicts
(2014/06/17 20:04), Masami Hiramatsu wrote:
Hi,
Here is the version 2 of the series of patches which introduces
IPMODIFY flag for ftrace_ops to detect conflicts of ftrace users
who can modify regs-ip in their handler.
In this version, I fixed some bugs in previous version and
added a patch which made kprobe itself free from IPMODIFY
except for jprobe.
Currently, only kprobes can change the regs-ip in the handler,
but recently kpatch is also want to change it. Moreover, since
the ftrace itself exported to modules, it might be considerable
senario.
Here we talked on github.
https://github.com/dynup/kpatch/issues/47
To protect modified regs-ip from each other, this series
introduces FTRACE_OPS_FL_IPMODIFY flag and ftrace now ensures
the flag can be set on each function entry location. If there
is someone who already reserve regs-ip on target function
entry, ftrace_set_filter_ip or register_ftrace_function will
return -EBUSY. Users must handle that.
The 3rd patch adds a special reservation of IPMODIFY on the
jprobed address, since it is the only user who will change
the regs-ip. Other kprobes do not change it anymore.
Testing:
BTW, I've tested it with samples/kprobes/{k,j}probe_example.ko
and a small test module which I added to the last of this mail.
Here is the test script. I think I should put this under
tools/testing, maybe kprobes? or selftest/kprobes?
#!/bin/bash
echo IPMODIFY test
ADDR=0x`grep do_fork /proc/kallsyms | cut -f 1 -d `
echo Case1: kprobe-jprobe-ipmodify(fail)
modprobe kprobe_example
modprobe jprobe_example
insmod ./ipmodify.ko echo Case1: [FAIL] || echo Case1: [OK]
Oops, I missed the target=$ADDR for these insmod...
Anyway, without passing target=$ADDR option, ipmodify.ko tries to
register ftrace_ops with an empty filter hash which means
trace all functions. So anyway jprobe and that ftrace_ops collide
with each other on the do_fork and the 2nd one should be failed.
Thus, of course this test doesn't fail with or without target=
param.
Thank you,
--
Masami HIRAMATSU
IT Management Research Dept. Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hiramatsu...@hitachi.com
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
