Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-11 Thread Alexei Starovoitov
On Tue, Dec 10, 2013 at 7:35 PM, Masami Hiramatsu wrote: > (2013/12/11 11:32), Alexei Starovoitov wrote: >> On Tue, Dec 10, 2013 at 7:47 AM, Ingo Molnar wrote: >>> >>> * Alexei Starovoitov wrote: >>> > I'm fine if it becomes a requirement to have a vmlinux built with > DEBUG_INFO to use

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-11 Thread Alexei Starovoitov
On Tue, Dec 10, 2013 at 7:35 PM, Masami Hiramatsu masami.hiramatsu...@hitachi.com wrote: (2013/12/11 11:32), Alexei Starovoitov wrote: On Tue, Dec 10, 2013 at 7:47 AM, Ingo Molnar mi...@kernel.org wrote: * Alexei Starovoitov a...@plumgrid.com wrote: I'm fine if it becomes a requirement to

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-10 Thread Masami Hiramatsu
(2013/12/11 11:32), Alexei Starovoitov wrote: > On Tue, Dec 10, 2013 at 7:47 AM, Ingo Molnar wrote: >> >> * Alexei Starovoitov wrote: >> I'm fine if it becomes a requirement to have a vmlinux built with DEBUG_INFO to use BPF and have a tool like perf to translate the filters. But

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-10 Thread Alexei Starovoitov
On Tue, Dec 10, 2013 at 7:47 AM, Ingo Molnar wrote: > > * Alexei Starovoitov wrote: > >> > I'm fine if it becomes a requirement to have a vmlinux built with >> > DEBUG_INFO to use BPF and have a tool like perf to translate the >> > filters. But it that must not replace what the current filters

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-10 Thread Ingo Molnar
* Alexei Starovoitov wrote: > > I'm fine if it becomes a requirement to have a vmlinux built with > > DEBUG_INFO to use BPF and have a tool like perf to translate the > > filters. But it that must not replace what the current filters do > > now. That is, it can be an add on, but not a

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-10 Thread Alexei Starovoitov
On Tue, Dec 10, 2013 at 7:47 AM, Ingo Molnar mi...@kernel.org wrote: * Alexei Starovoitov a...@plumgrid.com wrote: I'm fine if it becomes a requirement to have a vmlinux built with DEBUG_INFO to use BPF and have a tool like perf to translate the filters. But it that must not replace what

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-10 Thread Masami Hiramatsu
(2013/12/11 11:32), Alexei Starovoitov wrote: On Tue, Dec 10, 2013 at 7:47 AM, Ingo Molnar mi...@kernel.org wrote: * Alexei Starovoitov a...@plumgrid.com wrote: I'm fine if it becomes a requirement to have a vmlinux built with DEBUG_INFO to use BPF and have a tool like perf to translate the

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-10 Thread Ingo Molnar
* Alexei Starovoitov a...@plumgrid.com wrote: I'm fine if it becomes a requirement to have a vmlinux built with DEBUG_INFO to use BPF and have a tool like perf to translate the filters. But it that must not replace what the current filters do now. That is, it can be an add on, but not

Re: binary blob no more! Was: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-09 Thread Steven Rostedt
On Sun, 8 Dec 2013 19:36:18 -0800 Alexei Starovoitov wrote: > Actually I think there are few ways to include the source equivalent > in bpf image. > > Approach #1 > include original C source code into bpf image: > bpf_image = bpf_insns + original_C > this will imply that C code can have

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-09 Thread Masami Hiramatsu
(2013/12/09 16:29), Namhyung Kim wrote: > Hi Masami, > > On Wed, 04 Dec 2013 10:13:37 +0900, Masami Hiramatsu wrote: >> (2013/12/04 3:26), Alexei Starovoitov wrote: >>> the only inconvenience so far is to know how parameters are getting >>> into registers. >>> on x86-64, arg1 is in rdi, arg2 is

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-09 Thread Masami Hiramatsu
(2013/12/09 16:29), Namhyung Kim wrote: Hi Masami, On Wed, 04 Dec 2013 10:13:37 +0900, Masami Hiramatsu wrote: (2013/12/04 3:26), Alexei Starovoitov wrote: the only inconvenience so far is to know how parameters are getting into registers. on x86-64, arg1 is in rdi, arg2 is in rsi,... I

Re: binary blob no more! Was: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-09 Thread Steven Rostedt
On Sun, 8 Dec 2013 19:36:18 -0800 Alexei Starovoitov a...@plumgrid.com wrote: Actually I think there are few ways to include the source equivalent in bpf image. Approach #1 include original C source code into bpf image: bpf_image = bpf_insns + original_C this will imply that C code

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-08 Thread Namhyung Kim
Hi Masami, On Wed, 04 Dec 2013 10:13:37 +0900, Masami Hiramatsu wrote: > (2013/12/04 3:26), Alexei Starovoitov wrote: >> the only inconvenience so far is to know how parameters are getting >> into registers. >> on x86-64, arg1 is in rdi, arg2 is in rsi,... I want to improve that >> after first

Re: Re: Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-08 Thread Masami Hiramatsu
(2013/12/08 1:21), Jovi Zhangwei wrote: > On Sat, Dec 7, 2013 at 7:58 AM, Masami Hiramatsu > wrote: >> (2013/12/06 14:19), Jovi Zhangwei wrote: >>> Hi Alexei, >>> >>> On Thu, Dec 5, 2013 at 12:40 PM, Alexei Starovoitov >>> wrote: > On Tue, Dec 3, 2013 at 4:01 PM, Andi Kleen wrote: >>

binary blob no more! Was: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-08 Thread Alexei Starovoitov
On Thu, Dec 5, 2013 at 9:43 PM, Alexei Starovoitov wrote: > On Thu, Dec 5, 2013 at 2:38 AM, Ingo Molnar wrote: >> >>> Also I'm thinking to add 'license_string' section to bpf binary format >>> and call license_is_gpl_compatible() on it during load. >>> If false, then just reject it…. not even

binary blob no more! Was: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-08 Thread Alexei Starovoitov
On Thu, Dec 5, 2013 at 9:43 PM, Alexei Starovoitov a...@plumgrid.com wrote: On Thu, Dec 5, 2013 at 2:38 AM, Ingo Molnar mi...@kernel.org wrote: Also I'm thinking to add 'license_string' section to bpf binary format and call license_is_gpl_compatible() on it during load. If false, then just

Re: Re: Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-08 Thread Masami Hiramatsu
(2013/12/08 1:21), Jovi Zhangwei wrote: On Sat, Dec 7, 2013 at 7:58 AM, Masami Hiramatsu masami.hiramatsu...@hitachi.com wrote: (2013/12/06 14:19), Jovi Zhangwei wrote: Hi Alexei, On Thu, Dec 5, 2013 at 12:40 PM, Alexei Starovoitov a...@plumgrid.com wrote: On Tue, Dec 3, 2013 at 4:01 PM,

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-08 Thread Namhyung Kim
Hi Masami, On Wed, 04 Dec 2013 10:13:37 +0900, Masami Hiramatsu wrote: (2013/12/04 3:26), Alexei Starovoitov wrote: the only inconvenience so far is to know how parameters are getting into registers. on x86-64, arg1 is in rdi, arg2 is in rsi,... I want to improve that after first step is

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-07 Thread Jovi Zhangwei
On Sat, Dec 7, 2013 at 9:12 AM, Alexei Starovoitov wrote: > On Thu, Dec 5, 2013 at 5:20 PM, Andi Kleen wrote: >> "H. Peter Anvin" writes: >>> >>> Not to mention that in that case we might as well -- since we need a >>> compiler anyway -- generate the machine code in user space; the JIT >>>

Re: Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-07 Thread Jovi Zhangwei
On Sat, Dec 7, 2013 at 7:58 AM, Masami Hiramatsu wrote: > (2013/12/06 14:19), Jovi Zhangwei wrote: >> Hi Alexei, >> >> On Thu, Dec 5, 2013 at 12:40 PM, Alexei Starovoitov >> wrote: On Tue, Dec 3, 2013 at 4:01 PM, Andi Kleen wrote: > > Can you do some performance comparison

Re: Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-07 Thread Jovi Zhangwei
On Sat, Dec 7, 2013 at 7:58 AM, Masami Hiramatsu masami.hiramatsu...@hitachi.com wrote: (2013/12/06 14:19), Jovi Zhangwei wrote: Hi Alexei, On Thu, Dec 5, 2013 at 12:40 PM, Alexei Starovoitov a...@plumgrid.com wrote: On Tue, Dec 3, 2013 at 4:01 PM, Andi Kleen a...@firstfloor.org wrote:

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-07 Thread Jovi Zhangwei
On Sat, Dec 7, 2013 at 9:12 AM, Alexei Starovoitov a...@plumgrid.com wrote: On Thu, Dec 5, 2013 at 5:20 PM, Andi Kleen a...@firstfloor.org wrote: H. Peter Anvin h...@zytor.com writes: Not to mention that in that case we might as well -- since we need a compiler anyway -- generate the machine

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-06 Thread Alexei Starovoitov
On Thu, Dec 5, 2013 at 5:20 PM, Andi Kleen wrote: > "H. Peter Anvin" writes: >> >> Not to mention that in that case we might as well -- since we need a >> compiler anyway -- generate the machine code in user space; the JIT >> solution really only is useful if it can provide something that we

Re: Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-06 Thread Alexei Starovoitov
On Fri, Dec 6, 2013 at 3:54 PM, Masami Hiramatsu wrote: > (2013/12/06 14:16), Alexei Starovoitov wrote: >> On Thu, Dec 5, 2013 at 5:20 PM, Andi Kleen wrote: the difference is bigger now: 484-145 vs 185-145 >>> >>> This is a obvious improvement, but imho not big enough to be extremely >>>

Re: Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-06 Thread Masami Hiramatsu
(2013/12/06 14:19), Jovi Zhangwei wrote: > Hi Alexei, > > On Thu, Dec 5, 2013 at 12:40 PM, Alexei Starovoitov wrote: >>> On Tue, Dec 3, 2013 at 4:01 PM, Andi Kleen wrote: Can you do some performance comparison compared to e.g. ktap? How much faster is it? >> >> Did simple ktap

Re: Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-06 Thread Masami Hiramatsu
(2013/12/06 14:16), Alexei Starovoitov wrote: > On Thu, Dec 5, 2013 at 5:20 PM, Andi Kleen wrote: >>> the difference is bigger now: 484-145 vs 185-145 >> >> This is a obvious improvement, but imho not big enough to be extremely >> compelling (< cost 1-2 cache misses, no orders of magnitude

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-06 Thread Frank Ch. Eigler
hpa wrote: >> I can see there may be some setups which don't have a compiler >> (e.g. I know some people don't use systemtap because of that) >> But this needs a custom gcc install too as far as I understand. > > Yes... but no compiler and secure boot tend to go together, or at > least will in

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-06 Thread Frank Ch. Eigler
hpa wrote: I can see there may be some setups which don't have a compiler (e.g. I know some people don't use systemtap because of that) But this needs a custom gcc install too as far as I understand. Yes... but no compiler and secure boot tend to go together, or at least will in the

Re: Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-06 Thread Masami Hiramatsu
(2013/12/06 14:16), Alexei Starovoitov wrote: On Thu, Dec 5, 2013 at 5:20 PM, Andi Kleen a...@firstfloor.org wrote: the difference is bigger now: 484-145 vs 185-145 This is a obvious improvement, but imho not big enough to be extremely compelling ( cost 1-2 cache misses, no orders of

Re: Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-06 Thread Masami Hiramatsu
(2013/12/06 14:19), Jovi Zhangwei wrote: Hi Alexei, On Thu, Dec 5, 2013 at 12:40 PM, Alexei Starovoitov a...@plumgrid.com wrote: On Tue, Dec 3, 2013 at 4:01 PM, Andi Kleen a...@firstfloor.org wrote: Can you do some performance comparison compared to e.g. ktap? How much faster is it? Did

Re: Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-06 Thread Alexei Starovoitov
On Fri, Dec 6, 2013 at 3:54 PM, Masami Hiramatsu masami.hiramatsu...@hitachi.com wrote: (2013/12/06 14:16), Alexei Starovoitov wrote: On Thu, Dec 5, 2013 at 5:20 PM, Andi Kleen a...@firstfloor.org wrote: the difference is bigger now: 484-145 vs 185-145 This is a obvious improvement, but imho

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-06 Thread Alexei Starovoitov
On Thu, Dec 5, 2013 at 5:20 PM, Andi Kleen a...@firstfloor.org wrote: H. Peter Anvin h...@zytor.com writes: Not to mention that in that case we might as well -- since we need a compiler anyway -- generate the machine code in user space; the JIT solution really only is useful if it can provide

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Jovi Zhangwei
On Thu, Dec 5, 2013 at 12:40 PM, Alexei Starovoitov wrote: >> On Tue, Dec 3, 2013 at 4:01 PM, Andi Kleen wrote: >>> >>> Can you do some performance comparison compared to e.g. ktap? >>> How much faster is it? > > Did simple ktap test with 1M alloc_skb/kfree_skb toy test from earlier email: >

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Jovi Zhangwei
On Fri, Dec 6, 2013 at 9:20 AM, Andi Kleen wrote: > "H. Peter Anvin" writes: >> >> Not to mention that in that case we might as well -- since we need a >> compiler anyway -- generate the machine code in user space; the JIT >> solution really only is useful if it can provide something that we

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Alexei Starovoitov
On Thu, Dec 5, 2013 at 2:38 AM, Ingo Molnar wrote: > >> Also I'm thinking to add 'license_string' section to bpf binary format >> and call license_is_gpl_compatible() on it during load. >> If false, then just reject it…. not even messing with taint flags... >> That would be way stronger

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Jovi Zhangwei
Hi Alexei, On Thu, Dec 5, 2013 at 12:40 PM, Alexei Starovoitov wrote: >> On Tue, Dec 3, 2013 at 4:01 PM, Andi Kleen wrote: >>> >>> Can you do some performance comparison compared to e.g. ktap? >>> How much faster is it? > > Did simple ktap test with 1M alloc_skb/kfree_skb toy test from earlier

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Alexei Starovoitov
On Thu, Dec 5, 2013 at 5:20 PM, Andi Kleen wrote: >> the difference is bigger now: 484-145 vs 185-145 > > This is a obvious improvement, but imho not big enough to be extremely > compelling (< cost 1-2 cache misses, no orders of magnitude improvements > that would justify a lot of code) hmm.

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Alexei Starovoitov
On Thu, Dec 5, 2013 at 3:37 PM, Steven Rostedt wrote: > On Thu, 5 Dec 2013 14:36:58 -0800 > Alexei Starovoitov wrote: > >> On Thu, Dec 5, 2013 at 5:46 AM, Steven Rostedt wrote: >> > >> > I know that it would be great to have the bpf filter run before >> > recording of the tracepoint, but as

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread H. Peter Anvin
On 12/05/2013 05:20 PM, Andi Kleen wrote: > "H. Peter Anvin" writes: >> >> Not to mention that in that case we might as well -- since we need a >> compiler anyway -- generate the machine code in user space; the JIT >> solution really only is useful if it can provide something that we can't >> do

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Andi Kleen
"H. Peter Anvin" writes: > > Not to mention that in that case we might as well -- since we need a > compiler anyway -- generate the machine code in user space; the JIT > solution really only is useful if it can provide something that we can't > do otherwise, e.g. enable it in secure boot

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread H. Peter Anvin
On 12/05/2013 04:14 PM, Andi Kleen wrote: > > In my experience there are roughly two groups of trace users: > kernel hackers and users. The kernel hackers want something > convenient and fast, but for anything complicated or performance > critical they can always hack the kernel to include custom

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Andi Kleen
> 1M skb alloc/free 185660 (usecs) > > the difference is bigger now: 484-145 vs 185-145 Thanks for the data. This is a obvious improvement, but imho not big enough to be extremely compelling (< cost 1-2 cache misses, no orders of magnitude improvements that would justify a lot of code) One

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Steven Rostedt
On Thu, 5 Dec 2013 14:36:58 -0800 Alexei Starovoitov wrote: > On Thu, Dec 5, 2013 at 5:46 AM, Steven Rostedt wrote: > > > > I know that it would be great to have the bpf filter run before > > recording of the tracepoint, but as that becomes quite awkward for a > > user interface, because it

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Alexei Starovoitov
On Thu, Dec 5, 2013 at 5:46 AM, Steven Rostedt wrote: > > I know that it would be great to have the bpf filter run before > recording of the tracepoint, but as that becomes quite awkward for a > user interface, because it requires intimate knowledge of the kernel > source, this speed up on the

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Alexei Starovoitov
On Thu, Dec 5, 2013 at 8:11 AM, Frank Ch. Eigler wrote: > > ast wrote: > >>>[...] >> Did simple ktap test with 1M alloc_skb/kfree_skb toy test from earlier email: >> trace skb:kfree_skb { >> if (arg2 == 0x100) { >> printf("%x %x\n", arg1, arg2) >> } >> } >> [...] >

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Frank Ch. Eigler
Andi Kleen writes: > [...] While it sounds interesting, I would strongly advise to make > this capability only available to root. Traditionally lots of > complex byte code languages which were designed to be "safe" and > verifiable weren't really. e.g. i managed to crash things with > "safe"

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Frank Ch. Eigler
ast wrote: >>[...] > Did simple ktap test with 1M alloc_skb/kfree_skb toy test from earlier email: > trace skb:kfree_skb { > if (arg2 == 0x100) { > printf("%x %x\n", arg1, arg2) > } > } > [...] For reference, you might try putting systemtap into the performance

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Steven Rostedt
On Thu, 5 Dec 2013 11:41:13 +0100 Ingo Molnar wrote: > > so with one 'if' condition the difference ktap vs bpf is 350-145 vs 183-145 > > > > obviously ktap is an interpreter, so it's not really fair. > > > > To make it really unfair I did: > > trace skb:kfree_skb { > > if (arg2 ==

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Ingo Molnar
* Alexei Starovoitov wrote: > > On Tue, Dec 3, 2013 at 4:01 PM, Andi Kleen wrote: > >> > >> Can you do some performance comparison compared to e.g. ktap? > >> How much faster is it? > > Did simple ktap test with 1M alloc_skb/kfree_skb toy test from earlier email: > trace skb:kfree_skb { >

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Ingo Molnar
* Alexei Starovoitov wrote: > > I mean more than that, I mean the licensing of BFP filters a user > > can find on his own system's kernel should be very clear: by the > > act of loading a BFP script into the kernel the user doing the > > 'upload' gives permission for it to be redistributed

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Ingo Molnar
* Alexei Starovoitov a...@plumgrid.com wrote: I mean more than that, I mean the licensing of BFP filters a user can find on his own system's kernel should be very clear: by the act of loading a BFP script into the kernel the user doing the 'upload' gives permission for it to be

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Ingo Molnar
* Alexei Starovoitov a...@plumgrid.com wrote: On Tue, Dec 3, 2013 at 4:01 PM, Andi Kleen a...@firstfloor.org wrote: Can you do some performance comparison compared to e.g. ktap? How much faster is it? Did simple ktap test with 1M alloc_skb/kfree_skb toy test from earlier email: trace

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Steven Rostedt
On Thu, 5 Dec 2013 11:41:13 +0100 Ingo Molnar mi...@kernel.org wrote: so with one 'if' condition the difference ktap vs bpf is 350-145 vs 183-145 obviously ktap is an interpreter, so it's not really fair. To make it really unfair I did: trace skb:kfree_skb { if (arg2 ==

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Frank Ch. Eigler
ast wrote: [...] Did simple ktap test with 1M alloc_skb/kfree_skb toy test from earlier email: trace skb:kfree_skb { if (arg2 == 0x100) { printf(%x %x\n, arg1, arg2) } } [...] For reference, you might try putting systemtap into the performance comparison

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Frank Ch. Eigler
Andi Kleen a...@firstfloor.org writes: [...] While it sounds interesting, I would strongly advise to make this capability only available to root. Traditionally lots of complex byte code languages which were designed to be safe and verifiable weren't really. e.g. i managed to crash things

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Alexei Starovoitov
On Thu, Dec 5, 2013 at 8:11 AM, Frank Ch. Eigler f...@redhat.com wrote: ast wrote: [...] Did simple ktap test with 1M alloc_skb/kfree_skb toy test from earlier email: trace skb:kfree_skb { if (arg2 == 0x100) { printf(%x %x\n, arg1, arg2) } } [...] For

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Alexei Starovoitov
On Thu, Dec 5, 2013 at 5:46 AM, Steven Rostedt rost...@goodmis.org wrote: I know that it would be great to have the bpf filter run before recording of the tracepoint, but as that becomes quite awkward for a user interface, because it requires intimate knowledge of the kernel source, this

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Steven Rostedt
On Thu, 5 Dec 2013 14:36:58 -0800 Alexei Starovoitov a...@plumgrid.com wrote: On Thu, Dec 5, 2013 at 5:46 AM, Steven Rostedt rost...@goodmis.org wrote: I know that it would be great to have the bpf filter run before recording of the tracepoint, but as that becomes quite awkward for a

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Andi Kleen
1M skb alloc/free 185660 (usecs) the difference is bigger now: 484-145 vs 185-145 Thanks for the data. This is a obvious improvement, but imho not big enough to be extremely compelling ( cost 1-2 cache misses, no orders of magnitude improvements that would justify a lot of code) One larger

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread H. Peter Anvin
On 12/05/2013 04:14 PM, Andi Kleen wrote: In my experience there are roughly two groups of trace users: kernel hackers and users. The kernel hackers want something convenient and fast, but for anything complicated or performance critical they can always hack the kernel to include custom

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Andi Kleen
H. Peter Anvin h...@zytor.com writes: Not to mention that in that case we might as well -- since we need a compiler anyway -- generate the machine code in user space; the JIT solution really only is useful if it can provide something that we can't do otherwise, e.g. enable it in secure boot

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread H. Peter Anvin
On 12/05/2013 05:20 PM, Andi Kleen wrote: H. Peter Anvin h...@zytor.com writes: Not to mention that in that case we might as well -- since we need a compiler anyway -- generate the machine code in user space; the JIT solution really only is useful if it can provide something that we can't do

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Alexei Starovoitov
On Thu, Dec 5, 2013 at 3:37 PM, Steven Rostedt rost...@goodmis.org wrote: On Thu, 5 Dec 2013 14:36:58 -0800 Alexei Starovoitov a...@plumgrid.com wrote: On Thu, Dec 5, 2013 at 5:46 AM, Steven Rostedt rost...@goodmis.org wrote: I know that it would be great to have the bpf filter run before

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Alexei Starovoitov
On Thu, Dec 5, 2013 at 5:20 PM, Andi Kleen a...@firstfloor.org wrote: the difference is bigger now: 484-145 vs 185-145 This is a obvious improvement, but imho not big enough to be extremely compelling ( cost 1-2 cache misses, no orders of magnitude improvements that would justify a lot of

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Jovi Zhangwei
Hi Alexei, On Thu, Dec 5, 2013 at 12:40 PM, Alexei Starovoitov a...@plumgrid.com wrote: On Tue, Dec 3, 2013 at 4:01 PM, Andi Kleen a...@firstfloor.org wrote: Can you do some performance comparison compared to e.g. ktap? How much faster is it? Did simple ktap test with 1M alloc_skb/kfree_skb

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Alexei Starovoitov
On Thu, Dec 5, 2013 at 2:38 AM, Ingo Molnar mi...@kernel.org wrote: Also I'm thinking to add 'license_string' section to bpf binary format and call license_is_gpl_compatible() on it during load. If false, then just reject it…. not even messing with taint flags... That would be way stronger

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Jovi Zhangwei
On Fri, Dec 6, 2013 at 9:20 AM, Andi Kleen a...@firstfloor.org wrote: H. Peter Anvin h...@zytor.com writes: Not to mention that in that case we might as well -- since we need a compiler anyway -- generate the machine code in user space; the JIT solution really only is useful if it can provide

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-05 Thread Jovi Zhangwei
On Thu, Dec 5, 2013 at 12:40 PM, Alexei Starovoitov a...@plumgrid.com wrote: On Tue, Dec 3, 2013 at 4:01 PM, Andi Kleen a...@firstfloor.org wrote: Can you do some performance comparison compared to e.g. ktap? How much faster is it? Did simple ktap test with 1M alloc_skb/kfree_skb toy test

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-04 Thread Alexei Starovoitov
> On Tue, Dec 3, 2013 at 4:01 PM, Andi Kleen wrote: >> >> Can you do some performance comparison compared to e.g. ktap? >> How much faster is it? Did simple ktap test with 1M alloc_skb/kfree_skb toy test from earlier email: trace skb:kfree_skb { if (arg2 == 0x100) {

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-04 Thread Alexei Starovoitov
On Wed, Dec 4, 2013 at 1:34 AM, Ingo Molnar wrote: > > * Alexei Starovoitov wrote: > >> On Tue, Dec 3, 2013 at 1:16 AM, Ingo Molnar wrote: >> > >> > Very cool! (Added various other folks who might be interested in >> > this to the Cc: list.) >> > >> > I have one generic concern: >> > >> > It

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-04 Thread Ingo Molnar
* Alexei Starovoitov wrote: > On Tue, Dec 3, 2013 at 1:16 AM, Ingo Molnar wrote: > > > > Very cool! (Added various other folks who might be interested in > > this to the Cc: list.) > > > > I have one generic concern: > > > > It would be important to make it easy to extract loaded BPF code >

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-04 Thread Ingo Molnar
* Alexei Starovoitov a...@plumgrid.com wrote: On Tue, Dec 3, 2013 at 1:16 AM, Ingo Molnar mi...@kernel.org wrote: Very cool! (Added various other folks who might be interested in this to the Cc: list.) I have one generic concern: It would be important to make it easy to extract

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-04 Thread Alexei Starovoitov
On Wed, Dec 4, 2013 at 1:34 AM, Ingo Molnar mi...@kernel.org wrote: * Alexei Starovoitov a...@plumgrid.com wrote: On Tue, Dec 3, 2013 at 1:16 AM, Ingo Molnar mi...@kernel.org wrote: Very cool! (Added various other folks who might be interested in this to the Cc: list.) I have one

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-04 Thread Alexei Starovoitov
On Tue, Dec 3, 2013 at 4:01 PM, Andi Kleen a...@firstfloor.org wrote: Can you do some performance comparison compared to e.g. ktap? How much faster is it? Did simple ktap test with 1M alloc_skb/kfree_skb toy test from earlier email: trace skb:kfree_skb { if (arg2 == 0x100) {

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-03 Thread Alexei Starovoitov
On Tue, Dec 3, 2013 at 4:01 PM, Andi Kleen wrote: > Alexei Starovoitov writes: > > Can you do some performance comparison compared to e.g. ktap? > How much faster is it? imo the most interesting ktap scripts (like kmalloc-top.kp) need tables and timers. tables are almost ready for prime time,

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-03 Thread Masami Hiramatsu
(2013/12/04 3:26), Alexei Starovoitov wrote: > On Tue, Dec 3, 2013 at 7:33 AM, Steven Rostedt wrote: >> On Tue, 3 Dec 2013 10:16:55 +0100 >> Ingo Molnar wrote: >> >> >>> So, to do the math: >>> >>>tracing 'all' overhead: 95 nsecs per event >>>tracing 'eth5 + old filter'

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-03 Thread Andi Kleen
Alexei Starovoitov writes: Can you do some performance comparison compared to e.g. ktap? How much faster is it? While it sounds interesting, I would strongly advise to make this capability only available to root. Traditionally lots of complex byte code languages which were designed to be "safe"

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-03 Thread Alexei Starovoitov
On Tue, Dec 3, 2013 at 7:33 AM, Steven Rostedt wrote: > On Tue, 3 Dec 2013 10:16:55 +0100 > Ingo Molnar wrote: > > >> So, to do the math: >> >>tracing 'all' overhead: 95 nsecs per event >>tracing 'eth5 + old filter' overhead: 157 nsecs per event >>tracing 'eth5 + BPF

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-03 Thread Alexei Starovoitov
On Tue, Dec 3, 2013 at 1:16 AM, Ingo Molnar wrote: > > Very cool! (Added various other folks who might be interested in this > to the Cc: list.) > > I have one generic concern: > > It would be important to make it easy to extract loaded BPF code from > the kernel in source code equivalent form,

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-03 Thread Steven Rostedt
On Tue, 3 Dec 2013 10:16:55 +0100 Ingo Molnar wrote: > So, to do the math: > >tracing 'all' overhead: 95 nsecs per event >tracing 'eth5 + old filter' overhead: 157 nsecs per event >tracing 'eth5 + BPF filter' overhead: 54 nsecs per event > > So via BPF and a

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-03 Thread Masami Hiramatsu
(2013/12/03 13:28), Alexei Starovoitov wrote: > Hi All, > > the following set of patches adds BPF support to trace filters. > > Trace filters can be written in C and allow safe read-only access to any > kernel data structure. Like systemtap but with safety guaranteed by kernel. > > The user can

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-03 Thread Ingo Molnar
* Alexei Starovoitov wrote: > Hi All, > > the following set of patches adds BPF support to trace filters. > > Trace filters can be written in C and allow safe read-only access to > any kernel data structure. Like systemtap but with safety guaranteed > by kernel. Very cool! (Added various

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-03 Thread Ingo Molnar
* Alexei Starovoitov a...@plumgrid.com wrote: Hi All, the following set of patches adds BPF support to trace filters. Trace filters can be written in C and allow safe read-only access to any kernel data structure. Like systemtap but with safety guaranteed by kernel. Very cool! (Added

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-03 Thread Masami Hiramatsu
(2013/12/03 13:28), Alexei Starovoitov wrote: Hi All, the following set of patches adds BPF support to trace filters. Trace filters can be written in C and allow safe read-only access to any kernel data structure. Like systemtap but with safety guaranteed by kernel. The user can do:

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-03 Thread Steven Rostedt
On Tue, 3 Dec 2013 10:16:55 +0100 Ingo Molnar mi...@kernel.org wrote: So, to do the math: tracing 'all' overhead: 95 nsecs per event tracing 'eth5 + old filter' overhead: 157 nsecs per event tracing 'eth5 + BPF filter' overhead: 54 nsecs per event So via BPF

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-03 Thread Alexei Starovoitov
On Tue, Dec 3, 2013 at 1:16 AM, Ingo Molnar mi...@kernel.org wrote: Very cool! (Added various other folks who might be interested in this to the Cc: list.) I have one generic concern: It would be important to make it easy to extract loaded BPF code from the kernel in source code equivalent

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-03 Thread Alexei Starovoitov
On Tue, Dec 3, 2013 at 7:33 AM, Steven Rostedt rost...@goodmis.org wrote: On Tue, 3 Dec 2013 10:16:55 +0100 Ingo Molnar mi...@kernel.org wrote: So, to do the math: tracing 'all' overhead: 95 nsecs per event tracing 'eth5 + old filter' overhead: 157 nsecs per event

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-03 Thread Andi Kleen
Alexei Starovoitov a...@plumgrid.com writes: Can you do some performance comparison compared to e.g. ktap? How much faster is it? While it sounds interesting, I would strongly advise to make this capability only available to root. Traditionally lots of complex byte code languages which were

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-03 Thread Masami Hiramatsu
(2013/12/04 3:26), Alexei Starovoitov wrote: On Tue, Dec 3, 2013 at 7:33 AM, Steven Rostedt rost...@goodmis.org wrote: On Tue, 3 Dec 2013 10:16:55 +0100 Ingo Molnar mi...@kernel.org wrote: So, to do the math: tracing 'all' overhead: 95 nsecs per event tracing 'eth5 +

Re: [RFC PATCH tip 0/5] tracing filters with BPF

2013-12-03 Thread Alexei Starovoitov
On Tue, Dec 3, 2013 at 4:01 PM, Andi Kleen a...@firstfloor.org wrote: Alexei Starovoitov a...@plumgrid.com writes: Can you do some performance comparison compared to e.g. ktap? How much faster is it? imo the most interesting ktap scripts (like kmalloc-top.kp) need tables and timers. tables

[RFC PATCH tip 0/5] tracing filters with BPF

2013-12-02 Thread Alexei Starovoitov
Hi All, the following set of patches adds BPF support to trace filters. Trace filters can be written in C and allow safe read-only access to any kernel data structure. Like systemtap but with safety guaranteed by kernel. The user can do: cat bpf_program > /sys/kernel/debug/tracing/.../filter if

[RFC PATCH tip 0/5] tracing filters with BPF

2013-12-02 Thread Alexei Starovoitov
Hi All, the following set of patches adds BPF support to trace filters. Trace filters can be written in C and allow safe read-only access to any kernel data structure. Like systemtap but with safety guaranteed by kernel. The user can do: cat bpf_program /sys/kernel/debug/tracing/.../filter if