Re: [PATCH bpf-next 2/3] bpf: emit RECORD_MMAP events for bpf prog load/unload

2018-11-01 Thread Song Liu
Hi Arnaldo, On Wed, Oct 17, 2018 at 5:11 AM Arnaldo Carvalho de Melo wrote: > > Adding Alexey, Jiri and Namhyung as they worked/are working on > multithreading 'perf record'. I have read Alexey's work on enabling aio for perf-record (https://lkml.org/lkml/2018/10/15/169). But I feel it is not

Re: [PATCH bpf-next 2/3] bpf: emit RECORD_MMAP events for bpf prog load/unload

2018-10-17 Thread Alexei Starovoitov
On 10/17/18 2:31 PM, Arnaldo Carvalho de Melo wrote: > > Keep all that info in a file, as I described above. Or keep it for a > while, to give that thread in userspace time to get it and tell the > kernel that it can trow it away. stashing by kernel into a file is a huge headache, since format of

Re: [PATCH bpf-next 2/3] bpf: emit RECORD_MMAP events for bpf prog load/unload

2018-10-17 Thread Arnaldo Carvalho de Melo
Em Wed, Oct 17, 2018 at 07:08:37PM +, Alexei Starovoitov escreveu: > On 10/17/18 11:53 AM, Arnaldo Carvalho de Melo wrote: > > Em Wed, Oct 17, 2018 at 04:36:08PM +, Alexei Starovoitov escreveu: > >> On 10/17/18 8:09 AM, David Ahern wrote: > >>> On 10/16/18 11:43 PM, Song Liu wrote: >

Re: [PATCH bpf-next 2/3] bpf: emit RECORD_MMAP events for bpf prog load/unload

2018-10-17 Thread Alexei Starovoitov
On 10/17/18 11:53 AM, Arnaldo Carvalho de Melo wrote: > Em Wed, Oct 17, 2018 at 04:36:08PM +, Alexei Starovoitov escreveu: >> On 10/17/18 8:09 AM, David Ahern wrote: >>> On 10/16/18 11:43 PM, Song Liu wrote: I agree that processing events while recording has significant overhead. In

Re: [PATCH bpf-next 2/3] bpf: emit RECORD_MMAP events for bpf prog load/unload

2018-10-17 Thread Arnaldo Carvalho de Melo
Em Wed, Oct 17, 2018 at 04:36:08PM +, Alexei Starovoitov escreveu: > On 10/17/18 8:09 AM, David Ahern wrote: > > On 10/16/18 11:43 PM, Song Liu wrote: > >> I agree that processing events while recording has significant overhead. > >> In this case, perf user space need to know details about the

Re: [PATCH bpf-next 2/3] bpf: emit RECORD_MMAP events for bpf prog load/unload

2018-10-17 Thread Alexei Starovoitov
On 10/17/18 8:09 AM, David Ahern wrote: > On 10/16/18 11:43 PM, Song Liu wrote: >> I agree that processing events while recording has significant overhead. >> In this case, perf user space need to know details about the the jited BPF >> program. It is impossible to pass all these details to user

Re: [PATCH bpf-next 2/3] bpf: emit RECORD_MMAP events for bpf prog load/unload

2018-10-17 Thread Song Liu
Hi David, On Wed, Oct 17, 2018 at 8:09 AM David Ahern wrote: > > On 10/16/18 11:43 PM, Song Liu wrote: > > I agree that processing events while recording has significant overhead. > > In this case, perf user space need to know details about the the jited BPF > > program. It is impossible to pass

Re: [PATCH bpf-next 2/3] bpf: emit RECORD_MMAP events for bpf prog load/unload

2018-10-17 Thread Song Liu
On Wed, Oct 17, 2018 at 5:50 AM Arnaldo Carvalho de Melo wrote: > > Em Wed, Oct 17, 2018 at 09:11:40AM -0300, Arnaldo Carvalho de Melo escreveu: > > Adding Alexey, Jiri and Namhyung as they worked/are working on > > multithreading 'perf record'. > > > > Em Tue, Oct 16, 2018 at 11:43:11PM -0700,

Re: [PATCH bpf-next 2/3] bpf: emit RECORD_MMAP events for bpf prog load/unload

2018-10-17 Thread David Ahern
On 10/16/18 11:43 PM, Song Liu wrote: > I agree that processing events while recording has significant overhead. > In this case, perf user space need to know details about the the jited BPF > program. It is impossible to pass all these details to user space through > the relatively stable

Re: [PATCH bpf-next 2/3] bpf: emit RECORD_MMAP events for bpf prog load/unload

2018-10-17 Thread Arnaldo Carvalho de Melo
Em Wed, Oct 17, 2018 at 09:11:40AM -0300, Arnaldo Carvalho de Melo escreveu: > Adding Alexey, Jiri and Namhyung as they worked/are working on > multithreading 'perf record'. > > Em Tue, Oct 16, 2018 at 11:43:11PM -0700, Song Liu escreveu: > > On Tue, Oct 16, 2018 at 4:43 PM David Ahern wrote: >

Re: [PATCH bpf-next 2/3] bpf: emit RECORD_MMAP events for bpf prog load/unload

2018-10-17 Thread Arnaldo Carvalho de Melo
Adding Alexey, Jiri and Namhyung as they worked/are working on multithreading 'perf record'. Em Tue, Oct 16, 2018 at 11:43:11PM -0700, Song Liu escreveu: > On Tue, Oct 16, 2018 at 4:43 PM David Ahern wrote: > > On 10/15/18 4:33 PM, Song Liu wrote: > > > I am working with Alexei on the idea of

Re: [PATCH bpf-next 2/3] bpf: emit RECORD_MMAP events for bpf prog load/unload

2018-10-17 Thread Song Liu
Hi David, On Tue, Oct 16, 2018 at 4:43 PM David Ahern wrote: > > On 10/15/18 4:33 PM, Song Liu wrote: > > I am working with Alexei on the idea of fetching BPF program information via > > BPF_OBJ_GET_INFO_BY_FD cmd. I added PERF_RECORD_BPF_EVENT > > to perf_event_type, and dumped these events to

Re: [PATCH bpf-next 2/3] bpf: emit RECORD_MMAP events for bpf prog load/unload

2018-10-16 Thread David Ahern
On 10/15/18 4:33 PM, Song Liu wrote: > I am working with Alexei on the idea of fetching BPF program information via > BPF_OBJ_GET_INFO_BY_FD cmd. I added PERF_RECORD_BPF_EVENT > to perf_event_type, and dumped these events to perf event ring buffer. > > I found that perf will not process event

Re: [PATCH bpf-next 2/3] bpf: emit RECORD_MMAP events for bpf prog load/unload

2018-10-15 Thread Song Liu
On Fri, Sep 21, 2018 at 3:15 PM Alexei Starovoitov wrote: > > On Fri, Sep 21, 2018 at 09:25:00AM -0300, Arnaldo Carvalho de Melo wrote: > > > > > I have considered adding MUNMAP to match existing MMAP, but went > > > without it because I didn't want to introduce new bit in perf_event_attr > > >

Re: [PATCH bpf-next 2/3] bpf: emit RECORD_MMAP events for bpf prog load/unload

2018-09-21 Thread Alexei Starovoitov
On Fri, Sep 21, 2018 at 09:25:00AM -0300, Arnaldo Carvalho de Melo wrote: > > > I have considered adding MUNMAP to match existing MMAP, but went > > without it because I didn't want to introduce new bit in perf_event_attr > > and emit these new events in a misbalanced conditional way for prog >

Re: [PATCH bpf-next 2/3] bpf: emit RECORD_MMAP events for bpf prog load/unload

2018-09-21 Thread Peter Zijlstra
On Fri, Sep 21, 2018 at 09:25:00AM -0300, Arnaldo Carvalho de Melo wrote: > There is another longstanding TODO list entry: PERF_RECORD_MMAP records > should include a build-id I throught the problem was that the kernel doesn't have the build-id in the first place. So it cannot hand them out.

Re: [PATCH bpf-next 2/3] bpf: emit RECORD_MMAP events for bpf prog load/unload

2018-09-21 Thread Peter Zijlstra
On Fri, Sep 21, 2018 at 09:25:00AM -0300, Arnaldo Carvalho de Melo wrote: > > I consider synthetic perf events to be non-ABI. Meaning they're > > emitted by perf user space into perf.data and there is a convention > > on names, but it's not a kernel abi. Like RECORD_MMAP with > > event.filename ==

Re: [PATCH bpf-next 2/3] bpf: emit RECORD_MMAP events for bpf prog load/unload

2018-09-21 Thread Arnaldo Carvalho de Melo
Em Thu, Sep 20, 2018 at 08:14:46PM -0700, Alexei Starovoitov escreveu: > On Thu, Sep 20, 2018 at 03:56:51PM +0200, Peter Zijlstra wrote: > > On Thu, Sep 20, 2018 at 10:25:45AM -0300, Arnaldo Carvalho de Melo wrote: > > > PeterZ provided a patch introducing PERF_RECORD_MUNMAP, went nowhere due > >

Re: [PATCH bpf-next 2/3] bpf: emit RECORD_MMAP events for bpf prog load/unload

2018-09-20 Thread Alexei Starovoitov
On Thu, Sep 20, 2018 at 03:56:51PM +0200, Peter Zijlstra wrote: > On Thu, Sep 20, 2018 at 10:25:45AM -0300, Arnaldo Carvalho de Melo wrote: > > PeterZ provided a patch introducing PERF_RECORD_MUNMAP, went nowhere due > > to having to cope with munmapping parts of existing mmaps, etc. > > > > I'm

Re: [PATCH bpf-next 2/3] bpf: emit RECORD_MMAP events for bpf prog load/unload

2018-09-20 Thread Peter Zijlstra
On Thu, Sep 20, 2018 at 10:25:45AM -0300, Arnaldo Carvalho de Melo wrote: > PeterZ provided a patch introducing PERF_RECORD_MUNMAP, went nowhere due > to having to cope with munmapping parts of existing mmaps, etc. > > I'm still more in favour of introduce PERF_RECORD_MUNMAP, even if for > now it

Re: [PATCH bpf-next 2/3] bpf: emit RECORD_MMAP events for bpf prog load/unload

2018-09-20 Thread Peter Zijlstra
On Thu, Sep 20, 2018 at 10:44:24AM +0200, Peter Zijlstra wrote: > On Wed, Sep 19, 2018 at 03:39:34PM -0700, Alexei Starovoitov wrote: > > void bpf_prog_kallsyms_del(struct bpf_prog *fp) > > { > > + unsigned long symbol_start, symbol_end; > > + /* mmap_record.filename cannot be NULL and has

Re: [PATCH bpf-next 2/3] bpf: emit RECORD_MMAP events for bpf prog load/unload

2018-09-20 Thread Arnaldo Carvalho de Melo
Em Thu, Sep 20, 2018 at 10:44:24AM +0200, Peter Zijlstra escreveu: > On Wed, Sep 19, 2018 at 03:39:34PM -0700, Alexei Starovoitov wrote: > > void bpf_prog_kallsyms_del(struct bpf_prog *fp) > > { > > + unsigned long symbol_start, symbol_end; > > + /* mmap_record.filename cannot be NULL and

Re: [PATCH bpf-next 2/3] bpf: emit RECORD_MMAP events for bpf prog load/unload

2018-09-20 Thread Peter Zijlstra
On Wed, Sep 19, 2018 at 03:39:34PM -0700, Alexei Starovoitov wrote: > void bpf_prog_kallsyms_del(struct bpf_prog *fp) > { > + unsigned long symbol_start, symbol_end; > + /* mmap_record.filename cannot be NULL and has to be u64 aligned */ > + char buf[sizeof(u64)] = {}; > + > if

Re: [PATCH bpf-next 2/3] bpf: emit RECORD_MMAP events for bpf prog load/unload

2018-09-19 Thread Song Liu
> On Sep 19, 2018, at 5:59 PM, Alexei Starovoitov wrote: > > On 9/19/18 4:44 PM, Song Liu wrote: >> >> >>> On Sep 19, 2018, at 3:39 PM, Alexei Starovoitov wrote: >>> >>> use perf_event_mmap_bpf_prog() helper to notify user space >>> about JITed bpf programs. >>> Use RECORD_MMAP perf event

Re: [PATCH bpf-next 2/3] bpf: emit RECORD_MMAP events for bpf prog load/unload

2018-09-19 Thread Alexei Starovoitov
On 9/19/18 4:44 PM, Song Liu wrote: On Sep 19, 2018, at 3:39 PM, Alexei Starovoitov wrote: use perf_event_mmap_bpf_prog() helper to notify user space about JITed bpf programs. Use RECORD_MMAP perf event to tell user space where JITed bpf program was loaded. Use empty program name as unload

Re: [PATCH bpf-next 2/3] bpf: emit RECORD_MMAP events for bpf prog load/unload

2018-09-19 Thread Song Liu
> On Sep 19, 2018, at 3:39 PM, Alexei Starovoitov wrote: > > use perf_event_mmap_bpf_prog() helper to notify user space > about JITed bpf programs. > Use RECORD_MMAP perf event to tell user space where JITed bpf program was > loaded. > Use empty program name as unload indication. > >