[iovisor-dev] minutes: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Hi all,

Thanks for joining in the discussion today. As usual, here are my notes.

-Brenden

=== Discussion ===

Brenden:
- Adding fedora 28 buildbot/packages
   - debugging some test failures
- Tag release 0.7.0 after that

Yonghong:
- Some more python3 fixes
- Some rewriter changes, breakages
- Tool to do introspection for kprobe, uprobe

John:
- Working for certain classes of loops
   - some caveats with searching for loop variable declarations
   - only considers first order induction variables

Bjorn:
- some concerns about descriptor format
   - target to update next week
- to disable support in the meantime
   - kconfig/uapi header removal?
   - return not supported for socket options in meantime
- support for brcm, mlnx next merge window

Saeed:
- for AF_XDP, starting work on xdp offload for metadata/flow-mark concept
   - when loading xdp program, request specific offloads
 - e.g. vlan stripping, modifies headroom calculation
 - flow hash in metadata, avoid caching packet before forwarding w/
cpuhash
   - to come up with a proposal next week

=== Attendees ===
Brenden Blanco
Alexei Starovoitov
Andy Gospodarek
Bjorn Topel
Brendan Gregg
Jesper Brouer
Magnus
Yonghong Song
Quentin Monnet
Saeed
Kuba Kicinski
Alexander Duyck
JohnF
William Tu
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] reminder: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Please join us tomorrow for our bi-weekly call. As usual, this meeting is
open to everybody and completely optional.
You might be interested to join if:
You want to know what is going on in BPF land
You are doing something interesting yourself with BPF and would like to
share
You want to know what the heck BPF is

=== IO Visor Dev/TSC Meeting ===

Every 2 weeks on Wednesday, from Wednesday, January 25, 2017, to no end date
11:00 am  |  Pacific Daylight Time (San Francisco, GMT-07:00)  |  30 min

https://bluejeans.com/568677804/

https://www.timeanddate.com/worldclock/meetingdetails.html?year=2018=5=30=18=0=0=900
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] minutes: IO Visor TSC / Dev Meeting

[Brenden Blanco via iovisor-dev]

Hi All,

Thanks for joining the discussion today. As usual, here are my notes.

-Brenden

=== Discussion ===

John:
- sockmap fixes pushed
  - fixed broken tests
- sockmap hash impl next
- bounded loop rfc making progress
- cilium code using sockhash almost finished as well

Yonghong:
- python3 test cleanup
  - except memleak tool
- raw tracepoint support in bcc
  - runqlat tool
  - bio tracing
  - more precise than kprobes
- next work: introspection - bpf trace program in perf

Martin:
- BTF id patch WIP
  - BTF id -> BTF map
- next: support function pointer type in BTF

William:
- moving bpf tunnel example -> selftest
- hitting issue with long running test tunnel cases
  - try enabling kasan to debug
- verifier issue bugfix (ctx + constant arithmetic) WIP

Daniel:
- finalizing removal of some inst from jit
- few bugfixes in x86 jit

Alexei:
- AF_XDP patches being reviewed, looks ok

Jesper:
- Testing AF_XDP patches currently
  - no driver changes needed with xdp_redirect
  - driver changes still needed for zerocopy
  - intel: 12mpps, mlnx: 9mpps drop

Brendan:
- CFP for USENIX LISA in October, Nashville TN

=== Attendees ===
Brenden Blanco
Alex Reece
Jakub Kicinski
Martin Lau
Quentin Monnet
Sandipan
Jiong Wang
William Tu
Alexei Starovoitov
Alexander Duyck
John Fastabend
Jesper Brouer
Daniel Borkmann
Yonghong Song
Brendan Gregg
Ferris
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] reminder: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Please join us tomorrow for our bi-weekly call. As usual, this meeting is
open to everybody and completely optional.
You might be interested to join if:
You want to know what is going on in BPF land
You are doing something interesting yourself with BPF and would like to share
You want to know what the heck BPF is

=== IO Visor Dev/TSC Meeting ===

Every 2 weeks on Wednesday, from Wednesday, January 25, 2017, to no end date
11:00 am  |  Pacific Daylight Time (San Francisco, GMT-07:00)  |  30 min

https://bluejeans.com/568677804/

https://www.timeanddate.com/worldclock/meetingdetails.html?year=2018=5=2=18=0=0=900
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] minutes: IO Visort TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Hi All,

Thanks for attending the meeting today. As usual, here are my notes.

Cheers,
Brenden

=== Discussion ===

Alexei
- bpf-next queue is quite large, please have patience with delays in
clearing
  the backlog
- please help review others patches

Brenden
- Fixing image test issues in bcc
- To add python3 test infra

Yonghong
- some kernel fixes
- python3 script fixes
- asking for help adding python3 tests
- please help review bpfd changes

Jesper
- debugging performance regression, retpoline issue
  - getting help from dma experts
  - other issues may come with map lookup indirect accesses
- followup patches from xdp page return api

Sebastiano
- add map+prog id to tracepoint
  - can it be done with kprobe + tracepoint?
  - what is the use case, since it can already be done other ways?

Bjorn
- AF_XDP updates
  - non-rfc patchset to push asap
  - looking at different ring implementations
  - alternatives are each visible api change

Daniel
- Documentation guide!
  - http://cilium.readthedocs.io/en/latest/bpf/

William
- Trying AF_XDP
  - seeing if can be used by ovs datapath
  - seeing different perf numbers, 10g ixgbe
  - will try again with next patch spin
- Moving around examples/selftest code
- Some missing ipv6 tunnel test code

Jiong
- Control flow graph in verifier
  - dom tree to detect loops
  - please post rfc patches

Martin
- BTF patches posted
  - whether to have embedded pretty printer in the kernel
  - possible use in embedded environments to avoid extra binaries
  - should never support writing to maps from bpffs
  - bpftool should be main point of entry

=== Attendees ===
Alexei Starovoitov
Brenden Blanco
William Tu
Andy Gospodarek
Bjorn Topel
Daniel Borkmann
Jakub Kicinski
Jesper Brouer
Jiong Wang
Quentin Monnet
Sandipan
Sebastiano
Ferris
George Wilson
Yonghong Song
Martin Lau
Brendan Gregg
Alex Reece
Joe Stringer
Yi-Hung Wei
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] reminder: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Please join us tomorrow for our bi-weekly call. As usual, this meeting is
open to everybody and completely optional.
You might be interested to join if:
You want to know what is going on in BPF land
You are doing something interesting yourself with BPF and would like to
share
You want to know what the heck BPF is

=== IO Visor Dev/TSC Meeting ===

Every 2 weeks on Wednesday, from Wednesday, January 25, 2017, to no end date
11:00 am  |  Pacific Daylight Time (San Francisco, GMT-07:00)  |  30 min

https://bluejeans.com/568677804/

https://www.timeanddate.com/worldclock/meetingdetails.html?year=2018=4=18=18=0=0=900
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] minutes: IO VIsor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Hi All,

Thanks for attending the meeting today. Here are my notes from the call.

Cheers,
Brenden

=== Status ===

Yonghong
- Debugging incorrect user stack
  - fastpath vs slowpath register difference
  - should be updated in 4.9+ (LTS flavors)
- user stack id - doesn't have full access, trying to improve
- bpf perf event introspection

Daniel
cgroup bind and connect patches merged
- 3 new hooks for container management - used by fb
sockmap ingress support added
raw tracepoint
- use all tp arguments directly from bpf
some prep patches for upcoming features
- btf, few others
clang compiled kernel issue with bpffs
- c lang constant merging flag turned off

Joe
sock lookup progress
- improvements to verifier for tracking reference scope
- next rfc soon

John
Some patches merged, some just missed this window
- next support for lookup by hash
Adding Cilium support for sockmap soon
Rearranging selftests/examples for sockmap

Jakub
adding support for offloading of a few features
- inline map helpers
- atomic add
- some performance improvements
Next window - perf ring output support
Working on 32 bit register support in llvm

Martin
BTF support V2
- pretty print from kernel
- includes pahole converter
next: LLVM directly generate BTF with -g

=== Attendees ===
Dan Siemon
Brenden Blanco
Alex Reece
Alexander Duyck
Daniel Borkmann
Jakub Kicinski
Joe Stringer
Mauricio Vasquez
Yonghong Song
Martin
Quentin Monnet
Tom
George Wilson
Andy Gospodarek
Jiong Wang
Saeed
Francois
Joel
jcanseco
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] reminder: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Please join us tomorrow for our bi-weekly call. As usual, this meeting is
open to everybody and completely optional.
You might be interested to join if:
You want to know what is going on in BPF land
You are doing something interesting yourself with BPF and would like to share
You want to know what the heck BPF is

=== IO Visor Dev/TSC Meeting ===

Every 2 weeks on Wednesday, from Wednesday, January 25, 2017, to no end date
11:00 am  |  Pacific Daylight Time (San Francisco, GMT-07:00)  |  30 min

https://bluejeans.com/568677804/

https://www.timeanddate.com/worldclock/meetingdetails.html?year=2018=4=4=18=0=0=900
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] minutes: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Hi all,

Thanks for joining the discussion today, as usual my notes of the discussion
are included below.

Cheers,
Brenden

=== Discussion ===

Alexei
- microkernel/umh work - see lkml

Yonghong
- working on skb segmentation problem
  - ipv4/v6 transition
  - bpf on bpf clsact ingress

Brenden
- bcc release?
  - planning to go with a 1.0 tag soon

Ferris
- rust library for interfacing with bpf
  - more details to be announced in the future

Jesper
- Pushing internally to get bpf support officially enabled (rhel8)
- some progress detected on af_xdp patches

John
- sockmap sendmsg/sendfile merged
- ingress redirect upcoming
- sockmap hashmap lookup support
- fixes for mem charge leak
- reorganize samples into selftests
- documentation work

Brendan
- some minor tools update to use tracepoints
- some tools written again bpftrace

Martin
- CTF patches almost ready, to be posted in a couple weeks


=== Attendees ===
Brenden Blanco
Andy Gospodarek
Daniel Borkmann
Ferris
Logan Blyth
Tom
Alexei
Joe Stringer
Jesper Brouer
Jakub Kicinski
George Wilson
Yonghong Song
Martin Lau
Quentin Monnet
Edwin Peer
Brendan Gregg
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] reminder: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Please join us tomorrow for our bi-weekly call. As usual, this meeting is
open to everybody and completely optional.
You might be interested to join if:
You want to know what is going on in BPF land
You are doing something interesting yourself with BPF and would like to share
You want to know what the heck BPF is

=== IO Visor Dev/TSC Meeting ===

Every 2 weeks on Wednesday, from Wednesday, January 25, 2017, to no end date
11:00 am  |  Pacific Daylight Time (San Francisco, GMT-07:00)  |  30 min

https://bluejeans.com/568677804/

https://www.timeanddate.com/worldclock/meetingdetails.html?year=2018=3=21=18=0=0=900
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] minutes: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Hi all,

Thanks for joining the call today! Please see my notes from the
discussion included below.

=== Status ===

Sargun:
seccomp-ebpf - verifier hurdles
- wants synchronous filtering as minimum feature to allow the verifier risk
- current rcu locking approach was nacked
discussion about how bpf punts to userspace daemon
- problems with passing metadata back and forth

Jesper:
xdp redirect memory return api rfc
- patches under review on ml
- some questions about dma mapping in the api
- how to unmap page when device has already gone down?
- or when page pool is full?

Issue: enabling redirect to other device with no xdp rx program running
currently results in silent failure.
- Have an ethtool flag to enable tx devices -> reconfigure queues?

Brendan:
bpftrace - higher level frontend to bcc/bpf
- making good progress
  - 70% feature complete
  - symbol resolution missing
  - dereference of c structs
- depends on ctf?
- test suite exists
  - uses llvm-ir diff against previous builds to sanity check
- packaging?
  - tbd


=== Attendees ===
Brenden Blanco
Quentin Monnet
Tom
Alexander Duyck
Jiong Wang
Sandipan
Sargun Dhillon
Jakub Kicinski
Saeed
Daniel Borkmann
Ferris
Jesper Brouer
Brendan Gregg
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] reminder: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Please join us tomorrow for our bi-weekly call. As usual, this meeting is
open to everybody and completely optional.
You might be interested to join if:
You want to know what is going on in BPF land
You are doing something interesting yourself with BPF and would like to share
You want to know what the heck BPF is

=== IO Visor Dev/TSC Meeting ===

Every 2 weeks on Wednesday, from Wednesday, January 25, 2017, to no end date
11:00 am  |  Pacific Daylight Time (San Francisco, GMT-08:00)  |  30 min

https://bluejeans.com/568677804/

https://www.timeanddate.com/worldclock/meetingdetails.html?year=2018=3=7=19=0=0=900
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] minutes: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Hi all,

Thanks for attending the call this week. My notes are included below.

=== Updates ===

DTrace GPLed in Dec 2017
- To help with Oracle Linux port
- Should we take dtrace kernel code into linux?
  - opinion: probably not necessary now, more applicable 5 years ago
- How about dtrace frontend?
  - may have more short term value
  - combine with bpftrace?
  - who would work on it?
  - userspace tools are now UPL - how does that work with apache2?
- CTF
  - currently forking the format into BTF
  - exceed 32k types, other limitations
  - still, is the code useful to combine?

bpfilter announcement
- lots of lwn/reddit/hackernews comments

Brenden:
- Fixes for python3 string encoding
  - please test tools with python3 -W and submit PRs/issues
- todo: push clang ir rewriter to github branch

Yonghong:
- Kernel related bugfixes
  - memory leak in shared lpm map

Daniel:
- bpfilter work
- retpoline work in tail call code of x86 jit
  - also arm64

John:
- some more sockmap changes
  - tx hooks
  - more testing, cleanup
  - optimizations in followup patches

Joe:
- Soliciting feedback for socket sharing patches
  - looking into storing data along with socket
- scratch accessible from bpf program
- How to protect scratch area?
  - new lock helper idea

Sargun:
- Asking for feedback on seccomp ebpf changes
  - desires access to read-only map for
  - use case for scratch buffer primitive as well

Brendan:
- New pcp-bcc-pmda plugin by redhat
  - tbd: integrate into vector

Jesper:
- XDP redirect work
  - bugfixes
  - page free and return API next
- getting help from mellanox devs
- XDP bulk xmit for rx

=== Attendees ===
Brenden Blanco
Eduard Serra
Alexei Starovoitov
Joe String
Jakub Kicinski
Jiong Wang
Mauricio Vasquez
Quentin Monnet
Saeed
Sandipan
Sargun Dhillon
Brendan Gregg
Yonghong Song
Jesper Brouer
Alexander Duyck
Edwin Peer
PJ Waskiewicz
Daniel Borkmann
Nic Viljoen
John Fastabend
Alexander Duyck
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

Re: [iovisor-dev] minutes: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

On Wed, Feb 21, 2018 at 11:55 AM, Brenden Blanco  wrote:
> Hi all,
>
> Thanks for attending the call this week. My notes are included below.
>
> === Updates ===
>
> DTrace GPLed in Dec 2017
> - To help with Oracle Linux port
> - Should we take dtrace kernel code into linux?
>   - opinion: probably not necessary now, more applicable 5 years ago
> - How about dtrace frontend?
>   - may have more short term value
>   - combine with bpftrace?
>   - who would work on it?
>   - userspace tools are now UPL - how does that work with apache2?
> - CTF
>   - currently forking the format into BTF
>   - exceed 32k types, other limitations
>   - still, is the code useful to combine?
>
> bpfilter announcement
> - lots of lwn/reddit/hackernews comments
>
> Brenden:
> - Fixes for python3 string encoding
>   - please test tools with python3 -W and submit PRs/issues
> - todo: push clang ir rewriter to github branch

As requested, I have pushed the WIP code to
https://github.com/iovisor/bcc/tree/ir-probe-rewrite. Only one of the
two test cases currently works. The basic idea is to use the pointer
alias info to track which loads read from on-stack memory and maps,
and treat the rest as unsafe reads and rewrite them into
bpf_probe_read calls. Map data is not yet handled.

>
> Yonghong:
> - Kernel related bugfixes
>   - memory leak in shared lpm map
>
> Daniel:
> - bpfilter work
> - retpoline work in tail call code of x86 jit
>   - also arm64
>
> John:
> - some more sockmap changes
>   - tx hooks
>   - more testing, cleanup
>   - optimizations in followup patches
>
> Joe:
> - Soliciting feedback for socket sharing patches
>   - looking into storing data along with socket
> - scratch accessible from bpf program
> - How to protect scratch area?
>   - new lock helper idea
>
> Sargun:
> - Asking for feedback on seccomp ebpf changes
>   - desires access to read-only map for
>   - use case for scratch buffer primitive as well
>
> Brendan:
> - New pcp-bcc-pmda plugin by redhat
>   - tbd: integrate into vector
>
> Jesper:
> - XDP redirect work
>   - bugfixes
>   - page free and return API next
> - getting help from mellanox devs
> - XDP bulk xmit for rx
>
> === Attendees ===
> Brenden Blanco
> Eduard Serra
> Alexei Starovoitov
> Joe String
> Jakub Kicinski
> Jiong Wang
> Mauricio Vasquez
> Quentin Monnet
> Saeed
> Sandipan
> Sargun Dhillon
> Brendan Gregg
> Yonghong Song
> Jesper Brouer
> Alexander Duyck
> Edwin Peer
> PJ Waskiewicz
> Daniel Borkmann
> Nic Viljoen
> John Fastabend
> Alexander Duyck
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] reminder: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Please join us tomorrow for our bi-weekly call. As usual, this meeting is
open to everybody and completely optional.
You might be interested to join if:
You want to know what is going on in BPF land
You are doing something interesting yourself with BPF and would like to share
You want to know what the heck BPF is

=== IO Visor Dev/TSC Meeting ===

Every 2 weeks on Wednesday, from Wednesday, January 25, 2017, to no end date
11:00 am  |  Pacific Daylight Time (San Francisco, GMT-08:00)  |  30 min

https://bluejeans.com/568677804/

https://www.timeanddate.com/worldclock/meetingdetails.html?year=2018=2=21=19=0=0=900
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] reminder: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

 Please join us tomorrow for our bi-weekly call. As usual, this meeting is
open to everybody and completely optional.
You might be interested to join if:
You want to know what is going on in BPF land
You are doing something interesting yourself with BPF and would like to
share
You want to know what the heck BPF is

=== IO Visor Dev/TSC Meeting ===

Every 2 weeks on Wednesday, from Wednesday, January 25, 2017, to no end date
11:00 am  |  Pacific Daylight Time (San Francisco, GMT-08:00)  |  30 min

https://bluejeans.com/568677804/

https://www.timeanddate.com/worldclock/meetingdetails.html?year=2018=2=7=19=0=0=900
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] reminder: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Please join us tomorrow for our bi-weekly call. As usual, this meeting is
open to everybody and completely optional.
You might be interested to join if:
You want to know what is going on in BPF land
You are doing something interesting yourself with BPF and would like to share
You want to know what the heck BPF is

=== IO Visor Dev/TSC Meeting ===

Every 2 weeks on Wednesday, from Wednesday, January 25, 2017, to no end date
11:00 am  |  Pacific Daylight Time (San Francisco, GMT-08:00)  |  30 min

https://bluejeans.com/568677804/

https://www.timeanddate.com/worldclock/meetingdetails.html?year=2018=1=24=19=0=0=900
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] minutes: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Hi All,

Thanks for attending today. Below you will find my notes from the call.

Cheers,
Brenden

=== Status ===

Alexei:
- Security updates for BPF
- Patching uses of BPF for speculative execution
  - variant 1, one more patch in progress
  - variant 2, option to toggle off interpreter

Yonghong:
- Fixes for bcc
- kernel fixes for get next key
  - stack map iteration
  - goal: keep consistent with hash map iteration
- issues for usdt, probe name overlap

Brenden:
- Some work on CTF generation

Daniel:
- Working on spectre patches and bugfixes

JohnF:
- sockmap tx hook
  - monitor sendmsg/sendpage
- sample improvements to ease development/debugging/benchmarking

Jesper:
- xdp rx queue merged
  - tracking followup fixes
  - need to extend for memory return api
- todo: cpumap for generic xdp
- add cpumap support into suricata
- xdp_data_meta - want to combine skb->mark+cpumap to fine tune multicore
  handling of packets


=== Attendees ===
Brenden Blanco
Alexander Duyck
Daniel Borkmann
Edwin Peer
Jakub Kicinski
Jesper Brouer
Jiong Wang
Quentin Monnet
Sandipan
Marco Leogrande
Mauricio Vasquez
Nic Viljoen
Andy Gospodarek
Maciej Fijalkowski
Yonghong Song
Rony Efraim
Alexei Starovoitov
Gordon Marler
John Fastabend
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] reminder: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Please join us tomorrow for our bi-weekly call. As usual, this meeting is
open to everybody and completely optional.
You might be interested to join if:
You want to know what is going on in BPF land
You are doing something interesting yourself with BPF and would like to share
You want to know what the heck BPF is

=== IO Visor Dev/TSC Meeting ===

Every 2 weeks on Wednesday, from Wednesday, January 25, 2017, to no end date
11:00 am  |  Pacific Daylight Time (San Francisco, GMT-08:00)  |  30 min

https://bluejeans.com/568677804/

https://www.timeanddate.com/worldclock/meetingdetails.html?year=2018=1=10=19=0=0=900
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] minutes: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Thanks all for attending today. Below you may find the notes from the call. We
will be skipping the next call and resuming on Jan 10. See you all in the new
year!

=== Status ===
Yonghong:
- bcc: Fixed struct with alignment/int128 quirks
  - match in python
- kernel: query prog array
  - used for event hooks with multiple programs attached
  - debugging use case
- kernel: uprobe speedup patch landed

Jesper:
- submitted

Quentin:
- working on bpf syscall man page / helpers man page
  - where should this work go?
  - A: should go in kernel.org somewhere
 git.kernel.org (this includes kernel, libc, etc.)
if automatically generated, need to occassionally sync from kernel to
  manpage repo
sync once per release
  - contact Michael Kerisk

Jiong:
- Continuing 32bit bpf support in clang
  - support for rotate instruction? more optimal than shift combinations
- benchmark with jhash inlined use cases
  - send experimental patches to yonghong/daniel

Daniel:
- Some ptregs fixes into bpf tree
- Adding development documentation
- Fixes for some jits
- Work continues on bpf qdisc

John:
- Porting sockmap for other use case (ULP layer)
- Reviewing call patches from Alexei

Rony:
- Q: Progress on BPF offload?
  - A: not much discussion yet
  - What does a BPF (offloaded) -> BPF (XDP) interface look like?
- some vendors may accept P4 -> microcode (through firmware api)
  that can as fallback be compiled into BPF
- should we avoid an out of band api for offloading the parsing program
  into silicon?
  - John has tried such APIs in the past, and rejected due to not wanting
to add new api besides bpf+tc

=== Attendees ===
Brenden Blanco
Jakub Kicinski
Jiong Wang
Quentin Monnet
Rony Efraim
Mauricio Vasquez
Andy Gospodarek
Alexei Starovoitov
Yonghong Song
Daniel Borkmann
John Fastabend
Brendan Gregg
Nic Viljoen
Edwin Peer
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] reminder: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Please note, California observes daylight saving time, and hence the
time for some timezones may have changed since last meeting. Please
confirm using the timeanddate.com link at the bottom.

Please join us tomorrow for our bi-weekly call. As usual, this meeting is
open to everybody and completely optional.
You might be interested to join if:
You want to know what is going on in BPF land
You are doing something interesting yourself with BPF and would like to share
You want to know what the heck BPF is

=== IO Visor Dev/TSC Meeting ===

Every 2 weeks on Wednesday, from Wednesday, January 25, 2017, to no end date
11:00 am  |  Pacific Daylight Time (San Francisco, GMT-08:00)  |  30 min

https://bluejeans.com/568677804/

https://www.timeanddate.com/worldclock/meetingdetails.html?year=2017=11=15=19=0=0=900
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] minutes: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Hi All,

Thanks for attending the meeting today, here are my notes.

=== Status ===
Netdev 2.2 next week
- Live streaming available
  - link to be sent out when available

Brenden:
- Increase cadence for updating package version
  - todo: update 0.4.1

Alexei:
- Work proceeds on call between programs
  - bpf loader will start to look like a linker
  - still no loops
  - arbitrary argument definition

JohnF:
- work on sockmap
  - tcp hook
  - multiple cgroups
- bounded for loops
  - rough sketch
  - detect nested loops = reject
  - finding linearly increasing/decreasing loops
- finally, what is the bound of the loop
  - what about counters that are on stack instead of register?

Yonghong:
- fixing arm64 issues
  - netperf missing, other tests passing
- uprobe performance improvements

Martin:
- added small bpf debugging tool to bcc

Kenny:
- TPM from LF
  - Hi!

JohnB:
- Also saying Hi from Netflix

=== Attendees ===
Jakub Kicinski
Kenny Paul
Marco Leogrande
Panagiotis Moustafe
Quentin Monnet
Alexei Starovoitov
Brenden Blanco
John Bennett
Alexander Duyck
Yonghong Song
Prem
Nic Viljoen
John Fastabend
Teng Qin
Kenny Paul
Jesper Brouer
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] reminder: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Please join us tomorrow for our bi-weekly call. As usual, this meeting is
open to everybody and completely optional.
You might be interested to join if:
You want to know what is going on in BPF land
You are doing something interesting yourself with BPF and would like to share
You want to know what the heck BPF is

=== IO Visor Dev/TSC Meeting ===

Every 2 weeks on Wednesday, from Wednesday, January 25, 2017, to no end date
11:00 am  |  Pacific Daylight Time (San Francisco, GMT-07:00)  |  30 min

https://bluejeans.com/568677804/

https://www.timeanddate.com/worldclock/meetingdetails.html?year=2017=11=1=18=0=0=900
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

Re: [iovisor-dev] iovisor.org certificates

[Brenden Blanco via iovisor-dev]

Hi Asif,

The repo server has a certificate from Lets Encrypt. I would check with the
list of CAs installed in your distro to confirm that Lets Encrypt is
supported, and Google around for packages to update from.

On Wed, Oct 25, 2017 at 7:06 PM, Haswarey, Asif via iovisor-dev <
iovisor-dev@lists.iovisor.org> wrote:

> Hi!
>
>
>
> I am trying to do an apt-file update, but I get the following
>
> error:
>
>
>
>  ERROR MESSAGE BEGIN
>
> curl: (60) server certificate verification failed. CAfile:
> /etc/ssl/certs/ca-certificates.crt CRLfile: none
>
> More details here: http://curl.haxx.se/docs/sslcerts.html
>
>
>
> curl performs SSL certificate verification by default, using a "bundle"
>
> of Certificate Authority (CA) public keys (CA certs). If the default
>
> bundle file isn't adequate, you can specify an alternate file
>
> using the --cacert option.
>
> If this HTTPS server uses a certificate signed by a CA represented in
>
> the bundle, the certificate verification probably failed due to a
>
> problem with the certificate (it might be expired, or the name might
>
> not match the domain name in the URL).
>
> If you'd like to turn off curl's verification of the certificate, use
>
> the -k (or --insecure) option.
>
> Download of https://repo.iovisor.org/apt/xenial/dists/xenial-nightly/
> Contents-amd64.gz failed
>
>  ERROR MESSAGE END
>
>
>
> Is there a way for me to get iovisor.org’s certificate to make this work?
>
> I **do not** want to use the –k insecure option in /etc/apt/apt-file.conf:
>
>
>
> https = diffindex-download –k -i "/dists//Contents-.gz"
> /
>
>
>
> Thanks for your time!
>
> _
>
> Asif
>
> ___
> iovisor-dev mailing list
> iovisor-dev@lists.iovisor.org
> https://lists.iovisor.org/mailman/listinfo/iovisor-dev
>
>
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] minutes: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Hi All,

Thanks for attending the call today. Below are my notes.

=== Status ===

Alexei:
- 1 patch / day
- burdensome on alexei/daniel
  - how to scale?
  - to be discussed at netconf
- working on calls
  - can load relatively large files
  - rfc to come before netdev
  - memory pressure in the verifier
- First TCP tracepoints accepted
  - 5 more coming soon
  - tcp set state has some corner case where it is missed

Daniel:
- fixes for verifier
  - llvm generates some compare/jumps that aren't covered
  - off by 1 in xdp data_end checks
- lpm needs get next key helper for dumping to userspace
- plan to support more drivers for xdp metadata

Brendan:
- LISA in a couple weeks - 2 bpf talks from Sasha, 1 from Brendan

Jakub:
- How to print 2nd phase bpf verifier output from offloaded program?
  - quirk: it runs at attach time, not load time
  - Maybe try a dry run somehow?
  - Intermixing of driver specific errors?
  - Same problem for maps
  - Include offload check at load time instead
  - New system call for "BPF_PROG_PREPARE", in between load and offload
- jit a loaded bpf prog for a specific arch/offload

=== Attendees ===
Marco Leogrande
Brenden Blanco
Alexander Duyck
Daniel Borkmann
Jakub Kicinski
Carlos Neira
Quentin Monnet
Alexei Starovoitov
Jiong
Brendan Gregg
Mauricio Vasquez
Nic Viljoen
Gordon Marler
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] reminder: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Please join us tomorrow for our bi-weekly call. As usual, this meeting is
open to everybody and completely optional.
You might be interested to join if:
You want to know what is going on in BPF land
You are doing something interesting yourself with BPF and would like to share
You want to know what the heck BPF is

=== IO Visor Dev/TSC Meeting ===

Every 2 weeks on Wednesday, from Wednesday, January 25, 2017, to no end date
11:00 am  |  Pacific Daylight Time (San Francisco, GMT-07:00)  |  30 min

https://bluejeans.com/568677804/

https://www.timeanddate.com/worldclock/meetingdetails.html?year=2017=10=18=18=0=0=900
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] minutes: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Hi all,

Thanks for attending the call today. Here are my notes. Also, please be advised
that we will be moving to a 30 min time slot going forward.

=== Status ===

Yonghong:
- Added source debugging printer for bcc
  - Requires 6.0 (possible but needs volunteers to backport to 5.0)
- Permit multiple programs to attach to single tracepoint/probe
  - Uses ast's prog array for cgroups style attachment
- Validating arm64 support
  - May help with arm32 support too

Alexei:
- Still working on calling routines (6 months and counting)
  - Big changes in verifier

Jesper:
- cpumap type available with xdp redirect

Daniel:
- Metadata->skb landed
  - Other drivers to come
- Integrating xdp support into cilium

John:
- Fixes for sockmap
  - Adding support for ingress flags

To-do list @ github.com/iovisor/bcc/issues/574 has been updated
- stringmap and bounded loops are important
- network protocol parsing - lots of looping requirements
  - http2, etc.

Panagiotis (elasticcloud)
- Interested in security (networking/sandboxing)
- hitting limitations of seccomp-bpf and no pointer dereferences
  - is aware of landlock, to be looked into
  - should address needs

Andy:
- Close to adding bcm xdp_redirect suppport

=== Attendees ===
Brenden Blanco
Marco Leogrande
Quentin Monnet
Jakub Kicinski
Mauricio Vasquez
Alexander Duyck
Yonghong Song
Alexei Starovoitov
Andy Gospodarek
Panagiotis Moustafellos
Jesper Brouer
Nic Viljoen
Daniel Borkmann
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] minutes: IO Visor TSC/Dev meeting

[Brenden Blanco via iovisor-dev]

Hi All,

Thanks for attending the call yesterday, here are my notes:


=== News ===

Linux Plumbers was last week, and included an increasing focus on BPF for
tracing and other use cases over previous years. The tracing/BPF track on Friday
had a very packed agenda, only allowing 10-15 minutes per talk. For full notes
on the discussions, please check out the etherpad at:
https://etherpad.openstack.org/p/LPC2017_Tracing

Other upcoming conferences:
Kernel Recipes - Sept 27-29
https://kernel-recipes.org/en/2017/
Brendan - Performance Analysis with BPF

Netdev 2.2 - Nov 8-10
https://netdevconf.org/2.2/
Last day to submit a talk was yesterday, registration is still open


=== Developer Status ===
Brenden
- Work started on a LLVM Debug Info traverser for BCC
  - Should be able to generate CTF data

Martin
- Investigating CTF converter for use in kernel

Jesper
- new map type - cpu
  - like netdev map
  - redirecting to another cpu
  - example: ddos on first cpu
- queue non-dropped packets to other cpu
- other cpu allocs skb

Martin
- worked on tool to iterate over bpf programs and which maps they are using
  - Nic suggests a similar tool from Jakub to iterate programs
   - https://github.com/Netronome/bpf-tool

Jiong
- Working on 32 bit bpf backend in llvm
  - (makes netronome happy)
- Please let the community know when something is ready and we can help with
  testing :)

=== Attendees ===

Panagiotis Moustafellos
Nic Viljoen
Mauricio Vasquez
Martin Lau
Marco Leogrande
Jiong
Jesper Brouer
Jakub Kicinski
Edwin Peer
Brenden Blanco
Brendan Gregg
Alexander Duyck
Abder
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] reminder: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Please join us tomorrow for our bi-weekly call. As usual, this meeting is
open to everybody and completely optional.
You might be interested to join if:
You want to know what is going on in BPF land
You are doing something interesting yourself with BPF and would like to share
You want to know what the heck BPF is

=== IO Visor Dev/TSC Meeting ===

Every 2 weeks on Wednesday, from Wednesday, January 25, 2017, to no end date
11:00 am  |  Pacific Daylight Time (San Francisco, GMT-07:00)  |  1 hr

https://bluejeans.com/568677804/

https://www.timeanddate.com/worldclock/meetingdetails.html?year=2017=9=20=18=0=0=900
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] minutes: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Hi All,

Thanks for attending the call today. Here are the notes from the meeting.

Netdev 1.2
reminder: paper submissions
 deadline is Sept 20

Linux Plumbers microconference (next week)
If you have volunteered to lead discussion of a topic, Alexei would like to
remind you that we don't want to be power pointed to death. If you have slides,
they should be for fostering discussion, so please bring only a few :)
We believe the session will be 2-5pm on Friday, but couldn't find confirmation
on the website yet.

=== Status updates ===

Jakub:
- created tool for bpf verifier disassembly
  - verifier -> elf
  - to be shared in near future

William:
- conn track RFC: feedback being addressed
- wildcard map - looking at decision tree algorithm
  - some performance evaluation tbd
  - feedback: ok to make some problem space assumptions to simplify
  implementation complexity

Yonghong:
- looking at uprobe performance issues before plumbers
  - vdso, lttng
- kernel tracepoints - only 1 id limitation
  - improvements?
  - bug with fd closing
- pid inside container
  - helper to manage pid

John:
- Fixes for XDP
  - sockmap detach bug for net
- redirect to ingress support for sockmap
- parsing problems with complex protocols
  - fsm helper?

Daniel:
- Ran some statistics on bpf 4.0 -> 4.13 kernel commits
  - 123 patches (if the note-taker heard correctly)
  - 27 unique contributors
  - last 3 releases were busiest and growing
- XDP to skb metadata transfer
  - tests and reviews

Brendan:
-  bpf automation improvements @netflix
  - flamegraphs with oneclick

Misc:
- Landlock talk at security miniconf during plumbers

Alexei:
- focus is on visibility/debug
- wants to resume function call work

=== Attendees ===
Alexei Starovoitov
Andy Gospodarek
Brendan Gregg
Brenden Blanco
Daniel Borkmann
Flavio Crisciani
Fulvio Russo
Jakub Kicinski
John Fastabend
Mauricio Vasquez
Suchakra Sharma
William Tu
Yonghong Song
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] reminder: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Please join us tomorrow for our bi-weekly call. As usual, this meeting is
open to everybody and completely optional.
You might be interested to join if:
You want to know what is going on in BPF land
You are doing something interesting yourself with BPF and would like to share
You want to know what the heck BPF is

=== IO Visor Dev/TSC Meeting ===

Every 2 weeks on Wednesday, from Wednesday, January 25, 2017, to no end date
11:00 am  |  Pacific Daylight Time (San Francisco, GMT-07:00)  |  1 hr

https://bluejeans.com/568677804/

https://www.timeanddate.com/worldclock/meetingdetails.html?year=2017=9=6=18=0=0=900
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

Re: [iovisor-dev] modifying packets in XDP

[Brenden Blanco via iovisor-dev]

On Thu, Aug 24, 2017 at 12:25 PM, Y Song  wrote:
> On Thu, Aug 24, 2017 at 12:17 PM, Ilya Baldin  wrote:
>> Thank you, everyone for the replies. This explains about bpf_probe_read.
>>
>> I tried the approach below (changed the check to offset +1) however that
>> didn’t help, I still get the same error and BCC refuses to load the program
>>
>>  R0=inv,min_value=2,max_value=65537,min_align=1,aux_off_align=2
>> R1=inv,min_value=6,max_value=65541,min_align=1,aux_off_align=2 R2=pkt_end
>> R3=pkt(id=1,off=0,r=20),aux_off_align=2
>> R4=imm0,min_value=0,max_value=0,min_align=2147483648,aux_off_align=2
>> R5=imm0,min_value=0,max_value=0,min_align=2147483648,aux_off_align=2
>> R6=inv,min_value=4,max_value=65539,min_align=1,aux_off_align=2 R10=fp
>> 65: (69) r1 = *(u16 *)(r6 +0)
>> R6 invalid mem access 'inv'

Here is a full BCC version that worked for me:

#!/usr/bin/env python
import bcc
text = """
static inline int swapu16(uint16_t *start, void *data_end, int off1, int off2) {
  int len = (uint16_t*)data_end - start;
  uint16_t poff1, poff2;

  if (((void*)[off1 + 1] > data_end) || ((void*)[off2 + 1]
> data_end))
return 0;

  poff1 = start[off1];
  poff2 = start[off2];

  start[off2] = poff1;
  start[off1] = poff2;

  return 1;
}
int on_xdp_rx(struct xdp_md *xdp) {
  void *data_end = (void *)(long)xdp->data_end;
  void *data = (void *)(long)xdp->data;

  if (!swapu16(data, data_end, 4, 6))
return XDP_DROP;
  return XDP_PASS;
}
"""
b = bcc.BPF(text=text)
b.attach_xdp("eno1", b.load_func("on_xdp_rx", b.XDP), 0)

>
> Note that:
> R6=inv,min_value=4,max_value=65539,min_align=1,aux_off_align=2
> "inv" means unknown value.
>
> Could you double check your program?
>
>>
>> HINT: The invalid mem access 'inv' error can happen if you try to
>> dereference memory without first using bpf_probe_read() to copy it to the
>> BPF stack. Sometimes the bpf_probe_read is automatic by the bcc rewriter,
>> other times you'll need to be explicit.
>>
>> and it is the ‘write’ statements that are causing it as in when I try to
>> write into the packet
>>
>> start[off1] = val;
>>
>> is when I get the error.
>>
>> Is the solution to use the tc loader instead of bcc? Less convenient, but
>> doable, I suppose.
>>
>> -ilya
>>
>> Ilya Baldin
>>
>>
>> On Aug 24, 2017, at 12:43 PM, Brenden Blanco  wrote:
>>
>> On Thu, Aug 24, 2017 at 9:26 AM, Daniel Borkmann via iovisor-dev
>>  wrote:
>>
>> On 08/24/2017 06:17 PM, Y Song wrote:
>>
>>
>> CC to bcc mailing list as well.
>>
>> bpf_probe_read is not allowed in XDP programs.
>>
>> Your comparison may need to comparison not just starting offset, but
>> also including the memory size so the
>> whole write won't fall beyond the "data_end".
>>
>> Regarding to bcc translates "start[off2]" to bpf_probe_read, it is
>> possible that bcc rewriter tries to infer bpf_probe candidate and
>> finds this one ...
>>
>>
>>
>> I was wondering about this last one, and where this suggestion
>> comes from exactly (bcc for sure?). How does the error message
>> look like?
>>
>>
>> The error comes from here:
>> https://github.com/iovisor/bcc/blob/master/src/cc/libbpf.c#L210
>>
>> But as Yonghong said, you cannot use bpf_probe_read from XDP and hence
>> should ignore the warning.
>>
>> I would suggest rewriting your check to be something like:
>>
>>  if (((void*)[off1 + 1] > data_end) || ((void*)[off2 + 1]
>>
>> data_end))
>>
>>return 0;
>>
>>
>>
>> On Thu, Aug 24, 2017 at 8:22 AM, Ilya Baldin  wrote:
>>
>>
>> Hello everyone,
>>
>> A couple of questions. I’m using XDP with BCC on a Fedora 25 with kernel
>> 4.13.0-0.rc5.git0.2.fc27.x86_64 with e1000 driver. Basic XDP examples appear
>> to work, I was able to create my own simple example counting TCP SYN
>> packets, so the setup is at least partially correct.
>>
>> I’m playing around now with modifying TCP payload on the fly and failing
>> miserably. The function that is supposed to swap two u16s in TCP payload is
>> below
>>
>> static inline int swapu16(uint16_t *start, void *data_end, int off1, int
>> off2) {
>>   int len = (uint16_t*)data_end - start;
>>   uint16_t poff1, poff2;
>>
>>   if (((void*)[off1] > data_end) || ((void*)[off2] >
>> data_end))
>> return 0;
>>
>>   poff1 = start[off1];
>>   poff2 = start[off2];
>>
>>   //start[off2] = poff1;
>>   //start[off1] = poff2;
>>
>>   return 1;
>> }
>>
>> and it gets called with start pointing to the beginning of the payload
>> (I’m reasonably sure that is correct).
>>
>> If I *uncomment* either of the two lines in the function I get verifier
>> errors suggesting I use bpf_probe_read. First question
>>
>> 1. Is what I’m attempting even possible - am I allowed to modify the
>> packet in XDP hook? If no this may be a short conversation.
>>
>> 2. If it is possible, is there a canonical way of doing it compared to
>> what I’m trying to do?
>>
>> 3. I actually attempted to use bpf_probe_read() (even though with 

[iovisor-dev] minutes: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Hi All,

Thanks for joining in to the call today. We shared a round of updates, which are
included in the notes below.

Also please remember that Plumbers is right around the corner! If you have
signed up to lead discussion of a topic at the Tracing+BPF microconference,
Alexei would like to have a brief description of what will be discussed so that
organizers can update the bylines on the schedule. Please send that soon!



John F:
- currently working on improvements to sockmap
  - api changes merged
  - sockets can now be in multiple maps
  - bug: tracking down warning in socket cleanup

Daniel:
- currently working on misc net-next fixes
- taking XDP measurements for early drop filters
- next: XDP->CB metadata sharing

Brenden:
- working on some lib cleanups
- starting investigation into including compact c types in maps for plumbers
  conf
  - note from Alexei: Martin is also looking at this, we should sync up
- requests to share WIP patches for bpf+pktgen (done)

Yonghong:
- starting investigation into uprobe performance for plumbers
- planning to also give llvm bpf compiler change overview

William:
- feature request: xdp_redirect with support for multi-dev output/multicast
  - response: has been discssed, and the feature makes sense
  - will take the form of a port group/multi-dev-map with kernel doing the
duplication in the backend

Joe:
- working on rebasing bpf tuple space search table prototype
  - hitting gcc issues with container_of + rcu

Attendees:
Zhiting Zhu
Yonghong Song
William Tu
John Fastabend
Joe Stringer
Flavio Crisciani
Dongsu Park
Daniel Borkmann
Brenden Blanco
Alexei Starovoitov
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] reminder: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Please join us tomorrow for our bi-weekly call. As usual, this meeting is
open to everybody and completely optional.
You might be interested to join if:
You want to know what is going on in BPF land
You are doing something interesting yourself with BPF and would like to share
You want to know what the heck BPF is

=== IO Visor Dev/TSC Meeting ===

Every 2 weeks on Wednesday, from Wednesday, January 25, 2017, to no end date
11:00 am  |  Pacific Daylight Time (San Francisco, GMT-07:00)  |  1 hr

https://bluejeans.com/568677804/

https://www.timeanddate.com/worldclock/meetingdetails.html?year=2017=8=23=18=0=0=900
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] minutes: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Hi All,

Thanks for joining the meeting today. Here are the notes!

=== Updates ===

Yonghong:
- allow >2 pids attach to same usdt probe - wip
- next: looking at uprobe performance
  - looked into dyninst
  - researched systemtap approach
- c prog compiled into .so
- instrumentation of binary with jmp
  - limited helpers available
- maps?
- stacks?
  - filters in userspace
- sample rate
- malloc size
  - ongoing

John
- xdp redirect merged!
- ongoing: some minor bugfixes
- next: socket redirect

Alastair:
Welcome to the call! He is working on a new tool called bpftrace, which has a
promising starting codebase.
- new tool for tracing, standalone
- check out github.com/ajor/bpftrace

Daniel:
- adjusted map value verifier fixes
- iproute2 fixes

Joe:
- resuming work on ovs/bpf
- looking at integrating cilium connection tracker with ovs-bpf

!!
New feature request:
- add bpf id printing support to bcc

=== Attendees ===
Alastair
Andy Gospodarek
Brendan Gregg
Brenden Blanco
Daniel Borkmann
Genevieve Bastien
Jakub Kicinski
Joe Stringer
John Fastabend
Marco Leogrande
Mauricio Vasquez
Yonghong Song
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] reminder: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Please join us tomorrow for our bi-weekly call. As usual, this meeting is
open to everybody and completely optional.
You might be interested to join if:
You want to know what is going on in BPF land
You are doing something interesting yourself with BPF and would like to share
You want to know what the heck BPF is

=== IO Visor Dev/TSC Meeting ===

Every 2 weeks on Wednesday, from Wednesday, January 25, 2017, to no end date
11:00 am  |  Pacific Daylight Time (San Francisco, GMT-07:00)  |  1 hr

https://bluejeans.com/568677804/

https://www.timeanddate.com/worldclock/meetingdetails.html?year=2017=7=26=18=0=0=900
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] reminder: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Please join us tomorrow for our bi-weekly call. As usual, this meeting is
open to everybody and completely optional.
You might be interested to join if:
You want to know what is going on in BPF land
You are doing something interesting yourself with BPF and would like to share
You want to know what the heck BPF is

=== IO Visor Dev/TSC Meeting ===

Every 2 weeks on Wednesday, from Wednesday, January 25, 2017, to no end date
11:00 am  |  Pacific Daylight Time (San Francisco, GMT-07:00)  |  1 hr

https://bluejeans.com/568677804/

https://www.timeanddate.com/worldclock/meetingdetails.html?year=2017=7=12=18=0=0=900
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] reminder: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Please join us tomorrow for our bi-weekly call. As usual, this meeting is
open to everybody and completely optional.
You might be interested to join if:
You want to know what is going on in BPF land
You are doing something interesting yourself with BPF and would like to share
You want to know what the heck BPF is

=== IO Visor Dev/TSC Meeting ===

Every 2 weeks on Wednesday, from Wednesday, January 25, 2017, to no end date
11:00 am  |  Pacific Daylight Time (San Francisco, GMT-07:00)  |  1 hr

https://bluejeans.com/568677804/

https://www.timeanddate.com/worldclock/meetingdetails.html?year=2017=6=28=18=0=0=900
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

Re: [iovisor-dev] eBPF hashtables

[Brenden Blanco via iovisor-dev]

On Wed, Jun 21, 2017 at 6:40 AM, Douglas Caetano dos Santos via
iovisor-dev  wrote:
>
> Hi,
>
> We're starting to extensively use eBPF on our servers and we've got a couple 
> of
> questions on specific internals that I hope you can help clarify.
>
> The problem we're trying to solve is this: we want to mark packets from 
> selected
> incoming TCP flows to act on them in different ways. This marking must be done
> before the packets enter the IP stack.

This should work, just be aware that other subsystems may also mark
the packet (nft, etc.) so your system configuration should be sane.
>
>
> We plan on receiving around 1Mpps, so we're doing the decision making with 
> eBPF.
>
> Our idea is to use a eBPF hashtable whose keys are tuples and values are some
> flags. Once we receive a SYN packet, we decide if we start marking this new 
> flow
> from now on. If no marking is going to be done, we do nothing, but if we 
> decide
> to mark, we add this tuple to the hashtable with some internal flags (to be 
> used
> by a user space program) and add a mark to this packet. For other packets, we
> check if the tuple exists in the hashtable and, if it exists, we add a mark to
> these packets.
>
> So our questions are:
>
> 1) considering we're going to receive 1Mpps, are eBPF hashtables appropriate 
> for
> this task?

Yes, though I haven't recently done specific microbenchmarking (others
have, maybe they can chime in?), I can say anecdotally that a use case
with say 10Mpps across 16 cores and an insert/delete for each packet
would be achievable. My use case was doing a bunch of other stuff at
the time so I can't say how much of that workload was hash table
specific time.
>
> and 2) are the values written into the hashtable or the insertion/deletion of
> entries immediately propagated to other CPUs? This is important to avoid a 
> race
> condition where two packets of the same flow are received in different CPUs,
> where one could get marked and the other don't.

The hash table implementation uses RCU semantics to perform the
synchronization. There is a fair amount of documentation on that
subject, but in a nutshell, writes on one core will be made available
to other cores after a small grace period. Reads will always see a
consistent view of the hash table, but might read slightly older data.
If ntuple filtering is turned on in your nic, I would expect that all
packets for the same flow arrive to the same queue/CPU.

Also, note that there are two flavors of hash table, one which is
shared across all cpus (with semantics as just described) and one
which is per-cpu (faster but not shared). I expect you want to use the
former.
>
> (Thanks, Brendan, for pointing me to the iovisor mailing list.)
>
> Thanks for your help with these questions.
> Best regards,
> Douglas Santos.
> ___
> iovisor-dev mailing list
> iovisor-dev@lists.iovisor.org
> https://lists.iovisor.org/mailman/listinfo/iovisor-dev
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

Re: [iovisor-dev] [iovisor/bcc] USDT broken by b0f891d129a9b372

[Brenden Blanco via iovisor-dev]

Also, I would suggest that after that fix, we also update the test to
include actually testing for proper values. Probably you can send the
values over the perf ring buffer and assert at the end that all of the
values are collected properly.

On Tue, Jun 6, 2017 at 8:59 PM, Brenden Blanco  wrote:

> Yonghong, can you send a PR for your branch?
>
> On Tue, Jun 6, 2017 at 12:20 AM, Y Song via iovisor-dev <
> iovisor-dev@lists.iovisor.org> wrote:
>
>> Hi, Tetsuo,
>>
>> You are right. The bug is actually introduced by my last patch. I just
>> focused one aspect of issue and inadvertently introduced another
>> *blocker* bug.
>>
>> I just pushed a patch. Thanks a lot for reporting the issue and
>> testing out the fix.
>>
>> Yonghong
>>
>> On Mon, Jun 5, 2017 at 5:25 PM, Tetsuo Handa
>>  wrote:
>> > Hello.
>> >
>> > I changed the hook as below and confirmed that cp == NULL at
>> bpf_probe_read().
>> > That is, it is bpf_usdt_readarg() which got broken.
>> >
>> > --
>> > int do_start(struct pt_regs *ctx) {
>> > char *cp = NULL;
>> > bpf_usdt_readarg(1, ctx, );
>> > struct { char query[QUERY_MAX]; } data = { };
>> > if (cp != NULL)
>> > bpf_probe_read(, sizeof(data.query), (void *) cp);
>> > else
>> > data.query[0] = '!';
>> > events.perf_submit(ctx, , sizeof(data));
>> > return 0;
>> > };
>> > --
>> >
>> > If I do
>> >
>> > -std::string cptr = tfm::format("*((volatile %s *)dest)", ctype);
>> > +std::string cptr = tfm::format("*((%s *)dest)", ctype);
>> >
>> > (i.e. drop only "volatile " part) in that commit like below
>> >
>> > --
>> > diff --git a/src/cc/usdt.cc b/src/cc/usdt.cc
>> > --- a/src/cc/usdt.cc
>> > +++ b/src/cc/usdt.cc
>> > @@ -152,7 +152,7 @@ bool Probe::usdt_getarg(std::ostream ) {
>> >
>> >for (size_t arg_n = 0; arg_n < arg_count; ++arg_n) {
>> >  std::string ctype = largest_arg_type(arg_n);
>> > -std::string cptr("dest");
>> > +std::string cptr = tfm::format("*((%s *)dest)", ctype);
>> >
>> >  tfm::format(stream,
>> >  "static __always_inline int _bpf_readarg_%s_%d("
>> > --
>> >
>> > USDT probe works again as well as build warning shown below goes away.
>> >
>> > --
>> > /virtual/main.c:4:8: warning: incompatible integer to pointer
>> conversion assigning to 'void *' from 'volatile uint64_t' (aka 'volatile
>> unsigned long long') [-Wint-conversion]
>> >   dest = *(volatile uint64_t *)>bx;
>> >^ ~~
>> > --
>> ___
>> iovisor-dev mailing list
>> iovisor-dev@lists.iovisor.org
>> https://lists.iovisor.org/mailman/listinfo/iovisor-dev
>>
>
>
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

Re: [iovisor-dev] [iovisor/bcc] USDT broken by b0f891d129a9b372

[Brenden Blanco via iovisor-dev]

Yonghong, can you send a PR for your branch?

On Tue, Jun 6, 2017 at 12:20 AM, Y Song via iovisor-dev <
iovisor-dev@lists.iovisor.org> wrote:

> Hi, Tetsuo,
>
> You are right. The bug is actually introduced by my last patch. I just
> focused one aspect of issue and inadvertently introduced another
> *blocker* bug.
>
> I just pushed a patch. Thanks a lot for reporting the issue and
> testing out the fix.
>
> Yonghong
>
> On Mon, Jun 5, 2017 at 5:25 PM, Tetsuo Handa
>  wrote:
> > Hello.
> >
> > I changed the hook as below and confirmed that cp == NULL at
> bpf_probe_read().
> > That is, it is bpf_usdt_readarg() which got broken.
> >
> > --
> > int do_start(struct pt_regs *ctx) {
> > char *cp = NULL;
> > bpf_usdt_readarg(1, ctx, );
> > struct { char query[QUERY_MAX]; } data = { };
> > if (cp != NULL)
> > bpf_probe_read(, sizeof(data.query), (void *) cp);
> > else
> > data.query[0] = '!';
> > events.perf_submit(ctx, , sizeof(data));
> > return 0;
> > };
> > --
> >
> > If I do
> >
> > -std::string cptr = tfm::format("*((volatile %s *)dest)", ctype);
> > +std::string cptr = tfm::format("*((%s *)dest)", ctype);
> >
> > (i.e. drop only "volatile " part) in that commit like below
> >
> > --
> > diff --git a/src/cc/usdt.cc b/src/cc/usdt.cc
> > --- a/src/cc/usdt.cc
> > +++ b/src/cc/usdt.cc
> > @@ -152,7 +152,7 @@ bool Probe::usdt_getarg(std::ostream ) {
> >
> >for (size_t arg_n = 0; arg_n < arg_count; ++arg_n) {
> >  std::string ctype = largest_arg_type(arg_n);
> > -std::string cptr("dest");
> > +std::string cptr = tfm::format("*((%s *)dest)", ctype);
> >
> >  tfm::format(stream,
> >  "static __always_inline int _bpf_readarg_%s_%d("
> > --
> >
> > USDT probe works again as well as build warning shown below goes away.
> >
> > --
> > /virtual/main.c:4:8: warning: incompatible integer to pointer conversion
> assigning to 'void *' from 'volatile uint64_t' (aka 'volatile unsigned long
> long') [-Wint-conversion]
> >   dest = *(volatile uint64_t *)>bx;
> >^ ~~
> > --
> ___
> iovisor-dev mailing list
> iovisor-dev@lists.iovisor.org
> https://lists.iovisor.org/mailman/listinfo/iovisor-dev
>
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] reminder: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Please join us tomorrow for our bi-weekly call. As usual, this meeting is
open to everybody and completely optional.
You might be interested to join if:
You want to know what is going on in BPF land
You are doing something interesting yourself with BPF and would like to
share
You want to know what the heck BPF is

=== IO Visor Dev/TSC Meeting ===

Every 2 weeks on Wednesday, from Wednesday, January 25, 2017, to no end date
11:00 am  |  Pacific Daylight Time (San Francisco, GMT-07:00)  |  1 hr

https://bluejeans.com/568677804/

https://www.timeanddate.com/worldclock/meetingdetails.html?year=2017=5=31=18=0=0=900
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] minutes: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Hi All,

Thanks for attending the call today, here are the notes.

==

CFP for Linux Plumbers
Sept 13-15, Los Angeles
Tracing microconference accepted
 (Alexei, Brendan, Josef leading topics)
 http://wiki.linuxplumbersconf.org/2017:tracing
Last year was successful, looking forward to more topics
 (tbd: find previous etherpad)

==
BCC has grown, albeit with some maturity/sore spots

We have a well established core group, but probably need to expand our
feedback
to incorporate those that may not be speaking up. Think users of the tools
rather than developers.

How do we get broader user-group feedback besides mailing list/github
issues?

Brendan:
#1 feedback he has heard - simplify syntax, still compared to dtrace
 awareness via conferences, etc.

Alexei:
 internally, writing programs is hard, struggling with verifier
  - working on llvm improvements
 many kernel hackers @fb have hacked up bcc tools for their own use case

Brenden:
 reach larger user group => collectd integration?
  command line users are the smaller subset of our potential audience
 intel snapd
 collectd
 others?

Going forward:
It's a big library, needs shrink/split
 both in terms of memory usage and disk size
 split out clang/llvm
 split out loader
AI: get some proposals into bcc/issues, either in documentation form to
start
 with, or code
 split header files
 split libraries
 start thinking about external tool integration

==
Alexei:
has started thinking about program chaining (generically)

mistake: tail call solved a need, but helper/map model makes it inflexible

prog chaining should be a single native instruction
 (not done until now because "too hard to verify")
now should be possible to introduce and verify cleanly
direct, indirect, and tail call support
 (stack preserving)
3 new bpf instructions
introduce global variables
verifier changes started
llvm support
 (looks like normal c)
configurable stack upper bound of function
loader would need to parse elf file
main() function concept, still callable via tail call

So far, the above is perhaps 10% compelete, so stay tuned.

==
Attendees:
Alexei Starovoitov
Bilal Anwer
Jesper Brouer
Martin Lau
Mauricio Vasquez
Brenden Blanco
Andy Gospodarek
Nic Viljoen
Daniel borkmann
Suchackra Sharma
Brendan Gregg
John Fastabend
Teng Qin
Jakub Kicinski
Marco Leogrande
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] reminder: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Please join us tomorrow for our bi-weekly call. As usual, this meeting is
open to everybody and completely optional.
You might be interested to join if:
You want to know what is going on in BPF land
You are doing something interesting yourself with BPF and would like to
share
You want to know what the heck BPF is

=== IO Visor Dev/TSC Meeting ===

Every 2 weeks on Wednesday, from Wednesday, January 25, 2017, to no end date
11:00 am  |  Pacific Daylight Time (San Francisco, GMT-07:00)  |  1 hr

https://bluejeans.com/568677804/

https://www.timeanddate.com/worldclock/meetingdetails.html?year=2017=5=17=18=0=0=900
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] [cancelled] IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Hi all,

Due to many members being out of town, some concurrently with the netdev
conference, the meeting today is cancelled. See you all again in two weeks!

-Brenden
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] minutes: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Hi All,

Thanks for joining the meeting this week. Here are some status updates
from various folks.

William:
Discussed issue he is having with the verifier/llvm
 complex program becomes stack limited
 as a workaround, tried using volatile modifier for some stack variables
 but this causes the verifier to hit the 64k simulated instruction
complexity limit
 reason: no heuristics to prune states involving on-stack variables
 suggested workaround: use per-cpu array to hold state

Brenden:
Saw issue with xdp in virtio
warn on once in drivers/pci/msi.c:1261 pci_irq_vector+0xd4/0xe0
will send description to netdev list

ctf (tracing and types) topics:
Suchakra (tracing)
 - started with babeltrace python bindings
 - discussed optional compile-time dependency on libbabeltrace
 - we agreed for now to work on a separate bcc branch for integration testing

Alexei:
 libraries exist as part of pahole (maintained by Arnaldo):
 payhole is tool for analyzing holes in c datastructures
 supports dwarf+ctf
 parser and emitter
 may need to write emitter from scratch due to licensing, but not too hard
 bitfield representation may need some work
 api: extra section at map create time
  ability to dump as well via syscall
 new name to avoid ambiguity with tracing-ctf?

Andy:
 arm64 compiler issues
 inline assembly parser in llvm
 must-have for tracing to get accurate header definitions
 open PR already exists for hacking it in BCC to ifdef out inline
assembly (ugly)
 suggestion to respin patches to arm list to cleanup defines

Sargun:
 new landlock patches a month ago, progress?
 some discussion, seems stalled (unprivileged requirement)
 alternative: seccomp syscall filtering at global level (with eBPF)
  avoids criu problem

Attendees:
William Tu
Teng
Suchakra Sharma
Sargun Dhillon
Michael Schubert
Mauricio Vasquez
Matteo Bertrone
Martin Lau
Marco Leogrande
Lorenzo David
Joe Stringer
Jan-Erik
Jakub Kicinski
Gary Berger
Flavio Crisciani
Ferris
Daniel Borkmann
Brenden Blanco
Brendan Gregg
Andy Gospodarek
Alexei Starvoitov
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] reminder: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

WARNING: please note that California has recently observed Daylight
Saving Time, so remember to check your local time conversion.

WARNING 2: we're using Bluejeans instead of Webex this time.

Please join us tomorrow for our bi-weekly call. As usual, this meeting is
open to everybody and completely optional.
You might be interested to join if:
You want to know what is going on in BPF land
You are doing something interesting yourself with BPF and would like to share
You want to know what the heck BPF is

=== IO Visor Dev/TSC Meeting ===

Every 2 weeks on Wednesday, from Wednesday, January 25, 2017, to no end date
11:00 am  |  Pacific Daylight Time (San Francisco, GMT-07:00)  |  1 hr

https://bluejeans.com/568677804/

https://www.timeanddate.com/worldclock/meetingdetails.html?year=2017=3=22=18=0=0=900
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] minutes: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Hi all,

Thanks to those who attended, here are my minutes from the meeting.

= Conferences past and future =

IO Visor Summit
We had this last week, the conference room was packed and we had a few remote
attendees as well. We are working to upload the videos, collect the slides
(please send them to Debra if you haven't already!), and maybe write a blog
post. If you weren't able to attend, look out for an email shortly.

SCaLEx 15x
Linux 4.x Tracing: Performance Analysis with bcc/BPF
https://www.socallinuxexpo.org/scale/15x/presentations/linux-4x-tracing-performance-analysis-bccbpf
Brendan gave this featured talk, with an estimated 200-300 attendees in the
room. Many people were surprised and impressed with the capabilities, which
means that there are still too many people who don't know about BPF yet! Get
out there and give talks if you can.

O'Reilly Velocity - June 19-22 - Santa Clara, CA
Accepted talk by Brendan, and tutorial by Sasha

Netdev 2.1 - April 6-8 - Montreal, CA
CFP is over, several BPF/XDP topics on the agenda
Jesper: Tutorial on how to program BPF

Dockercon - April 17-20 - Austin, TX
Thomas: Cilium - Network and Application Security with BPF and XDP
Brendan: Container Performance Analysis (not BPF specific)

Other (I didn't catch the name) - Denmark
Jesper: how to use bpf for ddos mitigation

= Development =

Brenden:
Working on changing the way bpf maps are stored in the BPFModule and related
classes, with the hope of having different backend implementations based on
feature requirements (persistence, namespaces). The first user of the change
should be the json type generator, followed by the MapTypesVisitor proposal
(bcc #960).

CTF
Discussion kicked off regarding encoding of types. The use of CTF (compact C
type format) has been mentioned before, and still seems to be the way to go.
Alexei will kick off a discussion thread with some kernel folks who supposedly
have some baseline code work off of.

Suchakra:
 Working on something sip (voice protocol) related, details to come soon
 Some ctf (t=trace) work ongoing, depending on Babeltrace 2.0

= Attendees =
Brenden Blanco
Joe Stringer
Marco Leogrande
Suchakra
Brendan Gregg
Alexei Starovoitov
Matteo Bertrone
Mauricio Vasquez
William Tu
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

Re: [iovisor-dev] Problem with ebpf program array map

[Brenden Blanco via iovisor-dev]

No, really the only thing you are allowed to do from bpf on a prog
array is call() and delete(). It is a one-way street.

Unfortunately you will have to keep track of which programs are in
which slots from the userspace side entirely.

On Fri, Mar 3, 2017 at 12:52 AM, Tanja Ulmen via iovisor-dev
 wrote:
> Is there another way to look into a program array? I'm currently using the
> program below:
>
> prog = """
>   BPF_TABLE("prog", int, int, prog_array, 5);
>
>   int hello(struct __sk_buff* ebpf_packet) {
> int key = 1;
> int *value;
>
> bpf_trace_printk("Hello, World!\\n");
>
> value = bpf_map_lookup_elem(_array, );
>
> prog_array.call(ebpf_packet, 1);
> bpf_trace_printk("Tail-call not executed!\\n");
> return 1;
>   }
>   """
>
> And I get the following error:
>
> bpf: Invalid argument
> 0: (bf) r6 = r1
> 1: (b7) r1 = 1
> 2: (63) *(u32 *)(r10 -4) = r1
> 3: (b7) r1 = 2593
> 4: (6b) *(u16 *)(r10 -12) = r1
> 5: (b7) r1 = 1684828783
> 6: (63) *(u32 *)(r10 -16) = r1
> 7: (18) r1 = 0x6c6c6548
> 9: (7b) *(u64 *)(r10 -24) = r1
> 10: (b7) r7 = 0
> 11: (73) *(u8 *)(r10 -10) = r7
> 12: (bf) r1 = r10
> 13: (07) r1 += -24
> 14: (b7) r2 = 15
> 15: (85) call 6
> 16: (bf) r2 = r10
> 17: (07) r2 += -4
> 18: (08) r0 = 0x0
> invalid BPF_LD_IMM insn
>
> Regards,
> Tanja
>
>
> Am 02.03.2017 um 14:35 schrieb Mauricio Vasquez:
>
>
>
> On 02/28/2017 06:01 AM, Tanja Ulmen wrote:
>
> Hi Mauricio,
>
> thank you for the example program, it works and I get both outputs.
>
> Are you calling load_func() for the program in the .c file?
>
> Yes I'm calling load_func() for the program. My program is similar to the
> endToEndTest.py of the P4 frontend
> (https://github.com/iovisor/bcc/blob/0c8c179fc1283600887efa46fe428022efc4151b/src/cc/frontends/p4/test/endToEndTest.py).
> Thus a small virtual network where each node has its own namespace.
>
> I think the problem of my program is that the program with the tail-call is
> executed in a namespace, but I don't know exactly why this is a problem.  I
> will keep on looking for the problem, but maybe you have some further ideas?
>
> Unfortunately I don't have any idea for it.
>
> As a comment, it is normal that "print prog_array.items()" prints [], the
> prog type array does not implement the lookup function.
>
> Regards,
> Mauricio
>
>
> Regards,
> Tanja
>
>
> Am 23.02.2017 um 20:24 schrieb Mauricio Vasquez:
>
> Hello Tanja,
>
> On 02/20/2017 09:53 AM, Tanja Ulmen via iovisor-dev wrote:
>
> Hi,
>
> I'm new to eBPF and bcc and I hope this is the right platform for my
> question.
>
> Welcome!
>
> I have the following problem while working with a bpf program array map:
>
> I have a C file that contains a program map like this:
> BPF_TABLE("prog", int, int, prog_array, 2);
>
> Later on it is called with
> prog_array.call(ebpf_packet, 1);
>
> The respective part of my python file looks like this:
> b = BPF(src_file="./output.c")
>
> prog = """
> int hello(struct __sk_buff* ebpf_packet) {
> bpf_trace_printk("Hello, World!\\n");
> return 0;
> }
> """
>
> hw = BPF(text=prog)
> hello_fn = hw.load_func("hello", BPF.SCHED_CLS)
>
> prog_array = b["prog_array"]
> prog_array[ctypes.c_int(1)] = ctypes.c_int(hello_fn.fd)
>
> print prog_array.items()
>
> The output of the last line is just an empty array [] and the subprogram
> "hello" is not called when I run the program. There is no error. I have also
> hash bpf tables where I can assign key and leaf without any problems and
> these tables are accessed correctly during the run.
>
> Do you have any idea where my mistake could be?
>
> Are you calling load_func() for the program in the .c file?
> I modified one of the examples to add a tail call, you can see at [1].
> Although print(prog_array.items()) shows an empty array, the second eBPF
> program is called. I tried to print the prog array in some other examples
> and I always get an empty one, I don't know why.
>
> [1] https://gist.github.com/mvbpolito/6312896d2004efd189dff79e913df297
>
> Regards,
> Mauricio
>
>
> Best regards,
> Tanja
>
>
> ___
> iovisor-dev mailing list
> iovisor-dev@lists.iovisor.org
> https://lists.iovisor.org/mailman/listinfo/iovisor-dev
>
>
>
>
>
>
> ___
> iovisor-dev mailing list
> iovisor-dev@lists.iovisor.org
> https://lists.iovisor.org/mailman/listinfo/iovisor-dev
>
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

Re: [iovisor-dev] WebEx meeting invitation: IOVisor summit

[Brenden Blanco via iovisor-dev]

Hi Folks,

It looks like the link in the email for the global dial-in numbers got
filtered.

If you have trouble joining, please try one of the following:
https://www.myrcplus.com/cnums.asp?bwebid=8369444=6742974834=18667156501=17193252777

Thanks,
Brenden

On Sat, Feb 25, 2017 at 6:34 PM, Deepa Kalani via iovisor-dev <
iovisor-dev@lists.iovisor.org> wrote:

>  Please find the webex details below.
>
>
>
> Thx
>
> Deepa
>
>
>
>
>
>
>
>
>
> *IOVisor summit*
>
> Monday, February 27, 2017
>
> 8:30 am  |  Pacific Standard Time (San Francisco, GMT-08:00)  |  10 hrs
>
>
>
> Meeting number: 929 785 229
>
>
>
>
>
>
>
>
>
> Add to Calendar
> 
>
> When it's time, join the meeting
> 
> .
>
>
>
>
>
>
>
> *Join by phone*
>
> Call-in toll-free number (Premiere): 1-1-866715650  (US)
>
> Call-in number (Premiere): 1-1-7193252777 <(719)%20325-2777>  (US)
>
> Show global numbers
> 
>
> Attendee access code: 421 480 4609
>
>
>
>
>
>
>
> Can't join the meeting? 
>
>
>
>
>
>
>
> IMPORTANT NOTICE: Please note that this WebEx service allows audio and
> other information sent during the session to be recorded, which may be
> discoverable in a legal matter. By joining this session, you automatically
> consent to such recordings. If you do not consent to being recorded,
> discuss your concerns with the host or do not join the session.
>
>
>
> ___
> iovisor-dev mailing list
> iovisor-dev@lists.iovisor.org
> https://lists.iovisor.org/mailman/listinfo/iovisor-dev
>
>
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] cancelled: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Since we will be having the summit on Monday, I am cancelling
tomorrow's normally scheduled meeting. See you all next week!

FYI, we'll be posting the planned agenda shortly.
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] minutes: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

IO Visor summit coming, please prepare abstracts if you have something
to discuss

=== Conferences ===
netdev conf coming up, talks needed
Jesper - performance workshop
  DDoS topic
idea: reach out to nic vendors who support xdp
 - future xdp direction
idea: dpdk work
idea: specifically, xdp ring to userspace
 john's proposal for sharing pages

FOSDEM 2017
4 different BPF related talks:
 https://fosdem.org/2017/schedule/event/cilium/
 https://fosdem.org/2017/schedule/event/ebpf_xdp/
 https://fosdem.org/2017/schedule/event/stateful_ebpf/
 https://fosdem.org/2017/schedule/event/go_bpf/

linux.conf.au
BPF: Tracing and More
https://linux.conf.au/schedule/presentation/96/

BSidesSF
Linux Monitoring at Scale with eBPF
http://sched.co/9IKj

=== Dev Updates ===
bcc - buildbot/jenkins moved over to GCE
slower but cheaper/on demand

Daniel:
tracepoints for bpf accepted
xdp support (new drivers, new features)
verifier fixes
kallsym work wrapping up
lpm trie features and fixes

Jesper:
documentation work
 goal to enable customers to create own DDoS filters instead of RH coding on
 its own

cgroup work:
netns id helper

William:
hitting max stack size
(check printk strings)
unlikely to increase in kernel

=== Attendees ===
Daniel Borkmann
Eduard Serra
Brenden Blanco
Marco Leogrande
Joe Stringer
Davide Celotto
Mauricio Vasquez
Michael Schubert
William Tu
Jesper Brouer
Brendan Gregg
Sargun Dhillon
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] reminder: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Please join us tomorrow for our bi-weekly call. As usual, this meeting is
open to everybody and completely optional.
You might be interested to join if:
You want to know what is going on in BPF land
You are doing something interesting yourself with BPF and would like to share
You want to know what the heck BPF is

=== IO Visor Dev/TSC Meeting ===
Every 2 weeks on Wednesday, from Wednesday, January 25, 2017, to no end date
11:00 am  |  Pacific Standard Time (San Francisco, GMT-08:00)  |  1 hr
Meeting number: 928 532 538
Meeting password: iovisor

=== When it's time, join the meeting. ===
https://vmware.webex.com/vmware/j.php?MTID=me7e277c6a7646343ec6d302199d5f26b

=== Join by phone ===
Call-in toll-free number (Premiere): 1-8667156501  (US)
Call-in number (Premiere): 1-7193252777  (US)
Show global numbers: https://goo.gl/OXQabG
Attendee access code: 674 297 4834
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

Re: [iovisor-dev] Accessing a public eBPF map from different namespace

[Brenden Blanco via iovisor-dev]

On Sat, Feb 4, 2017 at 6:25 PM, Daehyeok Kim via iovisor-dev
 wrote:
> Hello,
>
> I am trying to use the eBPF map to share data between two different BPF 
> modules.
> In eBPF module A, I defined a table using BPF_TABLE_PUBLIC ("hash",..,...) and
> in the other module B, I define a table using BPF_TABLE ("extern", ..., ...).
> I confirmed that the module B can access a content of the original
> table defined in A when I loaded the modules in the same network
> namespace.
Are they also in the same process?
>
> I encountered a problem when I tried to load the module A in a network
> namespace A and load the module B in a namespace B; while the module A
> is loaded successfully, it fails to compile the module B with the
> following error:
I suspect that the error is due to attempting to share between
processes, rather than between namespaces. The former requires support
via map pinning whereas the latter should work (but likely depends on
the former in practice).
>
> /virtual/main.c:29:1: error: could not open bpf map: No such file or directory
> is maps/extern map type enabled in your kernel?
> BPF_TABLE("extern", struct Key, struct Val, test, 1000);
> ^
> /virtual/include/bcc/helpers.h:58:76: note: expanded from macro 'BPF_TABLE'
> #define BPF_TABLE(_table_type, _key_type, _leaf_type, _name, _max_entries) \
>^
> /virtual/include/bcc/helpers.h:54:4: note: expanded from macro '\
> BPF_F_TABLE'
> }; \
>
> It seems that it cannot find the public table from different namespace.
> Is there any way to solve this problem?
> Or is it invalid to access a public BPF table of another module
> running on a different namespace?
Were the externed maps to support pinning, then sharing across
namespaces should work with some hoop jumping.
>
> Thanks!
> ___
> iovisor-dev mailing list
> iovisor-dev@lists.iovisor.org
> https://lists.iovisor.org/mailman/listinfo/iovisor-dev
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] reminder: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Please join us for our bi-weekly call. As usual, this meeting is open
to everybody and completely optional.
You might be interested to join if:
You want to know what is going on in BPF land
You are doing something interesting yourself with BPF and would like to share
You want to know what the heck BPF is

Also, please note that the below is a new meeting URL.

=== IO Visor Dev/TSC Meeting ===
Every 2 weeks on Wednesday, from Wednesday, January 25, 2017, to no end date
11:00 am  |  Pacific Standard Time (San Francisco, GMT-08:00)  |  1 hr
Meeting number: 928 532 538
Meeting password: iovisor

=== When it's time, join the meeting. ===
https://vmware.webex.com/vmware/j.php?MTID=me7e277c6a7646343ec6d302199d5f26b

=== Join by phone ===
Call-in toll-free number (Premiere): 1-8667156501  (US)
Call-in number (Premiere): 1-7193252777  (US)
Show global numbers: https://goo.gl/OXQabG
Attendee access code: 674 297 4834
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] reminder: IO Visor TSC/Dev Meeting

[Brenden Blanco via iovisor-dev]

Happy New Year All!

Please join us for our first bi-weekly call of the year. As usual, this
meeting is open to everybody and completely optional. You might be
interested to join if:
You want to know what is going on in BPF land
You are doing something interesting yourself with BPF and would like to
share
You want to know what the heck BPF is

*IOVisor TSC/Dev Meeting*
Every 2 weeks on Wednesday, from Wednesday, May 18, 2016, to no end date
11:00 am  |  Pacific Daylight Time (San Francisco, GMT-07:00)  |  1 hr

*Join WebEx meeting*

Meeting number: 283 885 640
Meeting password: iovisor

*Join by phone*
*+1-415-655-0003 <%2B1-415-655-0003>* US TOLL
Access code: 283 885 640
Global call-in numbers

___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] reminder: IO Visor TSC and Dev Members Call

[Brenden Blanco via iovisor-dev]

Hi All,

Please join us for our bi-weekly call. This meeting is open to everybody
and completely optional.

*IOVisor TSC/Dev Meeting*
Every 2 weeks on Wednesday, from Wednesday, May 18, 2016, to no end date
11:00 am  |  Pacific Daylight Time (San Francisco, GMT-07:00)  |  1 hr

*Join WebEx meeting*

Meeting number: 283 885 640
Meeting password: iovisor

*Join by phone*
*+1-415-655-0003 <%2B1-415-655-0003>* US TOLL
Access code: 283 885 640
Global call-in numbers

___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

Re: [iovisor-dev] arm64: LLVM ERROR: Inline asm not supported

[Brenden Blanco via iovisor-dev]

On Mon, Dec 12, 2016 at 10:38 PM, Alexei Starovoitov <
alexei.starovoi...@gmail.com> wrote:

> On Mon, Dec 12, 2016 at 10:33 PM, Ming Lei  wrote:
> > On Tue, Dec 13, 2016 at 2:11 PM, Alexei Starovoitov
> >  wrote:
> >> On Mon, Dec 12, 2016 at 9:44 PM, Ming Lei 
> wrote:
> >>>
>  so you can send me a pre-processed .c file or even better .ll file ?
>  I don't have arm64 to reproduce it...
> >>>
> >>> Please see the whole log of building the prog.
> >>
> >> that raw asm outside of any function body comes from
> >> arch/arm64/include/asm/sysreg.h
> >> it obviously cannot work with any crosscompiler.
> >> That's why we have -D__ASM_SYSREG_H in samples/bpf/Makefile
> >
> > Yes, passing '-D__ASM_SYSREG_H' just avoids to include
> >  when building prog.
> >
> > But that workaround doesn't work any more recently, because
> > some macros defined in sysreg.h are needed by other headers,
> > please see the building failure in my 2nd email of this thread.
> >
> > I can be a headache by just fixing kernel, since old kernel is still
> > running and old header is still installed. And raw asm() can be
> > introduced in the future too.
> >
> > So could we deal with the issue in bcc?
>
> I guess we can try to filter out all raw asm after clang stage in bcc.
> Such fix would be better than doing similar filtering in the backend
> and asking everyone to upgrade to the latest llvm.
> Brenden, thoughts?
>

That probably won't work, since the clang rewriter only gets invoked after
a successful first pass compile. This seems to be happening before that
step.

Is it possible that arm64 kernel compilation via clang is broken in general
(without the LLVMLinux patches)? Is that project still active? It might be
worth a try to try bcc against such a patched kernel, or to take a look at
those patches to see if these inline asm's are mentioned anywhere.
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] canceled: IO Visor TSC and Dev Members Call

[Brenden Blanco via iovisor-dev]

A number of us are attending Linux Plumbers this week, so we will have the
next meeting 2 weeks from today. Sorry for the late notice.
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

Re: [iovisor-dev] kprobe/kretprobe on cls_bpf_classify

[Brenden Blanco via iovisor-dev]

On Sat, Oct 29, 2016 at 1:27 AM, William Tu via iovisor-dev <
iovisor-dev@lists.iovisor.org> wrote:

> Hi,
>
> I'm trying to measure the performance of my BPF programs, which attach
> to clsact. As a start, I thought I could simply measure the cycles
> spent on cls_bpf_classify(), because it's the wrapper calling into my
> BPF programs.
>
> However, it seems that kprobe can not probe cls_bpf_classify(), even
> though cls_bpf_classify isn't in NOKPROBE_SYMBOL. Is there any reason
> why we couldn't probe it?
>
> [root@vm-dev bpf]# echo 'p:myprobe3 cls_bpf_classify' >>
> /sys/kernel/debug/tracing/kprobe_events
> bash: echo: write error: No such file or directory
>

Worked for me:

root@localhost:/sys/kernel/debug/tracing# echo 'p:myprobe3
cls_bpf_classify' > kprobe_events
root@localhost:/sys/kernel/debug/tracing# cat events/kprobes/myprobe3/id
1256


>
> As an alternative, I can probe on "__bpf_prog_run".
> [root@vm-dev bpf]# echo 'p:myprobe4 __bpf_prog_run' >>
> /sys/kernel/debug/tracing/kprobe_events
>
> Thanks
> William
> ___
> iovisor-dev mailing list
> iovisor-dev@lists.iovisor.org
> https://lists.iovisor.org/mailman/listinfo/iovisor-dev
>
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] reminder: IO Visor TSC and Dev Members Call

[Brenden Blanco via iovisor-dev]

Hi All,

Please join us for our bi-weekly call. This meeting is open to everybody
and completely optional.

*IOVisor TSC/Dev Meeting*
Every 2 weeks on Wednesday, from Wednesday, May 18, 2016, to no end date
11:00 am  |  Pacific Daylight Time (San Francisco, GMT-07:00)  |  1 hr

*Join WebEx meeting*

Meeting number: 283 885 640
Meeting password: iovisor

*Join by phone*
*+1-415-655-0003 <%2B1-415-655-0003>* US TOLL
Access code: 283 885 640
Global call-in numbers

___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

[iovisor-dev] reminder: IO Visor TSC and Dev Members Call

[Brenden Blanco via iovisor-dev]

Hi All,

Please join us for our bi-weekly call. This meeting is open to everybody
and completely optional.

*IOVisor TSC/Dev Meeting*
Every 2 weeks on Wednesday, from Wednesday, May 18, 2016, to no end date
11:00 am  |  Pacific Daylight Time (San Francisco, GMT-07:00)  |  1 hr

*Join WebEx meeting*

Meeting number: 283 885 640
Meeting password: iovisor

*Join by phone*
*+1-415-655-0003 <%2B1-415-655-0003>* US TOLL
Access code: 283 885 640
Global call-in numbers

___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

Re: [iovisor-dev] read perf counters

[Brenden Blanco via iovisor-dev]

On Tue, Aug 9, 2016 at 9:59 AM, riya khanna 
wrote:

> On Tue, Aug 9, 2016 at 12:24 PM, Brenden Blanco 
> wrote:
> >
> >
> > On Tue, Aug 9, 2016 at 9:16 AM, riya khanna 
> > wrote:
> >>
> >> On Tue, Aug 9, 2016 at 12:05 PM, Brenden Blanco 
> >> wrote:
> >> > On Tue, Aug 9, 2016 at 8:54 AM, riya khanna  >
> >> > wrote:
> >> >>
> >> >> Hi Brenden,
> >> >>
> >> >> Saw test_perf_event.py in your branch. Its creates and enables per
> >> >> counters once during start. Is it also possible to
> >> >> enable/disable/reset counters on the fly? Perhaps we need a kernel
> >> >> patch for this?
> >> >
> >> > It doesn't "create" counters, it just attaches to the already
> available
> >> > counters provided by the hardware or OS.
> >>
> >> Yes, it enables monitoring when attached.
> >>
> >> > Any type of "reset" infrastructure
> >> > would adversely impact other users of those same counters (perf). I
> >> > consider
> >> > it the job of userspace or the program to compute deltas or other
> types
> >> > of
> >> > history.
> >>
> >> Well, there are limited counters. How to multiplex from userspace on
> >> the fly (e.g. monitoring a set of events first, followed by a
> >> different set)?
> >
> > I would just create a different BPF_PERF_ARRAY for each different one.
>
> Yes, but if you create more than available number of hardware counters
> (i.e. try to monitor more events concurrently than allowed by the
> hardware), counter value is reported as '0'. Verified with a test
> program (perf.c, attached) that uses perf_event_open() syscall.
> Depending upon the number of counters available on your platform,
> change NUM_REQ_HW_CNTRS to verify the behavior
>
> A way to enable/disable events at runtime will help userspace
> multiplex over available hardware counters and monitor more more
> events (similar to perf stat tool)
>
If you just call table.open_perf_event(NEW_TYPE) multiple times, it should
bind a new counter to the same table entry, allowing you to change the
monitored event over time. Have you tried that?

>
> >>
> >> Also, is it possible to handle counter overflow?
> >
> > What does it mean to "handle"? If computing deltas, for instance, the
> > subtraction will just underflow and wrap around to the correct value,
> > assuming the values are both unsigned.
> >>
> >>
> >> >>
> >> >>
> >> >> Thanks,
> >> >> Riya
> >> >>
> >> >> On Fri, Jul 29, 2016 at 1:57 PM, riya khanna <
> riyakhanna1...@gmail.com>
> >> >> wrote:
> >> >> > Thanks Brenden!
> >> >> >
> >> >> > I'm working with your branch for now. Additionally, I'm unable to
> >> >> > create software events (see exception below). Just wanted to bring
> >> >> > this to your attention.
> >> >> >
> >> >> > Traceback (most recent call last):
> >> >> >
> >> >> >   File "./test_bpf.py", line 176, in 
> >> >> > sw_clock.open_perf_event(1, 0)
> >> >> >
> >> >> >   File "/usr/lib/python2.7/dist-packages/bcc/table.py", line 410,
> in
> >> >> > open_perf_event
> >> >> > fd = self._open_perf_event(typ, config, i)
> >> >> >   File "/usr/lib/python2.7/dist-packages/bcc/table.py", line 416,
> in
> >> >> > _open_perf_event
> >> >> > self[self.Key(cpu)] = self.Leaf(fd)
> >> >> >   File "/usr/lib/python2.7/dist-packages/bcc/table.py", line 320,
> in
> >> >> > __setitem__
> >> >> > super(ArrayBase, self).__setitem__(key, leaf)
> >> >> >   File "/usr/lib/python2.7/dist-packages/bcc/table.py", line 169,
> in
> >> >> > __setitem__
> >> >> > raise Exception("Could not update table")
> >> >> > Exception: Could not update table
> >> >> >
> >> >> > On Fri, Jul 29, 2016 at 1:34 PM, Brenden Blanco
> >> >> > 
> >> >> > wrote:
> >> >> >> On Fri, Jul 29, 2016 at 10:21 AM, riya khanna
> >> >> >> 
> >> >> >> wrote:
> >> >> >>>
> >> >> >>> I'm testing perf counters on a 8-core machine.
> >> >> >>>
> >> >> >>> since BPF_PERF_ARRAY.perf_read(cpu) reads from local CPU, I'm
> >> >> >>> aggregating counters across all cpus by doing:
> >> >> >>>
> >> >> >>> BPF_PERF_ARRAY(counter, 32);
> >> >> >>>
> >> >> >>> for (key = 0; key < 8; key++)
> >> >> >>> counter.perf_read(key);
> >> >> >>
> >> >> >>
> >> >> >> I think it would make more sense to only read the counter on the
> cpu
> >> >> >> where
> >> >> >> the event is taking place. So:
> >> >> >>
> >> >> >> u64 key = cycles.perf_read(bpf_get_smp_processor_id());
> >> >> >>
> >> >> >> And then aggregate counters in userspace.
> >> >> >>
> >> >> >> I have spent some time over the past couple days cleaning up the
> >> >> >> code
> >> >> >> in
> >> >> >> that private branch, but have been distracted a bit so haven't
> >> >> >> finalized it.
> >> >> >> Hopefully a PR will come soon.
> >> >> >>
> >> >> >>>
> >> >> >>> However, this reports error:
> >> >> >>>
> >> >> >>> bpf: Invalid argument
> >> >> >>> back-edge from insn 69 to 17
> >> >> >>>
> >> >> >>> If I loop from 

Re: [iovisor-dev] read perf counters

[Brenden Blanco via iovisor-dev]

On Fri, Jul 29, 2016 at 10:21 AM, riya khanna 
wrote:

> I'm testing perf counters on a 8-core machine.
>
> since BPF_PERF_ARRAY.perf_read(cpu) reads from local CPU, I'm
> aggregating counters across all cpus by doing:
>
> BPF_PERF_ARRAY(counter, 32);
>
> for (key = 0; key < 8; key++)
> counter.perf_read(key);
>

I think it would make more sense to only read the counter on the cpu where
the event is taking place. So:

u64 key = cycles.perf_read(bpf_get_smp_processor_id());

And then aggregate counters in userspace.

I have spent some time over the past couple days cleaning up the code in
that private branch, but have been distracted a bit so haven't finalized
it. Hopefully a PR will come soon.


> However, this reports error:
>
> bpf: Invalid argument
> back-edge from insn 69 to 17
>
> If I loop from 0-4, it works. The code below works:
> for (key = 0; key < 4; key++)
> counter.perf_read(key);
>

> What could be wrong here?
>
The kernel verifier won't allow loops (i.e. back edges), and depending on
the loop unroll optimization decision made by llvm, this short loop may
have been automatically unrolled. Still, the solution should be to remove
the loop and just read the local cpu's perf counter as mentioned above.

>
>
> On Tue, Jul 26, 2016 at 7:29 PM, riya khanna 
> wrote:
> > From your patches I see that perf support is enabled per-cpu. Could
> > this be extended to enabling all or a group of perf counters on all
> > CPU cores similar to what perf_event_open provides (with args -1)?
> >
> > On Mon, Jul 25, 2016 at 9:55 PM, riya khanna 
> wrote:
> >> Thanks Brenden!
> >>
> >> I will try with your changes. Meanwhile please let me know if you add
> >> missing functionality.
> >>
> >>
> >> On Mon, Jul 25, 2016 at 8:14 PM, Brenden Blanco 
> wrote:
> >>> This needs support in bcc.
> >>>
> >>> I had a patch laying around that I never finished, you can find the
> partial
> >>> support here:
> >>> https://github.com/iovisor/bcc/tree/perf-counter
> >>>
> >>> It shouldn't be too hard to finalize that, let me see what I can do.
> >>>
> >>> On Mon, Jul 25, 2016 at 4:11 PM, riya khanna via iovisor-dev
> >>>  wrote:
> 
>  So I fixed the error above by using "count = my_map.perf_read(key);"
>  as opposed to "count = bpf_perf_event_read(_map, key);". However,
>  how do I selectively enable counters (e.g. instructions, cache misses,
>  etc.)?
> 
>  Thanks,
>  Riya
> 
>  On Mon, Jul 25, 2016 at 9:58 AM, riya khanna <
> riyakhanna1...@gmail.com>
>  wrote:
>  > Hi,
>  >
>  > I'm trying to read perf counters using bpf. However, adding
>  > BPF_PERF_ARRAY reports error:
>  >
>  > bpf: Invalid argument
>  > unrecognized bpf_ld_imm64 inns
>  >
>  > Is there an example/sample to read perf counters  that I can follow?
>  > The code below is what I'm trying to execute.
>  >
>  > Thanks,
>  > Riya
>  >
>  > # load BPF program
>  >
>  > bpf_text = """
>  >
>  > #include 
>  >
>  > BPF_PERF_ARRAY(my_map, 32);
>  >
>  > int start_counting(struct pt_regs *ctx) {
>  >
>  > if (!PT_REGS_PARM1(ctx))
>  >
>  > return 0;
>  >
>  > u64 count;
>  >
>  > u32 key = bpf_get_smp_processor_id();
>  >
>  > count = bpf_perf_event_read(_map, key);
>  >
>  > bpf_trace_printk("CPU-%d %llu", key, count);
>  >
>  > return 0;
>  >
>  > }
>  >
>  > """
>  ___
>  iovisor-dev mailing list
>  iovisor-dev@lists.iovisor.org
>  https://lists.iovisor.org/mailman/listinfo/iovisor-dev
> >>>
> >>>
>
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev

Re: [iovisor-dev] read perf counters

[Brenden Blanco via iovisor-dev]

This needs support in bcc.

I had a patch laying around that I never finished, you can find the partial
support here:
https://github.com/iovisor/bcc/tree/perf-counter

It shouldn't be too hard to finalize that, let me see what I can do.

On Mon, Jul 25, 2016 at 4:11 PM, riya khanna via iovisor-dev <
iovisor-dev@lists.iovisor.org> wrote:

> So I fixed the error above by using "count = my_map.perf_read(key);"
> as opposed to "count = bpf_perf_event_read(_map, key);". However,
> how do I selectively enable counters (e.g. instructions, cache misses,
> etc.)?
>
> Thanks,
> Riya
>
> On Mon, Jul 25, 2016 at 9:58 AM, riya khanna 
> wrote:
> > Hi,
> >
> > I'm trying to read perf counters using bpf. However, adding
> > BPF_PERF_ARRAY reports error:
> >
> > bpf: Invalid argument
> > unrecognized bpf_ld_imm64 inns
> >
> > Is there an example/sample to read perf counters  that I can follow?
> > The code below is what I'm trying to execute.
> >
> > Thanks,
> > Riya
> >
> > # load BPF program
> >
> > bpf_text = """
> >
> > #include 
> >
> > BPF_PERF_ARRAY(my_map, 32);
> >
> > int start_counting(struct pt_regs *ctx) {
> >
> > if (!PT_REGS_PARM1(ctx))
> >
> > return 0;
> >
> > u64 count;
> >
> > u32 key = bpf_get_smp_processor_id();
> >
> > count = bpf_perf_event_read(_map, key);
> >
> > bpf_trace_printk("CPU-%d %llu", key, count);
> >
> > return 0;
> >
> > }
> >
> > """
> ___
> iovisor-dev mailing list
> iovisor-dev@lists.iovisor.org
> https://lists.iovisor.org/mailman/listinfo/iovisor-dev
>
___
iovisor-dev mailing list
iovisor-dev@lists.iovisor.org
https://lists.iovisor.org/mailman/listinfo/iovisor-dev