[PATCH v5 3/4] libbpf: add bpf_prog_test_run_xattr

2018-12-03 Thread Lorenz Bauer
Add a new function, which encourages safe usage of the test interface. bpf_prog_test_run continues to work as before, but should be considered unsafe. Signed-off-by: Lorenz Bauer --- tools/lib/bpf/bpf.c | 23 +++ tools/lib/bpf/bpf.h | 19 +++ tools

[PATCH v5 2/4] tools: sync uapi/linux/bpf.h

2018-12-03 Thread Lorenz Bauer
Pull changes from "bpf: respect size hint to BPF_PROG_TEST_RUN if present". Signed-off-by: Lorenz Bauer --- tools/include/uapi/linux/bpf.h | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/b

[PATCH v5 1/4] bpf: respect size hint to BPF_PROG_TEST_RUN if present

2018-12-03 Thread Lorenz Bauer
Use data_size_out as a size hint when copying test output to user space. ENOSPC is returned if the output buffer is too small. Callers which so far did not set data_size_out are not affected. Signed-off-by: Lorenz Bauer --- include/uapi/linux/bpf.h | 7 +-- net/bpf/test_run.c | 15

[PATCH v5 0/4] Fix unsafe BPF_PROG_TEST_RUN interface

2018-12-03 Thread Lorenz Bauer
the existing function Changes in v2: * Make the syscall return ENOSPC if data_size_out is too small * Make bpf_prog_test_run return EINVAL if size_out is missing * Document the new behaviour of data_size_out Lorenz Bauer (4): bpf: respect size hint to BPF_PROG_TEST_RUN if present tools: sync

[PATCH v5 4/4] selftests: add a test for bpf_prog_test_run_xattr

2018-12-03 Thread Lorenz Bauer
Make sure that bpf_prog_test_run_xattr returns the correct length and that the kernel respects the output size hint. Also check that errno indicates ENOSPC if there is a short output buffer given. Signed-off-by: Lorenz Bauer --- tools/testing/selftests/bpf/test_progs.c | 55

[PATCH v4 0/4] Fix unsafe BPF_PROG_TEST_RUN interface

2018-11-28 Thread Lorenz Bauer
* Make bpf_prog_test_run return EINVAL if size_out is missing * Document the new behaviour of data_size_out Lorenz Bauer (4): bpf: respect size hint to BPF_PROG_TEST_RUN if present tools: sync uapi/linux/bpf.h libbpf: add bpf_prog_test_run_xattr selftests: add a test for bpf_prog_test_run_xattr

[PATCH v4 2/4] tools: sync uapi/linux/bpf.h

2018-11-28 Thread Lorenz Bauer
Pull changes from "bpf: respect size hint to BPF_PROG_TEST_RUN if present". Signed-off-by: Lorenz Bauer --- tools/include/uapi/linux/bpf.h | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/b

[PATCH v4 4/4] selftests: add a test for bpf_prog_test_run_xattr

2018-11-28 Thread Lorenz Bauer
Make sure that bpf_prog_test_run_xattr returns the correct length and that the kernel respects the output size hint. Also check that errno indicates ENOSPC if there is a short output buffer given. Signed-off-by: Lorenz Bauer --- tools/testing/selftests/bpf/test_progs.c | 55

[PATCH v4 1/4] bpf: respect size hint to BPF_PROG_TEST_RUN if present

2018-11-28 Thread Lorenz Bauer
Use data_size_out as a size hint when copying test output to user space. ENOSPC is returned if the output buffer is too small. Callers which so far did not set data_size_out are not affected. Signed-off-by: Lorenz Bauer --- include/uapi/linux/bpf.h | 7 +-- net/bpf/test_run.c | 15

[PATCH v4 3/4] libbpf: add bpf_prog_test_run_xattr

2018-11-28 Thread Lorenz Bauer
Add a new function, which encourages safe usage of the test interface. bpf_prog_test_run continues to work as before, but should be considered unsafe. Signed-off-by: Lorenz Bauer --- tools/lib/bpf/bpf.c | 23 +++ tools/lib/bpf/bpf.h | 19 +++ 2 files changed

Re: [PATCH v3 3/4] libbpf: add bpf_prog_test_run_xattr

2018-11-28 Thread Lorenz Bauer
On Wed, 28 Nov 2018 at 05:05, Alexei Starovoitov wrote: > > On Mon, Nov 26, 2018 at 4:45 AM Lorenz Bauer wrote: > > > > That's what I had initially, but that makes re-using test_attr really > > awkward. Either > > you need to reset data_out_size before every call be

Re: [PATCH v3 3/4] libbpf: add bpf_prog_test_run_xattr

2018-11-26 Thread Lorenz Bauer
On Sat, 24 Nov 2018 at 22:20, Alexei Starovoitov wrote: > > On Fri, Nov 23, 2018 at 11:25:11PM +0100, Daniel Borkmann wrote: > > On 11/22/2018 03:09 PM, Lorenz Bauer wrote: > > > Add a new function, which encourages safe usage of the test interface. > > > bpf_p

[PATCH v3 0/4] Fix unsafe BPF_PROG_TEST_RUN interface

2018-11-22 Thread Lorenz Bauer
in v3: * Introduce bpf_prog_test_run_xattr instead of modifying the existing function Changes in v2: * Make the syscall return ENOSPC if data_size_out is too small * Make bpf_prog_test_run return EINVAL if size_out is missing * Document the new behaviour of data_size_out Lorenz Bauer (4): bpf

[PATCH v3 4/4] selftests: add a test for bpf_prog_test_run_xattr

2018-11-22 Thread Lorenz Bauer
Make sure that bpf_prog_test_run_xattr returns the correct length and that the kernel respects the output size hint. Also check that errno indicates ENOSPC if there is a short output buffer given. Signed-off-by: Lorenz Bauer --- tools/testing/selftests/bpf/test_progs.c | 49

[PATCH v3 1/4] bpf: respect size hint to BPF_PROG_TEST_RUN if present

2018-11-22 Thread Lorenz Bauer
Use data_size_out as a size hint when copying test output to user space. ENOSPC is returned if the output buffer is too small. Callers which so far did not set data_size_out are not affected. Signed-off-by: Lorenz Bauer --- include/uapi/linux/bpf.h | 7 +-- net/bpf/test_run.c | 15

[PATCH v3 2/4] tools: sync uapi/linux/bpf.h

2018-11-22 Thread Lorenz Bauer
Pull changes from "bpf: respect size hint to BPF_PROG_TEST_RUN if present". Signed-off-by: Lorenz Bauer --- tools/include/uapi/linux/bpf.h | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/b

[PATCH v3 3/4] libbpf: add bpf_prog_test_run_xattr

2018-11-22 Thread Lorenz Bauer
Add a new function, which encourages safe usage of the test interface. bpf_prog_test_run continues to work as before, but should be considered unsafe. Signed-off-by: Lorenz Bauer --- tools/lib/bpf/bpf.c | 27 +++ tools/lib/bpf/bpf.h | 13 + 2 files changed

Re: [PATCH v2 3/4] libbpf: require size hint in bpf_prog_test_run

2018-11-20 Thread Lorenz Bauer
On Tue, 20 Nov 2018 at 19:18, Alexei Starovoitov wrote: > > On Tue, Nov 20, 2018 at 03:43:05PM +, Lorenz Bauer wrote: > > Require size_out to be non-NULL if data_out is given. This prevents > > accidental overwriting of process memory after the output buffer. >

[PATCH v2 4/4] selftests: add a test for bpf_prog_test_run output size

2018-11-20 Thread Lorenz Bauer
Make sure that bpf_prog_test_run returns the correct length in the size_out argument and that the kernel respects the output size hint. Also check that errno indicates ENOSPC. Signed-off-by: Lorenz Bauer --- tools/testing/selftests/bpf/test_progs.c | 34 1 file changed

[PATCH v2 3/4] libbpf: require size hint in bpf_prog_test_run

2018-11-20 Thread Lorenz Bauer
Require size_out to be non-NULL if data_out is given. This prevents accidental overwriting of process memory after the output buffer. Adjust callers of bpf_prog_test_run to this behaviour. Signed-off-by: Lorenz Bauer --- tools/lib/bpf/bpf.c | 7 ++- tools/testing

[PATCH v2 1/4] bpf: respect size hint to BPF_PROG_TEST_RUN if present

2018-11-20 Thread Lorenz Bauer
Use data_size_out as a size hint when copying test output to user space. ENOSPC is returned if the output buffer is too small. Callers which so far did not set data_size_out are not affected. Signed-off-by: Lorenz Bauer --- include/uapi/linux/bpf.h | 7 +-- net/bpf/test_run.c | 15

[PATCH v2 2/4] tools: sync uapi/linux/bpf.h

2018-11-20 Thread Lorenz Bauer
Pull changes from "bpf: respect size hint to BPF_PROG_TEST_RUN if present". Signed-off-by: Lorenz Bauer --- tools/include/uapi/linux/bpf.h | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/b

[PATCH v2 0/4] Fix unsafe BPF_PROG_TEST_RUN interface

2018-11-20 Thread Lorenz Bauer
in v2: * Make the syscall return ENOSPC if data_size_out is too small * Make bpf_prog_test_run return EINVAL if size_out is missing * Document the new behaviour of data_size_out Lorenz Bauer (4): bpf: respect size hint to BPF_PROG_TEST_RUN if present tools: sync uapi/linux/bpf.h libbpf

Re: [PATCH 3/3] selftests: add a test for bpf_prog_test_run output size

2018-11-20 Thread Lorenz Bauer
On Sun, 18 Nov 2018 at 05:59, Y Song wrote: > > On Fri, Nov 16, 2018 at 12:55 PM Lorenz Bauer wrote: > > > > Make sure that bpf_prog_test_run returns the correct length > > in the size_out argument and that the kernel respects the > > output size hint. >

Re: [PATCH 0/3] Fix unsafe BPF_PROG_TEST_RUN interface

2018-11-19 Thread Lorenz Bauer
dants (e.g katran) * github.com/iovisor/gobpf * github.com/newtools/ebpf That doesn't really guarantee anything of course. -- Lorenz Bauer | Systems Engineer 25 Lavington St., London SE1 0NZ www.cloudflare.com

[PATCH 3/3] selftests: add a test for bpf_prog_test_run output size

2018-11-16 Thread Lorenz Bauer
Make sure that bpf_prog_test_run returns the correct length in the size_out argument and that the kernel respects the output size hint. Signed-off-by: Lorenz Bauer --- tools/testing/selftests/bpf/test_progs.c | 34 1 file changed, 34 insertions(+) diff --git a/tools

[PATCH 2/3] libbpf: require size hint in bpf_prog_test_run

2018-11-16 Thread Lorenz Bauer
Require size_out to be non-NULL if data_out is given. This prevents accidental overwriting of process memory after the output buffer. Adjust callers of bpf_prog_test_run to this behaviour. Signed-off-by: Lorenz Bauer --- tools/lib/bpf/bpf.c | 4 +++- tools/testing

[PATCH 0/3] Fix unsafe BPF_PROG_TEST_RUN interface

2018-11-16 Thread Lorenz Bauer
this by using bpf_attr.test.data_size_out as a size hint. The old behaviour is retained if size_hint is zero. Interestingly, do_test_single() in test_verifier.c already assumes that this is the intended use of data_size_out, and sets it to the output buffer size. Lorenz Bauer (3): bpf: respect size

[PATCH 1/3] bpf: respect size hint to BPF_PROG_TEST_RUN if present

2018-11-16 Thread Lorenz Bauer
-by: Lorenz Bauer --- net/bpf/test_run.c | 9 - 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/net/bpf/test_run.c b/net/bpf/test_run.c index c89c22c49015..30c57b7f4ba4 100644 --- a/net/bpf/test_run.c +++ b/net/bpf/test_run.c @@ -74,8 +74,15 @@ static int bpf_test_finish(const

[PATCH v3 3/4] tools: sync linux/bpf.h

2018-11-16 Thread Lorenz Bauer
Synchronize changes to linux/bpf.h from * "bpf: allow zero-initializing hash map seed" * "bpf: move BPF_F_QUERY_EFFECTIVE after map flags" Signed-off-by: Lorenz Bauer --- tools/include/uapi/linux/bpf.h | 13 ++--- 1 file changed, 10 insertions(+), 3 deletions(-)

[PATCH v3 4/4] tools: add selftest for BPF_F_ZERO_SEED

2018-11-16 Thread Lorenz Bauer
Check that iterating two separate hash maps produces the same order of keys if BPF_F_ZERO_SEED is used. Signed-off-by: Lorenz Bauer --- tools/testing/selftests/bpf/test_maps.c | 68 + 1 file changed, 57 insertions(+), 11 deletions(-) diff --git a/tools/testing/selftests

[PATCH v3 2/4] bpf: move BPF_F_QUERY_EFFECTIVE after map flags

2018-11-16 Thread Lorenz Bauer
BPF_F_QUERY_EFFECTIVE is in the middle of the flags valid for BPF_MAP_CREATE. Move it to its own section to reduce confusion. Signed-off-by: Lorenz Bauer --- include/uapi/linux/bpf.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/include/uapi/linux/bpf.h b/include

[PATCH v3 1/4] bpf: allow zero-initializing hash map seed

2018-11-16 Thread Lorenz Bauer
Add a new flag BPF_F_ZERO_SEED, which forces a hash map to initialize the seed to zero. This is useful when doing performance analysis both on individual BPF programs, as well as the kernel's hash table implementation. Signed-off-by: Lorenz Bauer --- include/uapi/linux/bpf.h | 3 +++ kernel

[PATCH v3 0/4] bpf: allow zero-initialising hash map seed

2018-11-16 Thread Lorenz Bauer
Allow forcing the seed of a hash table to zero, for deterministic execution during benchmarking and testing. Changes from v2: * Change ordering of BPF_F_ZERO_SEED in linux/bpf.h Comments adressed from v1: * Add comment to discourage production use to linux/bpf.h * Require CAP_SYS_ADMIN Lorenz

Re: [PATCH v2 1/3] bpf: allow zero-initializing hash map seed

2018-10-25 Thread Lorenz Bauer
UILD_ID(1U << 5) I wanted to keep the flags for BPF_MAP_CREATE grouped together. Maybe the correct value is (1U << 3)? It seemed like the other flags were allocated to avoid overlap between different BPF commands, however, so I tried to follow suit. -- Lorenz Bauer | Systems Engineer 25 Lavington St., London SE1 0NZ www.cloudflare.com

Re: [PATCH v2 2/3] tools: sync linux/bpf.h

2018-10-25 Thread Lorenz Bauer
On Tue, 9 Oct 2018 at 01:12, Song Liu wrote: > > On Mon, Oct 8, 2018 at 3:34 AM Lorenz Bauer wrote: > > > > Synchronize changes to linux/bpf.h from > > commit 88db241b34bf ("bpf: allow zero-initializing hash map seed"). > I guess we cannot keep this hash du

[PATCH v2 2/3] tools: sync linux/bpf.h

2018-10-08 Thread Lorenz Bauer
Synchronize changes to linux/bpf.h from commit 88db241b34bf ("bpf: allow zero-initializing hash map seed"). Signed-off-by: Lorenz Bauer --- tools/include/uapi/linux/bpf.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/b

[PATCH v2 3/3] tools: add selftest for BPF_F_ZERO_SEED

2018-10-08 Thread Lorenz Bauer
Check that iterating two separate hash maps produces the same order of keys if BPF_F_ZERO_SEED is used. Signed-off-by: Lorenz Bauer --- tools/testing/selftests/bpf/test_maps.c | 68 + 1 file changed, 57 insertions(+), 11 deletions(-) diff --git a/tools/testing/selftests

[PATCH v2 1/3] bpf: allow zero-initializing hash map seed

2018-10-08 Thread Lorenz Bauer
Add a new flag BPF_F_ZERO_SEED, which forces a hash map to initialize the seed to zero. This is useful when doing performance analysis both on individual BPF programs, as well as the kernel's hash table implementation. Signed-off-by: Lorenz Bauer --- include/uapi/linux/bpf.h | 2 ++ kernel/bpf

[PATCH v2 0/3] bpf: allow zero-initialising hash map seed

2018-10-08 Thread Lorenz Bauer
Allow forcing the seed of a hash table to zero, for deterministic execution during benchmarking and testing. Comments adressed from v1: * Add comment to discourage production use to linux/bpf.h * Require CAP_SYS_ADMIN Lorenz Bauer (3): bpf: allow zero-initializing hash map seed tools: sync

Re: [PATCH 1/3] bpf: allow zero-initializing hash map seed

2018-10-08 Thread Lorenz Bauer
case for deterministic hashing is performance analysis. > Both while developing and tuning bpf program and while optimizing > kernel side implementation. > Local dos is a valid concern, so requiring root for this flag makes sense. > Ok, I'll respin and address the comments. -- Lorenz

Re: [PATCH 0/3] bpf: allow zero-initialising hash map seed

2018-10-05 Thread Lorenz Bauer
On Mon, 1 Oct 2018 at 20:12, Daniel Borkmann wrote: > > On 10/01/2018 12:45 PM, Lorenz Bauer wrote: > > This patch set adds a new flag BPF_F_ZERO_SEED, which allows > > forcing the seed used by hash maps to zero. This makes > > it possible to write deterministic tests.

Re: [PATCH 1/3] bpf: allow zero-initializing hash map seed

2018-10-05 Thread Lorenz Bauer
On Fri, 5 Oct 2018 at 15:12, Jann Horn wrote: > > On Fri, Oct 5, 2018 at 9:42 AM Lorenz Bauer wrote: > > On Tue, 2 Oct 2018 at 21:00, Jann Horn wrote: > > > > > > If this is for testing only, you can slap a capable(CAP_SYS_ADMIN) > > > check in here,

Re: [PATCH 1/3] bpf: allow zero-initializing hash map seed

2018-10-05 Thread Lorenz Bauer
kernel testing. That would mean all tests have to run as root / with CAP_SYS_ADMIN which isn't ideal. -- Lorenz Bauer | Systems Engineer 25 Lavington St., London SE1 0NZ www.cloudflare.com

[PATCH 1/3] bpf: allow zero-initializing hash map seed

2018-10-01 Thread Lorenz Bauer
Add a new flag BPF_F_ZERO_SEED, which forces a hash map to initialize the seed to zero. --- include/uapi/linux/bpf.h | 2 ++ kernel/bpf/hashtab.c | 8 ++-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index

[PATCH 3/3] tools: add selftest for BPF_F_ZERO_SEED

2018-10-01 Thread Lorenz Bauer
--- tools/testing/selftests/bpf/test_maps.c | 67 + 1 file changed, 56 insertions(+), 11 deletions(-) diff --git a/tools/testing/selftests/bpf/test_maps.c b/tools/testing/selftests/bpf/test_maps.c index 9b552c0fc47d..2a1cc00475f6 100644 ---

[PATCH 2/3] tools: sync linux/bpf.h

2018-10-01 Thread Lorenz Bauer
--- tools/include/uapi/linux/bpf.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index aa5ccd2385ed..9d15c8f179ac 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -252,6 +252,8 @@ enum

[PATCH 0/3] bpf: allow zero-initialising hash map seed

2018-10-01 Thread Lorenz Bauer
This patch set adds a new flag BPF_F_ZERO_SEED, which allows forcing the seed used by hash maps to zero. This makes it possible to write deterministic tests. Based on an off-list conversation with Alexei Starovoitov and Daniel Borkmann. Lorenz Bauer (3): bpf: allow zero-initializing hash map

Re: Allow bpf_perf_event_output to access packet data

2018-09-11 Thread Lorenz Bauer
! I managed to find samples/bpf/xdp_sample_pkts_kern.c as well, which was helpful. My next gotcha is that perf_event_output seems to ignore the sample_period parameter passed to perf_event_output. This is not a big problem since I can just implement the sampling in BPF, but am I missing something again? -

Allow bpf_perf_event_output to access packet data

2018-09-07 Thread Lorenz Bauer
Re-sent due to HTML e-mail mess up, apologies. -- Forwarded message -- From: Lorenz Bauer Date: 7 September 2018 at 15:53 Subject: Allow bpf_perf_event_output to access packet data To: netdev@vger.kernel.org, Alexei Starovoitov , Daniel Borkmann Hello list, I'm attempting

PROBLEM: Using BPF_PROG_TEST_RUN with data_out != NULL is unsafe

2018-04-04 Thread Lorenz Bauer
be larger than data_size_in due to bpf_xdp_adjust_head() and friends. bpf_test_finish doesn’t clamp size to data_size_out, which is what I was expecting. What is the correct way to use this interface? Best, Lorenz -- Lorenz Bauer | Systems Engineer 25 Lavington St., London SE1 0NZ