Re: [PATCH RFC 4/6] btrfs: Check if the filesystem is has mixed type of devices

2021-02-15 Thread Michal Rostecki
On Sat, Feb 13, 2021 at 12:36:02AM +0100, Michał Mirosław wrote: > On Fri, Feb 12, 2021 at 06:26:41PM +0000, Michal Rostecki wrote: > > On Wed, Feb 10, 2021 at 05:08:05AM +0100, Michał Mirosław wrote: > > > On Tue, Feb 09, 2021 at 09:30:38PM +0100, Michal Rostecki wrote: &g

Re: [PATCH RFC 5/6] btrfs: sysfs: Add directory for read policies

2021-02-15 Thread Michal Rostecki
On Sat, Feb 13, 2021 at 11:19:28AM +0100, Greg KH wrote: > On Tue, Feb 09, 2021 at 09:30:39PM +0100, Michal Rostecki wrote: > > From: Michal Rostecki > > > > Before this change, raid1 read policy could be selected by using the > > /sys/fs/btrfs/[fsid]/read_policy file.

Re: [PATCH RFC 4/6] btrfs: Check if the filesystem is has mixed type of devices

2021-02-12 Thread Michal Rostecki
On Wed, Feb 10, 2021 at 05:08:05AM +0100, Michał Mirosław wrote: > On Tue, Feb 09, 2021 at 09:30:38PM +0100, Michal Rostecki wrote: > > From: Michal Rostecki > > > > Add the btrfs_check_mixed() function which checks if the filesystem has > > the mixed type

Re: [PATCH RFC 6/6] btrfs: Add roundrobin raid1 read policy

2021-02-12 Thread Michal Rostecki
Hi Anand, re: inflight calculation On Thu, Feb 11, 2021 at 03:55:33PM +, Michal Rostecki wrote: > > It is better to have random workloads in the above three categories > > of configs. > > > > Apart from the above three configs, there is also > >

Re: [PATCH RFC 6/6] btrfs: Add roundrobin raid1 read policy

2021-02-11 Thread Michal Rostecki
On Wed, Feb 10, 2021 at 04:20:20PM +0800, Anand Jain wrote: > On 10/02/2021 04:30, Michal Rostecki wrote: > > The penalty value is an additional value added to the number of inflight > > requests when a scheduled request is non-local (which means it would > > start from t

Re: [PATCH RFC 6/6] btrfs: Add roundrobin raid1 read policy

2021-02-11 Thread Michal Rostecki
On Thu, Feb 11, 2021 at 03:27:38AM +0100, Michał Mirosław wrote: > On Wed, Feb 10, 2021 at 07:23:04PM +0000, Michal Rostecki wrote: > > On Wed, Feb 10, 2021 at 01:58:15PM +0100, Michał Mirosław wrote: > > > On Wed, Feb 10, 2021 at 12:29:25PM +0000, Michal Rostecki wrote: >

Re: [PATCH RFC 6/6] btrfs: Add roundrobin raid1 read policy

2021-02-10 Thread Michal Rostecki
On Wed, Feb 10, 2021 at 01:58:15PM +0100, Michał Mirosław wrote: > On Wed, Feb 10, 2021 at 12:29:25PM +0000, Michal Rostecki wrote: > > On Wed, Feb 10, 2021 at 05:24:28AM +0100, Michał Mirosław wrote: > > > This looks like it effectively decreases queue depth for non-last >

Re: [PATCH RFC 0/6] Add roundrobin raid1 read policy

2021-02-10 Thread Michal Rostecki
On Wed, Feb 10, 2021 at 12:18:53PM +, Michal Rostecki wrote: > > These patches look good. But as only round-robin policy requires > > to monitor the inflight and last-offset. Could you bring them under > > if policy=roundrobin? Otherwise, it is just a waste of CPU cycles

Re: [PATCH RFC 4/6] btrfs: Check if the filesystem is has mixed type of devices

2021-02-10 Thread Michal Rostecki
On Wed, Feb 10, 2021 at 10:09:10AM +, Filipe Manana wrote: > On Tue, Feb 9, 2021 at 9:32 PM Michal Rostecki wrote: > > > > From: Michal Rostecki > > > > Add the btrfs_check_mixed() function which checks if the filesystem has > > the mixed type of devic

Re: [PATCH RFC 4/6] btrfs: Check if the filesystem is has mixed type of devices

2021-02-10 Thread Michal Rostecki
On Wed, Feb 10, 2021 at 05:08:05AM +0100, Michał Mirosław wrote: > On Tue, Feb 09, 2021 at 09:30:38PM +0100, Michal Rostecki wrote: > > From: Michal Rostecki > > > > Add the btrfs_check_mixed() function which checks if the filesystem has > > the mixed type

Re: [PATCH RFC 6/6] btrfs: Add roundrobin raid1 read policy

2021-02-10 Thread Michal Rostecki
On Wed, Feb 10, 2021 at 05:24:28AM +0100, Michał Mirosław wrote: > On Tue, Feb 09, 2021 at 09:30:40PM +0100, Michal Rostecki wrote: > [...] > > For the array with 3 HDDs, not adding any penalty resulted in 409MiB/s > > (429MB/s) performance. Adding the penalty value 1 resulted in

Re: [PATCH RFC 0/6] Add roundrobin raid1 read policy

2021-02-10 Thread Michal Rostecki
On Wed, Feb 10, 2021 at 02:52:01PM +0800, Anand Jain wrote: > On 10/02/2021 04:30, Michal Rostecki wrote: > > From: Michal Rostecki > > > > This patch series adds a new raid1 read policy - roundrobin. For each > > request, it selects the mirror which has lower load

[PATCH RFC 6/6] btrfs: Add roundrobin raid1 read policy

2021-02-09 Thread Michal Rostecki
From: Michal Rostecki Add a new raid1 read policy `roundrobin`. For each read request, it selects the mirror which has lower load than queue depth and it starts iterating from the last used mirror (by the current CPU). Load is defined as the number of inflight requests + a potential penalty

[PATCH RFC 5/6] btrfs: sysfs: Add directory for read policies

2021-02-09 Thread Michal Rostecki
From: Michal Rostecki Before this change, raid1 read policy could be selected by using the /sys/fs/btrfs/[fsid]/read_policy file. Change it to /sys/fs/btrfs/[fsid]/read_policies/policy. The motivation behing creating the read_policies directory is that the next changes and new read policies

[PATCH RFC 1/6] btrfs: Add inflight BIO request counter

2021-02-09 Thread Michal Rostecki
From: Michal Rostecki Add a per-CPU inflight BIO counter to btrfs_device which stores the number of requests currently processed by the device. This information is going to be used in roundrobin raid1 read policy. Signed-off-by: Michal Rostecki --- fs/btrfs/volumes.c | 11 +-- fs

[PATCH RFC 3/6] btrfs: Add stripe_physical function

2021-02-09 Thread Michal Rostecki
From: Michal Rostecki Move the calculation of the physical address for a stripe to the new function - stripe_physical(). It can be used by raid1 read policies to calculate the offset and select mirrors based on I/O locality. Signed-off-by: Michal Rostecki --- fs/btrfs/volumes.c | 22

[PATCH RFC 4/6] btrfs: Check if the filesystem is has mixed type of devices

2021-02-09 Thread Michal Rostecki
From: Michal Rostecki Add the btrfs_check_mixed() function which checks if the filesystem has the mixed type of devices (non-rotational and rotational). This information is going to be used in roundrobin raid1 read policy. Signed-off-by: Michal Rostecki --- fs/btrfs/volumes.c | 44

[PATCH RFC 2/6] btrfs: Store the last device I/O offset

2021-02-09 Thread Michal Rostecki
From: Michal Rostecki Add an atomic field which stores the physical offset of the last I/O operation scheduled to the device. This information is going to be used to measure the locality of I/O requests. Signed-off-by: Michal Rostecki --- fs/btrfs/volumes.c | 4 fs/btrfs/volumes.h | 1

[PATCH RFC 0/6] Add roundrobin raid1 read policy

2021-02-09 Thread Michal Rostecki
From: Michal Rostecki This patch series adds a new raid1 read policy - roundrobin. For each request, it selects the mirror which has lower load than queue depth. Load is defined as the number of inflight requests + a penalty value (if the scheduled request is not local to the last processed

Re: [PATCH v2] btrfs: Avoid calling btrfs_get_chunk_map() twice

2021-01-29 Thread Michal Rostecki
On Fri, Jan 29, 2021 at 06:47:53PM +0100, David Sterba wrote: > On Fri, Jan 29, 2021 at 05:15:21PM +0000, Michal Rostecki wrote: > > On Fri, Jan 29, 2021 at 11:22:48AM -0500, Josef Bacik wrote: > > > On 1/27/21 8:57 AM, Michal Rostecki wrote: > > > > From: Michal Ros

Re: [PATCH v2] btrfs: Avoid calling btrfs_get_chunk_map() twice

2021-01-29 Thread Michal Rostecki
On Fri, Jan 29, 2021 at 11:22:48AM -0500, Josef Bacik wrote: > On 1/27/21 8:57 AM, Michal Rostecki wrote: > > From: Michal Rostecki > > > > Before this change, the btrfs_get_io_geometry() function was calling > > btrfs_get_chunk_map() to get the extent mapping, ne

[PATCH v2] btrfs: Avoid calling btrfs_get_chunk_map() twice

2021-01-27 Thread Michal Rostecki
From: Michal Rostecki Before this change, the btrfs_get_io_geometry() function was calling btrfs_get_chunk_map() to get the extent mapping, necessary for calculating the I/O geometry. It was using that extent mapping only internally and freeing the pointer after its execution. That resulted

Re: [PATCH] btrfs: Avoid calling btrfs_get_chunk_map() twice

2021-01-27 Thread Michal Rostecki
On Wed, Jan 27, 2021 at 11:20:55AM +, Filipe Manana wrote: > On Wed, Jan 27, 2021 at 9:59 AM Michal Rostecki wrote: > > > > From: Michal Rostecki > > > > Before this change, the btrfs_get_io_geometry() function was calling > > btrfs_get_chunk_map() to

[PATCH] btrfs: Avoid calling btrfs_get_chunk_map() twice

2021-01-27 Thread Michal Rostecki
From: Michal Rostecki Before this change, the btrfs_get_io_geometry() function was calling btrfs_get_chunk_map() to get the extent mapping, necessary for calculating the I/O geometry. It was using that extent mapping only internally and freeing the pointer after its execution. That resulted

Re: [PATCH v3 bpf] tools: bpftool: Add missing close before bpftool net attach exit

2020-11-11 Thread Michal Rostecki
jsonw_null(json_wtr); - return 0; + err = 0; + +cleanup: + close(progfd); + return err; } static int do_detach(int argc, char **argv) LGTM, thanks! Reviewed-By: Michal Rostecki

Re: [PATCH bpf] tools: bpftool: Add missing close before bpftool net attach exit

2020-11-09 Thread Michal Rostecki
On 11/9/20 8:04 AM, Wang Hai wrote: progfd is created by prog_parse_fd(), before 'bpftool net attach' exit, it should be closed. Fixes: 04949ccc273e ("tools: bpftool: add net attach command to attach XDP on interface") Signed-off-by: Wang Hai --- tools/bpf/bpftool/net.c | 2 ++ 1 file

[PATCH bpf-next] samples: bpf: Remove bpf_debug macro in favor of bpf_printk

2019-06-18 Thread Michal Rostecki
ibumad example was implementing the bpf_debug macro which is exactly the same as the bpf_printk macro available in bpf_helpers.h. This change makes use of bpf_printk instead of bpf_debug. Signed-off-by: Michal Rostecki --- samples/bpf/ibumad_kern.c | 18 ++ 1 file changed, 6

[PATCH bpf v4] libbpf: Return btf_fd for load_sk_storage_btf

2019-05-29 Thread Michal Rostecki
* functions. - Rename libbpf__probe_raw_btf function to libbpf__load_raw_btf and return a BTF descriptor. v4: - Fix typo in the commit message. Fixes: d7c4b3980c18 ("libbpf: detect supported kernel BTF features and sanitize BTF") Signed-off-by: Michal Rostecki Acked-by: Andrii Nakryiko

[PATCH bpf v3] libbpf: Return btf_fd for load_sk_storage_btf

2019-05-29 Thread Michal Rostecki
* functions. - Rename libbpf__probe_raw_btf function to libbpf__load_raw_btf and return a BTF descriptor. Fixes: d7c4b3980c18 ("libbpf: detect supported kernel BTF features and sanitize BTF") Signed-off-by: Michal Rostecki --- tools/lib/bpf/libbpf.c | 28 -

Re: [PATCH bpf] libbpf: Return btf_fd in libbpf__probe_raw_btf

2019-05-29 Thread Michal Rostecki
On Wed, May 29, 2019 at 08:35:25AM -0700, Andrii Nakryiko wrote: > On Wed, May 29, 2019 at 1:30 AM Michal Rostecki > wrote: > > > > Function load_sk_storage_btf expects that libbpf__probe_raw_btf is > > returning a btf descriptor, but before this change it was returning

[PATCH bpf v2] libbpf: Return btf_fd in libbpf__probe_raw_btf

2019-05-29 Thread Michal Rostecki
not returning a valid json object. v2: - Fix typo in the commit message. Fixes: d7c4b3980c18 ("libbpf: detect supported kernel BTF features and sanitize BTF") Signed-off-by: Michal Rostecki --- tools/lib/bpf/libbpf.c| 36 +-- tools/lib/bpf/libbpf_pro

Re: [PATCH bpf] libbpf: Return btf_fd in libbpf__probe_raw_btf

2019-05-29 Thread Michal Rostecki
On Wed, May 29, 2019 at 12:53:42PM +0300, Sergei Shtylyov wrote: > Hello! > > On 29.05.2019 11:29, Michal Rostecki wrote: > > > Function load_sk_storage_btf expects that libbpf__probe_raw_btf is > > returning a btf descriptor, but before this change it was returning

[PATCH bpf] libbpf: Return btf_fd in libbpf__probe_raw_btf

2019-05-29 Thread Michal Rostecki
not returning a valid json object. Fixes: d7c4b3980c18 ("libbpf: detect supported kernel BTF features and sanitize BTF") Signed-off-by: Michal Rostecki --- tools/lib/bpf/libbpf.c| 36 +-- tools/lib/bpf/libbpf_probes.c | 7 +-- 2 files changed, 23

[PATCH bpf-next v2 RESEND 0/2] Move bpf_printk to bpf_helpers.h

2019-05-23 Thread Michal Rostecki
This series of patches move the commonly used bpf_printk macro to bpf_helpers.h which is already included in all BPF programs which defined that macro on their own. v1->v2: - If HBM_DEBUG is not defined in hbm sample, undefine bpf_printk and set an empty macro for it. Michal Rostecki

[PATCH bpf-next v2 RESEND 2/2] samples: bpf: Do not define bpf_printk macro

2019-05-23 Thread Michal Rostecki
The bpf_printk macro was moved to bpf_helpers.h which is included in all example programs. Signed-off-by: Michal Rostecki --- samples/bpf/hbm_kern.h | 11 ++- samples/bpf/tcp_basertt_kern.c | 7 --- samples/bpf/tcp_bufs_kern.c| 7 --- samples/bpf

[PATCH bpf-next v2 RESEND 1/2] selftests: bpf: Move bpf_printk to bpf_helpers.h

2019-05-23 Thread Michal Rostecki
bpf_printk is a macro which is commonly used to print out debug messages in BPF programs and it was copied in many selftests and samples. Since all of them include bpf_helpers.h, this change moves the macro there. Signed-off-by: Michal Rostecki --- tools/testing/selftests/bpf/bpf_helpers.h

[PATCH bpf-next v2 1/2] selftests: bpf: Move bpf_printk to bpf_helpers.h

2019-05-16 Thread Michal Rostecki
bpf_printk is a macro which is commonly used to print out debug messages in BPF programs and it was copied in many selftests and samples. Since all of them include bpf_helpers.h, this change moves the macro there. Signed-off-by: Michal Rostecki --- tools/testing/selftests/bpf/bpf_helpers.h