Re: [PATCH 0/4] small random patches

2020-06-21 Thread Jens Axboe
ring: set @poll->file after @poll init > io_uring: kill NULL checks for submit state > > fs/io_uring.c | 38 +- > 1 file changed, 9 insertions(+), 29 deletions(-) Thanks, applied. -- Jens Axboe

Re: [PATCH][next] blk-iocost: Use struct_size() in kzalloc_node()

2020-06-20 Thread Jens Axboe
On 6/19/20 5:08 PM, Gustavo A. R. Silva wrote: > Make use of the struct_size() helper instead of an open-coded version > in order to avoid any potential type mistakes. > > This code was detected with the help of Coccinelle and, audited and > fixed manually. Applied, thanks. -- Jens Axboe

Re: [PATCH][next] block: bio: Use struct_size() in kmalloc()

2020-06-20 Thread Jens Axboe
On 6/19/20 6:49 PM, Gustavo A. R. Silva wrote: > Make use of the struct_size() helper instead of an open-coded version > in order to avoid any potential type mistakes. > > This code was detected with the help of Coccinelle and, audited and > fixed manually. Applied, thanks. -- Jens Axboe

Re: [PATCH v4] docs: block: Create blk-mq documentation

2020-06-20 Thread Jens Axboe
O: Introducing Multi-queue SSD Access on Multi-core > Systems", by Axboe et al. This looks fine to me now - Jon, would you care to queue it up? You can add my: Reviewed-by: Jens Axboe -- Jens Axboe

Re: [PATCH v7 0/8] blktrace: fix debugfs use after free

2020-06-20 Thread Jens Axboe
t; blktrace: ensure our debugfs dir exists > block: create the request_queue debugfs_dir on registration Applied for 5.9, thanks. -- Jens Axboe

Re: [PATCH 1/2] drivers: cdrom: fix all errors reported by checkpatch

2020-06-19 Thread Jens Axboe
al: 0 errors, 3 warnings, 882 lines checked > > Test: scripts/checkpatch.pl --file drivers/cdrom/*.c > Test: make drivers/cdrom/cdrom.o > Test: make CONFIG_GDROM=y drivers/cdrom/gdrom.o # partial > Test: compiles, boots on 5.7.4 > Test: CD ROM works (tested with Audio CD) Let's please not do this, it's a lot of churn for zero gain. -- Jens Axboe

Re: [PATCH] gdrom: Block comments use a trailing */ on a separate line

2020-06-19 Thread Jens Axboe
On 6/19/20 3:18 PM, Sudhip Nashi wrote: > Fix code style to conform with linux coding standards Let's please just leave these old things alone, not worth the trouble. -- Jens Axboe

Re: [PATCH v2] docs: block: Create blk-mq documentation

2020-06-19 Thread Jens Axboe
On 6/19/20 2:51 PM, André Almeida wrote: > On 6/19/20 5:49 PM, Jens Axboe wrote: >> On 6/19/20 1:47 PM, Randy Dunlap wrote: >>> On 6/19/20 12:45 PM, Jonathan Corbet wrote: >>>> On Fri, 5 Jun 2020 14:55:36 -0300 >>>> André Almeida wrote: >>>&g

Re: [PATCH v3] docs: block: Create blk-mq documentation

2020-06-19 Thread Jens Axboe
t performing any reordering. This is useful in the > following NOOP is a relic from the single queue days, the basic "doesn't do much" scheduler is NONE these days. And it doesn't provide FIFO ordering, requests will basically just end up in whatever software queue the process is running on. When someone runs the hardware queue, the software queues mapped to that hardware queue will be drained in sequence according to their mapping (generally from 0..N, if 0..N are mapped to that hardware queue). -- Jens Axboe

Re: [PATCH v2] docs: block: Create blk-mq documentation

2020-06-19 Thread Jens Axboe
gned-off-by: André Almeida >>> --- >>> Changes from v1: >>> - Fixed typos >>> - Reworked blk_mq_hw_ctx >> >> Jens, what's your pleasure on this one? Should I take it, or do you want >> it...? > > I wouldn't mind seeing a v3. Agree - Jon, I'd be happy with you taking it once a v3 is posted with the remaining issues ironed out. -- Jens Axboe

Re: [PATCH 04/15] io_uring: re-issue block requests that failed because of resources

2020-06-19 Thread Jens Axboe
On 6/19/20 8:30 AM, Pavel Begunkov wrote: > On 19/06/2020 17:22, Jens Axboe wrote: >> On 6/19/20 8:12 AM, Pavel Begunkov wrote: >>> On 18/06/2020 17:43, Jens Axboe wrote: >>>> Mark the plug with nowait == true, which will cause requests to avoid >>>> bloc

[GIT PULL] libata fixes for 5.8-rc2

2020-06-19 Thread Jens Axboe
/ata/libata-core.c | 11 +-- drivers/ata/libata-scsi.c | 9 ++--- drivers/ata/sata_rcar.c | 11 +++ include/linux/libata.h| 3 +++ 4 files changed, 21 insertions(+), 13 deletions(-) -- Jens Axboe

Re: [PATCH 3/3] io_uring: add support for zone-append

2020-06-19 Thread Jens Axboe
On 6/19/20 8:59 AM, Pavel Begunkov wrote: > On 19/06/2020 17:15, Jens Axboe wrote: >> On 6/19/20 3:41 AM, javier.g...@samsung.com wrote: >>> Jens, >>> >>> Would you have time to answer a question below in this thread? >>> >>> On 18.06.2020 11

Re: [PATCH 3/3] io_uring: add support for zone-append

2020-06-19 Thread Jens Axboe
On 6/19/20 9:14 AM, Matias Bjørling wrote: > On 19/06/2020 16.18, Jens Axboe wrote: >> On 6/19/20 5:15 AM, Matias Bjørling wrote: >>> On 19/06/2020 11.41, javier.g...@samsung.com wrote: >>>> Jens, >>>> >>>> Would you have time to answer a que

Re: [PATCH 3/3] io_uring: add support for zone-append

2020-06-19 Thread Jens Axboe
doesn't do anything for widening the result. > You need > > val |= (uint64_t)(cqe->flags >> 16) << 32; You're right of course, guess I should check my in-mail code a bit better :-) -- Jens Axboe

Re: [PATCH 3/3] io_uring: add support for zone-append

2020-06-19 Thread Jens Axboe
On 6/19/20 9:40 AM, Matias Bjørling wrote: > On 19/06/2020 17.20, Jens Axboe wrote: >> On 6/19/20 9:14 AM, Matias Bjørling wrote: >>> On 19/06/2020 16.18, Jens Axboe wrote: >>>> On 6/19/20 5:15 AM, Matias Bjørling wrote: >>>>> On 19/06/2020 11.4

[GIT PULL] io_uring fixes for 5.8-rc2

2020-06-19 Thread Jens Axboe
up to 6f2cc1664db20676069cff27a461ccc97dbfd114: io_uring: fix possible race condition against REQ_F_NEED_CLEANUP (2020-06-18 08:32:44 -0600) io_uring-5.8-2020-06-19 Jens Axboe (2): io_

Re: [PATCH 04/15] io_uring: re-issue block requests that failed because of resources

2020-06-19 Thread Jens Axboe
On 6/19/20 8:12 AM, Pavel Begunkov wrote: > On 18/06/2020 17:43, Jens Axboe wrote: >> Mark the plug with nowait == true, which will cause requests to avoid >> blocking on request allocation. If they do, we catch them and reissue >> them from a task_work based handler. >&g

Re: [PATCH 3/3] io_uring: add support for zone-append

2020-06-19 Thread Jens Axboe
acceptable change for you? We will of course add support for >> liburing when we agree on the right way to do this. > > I took a quick look at the code. No expert, but why not use the existing > userdata variable? use the lowest bits (40 bits) for the Zone Starting > LBA, and use the highest (24 bits) as index into the completion data > structure? > > If you want to pass the memory address (same as what fio does) for the > data structure used for completion, one may also play some tricks by > using a relative memory address to the data structure. For example, the > x86_64 architecture uses 48 address bits for its memory addresses. With > 24 bit, one can allocate the completion entries in a 32MB memory range, > and then use base_address + index to get back to the completion data > structure specified in the sqe. For any current request, sqe->user_data is just provided back as cqe->user_data. This would make these requests behave differently from everything else in that sense, which seems very confusing to me if I was an application writer. But generally I do agree with you, there are lots of ways to make < 64-bit work as a tag without losing anything or having to jump through hoops to do so. The lack of consistency introduced by having zone append work differently is ugly, though. -- Jens Axboe

Re: [PATCH 3/3] io_uring: add support for zone-append

2020-06-19 Thread Jens Axboe
ter to this at submission time through the sqe. One issue I do see with that is if we only have this information available at completion time, then we'd need some async punt to copy it to user space... Generally not ideal. A hackier approach would be to play some tricks with cqe->res and cqe->flags, setting aside a flag to denote an extension of cqe->res. That would mean excluding zone append (etc) from using buffer selection, which probably isn't a huge deal. It'd be more problematic for any other future flags. But if you just need 40 bits, then it could certainly work. Rigth now, if cqe->flags & 1 is set, then (cqe->flags >> 16) is the buffer ID. You could define IORING_CQE_F_ZONE_FOO to be bit 1, so that: uint64_t val = cqe->res; // assuming non-error here if (cqe->flags & IORING_CQE_F_ZONE_FOO) val |= (cqe->flags >> 16) << 32ULL; and hence use the upper 16 bits of cqe->flags for the upper bits of your (then) 48-bit total value. -- Jens Axboe

Re: [PATCH v3] libata: Use per port sync for detach

2020-06-18 Thread Jens Axboe
cheduled may wait for not-yet-scheduled > callbacks, causes a circular dependency. > > Instead of using big hammer like async_synchronize_full(), use async > cookie to make sure port probe are synced, without affecting other > scheduled PM callbacks. Queued up for 5.8, thanks. -- Jens Axboe

[PATCH 04/15] io_uring: re-issue block requests that failed because of resources

2020-06-18 Thread Jens Axboe
plete, and hence we cannot handle it inline as part of submission. This does potentially cause re-reads of parts of the range, as the whole request is reissued. There's currently no better way to handle this. Signed-off-by: Jens Axboe --- fs/i

[PATCH 06/15] mm: abstract out wake_page_match() from wake_page_function()

2020-06-18 Thread Jens Axboe
No functional changes in this patch, just in preparation for allowing more callers. Acked-by: Johannes Weiner Signed-off-by: Jens Axboe --- include/linux/pagemap.h | 37 + mm/filemap.c| 35 --- 2 files changed, 41

[PATCH 01/15] block: provide plug based way of signaling forced no-wait semantics

2020-06-18 Thread Jens Axboe
Provide a way for the caller to specify that IO should be marked with REQ_NOWAIT to avoid blocking on allocation. Signed-off-by: Jens Axboe --- block/blk-core.c | 6 ++ include/linux/blkdev.h | 1 + 2 files changed, 7 insertions(+) diff --git a/block/blk-core.c b/block/blk-core.c

[PATCHSET v7 0/12] Add support for async buffered reads

2020-06-18 Thread Jens Axboe
On 6/18/20 8:43 AM, Jens Axboe wrote: > We technically support this already through io_uring, but it's > implemented with a thread backend to support cases where we would > block. This isn't ideal. Apparently I'm cursed when I send these out, corrected the subject line t

[PATCH 09/15] fs: add FMODE_BUF_RASYNC

2020-06-18 Thread Jens Axboe
If set, this indicates that the file system supports IOCB_WAITQ for buffered reads. Signed-off-by: Jens Axboe --- include/linux/fs.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/linux/fs.h b/include/linux/fs.h index 6ac919b40596..3f9de90e0266 100644 --- a/include/linux/fs.h

[PATCH 07/15] mm: add support for async page locking

2020-06-18 Thread Jens Axboe
we define struct wait_page_async as the interface between the caller and the core. Acked-by: Johannes Weiner Signed-off-by: Jens Axboe --- include/linux/fs.h | 7 ++- include/linux/pagemap.h | 17 mm/filemap.c| 45

[PATCH 12/15] btrfs: flag files as supporting buffered async reads

2020-06-18 Thread Jens Axboe
btrfs uses generic_file_read_iter(), which already supports this. Acked-by: Chris Mason Signed-off-by: Jens Axboe --- fs/btrfs/file.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c index 2c14312b05e8..234a418eb6da 100644 --- a/fs/btrfs

[PATCH 08/15] mm: support async buffered reads in generic_file_buffered_read()

2020-06-18 Thread Jens Axboe
IO completion, the caller must retry the operation. Acked-by: Johannes Weiner Signed-off-by: Jens Axboe --- mm/filemap.c | 38 +++--- 1 file changed, 27 insertions(+), 11 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index e8aaf43bee9f..a5b1fa8f7ce4 100644

[PATCH 13/15] ext4: flag as supporting buffered async reads

2020-06-18 Thread Jens Axboe
ext4 uses generic_file_read_iter(), which already supports this. Cc: Theodore Ts'o Signed-off-by: Jens Axboe --- fs/ext4/file.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/ext4/file.c b/fs/ext4/file.c index 2a01e31a032c..1e827410e9e1 100644 --- a/fs/ext4/file.c

[PATCH 03/15] io_uring: catch -EIO from buffered issue request failure

2020-06-18 Thread Jens Axboe
some overhead by not first trying, then retrying from async context. We can go straight to async punt instead. Signed-off-by: Jens Axboe --- fs/io_uring.c | 28 +++- 1 file changed, 23 insertions(+), 5 deletions(-) diff --git a/fs/io_uring.c b/fs/io_uring.c index

[PATCH 05/15] mm: allow read-ahead with IOCB_NOWAIT set

2020-06-18 Thread Jens Axboe
The read-ahead shouldn't block, so allow it to be done even if IOCB_NOWAIT is set in the kiocb. Acked-by: Johannes Weiner Signed-off-by: Jens Axboe --- mm/filemap.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index f0ae9a6308cb..3378d4fca883 100644 ---

[PATCH 14/15] mm: add kiocb_wait_page_queue_init() helper

2020-06-18 Thread Jens Axboe
Checks if the file supports it, and initializes the values that we need. Caller passes in 'data' pointer, if any, and the callback function to be used. Acked-by: Johannes Weiner Signed-off-by: Jens Axboe --- include/linux/pagemap.h | 21 + 1 file changed, 21

[PATCH 10/15] block: flag block devices as supporting IOCB_WAITQ

2020-06-18 Thread Jens Axboe
Signed-off-by: Jens Axboe --- fs/block_dev.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/block_dev.c b/fs/block_dev.c index 47860e589388..54720c90dad0 100644 --- a/fs/block_dev.c +++ b/fs/block_dev.c @@ -1850,7 +1850,7 @@ static int blkdev_open(struct inode * inode

[PATCH 11/15] xfs: flag files as supporting buffered async reads

2020-06-18 Thread Jens Axboe
XFS uses generic_file_read_iter(), which already supports this. Acked-by: Darrick J. Wong Signed-off-by: Jens Axboe --- fs/xfs/xfs_file.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/xfs/xfs_file.c b/fs/xfs/xfs_file.c index 00db81eac80d..fdbff4860d61 100644 --- a/fs

[PATCH 15/15] io_uring: support true async buffered reads, if file provides it

2020-06-18 Thread Jens Axboe
ow we handle poll based retry. From the unlock callback, we simply queue the retry to a task_work based handler. Signed-off-by: Jens Axboe --- fs/io_uring.c | 145 +++--- 1 file changed, 137 insertions(+), 8 deletions(-) diff --git a/fs/io_uring.c b/fs/io_ur

[PATCH 02/15] io_uring: always plug for any number of IOs

2020-06-18 Thread Jens Axboe
Currently we only plug if we're doing more than two request. We're going to be relying on always having the plug there to pass down information, so plug unconditionally. Signed-off-by: Jens Axboe --- fs/io_uring.c | 15 +-- 1 file changed, 5 insertions(+), 10 deletion

[no subject]

2020-06-18 Thread Jens Axboe
e_page_match() in both callers Changes since v1: - Fix an issue with inline page locking - Fix a potential race with __wait_on_page_locked_async() - Fix a hang related to not setting page_match, thus missing a wakeup -- Jens Axboe

Re: linux-next: Tree for Jun 18 (fs/io_uring)

2020-06-18 Thread Jens Axboe
req)) { >^~~ >atomic_read_acquire > ../fs/io_uring.c: In function 'io_sq_thread': > ../fs/io_uring.c:6268:4: error: implicit declaration of function > 'io_sq_thread_drop_mm'; did you mean 'io_sq_thread'? > [-Werror=

Re: [RFC] io_uring: add restrictions to support untrusted applications and guests

2020-06-16 Thread Jens Axboe
On 6/16/20 3:12 AM, Stefano Garzarella wrote: > On Mon, Jun 15, 2020 at 11:00:25AM -0600, Jens Axboe wrote: >> On 6/15/20 7:33 AM, Stefano Garzarella wrote: >>> On Mon, Jun 15, 2020 at 11:04:06AM +0200, Jann Horn wrote: >>>> +Kees, Christian, Sargun, Aleksa, kerne

Re: [PATCH] trace/events/block.h: drop kernel-doc for dropped function parameter

2020-06-15 Thread Jens Axboe
on in 'trace_block_bio_complete' Applied, thanks. -- Jens Axboe

Re: [RFC] io_uring: add restrictions to support untrusted applications and guests

2020-06-15 Thread Jens Axboe
of things to register in the 'params', passed > to io_uring_setup(), should be feasible, if Jens agree :-) I wonder how best to deal with this, in terms of ring visibility vs registering restrictions. We could potentially start the ring in a disabled mode, if asked to. It'd still be visible in terms of having the fd installed, but it'd just error requests. That'd leave you with time to do the various setup routines needed before then flagging it as enabled. My only worry on that would be adding overhead for doing that. It'd be cheap enough to check for IORING_SETUP_DISABLED in ctx->flags in io_uring_enter(), and return -EBADFD or something if that's the case. That doesn't cover the SQPOLL case though, but maybe we just don't start the sq thread if IORING_SETUP_DISABLED is set. We'd need a way to clear IORING_SETUP_DISABLED through io_uring_register(). When clearing, that could then start the sq thread as well, when SQPOLL is set. -- Jens Axboe

Re: [PATCH 0/2] don't use pid for request cancellation

2020-06-15 Thread Jens Axboe
ght of not grabbing a ref to the task for the cancel case where we don't need to dereference it. -- Jens Axboe

Re: [PATCH v2 0/4][RESEND] cancel all reqs of an exiting task

2020-06-15 Thread Jens Axboe
matched request. The pathset is mainly > about fixing that. [1,2] are preps, [3/4] is the fix. > > The [4/4] tries to improve the worst case for io_uring_cancel_files(), > that's when they are a lot of inflights with ->files. Instead of doing > {kill(); wait();} one by one, it ca

Re: [PATCH 1/1] io_uring: fix lazy work init

2020-06-15 Thread Jens Axboe
922124] kthread+0x12c/0x170 > [ 140.922125] ? io_worker_handle_work+0x480/0x480 > [ 140.922126] ? kthread_park+0x90/0x90 > [ 140.922127] ret_from_fork+0x22/0x30 Applied, thanks. -- Jens Axboe

Re: [PATCH] blk-mq: Remove redundant 'return' statement

2020-06-15 Thread Jens Axboe
On 6/15/20 3:12 AM, Baolin Wang wrote: > The blk_mq_all_tag_iter() is a void function, thus remove > the redundant 'return' statement in this function. Applied, thanks. -- Jens Axboe

Re: [PATCH 6/6] smp: Cleanup smp_call_function*()

2020-06-15 Thread Jens Axboe
On 6/15/20 6:57 AM, Peter Zijlstra wrote: > Get rid of the __call_single_node union and cleanup the API a little > to avoid external code relying on the structure layout as much. core and block bits look good to me. -- Jens Axboe

Re: [RFC] io_uring: add restrictions to support untrusted applications and guests

2020-06-14 Thread Jens Axboe
R_RESTRICTIONS, > restrictions, ARRAY_SIZE(restrictions)); > > Limiting access to file descriptors > --- > The fixed files mechanism can be used to limit access to a set of file > descriptors: > > struct io_uring_restriction restrictions[] = { > { > .opcode = IORING_RESTRICTION_FIXED_FILES_ONLY, > }, > ... > }; > > io_uring_register(ringfd, IORING_REGISTER_RESTRICTIONS, > restrictions, ARRAY_SIZE(restrictions)); > > Only requests with the sqe->flags IOSQE_FIXED_FILE bit set will be allowed. I don't think this sounds unreasonable, but I'd really like to see a prototype hacked up before rendering any further opinions on it :-) -- Jens Axboe

[GIT PULL] io_uring fixes for 5.8-rc1

2020-06-11 Thread Jens Axboe
- io_uring-5.8-2020-06-11 Bijan Mottahedeh (1): io_uring: validate the full range of provided buffers for access Denis Efremov (1): io_uring: use kvfree() in io_sqe_buffer_register() Jens Axboe (3): io_uring: disallow close of ring itself io_u

Re: [PATCH] umem: remove redundant initialization of variable ret

2020-06-11 Thread Jens Axboe
On 6/11/20 8:35 AM, Colin King wrote: > From: Colin Ian King > > The variable ret is being initialized with a value that is never read > and it is being updated later with a new value. The initialization is > redundant and can be removed. Applied, thanks. -- Jens Axboe

Re: [PATCH] pktcdvd: remove redundant initialization of variable ret

2020-06-11 Thread Jens Axboe
On 6/11/20 8:30 AM, Colin King wrote: > From: Colin Ian King > > The variable ret is being initialized with a value that is never read > and it is being updated later with a new value. The initialization is > redundant and can be removed. Applied, thanks. -- Jens Axboe

Re: blk-softirq vs smp_call_function_single_async()

2020-06-08 Thread Jens Axboe
called from blk_mq_end_request which >> is called from the drivers ->complete_rq method, which is called >> from the block softirq. >> >> What is the method to guaranteed CSD completion? > > There isn't one, it was meant to be used with static allocations. > > Frederic proposed changing all these to irq_work, and I think I'll go do > that. First dinner though. That sounds good to me, and will also shrink struct request a bit, which is always nice. -- Jens Axboe

Re: [PATCH 0/4] remove work.func

2020-06-08 Thread Jens Axboe
n't arm a timeout through work.func > io_wq: add per-wq work handler instead of per work Thanks, this looks good and also nicely enable us to build on it to eliminate that extra overhead. I have applied it for 5.8. -- Jens Axboe

Re: [PATCH 0/4] cancel all reqs of an exiting task

2020-06-07 Thread Jens Axboe
e.kernel.org/io-uring/87a71jjbzr@x220.int.ebiederm.org/T/#u Might as well do it at the same time, imho, since the cancel-by-task is being reworked anyway. -- Jens Axboe

Re: [PATCHSET v5 0/12] Add support for async buffered reads

2020-06-06 Thread Jens Axboe
On 6/5/20 6:33 PM, Sedat Dilek wrote: > On Fri, Jun 5, 2020 at 11:24 PM Jens Axboe wrote: >> >> On 6/5/20 3:13 PM, Jens Axboe wrote: >>> On 6/5/20 2:53 PM, Jens Axboe wrote: >>>> On 6/5/20 2:36 PM, Andres Freund wrote: >>>>> Hi, >>&g

Re: [PATCHSET v5 0/12] Add support for async buffered reads

2020-06-05 Thread Jens Axboe
On 6/5/20 4:54 PM, Andres Freund wrote: > Hi, > > On 2020-06-05 16:49:24 -0600, Jens Axboe wrote: >> Yes that's expected, if we have to fallback to ->readpage(), then it'll >> go to a worker. read-ahead is what drives the async nature of it, as we >> iss

Re: [PATCHSET v5 0/12] Add support for async buffered reads

2020-06-05 Thread Jens Axboe
On 6/5/20 4:36 PM, Andres Freund wrote: > Hi, > > On 2020-06-05 15:30:44 -0700, Andres Freund wrote: >> On 2020-06-05 15:21:34 -0600, Jens Axboe wrote: >>>>> I can reproduce this, and I see what it is. I'll send out a patch soonish. >>>> >>&

Re: [PATCHSET v5 0/12] Add support for async buffered reads

2020-06-05 Thread Jens Axboe
On 6/5/20 3:13 PM, Jens Axboe wrote: > On 6/5/20 2:53 PM, Jens Axboe wrote: >> On 6/5/20 2:36 PM, Andres Freund wrote: >>> Hi, >>> >>> On 2020-06-05 13:20:28 -0700, Andres Freund wrote: >>>> I'll go and try to figure out why I don't see an

Re: [PATCHSET v5 0/12] Add support for async buffered reads

2020-06-05 Thread Jens Axboe
On 6/5/20 2:53 PM, Jens Axboe wrote: > On 6/5/20 2:36 PM, Andres Freund wrote: >> Hi, >> >> On 2020-06-05 13:20:28 -0700, Andres Freund wrote: >>> I'll go and try to figure out why I don't see an oops... >> >> Err, that probably was a typo on

Re: [PATCHSET v5 0/12] Add support for async buffered reads

2020-06-05 Thread Jens Axboe
? ip_reply_glue_bits+0x40/0x40 > [ 128.239951] ? _cond_resched+0x19/0x30 > [ 128.239951] ? unmap_page_range+0x678/0xa60 > [ 128.239951] ? __sys_sendto+0x108/0x190 > [ 128.239951] __sys_sendto+0x108/0x190 > [ 128.239952] ? __fput+0x1a5/0x240 > [ 128.239952] ? _cond_resched+0x19/0x30 > [ 128.239952] ? task_work_run+0x67/0x90 > [ 128.239952] __x64_sys_sendto+0x25/0x30 > [ 128.239953] do_syscall_64+0x48/0x130 > [ 128.239953] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > [ 128.239953] RIP: 0033:0x7f0b97a7826c > [ 128.239954] Code: c0 ff ff ff ff eb b9 0f 1f 80 00 00 00 00 41 89 ca 64 8b > 04 25 18 00 00 00 85 c0 75 19 45 31 c9 45 31 c0 b8 2c 00 00 00 0f 05 <48> 3d > 00 f0 ff ff 77 64 c3 0f 1f 00 55 48 83 ec 20 48 89 54 24 10 > [ 128.239954] RSP: 002b:7ffe80ffd7b8 EFLAGS: 0246 ORIG_RAX: > 002c > [ 128.239955] RAX: ffda RBX: 03a8 RCX: > 7f0b97a7826c > [ 128.239955] RDX: 03a8 RSI: 7ffe80ffd800 RDI: > 0009 > [ 128.239955] RBP: 7ffe80ffd800 R08: R09: > > [ 128.239956] R10: R11: 0246 R12: > 7ffe80ffd800 > [ 128.239956] R13: 7ffe80ffd800 R14: 000e R15: > 55fb76b37e58 I can reproduce this, and I see what it is. I'll send out a patch soonish. -- Jens Axboe

Re: [PATCHSET v5 0/12] Add support for async buffered reads

2020-06-05 Thread Jens Axboe
On 6/5/20 2:20 PM, Andres Freund wrote: > Hi, > > On 2020-06-05 08:42:28 -0600, Jens Axboe wrote: >> Can you try with async-buffered.7? I've rebased it on a new mechanism, >> and doing something like what you describe above I haven't been able >> to trigger

Re: [PATCHSET v5 0/12] Add support for async buffered reads

2020-06-05 Thread Jens Axboe
On 6/3/20 7:04 PM, Jens Axboe wrote: > On 6/3/20 6:59 PM, Andres Freund wrote: >> Hi, >> >> I was trying to benchmark the benefits of this for the io_uring using >> postgres I am working on. The initial results where quite promising >> (reducing cpu usage signi

Re: block: Fix use-after-free in blkdev_get()

2020-06-05 Thread Jens Axboe
rovements" aren't usually improvements, and it causes more confusion for the submitter. Maintainers generally do change commit messages to improve them, if needed. No reply is needed to this e-mail. -- Jens Axboe

Re: [PATCH] io_uring: use kvfree() in io_sqe_buffer_register()

2020-06-05 Thread Jens Axboe
On 6/5/20 3:32 AM, Denis Efremov wrote: > Use kvfree() to free the pages and vmas, since they are allocated by > kvmalloc_array() in a loop. Applied, thanks. -- Jens Axboe

Re: [PATCH v2 5/6] block: nr_sects_write(): Disable preemption on seqcount write

2020-06-04 Thread Jens Axboe
r's write side > critical section. Otherwise, the read side can preempt the write side > section and spin for the entire scheduler tick. If the reader belongs to > a real-time scheduling class, it can spin forever and the kernel will > livelock. Applied, thanks. -- Jens Axboe

Re: [PATCH] sata_rcar: handle pm_runtime_get_sync failure cases

2020-06-04 Thread Jens Axboe
On 6/4/20 9:06 PM, Navid Emamdoost wrote: > Calling pm_runtime_get_sync increments the counter even in case of > failure, causing incorrect ref count. Call pm_runtime_put if > pm_runtime_get_sync fails. Applied, thanks. -- Jens Axboe

Re: [PATCH] loop: Fix wrong masking of status flags

2020-06-04 Thread Jens Axboe
01.c:68: PASS: /sys/block/loop0/loop/autoclear = 1 > ioctl_loop01.c:77: PASS: access /dev/loop0p1 succeeds > ioctl_loop01.c:83: PASS: access /sys/block/loop0/loop0p1 succeeds > > Summary: > passed 8 > failed 0 > skipped 0 > warnings 0 Applied, thanks. -- Jens Axboe

Re: [PATCH v3 0/4] forbid fix {SQ,IO}POLL

2020-06-04 Thread Jens Axboe
On 6/4/20 2:12 PM, Pavel Begunkov wrote: > On 04/06/2020 22:52, Jens Axboe wrote: >> On 6/4/20 1:22 PM, Pavel Begunkov wrote: >>> On 04/06/2020 20:06, Jens Axboe wrote: >>>> On 6/3/20 12:51 PM, Jens Axboe wrote: >>>>> On 6/3/20 9:03 AM, Pavel Beg

Re: [PATCH v3 0/4] forbid fix {SQ,IO}POLL

2020-06-04 Thread Jens Axboe
On 6/4/20 1:22 PM, Pavel Begunkov wrote: > On 04/06/2020 20:06, Jens Axboe wrote: >> On 6/3/20 12:51 PM, Jens Axboe wrote: >>> On 6/3/20 9:03 AM, Pavel Begunkov wrote: >>>> The first one adds checks {SQPOLL,IOPOLL}. IOPOLL check can be >>>> moved in th

Re: [PATCH v3 0/4] forbid fix {SQ,IO}POLL

2020-06-04 Thread Jens Axboe
On 6/3/20 12:51 PM, Jens Axboe wrote: > On 6/3/20 9:03 AM, Pavel Begunkov wrote: >> The first one adds checks {SQPOLL,IOPOLL}. IOPOLL check can be >> moved in the common path later, or rethinked entirely, e.g. >> not io_iopoll_req_issued()'ed for unsupported opcode

Re: kernel/trace/blktrace.c:347:12: sparse: sparse: incorrect type in assignment (different address spaces)

2020-06-03 Thread Jens Axboe
s so that > we can stop these emails ? Which patch from Jan? I saw one, and it had issues. Then there's a second one, which is ordered behind a series that's not in my tree and wasn't queued for 5.8. And finally, there's your series, which seemed to be a subset of Jan's

Re: [PATCHSET v5 0/12] Add support for async buffered reads

2020-06-03 Thread Jens Axboe
hout fear... In one instance I did see some minor corruption on > another device & fs (ext4 on dm-crypt on nvme). It's all backed up, > but... I have a known issue with request starvation, wonder if that could be it. I'm going to rebase the branch on top of the aops->readahead() changes shortly, and fix that issue. Hopefully that's what's plaguing your run here, but if not, I'll hunt that one down. Thanks for testing! -- Jens Axboe

Re: [PATCH v3 0/4] forbid fix {SQ,IO}POLL

2020-06-03 Thread Jens Axboe
eduplicate io_openat{,2}_prep() > io_uring: move send/recv IOPOLL check into prep > > fs/io_uring.c | 94 ++- > 1 file changed, 48 insertions(+), 46 deletions(-) Thanks, applied. -- Jens Axboe

Re: [GIT PULL] io_uring updates for 5.8-rc1

2020-06-02 Thread Jens Axboe
On 6/2/20 5:03 PM, Linus Torvalds wrote: > On Mon, Jun 1, 2020 at 10:55 AM Jens Axboe wrote: >> >> git://git.kernel.dk/linux-block.git for-5.8/io_uring-2020-06-01 > > I'm not sure why pr-tracker-bot didn't like your io_uring pull request. > > It replied

Re: [PATCH] block/bio-integrity: don't free 'buf' if bio_integrity_add_page() failed

2020-06-02 Thread Jens Axboe
#x27; and > 'bio->bi_integrity' was set previousy in bio_integrity_alloc(). Applied, thanks. -- Jens Axboe

Re: linux-next: manual merge of the block tree with the rdma tree

2020-06-02 Thread Jens Axboe
On 6/2/20 1:09 PM, Jason Gunthorpe wrote: > On Tue, Jun 02, 2020 at 01:02:55PM -0600, Jens Axboe wrote: >> On 6/2/20 1:01 PM, Jason Gunthorpe wrote: >>> On Tue, Jun 02, 2020 at 11:37:26AM +0300, Max Gurtovoy wrote: >>>> >>>> On 6/2/2020 5:56 AM, Stephen

Re: linux-next: manual merge of the block tree with the rdma tree

2020-06-02 Thread Jens Axboe
s. I am going to drop the nvme part of this from RDMA. > > Normally I don't like applying partial series, but due to this tree > split, you can send the rebased nvme part through the nvme/block tree > at rc1 in two weeks.. Was going to comment that this is probably how it should have been done to begin with. If we have multiple conflicts like that between two trees, someone is doing something wrong... -- Jens Axboe

[GIT PULL] io_uring updates for 5.8-rc1

2020-06-01 Thread Jens Axboe
uring-2020-06-01 Bijan Mottahedeh (4): io_uring: add io_statx structure statx: allow system call to be invoked from io_uring io_uring: call statx directly statx: hide interfaces no longer used by io_uring Jens

Re: [PATCH 04/12] mm: add support for async page locking

2020-06-01 Thread Jens Axboe
On 6/1/20 8:26 AM, Matthew Wilcox wrote: > On Tue, May 26, 2020 at 01:51:15PM -0600, Jens Axboe wrote: >> +static int __wait_on_page_locked_async(struct page *page, >> + struct wait_page_queue *wait, bool set) >> +{ >> +

Re: [PATCHSET v5 0/12] Add support for async buffered reads

2020-06-01 Thread Jens Axboe
On 6/1/20 8:43 AM, Sedat Dilek wrote: > On Mon, Jun 1, 2020 at 4:35 PM Jens Axboe wrote: >> >> On 6/1/20 8:14 AM, Jens Axboe wrote: >>> On 6/1/20 8:13 AM, Sedat Dilek wrote: >>>> On Mon, Jun 1, 2020 at 4:04 PM Jens Axboe wrote: >>>>> >>

Re: [PATCHSET v5 0/12] Add support for async buffered reads

2020-06-01 Thread Jens Axboe
On 6/1/20 8:14 AM, Jens Axboe wrote: > On 6/1/20 8:13 AM, Sedat Dilek wrote: >> On Mon, Jun 1, 2020 at 4:04 PM Jens Axboe wrote: >>> >>> On 6/1/20 7:35 AM, Sedat Dilek wrote: >>>> Hi Jens, >>>> >>>> with Linux v5.7

Re: [PATCHSET v5 0/12] Add support for async buffered reads

2020-06-01 Thread Jens Axboe
On 6/1/20 8:13 AM, Sedat Dilek wrote: > On Mon, Jun 1, 2020 at 4:04 PM Jens Axboe wrote: >> >> On 6/1/20 7:35 AM, Sedat Dilek wrote: >>> Hi Jens, >>> >>> with Linux v5.7 final I switched to linux-block.git/for-next and reverted... >>> >>>

Re: [PATCHSET v5 0/12] Add support for async buffered reads

2020-06-01 Thread Jens Axboe
plied instead? Or pull my async-readahead branch from the same location. -- Jens Axboe >From 297f4d794780f7f55180fb0de6692bd1a1d81c58 Mon Sep 17 00:00:00 2001 From: Jens Axboe Date: Sun, 31 May 2020 21:31:08 -0600 Subject: [PATCH 6/6] Revert "block: read-ahead submission should imply

Re: [PATCHSET v5 0/12] Add support for async buffered reads

2020-05-30 Thread Jens Axboe
1ULL << __REQ_RAHEAD) #define REQ_BACKGROUND (1ULL << __REQ_BACKGROUND) #define REQ_NOWAIT (1ULL << __REQ_NOWAIT) #define REQ_CGROUP_PUNT(1ULL << __REQ_CGROUP_PUNT) -- Jens Axboe

Re: [PATCH v3 0/2] CQ-seq only based timeouts

2020-05-30 Thread Jens Axboe
- > 1 file changed, 27 insertions(+), 70 deletions(-) Applied, thanks. -- Jens Axboe

Re: [PATCH v2] io_uring: fix overflowed reqs cancellation

2020-05-30 Thread Jens Axboe
ref, don't do that. Also, don't need to do io_wq_cancel_work() > for overflowed reqs, they will be let go shortly anyway. Applied, thanks. -- Jens Axboe

Re: [PATCH v2] blkdev: Replace blksize_bits() with ilog2()

2020-05-29 Thread Jens Axboe
a two-parter: 1) Use the inode blkbits where appropriate 2) Then do this change I'm leaning towards just doing it in blksize_bits() instead of updating every caller, unless there aren't that many left once we've gone through patch 1. -- Jens Axboe

Re: [PATCH for-next 1/1] null_blk: force complete for timeout request

2020-05-29 Thread Jens Axboe
ev/nullb0 bs=512 count=1 oflag=direct > > With this patch, the timeout handler is able to complete the IO. Applied, thanks. -- Jens Axboe

Re: [PATCH v2] blkdev: Replace blksize_bits() with ilog2()

2020-05-29 Thread Jens Axboe
message is handwavy at best. -- Jens Axboe

Re: [PATCH -next] block: add another struct gendisk to

2020-05-28 Thread Jens Axboe
xed already: https://git.kernel.dk/cgit/linux-block/commit/?h=for-5.8/block&id=dc35ada4251f183137ee3a524543c9329d7a4fa2 -- Jens Axboe

Re: [PATCH 09/12] xfs: flag files as supporting buffered async reads

2020-05-28 Thread Jens Axboe
On 5/28/20 11:53 AM, Darrick J. Wong wrote: > On Tue, May 26, 2020 at 01:51:20PM -0600, Jens Axboe wrote: >> XFS uses generic_file_read_iter(), which already supports this. >> >> Signed-off-by: Jens Axboe > > Er... I guess that looks ok? Assuming you've done en

Re: [PATCHSET v5 0/12] Add support for async buffered reads

2020-05-28 Thread Jens Axboe
On 5/28/20 11:12 AM, Sedat Dilek wrote: > On Thu, May 28, 2020 at 7:06 PM Jens Axboe wrote: >> >> On 5/28/20 11:02 AM, Sedat Dilek wrote: >>> On Tue, May 26, 2020 at 10:59 PM Jens Axboe wrote: >>>> >>>> We technically support this already through i

Re: [PATCHSET v5 0/12] Add support for async buffered reads

2020-05-28 Thread Jens Axboe
On 5/28/20 11:02 AM, Sedat Dilek wrote: > On Tue, May 26, 2020 at 10:59 PM Jens Axboe wrote: >> >> We technically support this already through io_uring, but it's >> implemented with a thread backend to support cases where we would >> block. This isn't ideal

Re: [PATCH][next] block: blk-crypto-fallback: remove redundant initialization of variable err

2020-05-27 Thread Jens Axboe
On 5/26/20 4:49 PM, Colin King wrote: > From: Colin Ian King > > The variable err is being initialized with a value that is never read > and it is being updated later with a new value. The initialization is > redundant and can be removed. Applied, thanks. -- Jens Axboe

Re: block I/O accounting improvements v2

2020-05-27 Thread Jens Axboe
us > performanc improvements. Most of this comes from a series Konstantin > sent a few weeks ago, rebased on changes that landed in your tree since > and my change to always use the percpu version of the disk stats. Applied, thanks. -- Jens Axboe

Re: [PATCH 04/12] mm: add support for async page locking

2020-05-26 Thread Jens Axboe
On 5/26/20 3:59 PM, Johannes Weiner wrote: > On Tue, May 26, 2020 at 01:51:15PM -0600, Jens Axboe wrote: >> Normally waiting for a page to become unlocked, or locking the page, >> requires waiting for IO to complete. Add support for lock_page_async() >> and wait_on_page_lock

Re: [PATCH] block/floppy: fix contended case in floppy_queue_rq()

2020-05-26 Thread Jens Axboe
ted device. > > Fix that by not queuing the request in the contended case, and return > BLK_STS_RESOURCE instead, so that blk core handles the request > rescheduling and let it pass properly non-contended later. Applied, thanks. -- Jens Axboe

[PATCHSET v5 0/12] Add support for async buffered reads

2020-05-26 Thread Jens Axboe
ct, just use wait_page_async - Add another prep handler, adding wake_page_match() - Use wake_page_match() in both callers Changes since v1: - Fix an issue with inline page locking - Fix a potential race with __wait_on_page_locked_async() - Fix a hang related to not setting page_match, thus missing a wakeup -- Jens Axboe

[PATCH 04/12] mm: add support for async page locking

2020-05-26 Thread Jens Axboe
we define struct wait_page_async as the interface between the caller and the core. Signed-off-by: Jens Axboe --- include/linux/fs.h | 7 ++- include/linux/pagemap.h | 9 + mm/filemap.c| 41 + 3 files changed, 56 insertions

[PATCH 02/12] mm: allow read-ahead with IOCB_NOWAIT set

2020-05-26 Thread Jens Axboe
The read-ahead shouldn't block, so allow it to be done even if IOCB_NOWAIT is set in the kiocb. Signed-off-by: Jens Axboe --- mm/filemap.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 23a051a7ef0f..80747f1377d5 100644 --- a/mm/filemap.c +++

<    5   6   7   8   9   10   11   12   13   14   >