Re: confusion about bio_split

On Mon, Jan 15 2018, Wang Sheng-Hui wrote: > Dear, > > Sorry to trouble you. > > I'm confused by the following code in bio.c/bio_split: > - > struct bio *bio_split(struct bio *bio, int sectors, > ... > if (bio_flagged(bio, BIO_TRACE_COMPLETION)) >

Re: [PATCH v5 1/2] Return bytes transferred for partial direct I/O

On Mon, Jan 22, 2018 at 08:28:54PM -0700, Jens Axboe wrote: > On 1/22/18 8:18 PM, Goldwyn Rodrigues wrote: > >> that their application was "already broken". I'd hate for a kernel > >> upgrade to break them. > >> > >> I do wish we could make the change, and maybe we can. But it probably > >> needs

Re: [PATCH v5 1/2] Return bytes transferred for partial direct I/O

On 1/22/18 8:18 PM, Goldwyn Rodrigues wrote: >> that their application was "already broken". I'd hate for a kernel >> upgrade to break them. >> >> I do wish we could make the change, and maybe we can. But it probably >> needs some safe guard proc entry to toggle the behavior, something we >> can

Re: [PATCH v5 1/2] Return bytes transferred for partial direct I/O

On 01/22/2018 01:13 PM, Jens Axboe wrote: > On 1/22/18 12:10 PM, Andreas Dilger wrote: >> >>> On Jan 20, 2018, at 8:07 PM, Jens Axboe wrote: >>> >>> On 1/20/18 7:23 PM, Goldwyn Rodrigues wrote: On 01/20/2018 08:11 PM, Andi Kleen wrote: > > It's likely

[PATCH RESEND] blk-throttle: dispatch more sync writes in block throttle layer

Cgroup writeback is supported since v4.2. But there exists a problem in the following case. A cgroup may send both buffer and direct/sync IOs. The foreground thread will be stalled when periodic writeback IOs is flushed because the service queue in block throttle layer already has a plenty of

Re: [PATCH 4/5] blk-mq: introduce blk_get_request_notify

On Mon, Jan 22, 2018 at 05:13:03PM +, Bart Van Assche wrote: > On Mon, 2018-01-22 at 11:35 +0800, Ming Lei wrote: > > DM-MPATH need to allocate request from underlying queue, but when the > > allocation fails, there is no way to make underlying queue's RESTART > > to restart DM's queue. > > >

Re: Hard LOCKUP on 4.15-rc9 + 'blkmq/for-next' branch

On 1/22/18 6:05 PM, David Zarzycki wrote: > > >> On Jan 22, 2018, at 18:34, Jens Axboe wrote: >> >> On 1/22/18 4:31 PM, David Zarzycki wrote: >>> Hello, >>> >>> I previously reported a hang when building LLVM+clang on a block >>> multi-queue device (NVMe _or_ loopback onto

Re: Hard LOCKUP on 4.15-rc9 + 'blkmq/for-next' branch

> On Jan 22, 2018, at 18:34, Jens Axboe wrote: > > On 1/22/18 4:31 PM, David Zarzycki wrote: >> Hello, >> >> I previously reported a hang when building LLVM+clang on a block multi-queue >> device (NVMe _or_ loopback onto tmpfs with the ’none’ scheduler). >> >> I’ve since

Re: [PATCH 1/5] blk-mq: introduce BLK_STS_DEV_RESOURCE

On Mon, Jan 22, 2018 at 04:49:54PM +, Bart Van Assche wrote: > On Mon, 2018-01-22 at 11:35 +0800, Ming Lei wrote: > > @@ -1280,10 +1282,18 @@ bool blk_mq_dispatch_rq_list(struct request_queue > > *q, struct list_head *list, > > * - Some but not all block drivers stop a queue

Re: [PATCH 1/3] nvme/pci: Start request after doorbell ring

On Wed, Jan 03, 2018 at 01:21:05PM -0700, Keith Busch wrote: > > I've removed the submission side poll in a local build, and amazingly I > am observing a not insignificant increase in latency without it on some > workloads with certain hardware. I will have to delay recommending/posting > removal

[RFC PATCH] BUG at block/blk-ioc.c:146 during bcache search_free

Hello all, In 4.14.13 we have hit the bug with the full backtrace noted below. This looks like a double free problem somewhere related to cached_dev_write_complete calling search_free. It looks like the compiler inlined cached_dev_bio_complete so the stack trace is not complete. This is the

Re: Hard LOCKUP on 4.15-rc9 + 'blkmq/for-next' branch

On 1/22/18 4:31 PM, David Zarzycki wrote: > Hello, > > I previously reported a hang when building LLVM+clang on a block multi-queue > device (NVMe _or_ loopback onto tmpfs with the ’none’ scheduler). > > I’ve since updated the kernel to 4.15-rc9, merged the ‘blkmq/for-next’ > branch, disabled

Hard LOCKUP on 4.15-rc9 + 'blkmq/for-next' branch

Hello, I previously reported a hang when building LLVM+clang on a block multi-queue device (NVMe _or_ loopback onto tmpfs with the ’none’ scheduler). I’ve since updated the kernel to 4.15-rc9, merged the ‘blkmq/for-next’ branch, disabled nohz_full parameter (used for testing), and tried again.

Re: [PATCH v5 1/2] Return bytes transferred for partial direct I/O

On 1/22/18 4:24 PM, Bart Van Assche wrote: > On Mon, 2018-01-22 at 16:14 -0700, Jens Axboe wrote: >> On 1/22/18 3:25 PM, Elliott, Robert (Persistent Memory) wrote: >>> fio engines/sg.c fio_sgio_rw_doio() has that pattern: >>> >>> ret = write(f->fd, hdr, sizeof(*hdr)); >>> if (ret < 0) >>>

Re: [PATCH v5 1/2] Return bytes transferred for partial direct I/O

On Mon, 2018-01-22 at 16:14 -0700, Jens Axboe wrote: > On 1/22/18 3:25 PM, Elliott, Robert (Persistent Memory) wrote: > > fio engines/sg.c fio_sgio_rw_doio() has that pattern: > > > > ret = write(f->fd, hdr, sizeof(*hdr)); > > if (ret < 0) > > return ret; > > ... > >

Re: [PATCH v5 1/2] Return bytes transferred for partial direct I/O

On 1/22/18 3:25 PM, Elliott, Robert (Persistent Memory) wrote: > > >> On 1/20/18 7:23 PM, Goldwyn Rodrigues wrote: >>> On 01/20/2018 08:11 PM, Andi Kleen wrote: Goldwyn Rodrigues writes: > From: Goldwyn Rodrigues > In case direct I/O encounters

RE: [PATCH v5 1/2] Return bytes transferred for partial direct I/O

> On 1/20/18 7:23 PM, Goldwyn Rodrigues wrote: > > On 01/20/2018 08:11 PM, Andi Kleen wrote: > >> Goldwyn Rodrigues writes: > >>> From: Goldwyn Rodrigues > >>> In case direct I/O encounters an error midway, it returns the error. > >>> Instead it should be

Re: [PATCH TRIVIAL] bsg: use pr_debug instead of hand crafted macros

On Mon, 2018-01-22 at 20:42 +, Bart Van Assche wrote: > On Mon, 2018-01-22 at 12:23 -0800, Joe Perches wrote: > > On Mon, 2018-01-22 at 08:53 +0100, Johannes Thumshirn wrote: > > > Use pr_debug instead of hand crafted macros. This way it is not needed to > > > re-compile the kernel to enable

Re: [PATCH TRIVIAL] bsg: use pr_debug instead of hand crafted macros

On Mon, 2018-01-22 at 12:23 -0800, Joe Perches wrote: > On Mon, 2018-01-22 at 08:53 +0100, Johannes Thumshirn wrote: > > Use pr_debug instead of hand crafted macros. This way it is not needed to > > re-compile the kernel to enable bsg debug outputs and it's possible to > > selectively enable

Re: [PATCH TRIVIAL] bsg: use pr_debug instead of hand crafted macros

On Mon, 2018-01-22 at 08:53 +0100, Johannes Thumshirn wrote: > Use pr_debug instead of hand crafted macros. This way it is not needed to > re-compile the kernel to enable bsg debug outputs and it's possible to > selectively enable specific prints. Perhaps better to use #define bsg_dbg(bd, fmt,

Re: [dm-devel] [PATCH 0/2] dm: add new loop and ram targets

On Wed, Jan 17, 2018 at 08:33:59PM +0100, Heinz Mauelshagen wrote: > Enhancing IO performance compared to the kernels existing > loop driver thus better suiting respective requirements in > test setups, this patch series adds new "loop" and "ram" targets. For a fair comparism vs loop you'll need

Re: [PATCH v5 1/2] Return bytes transferred for partial direct I/O

On 1/22/18 12:10 PM, Andreas Dilger wrote: > >> On Jan 20, 2018, at 8:07 PM, Jens Axboe wrote: >> >> On 1/20/18 7:23 PM, Goldwyn Rodrigues wrote: >>> >>> >>> On 01/20/2018 08:11 PM, Andi Kleen wrote: It's likely there's a lot of code in user space that does

Re: [PATCH v5 1/2] Return bytes transferred for partial direct I/O

> On Jan 20, 2018, at 8:07 PM, Jens Axboe wrote: > > On 1/20/18 7:23 PM, Goldwyn Rodrigues wrote: >> >> >> On 01/20/2018 08:11 PM, Andi Kleen wrote: >>> >>> It's likely there's a lot of code in user space that does >>> >>> if (write(..., N) < 0) handle error >>> >>>

Re: [PATCH v5 1/2] Return bytes transferred for partial direct I/O

> The only way I can think is that a DIO write should check early enough > that the write(N) will complete with N bytes without an error. Is it > possible to completely guarantee that? Probably not. > > Leaving it as it is incorrect as quoted in the artificial test case. You > should not be

Re: blk-mq hangs easily with LLVM+clang test suite

On 1/21/18 10:39 AM, David Zarzycki wrote: > Hi Bart, > > I can do [1] and [2] but I won’t be able to provide the command output > because the hang is almost total. No new commands can run because the > scheduler is hung. For example, see this backtrace: > > http://znu.io/IMG_0362.jpg > > Is

Re: blk-mq hangs easily with LLVM+clang test suite

On Sun, 2018-01-21 at 12:39 -0500, David Zarzycki wrote: > Hi Bart, > > I can do [1] and [2] but I won’t be able to provide the command output > because the hang is almost total. No new commands can run because the > scheduler is hung. For example, see this backtrace: > >

Re: [PATCH 4/5] blk-mq: introduce blk_get_request_notify

On Mon, 2018-01-22 at 11:35 +0800, Ming Lei wrote: > DM-MPATH need to allocate request from underlying queue, but when the > allocation fails, there is no way to make underlying queue's RESTART > to restart DM's queue. > > This patch introduces blk_get_request_notify() for this purpose, and >

Re: [PATCH 1/5] blk-mq: introduce BLK_STS_DEV_RESOURCE

On Mon, 2018-01-22 at 11:35 +0800, Ming Lei wrote: > @@ -1280,10 +1282,18 @@ bool blk_mq_dispatch_rq_list(struct request_queue *q, > struct list_head *list, >* - Some but not all block drivers stop a queue before >* returning BLK_STS_RESOURCE. Two exceptions are

Re: [PATCH TRIVIAL] bsg: use pr_debug instead of hand crafted macros

On 01/21/18 23:53, Johannes Thumshirn wrote: - dprintk("map hdr %llx/%u %llx/%u\n", (unsigned long long) hdr->dout_xferp, + pr_debug("map hdr %llx/%u %llx/%u\n", +(unsigned long long) hdr->dout_xferp, hdr->dout_xfer_len, (unsigned long long)

Re: [PATCH 1/5] blk-mq: introduce BLK_STS_DEV_RESOURCE

> - if (ret == BLK_STS_RESOURCE) { > + if ((ret == BLK_STS_RESOURCE) || (ret == BLK_STS_DEV_RESOURCE)) > { No need for the inner braces here. > + if ((ret == BLK_STS_RESOURCE) || (ret == BLK_STS_DEV_RESOURCE)) Same here. > +/* > + * This status is returned from

Re: [PATCH] blktrace: support enabling cgroup info per-device

Hi Jens, Could you please look at this patch and the related patch set for blktrace [1], and give some feedback ? Regards, Tao [1]: https://www.spinics.net/lists/linux-btrace/msg00790.html On 2018/1/17 14:10, Hou Tao wrote: > Hi Jens, > > Any comments on this patch and the related patch set

Re: [LSF/MM TOPIC] A high-performance userspace block driver

On Thu, Jan 18, 2018 at 5:21 AM, Matthew Wilcox wrote: > On Wed, Jan 17, 2018 at 10:49:24AM +0800, Ming Lei wrote: >> Userfaultfd might be another choice: >> >> 1) map the block LBA space into a range of process vm space > > That would limit the size of a block device to

Re: [LSF/MM TOPIC] A high-performance userspace block driver

On Wed, Jan 17, 2018 at 01:21:44PM -0800, Matthew Wilcox wrote: > On Wed, Jan 17, 2018 at 10:49:24AM +0800, Ming Lei wrote: > > Userfaultfd might be another choice: > > > > 1) map the block LBA space into a range of process vm space > > That would limit the size of a block device to ~200TB (with

Re: [PATCH 4/5] blk-mq: introduce blk_get_request_notify

On Mon, Jan 22, 2018 at 11:35:49AM +0800, Ming Lei wrote: > DM-MPATH need to allocate request from underlying queue, but when the > allocation fails, there is no way to make underlying queue's RESTART > to restart DM's queue. > > This patch introduces blk_get_request_notify() for this purpose,

[PATCH 4.14 28/89] delayacct: Account blkio completion on the correct task

4.14-stable review patch. If anyone has any objections, please let me know. -- From: Josh Snyder commit c96f5471ce7d2aefd0dda560cc23f08ab00bc65d upstream. Before commit: e33a9bba85a8 ("sched/core: move IO scheduling accounting from io_schedule_timeout()