Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On Thu, Jan 11, 2018 at 06:46:54PM +0100, Christoph Hellwig wrote: > Thanks for looking into this Ming, I had missed it in the my current > work overload. Can you send the updated series to Jens? OK, I will post it out soon. Thanks, Ming

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

Thanks for looking into this Ming, I had missed it in the my current work overload. Can you send the updated series to Jens?

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On 11.01.2018 12:44, Christian Borntraeger wrote: On 01/11/2018 10:13 AM, Ming Lei wrote: On Wed, Dec 20, 2017 at 04:47:21PM +0100, Christian Borntraeger wrote: On 12/18/2017 02:56 PM, Stefan Haberland wrote: On 07.12.2017 00:29, Christoph Hellwig wrote: On Wed, Dec 06, 2017 at 01:25:11PM

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On 01/11/2018 10:13 AM, Ming Lei wrote: > On Wed, Dec 20, 2017 at 04:47:21PM +0100, Christian Borntraeger wrote: >> On 12/18/2017 02:56 PM, Stefan Haberland wrote: >>> On 07.12.2017 00:29, Christoph Hellwig wrote: On Wed, Dec 06, 2017 at 01:25:11PM +0100, Christian Borntraeger wrote: t

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On 11.01.2018 10:13, Ming Lei wrote: On Wed, Dec 20, 2017 at 04:47:21PM +0100, Christian Borntraeger wrote: On 12/18/2017 02:56 PM, Stefan Haberland wrote: On 07.12.2017 00:29, Christoph Hellwig wrote: On Wed, Dec 06, 2017 at 01:25:11PM +0100, Christian Borntraeger wrote: t > commit

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On Wed, Dec 20, 2017 at 04:47:21PM +0100, Christian Borntraeger wrote: > On 12/18/2017 02:56 PM, Stefan Haberland wrote: > > On 07.12.2017 00:29, Christoph Hellwig wrote: > >> On Wed, Dec 06, 2017 at 01:25:11PM +0100, Christian Borntraeger wrote: > >> t > commit

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On 12/18/2017 02:56 PM, Stefan Haberland wrote: > On 07.12.2017 00:29, Christoph Hellwig wrote: >> On Wed, Dec 06, 2017 at 01:25:11PM +0100, Christian Borntraeger wrote: >> t > commit 11b2025c3326f7096ceb588c3117c7883850c068    -> bad >>> blk-mq: create a blk_mq_ctx for each possible CPU >>>

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On 07.12.2017 00:29, Christoph Hellwig wrote: On Wed, Dec 06, 2017 at 01:25:11PM +0100, Christian Borntraeger wrote: t > commit 11b2025c3326f7096ceb588c3117c7883850c068-> bad blk-mq: create a blk_mq_ctx for each possible CPU does not boot on DASD and commit

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On 12/07/2017 12:29 AM, Christoph Hellwig wrote: > On Wed, Dec 06, 2017 at 01:25:11PM +0100, Christian Borntraeger wrote: > t > commit 11b2025c3326f7096ceb588c3117c7883850c068-> bad >> blk-mq: create a blk_mq_ctx for each possible CPU >> does not boot on DASD and >> commit

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On Wed, Dec 06, 2017 at 01:25:11PM +0100, Christian Borntraeger wrote: t > commit 11b2025c3326f7096ceb588c3117c7883850c068-> bad > blk-mq: create a blk_mq_ctx for each possible CPU > does not boot on DASD and > commit 9c6ae239e01ae9a9f8657f05c55c4372e9fc8bcc-> good >

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On 12/04/2017 05:21 PM, Christoph Hellwig wrote: > On Wed, Nov 29, 2017 at 08:18:09PM +0100, Christian Borntraeger wrote: >> Works fine under KVM with virtio-blk, but still hangs during boot in an LPAR. >> FWIW, the system not only has scsi disks via fcp but also DASDs as a boot >> disk. >> Seems

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On Wed, Nov 29, 2017 at 08:18:09PM +0100, Christian Borntraeger wrote: > Works fine under KVM with virtio-blk, but still hangs during boot in an LPAR. > FWIW, the system not only has scsi disks via fcp but also DASDs as a boot > disk. > Seems that this is the place where the system stops. (see

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On 11/29/2017 08:18 PM, Christian Borntraeger wrote: > Works fine under KVM with virtio-blk, but still hangs during boot in an LPAR. > FWIW, the system not only has scsi disks via fcp but also DASDs as a boot > disk. > Seems that this is the place where the system stops. (see the sysrq-t output

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

Works fine under KVM with virtio-blk, but still hangs during boot in an LPAR. FWIW, the system not only has scsi disks via fcp but also DASDs as a boot disk. Seems that this is the place where the system stops. (see the sysrq-t output at the bottom). Message "[0.247484] Linux version

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On 11/23/2017 07:59 PM, Christian Borntraeger wrote: > > > On 11/23/2017 07:32 PM, Christoph Hellwig wrote: >> On Thu, Nov 23, 2017 at 07:28:31PM +0100, Christian Borntraeger wrote: >>> zfcp on s390. >> >> Ok, so it can't be the interrupt code, but probably is the blk-mq-cpumap.c >> changes.

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On 11/23/2017 07:32 PM, Christoph Hellwig wrote: > On Thu, Nov 23, 2017 at 07:28:31PM +0100, Christian Borntraeger wrote: >> zfcp on s390. > > Ok, so it can't be the interrupt code, but probably is the blk-mq-cpumap.c > changes. Can you try to revert just those for a quick test? Hmm, I get

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On Thu, Nov 23, 2017 at 07:28:31PM +0100, Christian Borntraeger wrote: > zfcp on s390. Ok, so it can't be the interrupt code, but probably is the blk-mq-cpumap.c changes. Can you try to revert just those for a quick test?

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

zfcp on s390. On 11/23/2017 07:25 PM, Christoph Hellwig wrote: > What HBA driver do you use in the host? >

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

What HBA driver do you use in the host?

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On 11/23/2017 03:34 PM, Christoph Hellwig wrote: > FYI, the patch below changes both the irq and block mappings to > always use the cpu possible map (should be split in two in due time). > > I think this is the right way forward. For every normal machine > those two are the same, but for VMs

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

Yes it seems to fix the bug. On 11/23/2017 03:34 PM, Christoph Hellwig wrote: > FYI, the patch below changes both the irq and block mappings to > always use the cpu possible map (should be split in two in due time). > > I think this is the right way forward. For every normal machine > those two

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

[fullquote deleted] > What will happen for the CPU hotplug case? > Wouldn't we route I/O to a disabled CPU with this patch? Why would we route I/O to a disabled CPU (we generally route I/O to devices to start with). How would including possible but not present cpus change anything?

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On 11/23/2017 03:34 PM, Christoph Hellwig wrote: > FYI, the patch below changes both the irq and block mappings to > always use the cpu possible map (should be split in two in due time). > > I think this is the right way forward. For every normal machine > those two are the same, but for VMs

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

FYI, the patch below changes both the irq and block mappings to always use the cpu possible map (should be split in two in due time). I think this is the right way forward. For every normal machine those two are the same, but for VMs with maxcpus above their normal count or some big iron that

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

Ok, it helps to make sure we're actually doing I/O from the CPU, I've reproduced it now.

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

I can't reproduce it in my VM with adding a new CPU. Do you have any interesting blk-mq like actually using multiple queues? I'll give that a spin next.

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On 11/22/2017 12:28 AM, Christoph Hellwig wrote: > Jens, please don't just revert the commit in your for-linus tree. > > On its own this will totally mess up the interrupt assignments. Give > me a bit of time to sort this out properly. I wasn't going to push it until I heard otherwise. I'll

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

Jens, please don't just revert the commit in your for-linus tree. On its own this will totally mess up the interrupt assignments. Give me a bit of time to sort this out properly.

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On 11/21/2017 01:31 PM, Christian Borntraeger wrote: > > > On 11/21/2017 09:21 PM, Jens Axboe wrote: >> On 11/21/2017 01:19 PM, Christian Borntraeger wrote: >>> >>> On 11/21/2017 09:14 PM, Jens Axboe wrote: On 11/21/2017 01:12 PM, Christian Borntraeger wrote: > > > On 11/21/2017

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On 11/21/2017 09:21 PM, Jens Axboe wrote: > On 11/21/2017 01:19 PM, Christian Borntraeger wrote: >> >> On 11/21/2017 09:14 PM, Jens Axboe wrote: >>> On 11/21/2017 01:12 PM, Christian Borntraeger wrote: On 11/21/2017 08:30 PM, Jens Axboe wrote: > On 11/21/2017 12:15 PM,

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On 11/21/2017 01:19 PM, Christian Borntraeger wrote: > > On 11/21/2017 09:14 PM, Jens Axboe wrote: >> On 11/21/2017 01:12 PM, Christian Borntraeger wrote: >>> >>> >>> On 11/21/2017 08:30 PM, Jens Axboe wrote: On 11/21/2017 12:15 PM, Christian Borntraeger wrote: > > > On

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On 11/21/2017 09:14 PM, Jens Axboe wrote: > On 11/21/2017 01:12 PM, Christian Borntraeger wrote: >> >> >> On 11/21/2017 08:30 PM, Jens Axboe wrote: >>> On 11/21/2017 12:15 PM, Christian Borntraeger wrote: On 11/21/2017 07:39 PM, Jens Axboe wrote: > On 11/21/2017 11:27 AM, Jens

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On 11/21/2017 01:12 PM, Christian Borntraeger wrote: > > > On 11/21/2017 08:30 PM, Jens Axboe wrote: >> On 11/21/2017 12:15 PM, Christian Borntraeger wrote: >>> >>> >>> On 11/21/2017 07:39 PM, Jens Axboe wrote: On 11/21/2017 11:27 AM, Jens Axboe wrote: > On 11/21/2017 11:12 AM,

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On 11/21/2017 08:30 PM, Jens Axboe wrote: > On 11/21/2017 12:15 PM, Christian Borntraeger wrote: >> >> >> On 11/21/2017 07:39 PM, Jens Axboe wrote: >>> On 11/21/2017 11:27 AM, Jens Axboe wrote: On 11/21/2017 11:12 AM, Christian Borntraeger wrote: > > > On 11/21/2017 07:09 PM,

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On 11/21/2017 12:15 PM, Christian Borntraeger wrote: > > > On 11/21/2017 07:39 PM, Jens Axboe wrote: >> On 11/21/2017 11:27 AM, Jens Axboe wrote: >>> On 11/21/2017 11:12 AM, Christian Borntraeger wrote: On 11/21/2017 07:09 PM, Jens Axboe wrote: > On 11/21/2017 10:27 AM, Jens

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On 11/21/2017 07:39 PM, Jens Axboe wrote: > On 11/21/2017 11:27 AM, Jens Axboe wrote: >> On 11/21/2017 11:12 AM, Christian Borntraeger wrote: >>> >>> >>> On 11/21/2017 07:09 PM, Jens Axboe wrote: On 11/21/2017 10:27 AM, Jens Axboe wrote: > On 11/21/2017 03:14 AM, Christian Borntraeger

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On 11/21/2017 11:27 AM, Jens Axboe wrote: > On 11/21/2017 11:12 AM, Christian Borntraeger wrote: >> >> >> On 11/21/2017 07:09 PM, Jens Axboe wrote: >>> On 11/21/2017 10:27 AM, Jens Axboe wrote: On 11/21/2017 03:14 AM, Christian Borntraeger wrote: > Bisect points to > >

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On 11/21/2017 11:12 AM, Christian Borntraeger wrote: > > > On 11/21/2017 07:09 PM, Jens Axboe wrote: >> On 11/21/2017 10:27 AM, Jens Axboe wrote: >>> On 11/21/2017 03:14 AM, Christian Borntraeger wrote: Bisect points to 1b5a7455d345b223d3a4658a9e5fce985b7998c1 is the first bad

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On 11/21/2017 07:09 PM, Jens Axboe wrote: > On 11/21/2017 10:27 AM, Jens Axboe wrote: >> On 11/21/2017 03:14 AM, Christian Borntraeger wrote: >>> Bisect points to >>> >>> 1b5a7455d345b223d3a4658a9e5fce985b7998c1 is the first bad commit >>> commit 1b5a7455d345b223d3a4658a9e5fce985b7998c1 >>>

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On 11/21/2017 10:27 AM, Jens Axboe wrote: > On 11/21/2017 03:14 AM, Christian Borntraeger wrote: >> Bisect points to >> >> 1b5a7455d345b223d3a4658a9e5fce985b7998c1 is the first bad commit >> commit 1b5a7455d345b223d3a4658a9e5fce985b7998c1 >> Author: Christoph Hellwig >> Date: Mon

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk (also 4.12 stable)

On 11/21/2017 10:50 AM, Christian Borntraeger wrote: > > > On 11/21/2017 09:35 AM, Christian Borntraeger wrote: >> >> >> On 11/20/2017 09:52 PM, Jens Axboe wrote: >>> On 11/20/2017 01:49 PM, Christian Borntraeger wrote: On 11/20/2017 08:42 PM, Jens Axboe wrote: > On

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk

On 11/21/2017 09:35 AM, Christian Borntraeger wrote: > > > On 11/20/2017 09:52 PM, Jens Axboe wrote: >> On 11/20/2017 01:49 PM, Christian Borntraeger wrote: >>> >>> >>> On 11/20/2017 08:42 PM, Jens Axboe wrote: On 11/20/2017 12:29 PM, Christian Borntraeger wrote: > > > On

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk

On 11/20/2017 09:52 PM, Jens Axboe wrote: > On 11/20/2017 01:49 PM, Christian Borntraeger wrote: >> >> >> On 11/20/2017 08:42 PM, Jens Axboe wrote: >>> On 11/20/2017 12:29 PM, Christian Borntraeger wrote: On 11/20/2017 08:20 PM, Bart Van Assche wrote: > On Fri, 2017-11-17 at

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk

On 11/20/2017 01:49 PM, Christian Borntraeger wrote: > > > On 11/20/2017 08:42 PM, Jens Axboe wrote: >> On 11/20/2017 12:29 PM, Christian Borntraeger wrote: >>> >>> >>> On 11/20/2017 08:20 PM, Bart Van Assche wrote: On Fri, 2017-11-17 at 15:42 +0100, Christian Borntraeger wrote: > This

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk

On 11/20/2017 08:42 PM, Jens Axboe wrote: > On 11/20/2017 12:29 PM, Christian Borntraeger wrote: >> >> >> On 11/20/2017 08:20 PM, Bart Van Assche wrote: >>> On Fri, 2017-11-17 at 15:42 +0100, Christian Borntraeger wrote: This is b7a71e66d (Jens Axboe2017-08-01

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk

On 11/20/2017 12:29 PM, Christian Borntraeger wrote: > > > On 11/20/2017 08:20 PM, Bart Van Assche wrote: >> On Fri, 2017-11-17 at 15:42 +0100, Christian Borntraeger wrote: >>> This is >>> >>> b7a71e66d (Jens Axboe2017-08-01 09:28:24 -0600 1141) * >>> are mapped to it. >>>

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk

On 11/20/2017 08:20 PM, Bart Van Assche wrote: > On Fri, 2017-11-17 at 15:42 +0100, Christian Borntraeger wrote: >> This is >> >> b7a71e66d (Jens Axboe2017-08-01 09:28:24 -0600 1141) * >> are mapped to it. >> b7a71e66d (Jens Axboe2017-08-01 09:28:24 -0600

Re: 4.14: WARNING: CPU: 4 PID: 2895 at block/blk-mq.c:1144 with virtio-blk

On Fri, 2017-11-17 at 15:42 +0100, Christian Borntraeger wrote: > This is > > b7a71e66d (Jens Axboe2017-08-01 09:28:24 -0600 1141) * > are mapped to it. > b7a71e66d (Jens Axboe2017-08-01 09:28:24 -0600 1142) */ > 6a83e74d2 (Bart Van Assche