Re: [linux-next][bock] WARNING: CPU: 22 PID: 0 at block/blk-core.c:2655 .blk_update_request+0x4f8/0x500
On Mon, May 08, 2017 at 08:00:41AM -0600, Jens Axboe wrote: > Christoph, Martin - any ideas? Trace from Abdul below. Btw, what page size does the system have? > > WARNING: CPU: 12 PID: 0 at block/blk-core.c:2651 > .blk_update_request+0x4cc/0x4e0 Any knowledge from tracing or printk on what command is complete? Both req_op type and SCSI command?
Re: [linux-next][bock] WARNING: CPU: 22 PID: 0 at block/blk-core.c:2655 .blk_update_request+0x4f8/0x500
On 05/08/2017 01:13 AM, Abdul Haleem wrote: > On Fri, 2017-05-05 at 08:02 -0600, Jens Axboe wrote: >> On 05/05/2017 12:25 AM, Abdul Haleem wrote: >>> Hi, >>> >>> 4.11.0 Linus mainline booted with Warnings on PowerPC. >>> >>> We did not see this on next-20170407 but on next-20170410 and later. >> >> Have you tried current Linus -git? Both of the -next versions you list >> are rather old. >> > > Hi Jens, > > Warning is still seen with next-20170505 and also with today's mainline. > > It was first seen on next-20170410, so the last good was next-20170407. The log between the known good and first bad version, condensed a bit for primary suspects, is below. Christoph Hellwig (4): sd: split sd_setup_discard_cmnd sd: implement REQ_OP_WRITE_ZEROES sd: implement unmapping Write Zeroes block: remove the discard_zeroes_data flag Martin K. Petersen (2): scsi: sd: Separate zeroout and discard command choices scsi: sd: Remove LBPRZ dependency for discards Christoph Hellwig (7): block: implement splitting of REQ_OP_WRITE_ZEROES bios block: stop using blkdev_issue_write_same for zeroing block: add a flags argument to (__)blkdev_issue_zeroout block: add a REQ_NOUNMAP flag for REQ_OP_WRITE_ZEROES block: add a new BLKDEV_ZERO_NOFALLBACK flag block: stop using discards for zeroing block: remove the discard_zeroes_data flag Christoph, Martin - any ideas? Trace from Abdul below. WARNING: CPU: 12 PID: 0 at block/blk-core.c:2651 .blk_update_request+0x4cc/0x4e0 Modules linked in: sg(E) nfsd(E) auth_rpcgss(E) nfs_acl(E) lockd(E) grace(E) sunrpc(E) binfmt_misc(E) ip_tables(E) ext4(E) mbcache(E) jbd2(E) sd_mod(E) ibmvscsi(E) scsi_transport_srp(E) ibmveth(E) CPU: 12 PID: 0 Comm: swapper/12 Tainted: GE 4.11.0-autotest #1 task: c009f455ee80 task.stack: c009fb2e8000 NIP: c050bd1c LR: c050b8ec CTR: c05114b0 REGS: c013fff73740 TRAP: 0700 Tainted: GE(4.11.0-autotest) MSR: 80029032 CR: 48042048 XER: 0001 CFAR: c050bb34 SOFTE: 1 GPR00: c050b8ec c013fff739c0 c1389c00 c009eca9c800 GPR04: 0001 0060 GPR08: 00067887 c009eca9c800 de5f7e30 GPR12: 88044044 ce9f6c00 c009fb2ebf90 00200042 GPR16: 9367 c013fff7 c0df4100 GPR20: c13c3b00 c0df4100 0005 GPR24: 2ee0 c17789f8 GPR28: c38ba400 c009eca9c800 NIP [c050bd1c] .blk_update_request+0x4cc/0x4e0 LR [c050b8ec] .blk_update_request+0x9c/0x4e0 Call Trace: [c013fff739c0] [c050b8ec] .blk_update_request+0x9c/0x4e0 (unreliable) [c013fff73a60] [c06b06fc] .scsi_end_request+0x4c/0x240 [c013fff73b10] [c06b4564] .scsi_io_completion+0x1d4/0x6c0 [c013fff73be0] [c06a8cd0] .scsi_finish_command+0x100/0x1b0 [c013fff73c70] [c06b3978] .scsi_softirq_done+0x188/0x1e0 [c013fff73d00] [c0516b44] .blk_done_softirq+0xc4/0xf0 [c013fff73d90] [c00daef8] .__do_softirq+0x158/0x3b0 [c013fff73e90] [c00db5b8] .irq_exit+0x1a8/0x1c0 [c013fff73f10] [c0014f84] .__do_irq+0x94/0x1f0 [c013fff73f90] [c0026cbc] .call_do_irq+0x14/0x24 [c009fb2eb7f0] [c001516c] .do_IRQ+0x8c/0x100 [c009fb2eb890] [c0008bf4] hardware_interrupt_common+0x114/0x120 --- interrupt: 501 at .plpar_hcall_norets+0x14/0x20 LR = .check_and_cede_processor+0x24/0x40 [c009fb2ebb80] [0002] 0x2 (unreliable) [c009fb2ebbf0] [c07c360c] .dedicated_cede_loop+0x4c/0x150 [c009fb2ebc70] [c07c1040] .cpuidle_enter_state+0xb0/0x3b0 [c009fb2ebd20] [c012d1bc] .call_cpuidle+0x3c/0x70 [c009fb2ebd90] [c012d550] .do_idle+0x280/0x2e0 [c009fb2ebe50] [c012d768] .cpu_startup_entry+0x28/0x40 [c009fb2ebed0] [c00428a4] .start_secondary+0x304/0x350 [c009fb2ebf90] [c000aa6c] start_secondary_prolog+0x10/0x14 Instruction dump: 3f82ff90 3b9cc190 4bfffd8c 3f82ff90 3b9cc1a8 4bfffd80 61290040 b13f0018 4bfffbd4 3cc2ff8b 38c63160 4bfffd9c <0fe0> 4bfffe18 6000 6000 ---[ end trace 0f80359f8fb9c5f4 ]--- EXT4-fs (sda3): Delayed block allocation failed for inode 11011467 at logical offset 0 with max blocks 7 with error 121 EXT4-fs (sda3): This should not happen!! Data will be lost -- Jens Axboe
Re: [linux-next][bock] WARNING: CPU: 22 PID: 0 at block/blk-core.c:2655 .blk_update_request+0x4f8/0x500
On Fri, 2017-05-05 at 08:02 -0600, Jens Axboe wrote: > On 05/05/2017 12:25 AM, Abdul Haleem wrote: > > Hi, > > > > 4.11.0 Linus mainline booted with Warnings on PowerPC. > > > > We did not see this on next-20170407 but on next-20170410 and later. > > Have you tried current Linus -git? Both of the -next versions you list > are rather old. > Hi Jens, Warning is still seen with next-20170505 and also with today's mainline. It was first seen on next-20170410, so the last good was next-20170407. -- Regard's Abdul Haleem IBM Linux Technology Centre
Re: [linux-next][bock] WARNING: CPU: 22 PID: 0 at block/blk-core.c:2655 .blk_update_request+0x4f8/0x500
On 05/05/2017 12:25 AM, Abdul Haleem wrote: > Hi, > > 4.11.0 Linus mainline booted with Warnings on PowerPC. > > We did not see this on next-20170407 but on next-20170410 and later. Have you tried current Linus -git? Both of the -next versions you list are rather old. -- Jens Axboe
[linux-next][bock] WARNING: CPU: 22 PID: 0 at block/blk-core.c:2655 .blk_update_request+0x4f8/0x500
Hi, 4.11.0 Linus mainline booted with Warnings on PowerPC. We did not see this on next-20170407 but on next-20170410 and later. Machine type: Power 7 PowerVM LPAR Kernel : 4.11.0 Config : file attched. IPv6: ADDRCONF(NETDEV_UP): net0: link is not ready Starting WPA Supplicant daemon... Starting Authorization Manager... [ cut here ] WARNING: CPU: 12 PID: 0 at block/blk-core.c:2651 .blk_update_request+0x4cc/0x4e0 Modules linked in: sg(E) nfsd(E) auth_rpcgss(E) nfs_acl(E) lockd(E) grace(E) sunrpc(E) binfmt_misc(E) ip_tables(E) ext4(E) mbcache(E) jbd2(E) sd_mod(E) ibmvscsi(E) scsi_transport_srp(E) ibmveth(E) CPU: 12 PID: 0 Comm: swapper/12 Tainted: GE 4.11.0-autotest #1 task: c009f455ee80 task.stack: c009fb2e8000 NIP: c050bd1c LR: c050b8ec CTR: c05114b0 REGS: c013fff73740 TRAP: 0700 Tainted: GE(4.11.0-autotest) MSR: 80029032 CR: 48042048 XER: 0001 CFAR: c050bb34 SOFTE: 1 GPR00: c050b8ec c013fff739c0 c1389c00 c009eca9c800 GPR04: 0001 0060 GPR08: 00067887 c009eca9c800 de5f7e30 GPR12: 88044044 ce9f6c00 c009fb2ebf90 00200042 GPR16: 9367 c013fff7 c0df4100 GPR20: c13c3b00 c0df4100 0005 GPR24: 2ee0 c17789f8 GPR28: c38ba400 c009eca9c800 NIP [c050bd1c] .blk_update_request+0x4cc/0x4e0 LR [c050b8ec] .blk_update_request+0x9c/0x4e0 Call Trace: [c013fff739c0] [c050b8ec] .blk_update_request+0x9c/0x4e0 (unreliable) [c013fff73a60] [c06b06fc] .scsi_end_request+0x4c/0x240 [c013fff73b10] [c06b4564] .scsi_io_completion+0x1d4/0x6c0 [c013fff73be0] [c06a8cd0] .scsi_finish_command+0x100/0x1b0 [c013fff73c70] [c06b3978] .scsi_softirq_done+0x188/0x1e0 [c013fff73d00] [c0516b44] .blk_done_softirq+0xc4/0xf0 [c013fff73d90] [c00daef8] .__do_softirq+0x158/0x3b0 [c013fff73e90] [c00db5b8] .irq_exit+0x1a8/0x1c0 [c013fff73f10] [c0014f84] .__do_irq+0x94/0x1f0 [c013fff73f90] [c0026cbc] .call_do_irq+0x14/0x24 [c009fb2eb7f0] [c001516c] .do_IRQ+0x8c/0x100 [c009fb2eb890] [c0008bf4] hardware_interrupt_common+0x114/0x120 --- interrupt: 501 at .plpar_hcall_norets+0x14/0x20 LR = .check_and_cede_processor+0x24/0x40 [c009fb2ebb80] [0002] 0x2 (unreliable) [c009fb2ebbf0] [c07c360c] .dedicated_cede_loop+0x4c/0x150 [c009fb2ebc70] [c07c1040] .cpuidle_enter_state+0xb0/0x3b0 [c009fb2ebd20] [c012d1bc] .call_cpuidle+0x3c/0x70 [c009fb2ebd90] [c012d550] .do_idle+0x280/0x2e0 [c009fb2ebe50] [c012d768] .cpu_startup_entry+0x28/0x40 [c009fb2ebed0] [c00428a4] .start_secondary+0x304/0x350 [c009fb2ebf90] [c000aa6c] start_secondary_prolog+0x10/0x14 Instruction dump: 3f82ff90 3b9cc190 4bfffd8c 3f82ff90 3b9cc1a8 4bfffd80 61290040 b13f0018 4bfffbd4 3cc2ff8b 38c63160 4bfffd9c <0fe0> 4bfffe18 6000 6000 ---[ end trace 0f80359f8fb9c5f4 ]--- EXT4-fs (sda3): Delayed block allocation failed for inode 11011467 at logical offset 0 with max blocks 7 with error 121 EXT4-fs (sda3): This should not happen!! Data will be lost Starting Network Manager Script Dispatcher Service... Started Network Manager Script Dispatcher Service -- Regard's Abdul Haleem IBM Linux Technology Centre # # Automatically generated file; DO NOT EDIT. # Linux/powerpc 4.10.0-rc5 Kernel Configuration # CONFIG_PPC64=y # # Processor support # CONFIG_PPC_BOOK3S_64=y # CONFIG_PPC_BOOK3E_64 is not set CONFIG_GENERIC_CPU=y # CONFIG_CELL_CPU is not set # CONFIG_POWER4_CPU is not set # CONFIG_POWER5_CPU is not set # CONFIG_POWER6_CPU is not set # CONFIG_POWER7_CPU is not set # CONFIG_POWER8_CPU is not set CONFIG_PPC_BOOK3S=y CONFIG_PPC_FPU=y CONFIG_ALTIVEC=y CONFIG_VSX=y CONFIG_PPC_ICSWX=y # CONFIG_PPC_ICSWX_PID is not set # CONFIG_PPC_ICSWX_USE_SIGILL is not set CONFIG_PPC_STD_MMU=y CONFIG_PPC_STD_MMU_64=y CONFIG_PPC_RADIX_MMU=y CONFIG_PPC_MM_SLICES=y CONFIG_PPC_HAVE_PMU_SUPPORT=y CONFIG_PPC_PERF_CTRS=y CONFIG_SMP=y CONFIG_NR_CPUS=2048 CONFIG_PPC_DOORBELL=y CONFIG_VDSO32=y CONFIG_CPU_BIG_ENDIAN=y # CONFIG_CPU_LITTLE_ENDIAN is not set CONFIG_64BIT=y CONFIG_ARCH_PHYS_ADDR_T_64BIT=y CONFIG_ARCH_DMA_ADDR_T_64BIT=y CONFIG_MMU=y CONFIG_HAVE_SETUP_PER_CPU_AREA=y CONFIG_NEED_PER_CPU_EMBED_FIRST_CHUNK=y CONFIG_NR_IRQS=512 CONFIG_STACKTRACE_SUPPORT=y CONFIG_TRACE_IRQFLAGS_SUPPORT=y CONFIG_LOCKDEP_SUPPORT=y CONFIG_RWSEM_XCHGADD_ALGORITHM=y CONFIG_ARCH_HAS_ILOG2_U32=y CONFIG_ARCH_HAS_ILOG2_U64=y CONFIG_GENERIC_HWEIGHT=y CONFIG_ARCH_HAS_DMA_SET_COHERENT_MASK=y CONFIG_PPC=y # CONFIG_GE
