Re: Panic String: solaris assert: (lsize != psize) implies ((flags & ZIO_FLAG_RAW) != 0), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c, line: 631
> Andriy: > > I am happy to report that the system no longer panics. As requested I > removed > the remaining logs (34G worth) and punished the file system as hard as I > could. > > A scrub of the pool completed without error > > Will the change be committed or do I need to open a PR? > > Please let me know if I can supply additional information or if there > are any > further tests you would like me to perform. > > Thanks again for you prompt reply and apparent solution. > > Regards, > > Michael Jung > ___ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to > "freebsd-current-unsubscr...@freebsd.org" Great news. I committed the fix this morning. Thanks for reporting the problem and testing the fix. -- Thanks, Josh Paetzel ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Panic String: solaris assert: (lsize != psize) implies ((flags & ZIO_FLAG_RAW) != 0), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c, line: 631
On 2017-04-28 17:42, Andriy Gapon wrote: On 28/04/2017 14:56, Michael Jung wrote: I have mad the requested change.. [root@bsd11 /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs]# diff zio.c ~mikej/zio.c.orig 965c965 < size, NULL, NULL, ZIO_TYPE_FREE, ZIO_PRIORITY_NOW, --- BP_GET_PSIZE(bp), NULL, NULL, ZIO_TYPE_FREE, ZIO_PRIORITY_NOW, Yes, that's the change that I had in mind. I was a little bit confused by the order of the original and modified files, though :-) [root@bsd11 /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs]# As to the pool size: [root@bsd11 /usr/home/mikej]# zpool list NAME SIZE ALLOC FREE EXPANDSZ FRAGCAP DEDUP HEALTH ALTROOT tank 199G 143G 55.9G -85%71% 1.00x ONLINE - [root@bsd11 /usr/home/mikej]# I should have also mentioned that besides poudriere running a build, it was removing old logs - There was some 43G of old logs files that were in the process of being removed. So, given that the panic was in the freeing path, you were probably low on the pool space back when those log files were created. I mean that the gang blocks are typically created when a pool is very fragmented. I will hammer the box with and report back first of the week whether the panic re-occurs or not. Please also try removing those old files again too. Running zpool scrub afterwards could be a good idea too. Thank you again! Andriy: I am happy to report that the system no longer panics. As requested I removed the remaining logs (34G worth) and punished the file system as hard as I could. A scrub of the pool completed without error Will the change be committed or do I need to open a PR? Please let me know if I can supply additional information or if there are any further tests you would like me to perform. Thanks again for you prompt reply and apparent solution. Regards, Michael Jung ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Panic String: solaris assert: (lsize != psize) implies ((flags & ZIO_FLAG_RAW) != 0), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c, line: 631
On 28/04/2017 14:56, Michael Jung wrote: > I have mad the requested change.. > > [root@bsd11 /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs]# diff > zio.c > ~mikej/zio.c.orig > 965c965 > < size, NULL, NULL, ZIO_TYPE_FREE, ZIO_PRIORITY_NOW, > --- >> BP_GET_PSIZE(bp), NULL, NULL, ZIO_TYPE_FREE, ZIO_PRIORITY_NOW, Yes, that's the change that I had in mind. I was a little bit confused by the order of the original and modified files, though :-) > [root@bsd11 /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs]# > > As to the pool size: > > [root@bsd11 /usr/home/mikej]# zpool list > NAME SIZE ALLOC FREE EXPANDSZ FRAGCAP DEDUP HEALTH ALTROOT > tank 199G 143G 55.9G -85%71% 1.00x ONLINE - > [root@bsd11 /usr/home/mikej]# > > I should have also mentioned that besides poudriere running a build, it was > removing old logs - There was some 43G of old logs files that were in the > process > of being removed. So, given that the panic was in the freeing path, you were probably low on the pool space back when those log files were created. I mean that the gang blocks are typically created when a pool is very fragmented. > I will hammer the box with and report back first of the week whether the panic > re-occurs or not. Please also try removing those old files again too. Running zpool scrub afterwards could be a good idea too. Thank you again! -- Andriy Gapon ___ freebsd-current@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"
Re: Panic String: solaris assert: (lsize != psize) implies ((flags & ZIO_FLAG_RAW) != 0), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c, line: 631
On 2017-04-27 17:57, Andriy Gapon wrote: On 27/04/2017 18:52, Michael Jung wrote: Hi: Recently upgraded from r315905 to r317435 and during a poudriere run got this panic which I have not seen before. https://charon.gopai.com/core.txt.1 https://charon.gopai.com/info.1 Let me know what additional information I might supply. Mike, could you please edit function zio_free_sync() in sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c so that the zio_create() call has "size, size" arguments instead of "size, BP_GET_PSIZE(bp)" and see if that helps? (Your pool is probably low on space too.) panic: solaris assert: (lsize != psize) implies ((flags & ZIO_FLAG_RAW) != 0), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c, line: 631 cpuid = 6 time = 1493306220 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfe086140e850 vpanic() at vpanic+0x19c/frame 0xfe086140e8d0 panic() at panic+0x43/frame 0xfe086140e930 assfail() at assfail+0x1a/frame 0xfe086140e940 zio_create() at zio_create+0x11f/frame 0xfe086140e9a0 zio_free_sync() at zio_free_sync+0x197/frame 0xfe086140ea50 zio_gang_tree_issue() at zio_gang_tree_issue+0x13f/frame 0xfe086140eaa0 zio_gang_issue() at zio_gang_issue+0x152/frame 0xfe086140ead0 zio_execute() at zio_execute+0x36c/frame 0xfe086140eb20 taskqueue_run_locked() at taskqueue_run_locked+0x13d/frame 0xfe086140eb80 taskqueue_thread_loop() at taskqueue_thread_loop+0x88/frame 0xfe086140ebb0 fork_exit() at fork_exit+0x84/frame 0xfe086140ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfe086140ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- KDB: enter: panic Reading symbols from /boot/kernel/vmm.ko...Reading symbols from /usr/lib/debug//boot/kernel/vmm.ko.debug...done. done. Loaded symbols for /boot/kernel/vmm.ko Reading symbols from /boot/kernel/filemon.ko...Reading symbols from /usr/lib/debug//boot/kernel/filemon.ko.debug...done. done. Loaded symbols for /boot/kernel/filemon.ko Reading symbols from /boot/kernel/zfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/zfs.ko.debug...done. done. Loaded symbols for /boot/kernel/zfs.ko Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done. done. Loaded symbols for /boot/kernel/opensolaris.ko Reading symbols from /usr/local/lib/vmware-tools/modules/drivers/vmmemctl.ko...done. Loaded symbols for /usr/local/lib/vmware-tools/modules/drivers/vmmemctl.ko Reading symbols from /usr/local/lib/vmware-tools/modules/drivers/vmxnet.ko...done. Loaded symbols for /usr/local/lib/vmware-tools/modules/drivers/vmxnet.ko Reading symbols from /usr/local/lib/vmware-tools/modules/drivers/vmblock.ko...done. Loaded symbols for /usr/local/lib/vmware-tools/modules/drivers/vmblock.ko Reading symbols from /usr/local/lib/vmware-tools/modules/drivers/vmhgfs.ko...done. Loaded symbols for /usr/local/lib/vmware-tools/modules/drivers/vmhgfs.ko Reading symbols from /boot/kernel/linux.ko...Reading symbols from /usr/lib/debug//boot/kernel/linux.ko.debug...done. done. Loaded symbols for /boot/kernel/linux.ko Reading symbols from /boot/kernel/linux_common.ko...Reading symbols from /usr/lib/debug//boot/kernel/linux_common.ko.debug...done. done. Loaded symbols for /boot/kernel/linux_common.ko Reading symbols from /boot/kernel/linux64.ko...Reading symbols from /usr/lib/debug//boot/kernel/linux64.ko.debug...done. done. Loaded symbols for /boot/kernel/linux64.ko Reading symbols from /boot/kernel/nullfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/nullfs.ko.debug...done. done. Loaded symbols for /boot/kernel/nullfs.ko Reading symbols from /boot/kernel/linprocfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/linprocfs.ko.debug...done. done. Loaded symbols for /boot/kernel/linprocfs.ko Reading symbols from /boot/kernel/tmpfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/tmpfs.ko.debug...done. done. Loaded symbols for /boot/kernel/tmpfs.ko Reading symbols from /boot/kernel/fdescfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/fdescfs.ko.debug...done. done. Loaded symbols for /boot/kernel/fdescfs.ko #0 doadump (textdump=0) at pcpu.h:232 232 pcpu.h: No such file or directory. in pcpu.h (kgdb) #0 doadump (textdump=0) at pcpu.h:232 #1 0x803a1f7b in db_dump (dummy=, dummy2=, dummy3=, dummy4=) at /usr/src/sys/ddb/db_command.c:546 #2 0x803a1d6f in db_command (cmd_table=) at /usr/src/sys/ddb/db_command.c:453 #3 0x803a1aa4 in db_command_loop () at /usr/src/sys/ddb/db_command.c:506 #4 0x803a4b6f in db_trap (type=, code=) at /usr/src/sys/ddb/db_main.c:248 #5 0x80a9 in kdb_trap (type=3, code=-61456, tf=) at /usr/src/sys/kern/subr_kdb.c:654 #6 0x80ed2de6 in trap (frame=0xfe086140e780) at /usr/src/sys/amd64/amd64/trap.c:537 #7 0x80eb54e1 in calltrap () at /usr/s
Re: Panic String: solaris assert: (lsize != psize) implies ((flags & ZIO_FLAG_RAW) != 0), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c, line: 631
On 27/04/2017 18:52, Michael Jung wrote: > Hi: > > Recently upgraded from r315905 to r317435 and during a poudriere run got this > panic which I have not seen before. > > https://charon.gopai.com/core.txt.1 > https://charon.gopai.com/info.1 > > Let me know what additional information I might supply. Mike, could you please edit function zio_free_sync() in sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c so that the zio_create() call has "size, size" arguments instead of "size, BP_GET_PSIZE(bp)" and see if that helps? (Your pool is probably low on space too.) > panic: solaris assert: (lsize != psize) implies ((flags & ZIO_FLAG_RAW) != 0), > file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c, line: 631 > cpuid = 6 > time = 1493306220 > KDB: stack backtrace: > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfe086140e850 > vpanic() at vpanic+0x19c/frame 0xfe086140e8d0 > panic() at panic+0x43/frame 0xfe086140e930 > assfail() at assfail+0x1a/frame 0xfe086140e940 > zio_create() at zio_create+0x11f/frame 0xfe086140e9a0 > zio_free_sync() at zio_free_sync+0x197/frame 0xfe086140ea50 > zio_gang_tree_issue() at zio_gang_tree_issue+0x13f/frame 0xfe086140eaa0 > zio_gang_issue() at zio_gang_issue+0x152/frame 0xfe086140ead0 > zio_execute() at zio_execute+0x36c/frame 0xfe086140eb20 > taskqueue_run_locked() at taskqueue_run_locked+0x13d/frame 0xfe086140eb80 > taskqueue_thread_loop() at taskqueue_thread_loop+0x88/frame 0xfe086140ebb0 > fork_exit() at fork_exit+0x84/frame 0xfe086140ebf0 > fork_trampoline() at fork_trampoline+0xe/frame 0xfe086140ebf0 > --- trap 0, rip = 0, rsp = 0, rbp = 0 --- > KDB: enter: panic > > Reading symbols from /boot/kernel/vmm.ko...Reading symbols from > /usr/lib/debug//boot/kernel/vmm.ko.debug...done. > done. > Loaded symbols for /boot/kernel/vmm.ko > Reading symbols from /boot/kernel/filemon.ko...Reading symbols from > /usr/lib/debug//boot/kernel/filemon.ko.debug...done. > done. > Loaded symbols for /boot/kernel/filemon.ko > Reading symbols from /boot/kernel/zfs.ko...Reading symbols from > /usr/lib/debug//boot/kernel/zfs.ko.debug...done. > done. > Loaded symbols for /boot/kernel/zfs.ko > Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from > /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done. > done. > Loaded symbols for /boot/kernel/opensolaris.ko > Reading symbols from > /usr/local/lib/vmware-tools/modules/drivers/vmmemctl.ko...done. > Loaded symbols for /usr/local/lib/vmware-tools/modules/drivers/vmmemctl.ko > Reading symbols from > /usr/local/lib/vmware-tools/modules/drivers/vmxnet.ko...done. > Loaded symbols for /usr/local/lib/vmware-tools/modules/drivers/vmxnet.ko > Reading symbols from > /usr/local/lib/vmware-tools/modules/drivers/vmblock.ko...done. > Loaded symbols for /usr/local/lib/vmware-tools/modules/drivers/vmblock.ko > Reading symbols from > /usr/local/lib/vmware-tools/modules/drivers/vmhgfs.ko...done. > Loaded symbols for /usr/local/lib/vmware-tools/modules/drivers/vmhgfs.ko > Reading symbols from /boot/kernel/linux.ko...Reading symbols from > /usr/lib/debug//boot/kernel/linux.ko.debug...done. > done. > Loaded symbols for /boot/kernel/linux.ko > Reading symbols from /boot/kernel/linux_common.ko...Reading symbols from > /usr/lib/debug//boot/kernel/linux_common.ko.debug...done. > done. > Loaded symbols for /boot/kernel/linux_common.ko > Reading symbols from /boot/kernel/linux64.ko...Reading symbols from > /usr/lib/debug//boot/kernel/linux64.ko.debug...done. > done. > Loaded symbols for /boot/kernel/linux64.ko > Reading symbols from /boot/kernel/nullfs.ko...Reading symbols from > /usr/lib/debug//boot/kernel/nullfs.ko.debug...done. > done. > Loaded symbols for /boot/kernel/nullfs.ko > Reading symbols from /boot/kernel/linprocfs.ko...Reading symbols from > /usr/lib/debug//boot/kernel/linprocfs.ko.debug...done. > done. > Loaded symbols for /boot/kernel/linprocfs.ko > Reading symbols from /boot/kernel/tmpfs.ko...Reading symbols from > /usr/lib/debug//boot/kernel/tmpfs.ko.debug...done. > done. > Loaded symbols for /boot/kernel/tmpfs.ko > Reading symbols from /boot/kernel/fdescfs.ko...Reading symbols from > /usr/lib/debug//boot/kernel/fdescfs.ko.debug...done. > done. > Loaded symbols for /boot/kernel/fdescfs.ko > #0 doadump (textdump=0) at pcpu.h:232 > 232 pcpu.h: No such file or directory. > in pcpu.h > (kgdb) #0 doadump (textdump=0) at pcpu.h:232 > #1 0x803a1f7b in db_dump (dummy=, > dummy2=, dummy3=, > dummy4=) at /usr/src/sys/ddb/db_command.c:546 > #2 0x803a1d6f in db_command (cmd_table=) > at /usr/src/sys/ddb/db_command.c:453 > #3 0x803a1aa4 in db_command_loop () > at /usr/src/sys/ddb/db_command.c:506 > #4 0x803a4b6f in db_trap (type=, > code=) at /usr/src/sys/ddb/db_main.c:248 > #5 0x80a9 in kdb_trap (type=3, code=-61456, > tf=) at /usr/src/sys/kern/subr_kdb.c:654 > #6 0x80e
Panic String: solaris assert: (lsize != psize) implies ((flags & ZIO_FLAG_RAW) != 0), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c, line: 631
Hi: Recently upgraded from r315905 to r317435 and during a poudriere run got this panic which I have not seen before. https://charon.gopai.com/core.txt.1 https://charon.gopai.com/info.1 Let me know what additional information I might supply. --mikej panic: solaris assert: (lsize != psize) implies ((flags & ZIO_FLAG_RAW) != 0), file: /usr/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zio.c, line: 631 cpuid = 6 time = 1493306220 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfe086140e850 vpanic() at vpanic+0x19c/frame 0xfe086140e8d0 panic() at panic+0x43/frame 0xfe086140e930 assfail() at assfail+0x1a/frame 0xfe086140e940 zio_create() at zio_create+0x11f/frame 0xfe086140e9a0 zio_free_sync() at zio_free_sync+0x197/frame 0xfe086140ea50 zio_gang_tree_issue() at zio_gang_tree_issue+0x13f/frame 0xfe086140eaa0 zio_gang_issue() at zio_gang_issue+0x152/frame 0xfe086140ead0 zio_execute() at zio_execute+0x36c/frame 0xfe086140eb20 taskqueue_run_locked() at taskqueue_run_locked+0x13d/frame 0xfe086140eb80 taskqueue_thread_loop() at taskqueue_thread_loop+0x88/frame 0xfe086140ebb0 fork_exit() at fork_exit+0x84/frame 0xfe086140ebf0 fork_trampoline() at fork_trampoline+0xe/frame 0xfe086140ebf0 --- trap 0, rip = 0, rsp = 0, rbp = 0 --- KDB: enter: panic Reading symbols from /boot/kernel/vmm.ko...Reading symbols from /usr/lib/debug//boot/kernel/vmm.ko.debug...done. done. Loaded symbols for /boot/kernel/vmm.ko Reading symbols from /boot/kernel/filemon.ko...Reading symbols from /usr/lib/debug//boot/kernel/filemon.ko.debug...done. done. Loaded symbols for /boot/kernel/filemon.ko Reading symbols from /boot/kernel/zfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/zfs.ko.debug...done. done. Loaded symbols for /boot/kernel/zfs.ko Reading symbols from /boot/kernel/opensolaris.ko...Reading symbols from /usr/lib/debug//boot/kernel/opensolaris.ko.debug...done. done. Loaded symbols for /boot/kernel/opensolaris.ko Reading symbols from /usr/local/lib/vmware-tools/modules/drivers/vmmemctl.ko...done. Loaded symbols for /usr/local/lib/vmware-tools/modules/drivers/vmmemctl.ko Reading symbols from /usr/local/lib/vmware-tools/modules/drivers/vmxnet.ko...done. Loaded symbols for /usr/local/lib/vmware-tools/modules/drivers/vmxnet.ko Reading symbols from /usr/local/lib/vmware-tools/modules/drivers/vmblock.ko...done. Loaded symbols for /usr/local/lib/vmware-tools/modules/drivers/vmblock.ko Reading symbols from /usr/local/lib/vmware-tools/modules/drivers/vmhgfs.ko...done. Loaded symbols for /usr/local/lib/vmware-tools/modules/drivers/vmhgfs.ko Reading symbols from /boot/kernel/linux.ko...Reading symbols from /usr/lib/debug//boot/kernel/linux.ko.debug...done. done. Loaded symbols for /boot/kernel/linux.ko Reading symbols from /boot/kernel/linux_common.ko...Reading symbols from /usr/lib/debug//boot/kernel/linux_common.ko.debug...done. done. Loaded symbols for /boot/kernel/linux_common.ko Reading symbols from /boot/kernel/linux64.ko...Reading symbols from /usr/lib/debug//boot/kernel/linux64.ko.debug...done. done. Loaded symbols for /boot/kernel/linux64.ko Reading symbols from /boot/kernel/nullfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/nullfs.ko.debug...done. done. Loaded symbols for /boot/kernel/nullfs.ko Reading symbols from /boot/kernel/linprocfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/linprocfs.ko.debug...done. done. Loaded symbols for /boot/kernel/linprocfs.ko Reading symbols from /boot/kernel/tmpfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/tmpfs.ko.debug...done. done. Loaded symbols for /boot/kernel/tmpfs.ko Reading symbols from /boot/kernel/fdescfs.ko...Reading symbols from /usr/lib/debug//boot/kernel/fdescfs.ko.debug...done. done. Loaded symbols for /boot/kernel/fdescfs.ko #0 doadump (textdump=0) at pcpu.h:232 232 pcpu.h: No such file or directory. in pcpu.h (kgdb) #0 doadump (textdump=0) at pcpu.h:232 #1 0x803a1f7b in db_dump (dummy=, dummy2=, dummy3=, dummy4=) at /usr/src/sys/ddb/db_command.c:546 #2 0x803a1d6f in db_command (cmd_table=) at /usr/src/sys/ddb/db_command.c:453 #3 0x803a1aa4 in db_command_loop () at /usr/src/sys/ddb/db_command.c:506 #4 0x803a4b6f in db_trap (type=, code=) at /usr/src/sys/ddb/db_main.c:248 #5 0x80a9 in kdb_trap (type=3, code=-61456, tf=) at /usr/src/sys/kern/subr_kdb.c:654 #6 0x80ed2de6 in trap (frame=0xfe086140e780) at /usr/src/sys/amd64/amd64/trap.c:537 #7 0x80eb54e1 in calltrap () at /usr/src/sys/amd64/amd64/exception.S:236 #8 0x80a92a6b in kdb_enter (why=0x8143c265 "panic", msg=) at cpufunc.h:63 #9 0x80a513c9 in vpanic (fmt=, ap=0xfe086140e910) at /usr/src/sys/kern/kern_shutdown.c:772 #10 0x80a51433 in panic (fmt=) at /usr/src/sys/kern/kern_shutdown.c:710 #11 0xff
