more info with panic dump: root@con1:/var/crash/myhost# cat 112.txt MESSAGE sd16 is /scsi_vhci/disk@g600144f07adeb1180000599164520004 /scsi_vhci/disk@g600144f07adeb1180000599164550005 (sd17) online /scsi_vhci/disk@g600144f07adeb1180000599164550005 (sd17) multipath status: degraded: path 18 iscsi0/[email protected]:02:f5a77de5-60c0-69ea-90a7-833c1ca9aeec0001,4 is online NOTICE: iscsi session(20) iqn.2010-08.org.illumos:02:f5da3fc0-05b2-4f4b-ed65-b401fbe69198 online
sd17 at scsi_vhci0: unit-address g600144f07adeb1180000599164550005: f_tpgs sd17 is /scsi_vhci/disk@g600144f07adeb1180000599164550005 /scsi_vhci/disk@g600144f07adedf00000059916fdd0001 (sd18) online /scsi_vhci/disk@g600144f07adedf00000059916fdd0001 (sd18) multipath status: degraded: path 19 iscsi0/[email protected]:02:f5da3fc0-05b2-4f4b-ed65-b401fbe691980001,0 is online sd18 at scsi_vhci0: unit-address g600144f07adedf00000059916fdd0001: f_tpgs sd18 is /scsi_vhci/disk@g600144f07adedf00000059916fdd0001 /scsi_vhci/disk@g600144f07adedf00000059916fe00002 (sd19) online /scsi_vhci/disk@g600144f07adedf00000059916fe00002 (sd19) multipath status: degraded: path 20 iscsi0/[email protected]:02:f5da3fc0-05b2-4f4b-ed65-b401fbe691980001,1 is online sd19 at scsi_vhci0: unit-address g600144f07adedf00000059916fe00002: f_tpgs sd19 is /scsi_vhci/disk@g600144f07adedf00000059916fe00002 /scsi_vhci/disk@g600144f07adedf00000059916fe40003 (sd20) online /scsi_vhci/disk@g600144f07adedf00000059916fe40003 (sd20) multipath status: degraded: path 21 iscsi0/[email protected]:02:f5da3fc0-05b2-4f4b-ed65-b401fbe691980001,2 is online sd20 at scsi_vhci0: unit-address g600144f07adedf00000059916fe40003: f_tpgs sd20 is /scsi_vhci/disk@g600144f07adedf00000059916fe40003 /scsi_vhci/disk@g600144f07adedf00000059916fe70004 (sd21) online /scsi_vhci/disk@g600144f07adedf00000059916fe70004 (sd21) multipath status: degraded: path 22 iscsi0/[email protected]:02:f5da3fc0-05b2-4f4b-ed65-b401fbe691980001,3 is online sd21 at scsi_vhci0: unit-address g600144f07adedf00000059916fe70004: f_tpgs sd21 is /scsi_vhci/disk@g600144f07adedf00000059916fe70004 /scsi_vhci/disk@g600144f07adedf00000059916fea0005 (sd22) online /scsi_vhci/disk@g600144f07adedf00000059916fea0005 (sd22) multipath status: degraded: path 23 iscsi0/[email protected]:02:f5da3fc0-05b2-4f4b-ed65-b401fbe691980001,4 is online sd22 at scsi_vhci0: unit-address g600144f07adedf00000059916fea0005: f_tpgs sd22 is /scsi_vhci/disk@g600144f07adedf00000059916fea0005 NOTICE: igb0 unregistered NOTICE: ixgbe4 unregistered NOTICE: ixgbe0 unregistered NOTICE: ixgbe3 unregistered NOTICE: ixgbe1 unregistered pseudo-device: devinfo0 devinfo0 is /pseudo/devinfo@0 ISA-device: asy0 asy0 is /pci@0,0/isa@1f/asy@1,3f8 ISA-device: asy1 asy1 is /pci@0,0/isa@1f/asy@1,2f8 pseudo-device: fcp0 fcp0 is /pseudo/fcp@0 pseudo-device: fct0 fct0 is /pseudo/fct@0 pseudo-device: stmf0 stmf0 is /pseudo/stmf@0 pseudo-device: fbt0 fbt0 is /pseudo/fbt@0 pseudo-device: lockstat0 lockstat0 is /pseudo/lockstat@0 pseudo-device: profile0 profile0 is /pseudo/profile@0 pseudo-device: sdt0 sdt0 is /pseudo/sdt@0 pseudo-device: systrace0 systrace0 is /pseudo/systrace@0 pseudo-device: fcsm0 fcsm0 is /pseudo/fcsm@0 pseudo-device: llc10 llc10 is /pseudo/llc1@0 NOTICE: igb0 registered pseudo-device: lofi0 lofi0 is /pseudo/lofi@0 pseudo-device: ramdisk1024 ramdisk1024 is /pseudo/ramdisk@1024 pseudo-device: ucode0 ucode0 is /pseudo/ucode@0 pseudo-device: bpf0 bpf0 is /pseudo/bpf@0 pseudo-device: eventfd0 eventfd0 is /pseudo/eventfd@0 pseudo-device: fssnap0 fssnap0 is /pseudo/fssnap@0 pseudo-device: inotify0 inotify0 is /pseudo/inotify@0 pseudo-device: signalfd0 signalfd0 is /pseudo/signalfd@0 pseudo-device: timerfd0 timerfd0 is /pseudo/timerfd@0 pseudo-device: pm0 pm0 is /pseudo/pm@0 pseudo-device: nsmb0 nsmb0 is /pseudo/nsmb@0 NOTICE: ixgbe4 registered NOTICE: ixgbe4: Intel 10Gb Ethernet NOTICE: ixgbe0 registered NOTICE: ixgbe0: Intel 10Gb Ethernet NOTICE: ixgbe3 registered NOTICE: ixgbe3: Intel 10Gb Ethernet NOTICE: ixgbe1 registered NOTICE: ixgbe1: Intel 10Gb Ethernet NOTICE: ixgbe4 link up, 10000 Mbps, full duplex NOTICE: ixgbe0 link up, 10000 Mbps, full duplex NOTICE: ixgbe1 link up, 10000 Mbps, full duplex NOTICE: ixgbe3 link up, 1000 Mbps, full duplex panic[cpu5]/thread=ffffd06319739440: BAD TRAP: type=e (#pf Page fault) rp=ffffd001ed1da870 addr=20 occurred in module "zfs" due to a NULL pointer dereference fio: #pf Page fault Bad kernel fault at addr=0x20 pid=652, pc=0xfffffffff79e634b, sp=0xffffd001ed1da960, eflags=0x10282 cr0: 80050033<pg,wp,ne,et,mp,pe> cr4: 3406f8<smap,smep,osxsav,xmme,fxsr,pge,mce,pae,pse,de> cr2: 20 cr3: 3dada8f000 cr8: 0 rdi: 862d8ecccf5bf6fe rsi: ffffd0646b290190 rdx: ffffd062e84c4b50 rcx: 0 r8: 165ec0e80495dc9c r9: ffffd0646b290190 rax: 5a876a rbx: 1 rbp: ffffd001ed1da970 r10: ffffd0646b290180 r11: ffffd06316114000 r12: ffffd0646b290180 r13: fffffffffbd2b0c0 r14: fffffffffbd2b080 r15: ffffd0646b386898 fsb: ffffbf7fff2c2a40 gsb: ffffd063167e7080 ds: 0 es: 0 fs: 0 gs: 0 trp: e err: 0 rip: fffffffff79e634b cs: 30 rfl: 10282 rsp: ffffd001ed1da960 ss: 38 ffffd001ed1da770 unix:real_mode_stop_cpu_stage2_end+ade4 () ffffd001ed1da860 unix:trap+12df () ffffd001ed1da870 unix:_cmntrap+e6 () ffffd001ed1da970 zfs:buf_hash_remove+3b () ffffd001ed1da9e0 zfs:arc_change_state+294 () ffffd001ed1daa60 zfs:arc_release+323 () ffffd001ed1daa90 zfs:dbuf_unoverride+6d () ffffd001ed1daac0 zfs:dbuf_redirty+24 () ffffd001ed1dab00 zfs:dmu_buf_will_dirty+62 () ffffd001ed1daba0 zfs:dmu_write_uio_dnode+6d () ffffd001ed1dabf0 zfs:dmu_write_uio_dbuf+62 () ffffd001ed1daca0 zfs:zvol_write+1dd () ffffd001ed1dacd0 genunix:cdev_write+24 () ffffd001ed1dada0 specfs:spec_write+55e () ffffd001ed1dae40 genunix:fop_write+fd () ffffd001ed1daf00 genunix:pwrite+214 () ffffd001ed1daf10 unix:brand_sys_syscall+21a () dumping to /dev/zvol/dsk/rpool/dump, offset 65536, content: kernel + curproc root@con1:/var/crash/myhost# cat 111.txt ffffd001ed1da970 buf_hash_remove+0x3b(ffffd0646b290180) ffffd001ed1da9e0 arc_change_state+0x294(fffffffffbd2b080, ffffd0646b290180, fffffffffbd2ccd0) ffffd001ed1daa60 arc_release+0x323(ffffd0646b353e78, ffffd0646b386898) ffffd001ed1daa90 dbuf_unoverride+0x6d(ffffd06319827380) ffffd001ed1daac0 dbuf_redirty+0x24(ffffd06319827380) ffffd001ed1dab00 dmu_buf_will_dirty+0x62(ffffd0646b386898, ffffd064b0d2da00) ffffd001ed1daba0 dmu_write_uio_dnode+0x6d(ffffd064b465ac70, ffffd001ed1dae80, 400, ffffd064b0d2da00) ffffd001ed1dabf0 dmu_write_uio_dbuf+0x62(ffffd064afa264b0, ffffd001ed1dae80, 400, ffffd064b0d2da00) ffffd001ed1daca0 zvol_write+0x1dd(8800000003, ffffd001ed1dae80, ffffd064af9693f8) ffffd001ed1dacd0 cdev_write+0x24(8800000003, ffffd001ed1dae80, ffffd064af9693f8) ffffd001ed1dada0 spec_write+0x55e(ffffd064b0d3ee00, ffffd001ed1dae80, 10, ffffd064af9693f8, 0) ffffd001ed1dae40 fop_write+0xfd(ffffd064b0d3ee00, ffffd001ed1dae80, 10, ffffd064af9693f8, 0) ffffd001ed1daf00 pwrite+0x214(4, 55ef80, 400, 19b800) ffffd001ed1daf10 sys_syscall+0x19f() i have panic dump if you need more info - ping me. -Igor > On Sep 11, 2017, at 2:37 PM, Igor Kozhukhov <[email protected]> wrote: > > Reproduced with iSCSI connected drives. > > DilOS (based on illumos) Version 1.3.7.202 2017-09-11 > > root@con1:~# zpool list > NAME SIZE ALLOC FREE EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT > rpool 372G 170G 202G - 1% 45% 1.00x ONLINE - > tstpool2 97.5T 6.17M 97.5T - 0% 0% 1.00x ONLINE - > tstpool3 65T 8.59G 65.0T - 0% 0% 1.00x ONLINE - > > root@con1:~# uptime > 14:11pm up 0:17, 2 users, load average: 0.01, 0.02, 0.05 > > root@con1:~# > root@con1:~# sudo zfs create -o logbias=throughput -o sync=always -V 64M > tstpool3/space4 > root@con1:~# > > root@con1:~# ls /dev/zvol/rdsk/tstpool3/space4 > /dev/zvol/rdsk/tstpool3/space4 > > root@con1:~# sudo fio --filename=/dev/zvol/rdsk/tstpool3/space4 --sync=1 > --rw=write --bs=1K --numjobs=8 --iodepth=1 --size=10MB --name=panic > panic: (g=0): rw=write, bs=1K-1K/1K-1K/1K-1K, ioengine=psync, iodepth=1 > ... > fio-2.10 > clock setaffinity failed: Operation not supported > clock setaffinity failed: No such process > clock setaffinity failed: No such process > clock setaffinity failed: No such process > clock setaffinity failed: No such process > clock setaffinity failed: Operation not supported > clock setaffinity failed: No such process > clock setaffinity failed: No such process > clock setaffinity failed: No such process > clock setaffinity failed: No such process > clock setaffinity failed: Operation not supported > clock setaffinity failed: Operation not supported > clock setaffinity failed: No such process > clock setaffinity failed: No such process > Starting 8 processes > Jobs: 8 (f=8) > > Result: con1 rebooted > dump > ls -l /var/crash/myhost > -rw-r--r-- 1 root root 2442199040 Sep 11 14:20 vmdump.11 > >> On Sep 9, 2017, at 6:31 PM, Prakash Surya <[email protected] >> <mailto:[email protected]>> wrote: >> >> perfect, thank you! >> >> — >> You are receiving this because you are subscribed to this thread. >> Reply to this email directly, view it on GitHub >> <https://github.com/openzfs/openzfs/pull/462#issuecomment-328284378>, or >> mute the thread >> <https://github.com/notifications/unsubscribe-auth/AOi0K7eOxYrsamho44fLpHeTXNZ-U7bsks5sgq9kgaJpZM4PRzfH>. >> >> openzfs-developer | Archives >> <https://openzfs.topicbox.com/groups/developer/discussions/Tffbfe5cf4d0c0904-Mfb94666e086118013e2cd115> >> | Powered by Topicbox <https://topicbox.com/> ------------------------------------------ openzfs-developer Archives: https://openzfs.topicbox.com/groups/developer/discussions/Tffbfe5cf4d0c0904-Mda250da0563555f1c36b8932 Powered by Topicbox: https://topicbox.com
