[Kernel-packages] [Bug 1881810] Re: ZFS import cause panic
** Changed in: zfs-linux (Ubuntu) Assignee: Dimitri John Ledkov (xnox) => (unassigned) ** Changed in: zfs-linux (Ubuntu) Status: Triaged => Invalid -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to zfs-linux in Ubuntu. https://bugs.launchpad.net/bugs/1881810 Title: ZFS import cause panic Status in zfs-linux package in Ubuntu: Invalid Bug description: 1) Ubuntu 20.04 LTS 2) apt policy zfsutils-linux zfsutils-linux: Installed: 0.8.3-1ubuntu12 Candidate:0.8.3-1ubuntu12 3) ZFS pool not possible to import, unless using read only. If trying to import normal way as write/read, casing panic message Expected to be able to import zpool without panic. Expected that scrub should not be able to run in read only. Expected that somehow be able to restore the zpool somehow. 4) # zpool status no pools available # zpool import pool: datastore id: 3190464655986727485 state: ONLINE status: Some supported features are not enabled on the pool. action: The pool can be imported using its name or numeric identifier, though some features will not be available without an explicit 'zpool upgrade'. config: datastore ONLINE raidz2-0 ONLINE sdb ONLINE sdc ONLINE sdd ONLINE sde ONLINE sdf ONLINE sdg ONLINE # zpool import -o readonly=on datastore # zpool status pool: datastore state: ONLINE status: Some supported features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(5) for details. scan: scrub in progress since Sun May 10 00:24:01 2020 15,1T scanned at 0B/s, 15,1T issued at 0B/s, 15,1T total 1,44M repaired, 100,01% done, no estimated completion time config: NAMESTATE READ WRITE CKSUM datastore ONLINE 0 0 0 raidz2-0 ONLINE 0 0 0 sdb ONLINE 0 0 0 sdc ONLINE 0 0 0 sdd ONLINE 0 0 0 sde ONLINE 0 0 0 sdf ONLINE 0 0 0 sdg ONLINE 0 0 0 errors: No known data errors Running zdb it always stops ad 2,23T and then aborts. # zdb -e -bcsvL datastore Traversing all blocks to verify checksums ... 2.23T completed ( 149MB/s) estimated time remaining: 25hr 06min 25sec free(): invalid pointer Abort (SIGABRT) #zpool import datastore VERIFY3(c < SPA_MAXBLOCKSIZE >> SPA_MINBLOCKSHIFT) failed (36028797018963967 < 32768) PANIC at zio.c:293:zio_buf_alloc() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/1881810/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1881810] Re: ZFS import cause panic
** Changed in: zfs-linux (Ubuntu) Assignee: Colin Ian King (colin-king) => Dimitri John Ledkov (xnox) -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to zfs-linux in Ubuntu. https://bugs.launchpad.net/bugs/1881810 Title: ZFS import cause panic Status in zfs-linux package in Ubuntu: Triaged Bug description: 1) Ubuntu 20.04 LTS 2) apt policy zfsutils-linux zfsutils-linux: Installed: 0.8.3-1ubuntu12 Candidate:0.8.3-1ubuntu12 3) ZFS pool not possible to import, unless using read only. If trying to import normal way as write/read, casing panic message Expected to be able to import zpool without panic. Expected that scrub should not be able to run in read only. Expected that somehow be able to restore the zpool somehow. 4) # zpool status no pools available # zpool import pool: datastore id: 3190464655986727485 state: ONLINE status: Some supported features are not enabled on the pool. action: The pool can be imported using its name or numeric identifier, though some features will not be available without an explicit 'zpool upgrade'. config: datastore ONLINE raidz2-0 ONLINE sdb ONLINE sdc ONLINE sdd ONLINE sde ONLINE sdf ONLINE sdg ONLINE # zpool import -o readonly=on datastore # zpool status pool: datastore state: ONLINE status: Some supported features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(5) for details. scan: scrub in progress since Sun May 10 00:24:01 2020 15,1T scanned at 0B/s, 15,1T issued at 0B/s, 15,1T total 1,44M repaired, 100,01% done, no estimated completion time config: NAMESTATE READ WRITE CKSUM datastore ONLINE 0 0 0 raidz2-0 ONLINE 0 0 0 sdb ONLINE 0 0 0 sdc ONLINE 0 0 0 sdd ONLINE 0 0 0 sde ONLINE 0 0 0 sdf ONLINE 0 0 0 sdg ONLINE 0 0 0 errors: No known data errors Running zdb it always stops ad 2,23T and then aborts. # zdb -e -bcsvL datastore Traversing all blocks to verify checksums ... 2.23T completed ( 149MB/s) estimated time remaining: 25hr 06min 25sec free(): invalid pointer Abort (SIGABRT) #zpool import datastore VERIFY3(c < SPA_MAXBLOCKSIZE >> SPA_MINBLOCKSHIFT) failed (36028797018963967 < 32768) PANIC at zio.c:293:zio_buf_alloc() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/1881810/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1881810] Re: ZFS import cause panic
I does seem that something on disc is corrupted. I'm still trying to find any viable solutions. -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to zfs-linux in Ubuntu. https://bugs.launchpad.net/bugs/1881810 Title: ZFS import cause panic Status in zfs-linux package in Ubuntu: Triaged Bug description: 1) Ubuntu 20.04 LTS 2) apt policy zfsutils-linux zfsutils-linux: Installed: 0.8.3-1ubuntu12 Candidate:0.8.3-1ubuntu12 3) ZFS pool not possible to import, unless using read only. If trying to import normal way as write/read, casing panic message Expected to be able to import zpool without panic. Expected that scrub should not be able to run in read only. Expected that somehow be able to restore the zpool somehow. 4) # zpool status no pools available # zpool import pool: datastore id: 3190464655986727485 state: ONLINE status: Some supported features are not enabled on the pool. action: The pool can be imported using its name or numeric identifier, though some features will not be available without an explicit 'zpool upgrade'. config: datastore ONLINE raidz2-0 ONLINE sdb ONLINE sdc ONLINE sdd ONLINE sde ONLINE sdf ONLINE sdg ONLINE # zpool import -o readonly=on datastore # zpool status pool: datastore state: ONLINE status: Some supported features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(5) for details. scan: scrub in progress since Sun May 10 00:24:01 2020 15,1T scanned at 0B/s, 15,1T issued at 0B/s, 15,1T total 1,44M repaired, 100,01% done, no estimated completion time config: NAMESTATE READ WRITE CKSUM datastore ONLINE 0 0 0 raidz2-0 ONLINE 0 0 0 sdb ONLINE 0 0 0 sdc ONLINE 0 0 0 sdd ONLINE 0 0 0 sde ONLINE 0 0 0 sdf ONLINE 0 0 0 sdg ONLINE 0 0 0 errors: No known data errors Running zdb it always stops ad 2,23T and then aborts. # zdb -e -bcsvL datastore Traversing all blocks to verify checksums ... 2.23T completed ( 149MB/s) estimated time remaining: 25hr 06min 25sec free(): invalid pointer Abort (SIGABRT) #zpool import datastore VERIFY3(c < SPA_MAXBLOCKSIZE >> SPA_MINBLOCKSHIFT) failed (36028797018963967 < 32768) PANIC at zio.c:293:zio_buf_alloc() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/1881810/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1881810] Re: ZFS import cause panic
Thanks for you fast reply. This is the same behaviour from bug: 1880580, that I wrote. I have updated and can only continue verify on the latest 20.04, not 18.04 anymore. This is what I have done so far now. I tried you suggested solution. I first had to remove /etc/zfs/zpool.cache in order to be able to do this. Then I tried your solution 2 times, once with no flags, and then with -f flag. The panic still occurs when doing this. Should I try zdb again? Any other suggestions ? # cat /sys/module/zfs/parameters/spa_load_verify_metadata 1 # echo 0 | sudo tee /sys/module/zfs/parameters/spa_load_verify_metadata 0 # echo 0 | sudo tee /sys/module/zfs/parameters/spa_load_verify_data 0 # zpool import pool: datastore id: 3190464655986727485 state: ONLINE status: Some supported features are not enabled on the pool. action: The pool can be imported using its name or numeric identifier, though some features will not be available without an explicit 'zpool upgrade'. config: datastore ONLINE raidz2-0 ONLINE sdb ONLINE sdc ONLINE sdd ONLINE sde ONLINE sdf ONLINE sdg ONLINE # zpool import datastore or # zpool import -f datastore Cause this output: VERIFY3(c < SPA_MAXBLOCKSIZE >> SPA_MINBLOCKSHIFT) failed (...) PANIC at zio.c:293:zio_buf_alloc() -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to zfs-linux in Ubuntu. https://bugs.launchpad.net/bugs/1881810 Title: ZFS import cause panic Status in zfs-linux package in Ubuntu: Triaged Bug description: 1) Ubuntu 20.04 LTS 2) apt policy zfsutils-linux zfsutils-linux: Installed: 0.8.3-1ubuntu12 Candidate:0.8.3-1ubuntu12 3) ZFS pool not possible to import, unless using read only. If trying to import normal way as write/read, casing panic message Expected to be able to import zpool without panic. Expected that scrub should not be able to run in read only. Expected that somehow be able to restore the zpool somehow. 4) # zpool status no pools available # zpool import pool: datastore id: 3190464655986727485 state: ONLINE status: Some supported features are not enabled on the pool. action: The pool can be imported using its name or numeric identifier, though some features will not be available without an explicit 'zpool upgrade'. config: datastore ONLINE raidz2-0 ONLINE sdb ONLINE sdc ONLINE sdd ONLINE sde ONLINE sdf ONLINE sdg ONLINE # zpool import -o readonly=on datastore # zpool status pool: datastore state: ONLINE status: Some supported features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(5) for details. scan: scrub in progress since Sun May 10 00:24:01 2020 15,1T scanned at 0B/s, 15,1T issued at 0B/s, 15,1T total 1,44M repaired, 100,01% done, no estimated completion time config: NAMESTATE READ WRITE CKSUM datastore ONLINE 0 0 0 raidz2-0 ONLINE 0 0 0 sdb ONLINE 0 0 0 sdc ONLINE 0 0 0 sdd ONLINE 0 0 0 sde ONLINE 0 0 0 sdf ONLINE 0 0 0 sdg ONLINE 0 0 0 errors: No known data errors Running zdb it always stops ad 2,23T and then aborts. # zdb -e -bcsvL datastore Traversing all blocks to verify checksums ... 2.23T completed ( 149MB/s) estimated time remaining: 25hr 06min 25sec free(): invalid pointer Abort (SIGABRT) #zpool import datastore VERIFY3(c < SPA_MAXBLOCKSIZE >> SPA_MINBLOCKSHIFT) failed (36028797018963967 < 32768) PANIC at zio.c:293:zio_buf_alloc() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/1881810/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1881810] Re: ZFS import cause panic
Looks like you may have data corruption. One can temporarily disable the spa verification to see if this allows the import to at least complete to see if this helps. echo 0 | sudo tee /sys/module/zfs/parameters/spa_load_verify_metadata echo 0 | sudo tee /sys/module/zfs/parameters/spa_load_verify_data do the import, and then re-enable: echo 1 | sudo tee /sys/module/zfs/parameters/spa_load_verify_metadata echo 1 | sudo tee /sys/module/zfs/parameters/spa_load_verify_data -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to zfs-linux in Ubuntu. https://bugs.launchpad.net/bugs/1881810 Title: ZFS import cause panic Status in zfs-linux package in Ubuntu: Triaged Bug description: 1) Ubuntu 20.04 LTS 2) apt policy zfsutils-linux zfsutils-linux: Installed: 0.8.3-1ubuntu12 Candidate:0.8.3-1ubuntu12 3) ZFS pool not possible to import, unless using read only. If trying to import normal way as write/read, casing panic message Expected to be able to import zpool without panic. Expected that scrub should not be able to run in read only. Expected that somehow be able to restore the zpool somehow. 4) # zpool status no pools available # zpool import pool: datastore id: 3190464655986727485 state: ONLINE status: Some supported features are not enabled on the pool. action: The pool can be imported using its name or numeric identifier, though some features will not be available without an explicit 'zpool upgrade'. config: datastore ONLINE raidz2-0 ONLINE sdb ONLINE sdc ONLINE sdd ONLINE sde ONLINE sdf ONLINE sdg ONLINE # zpool import -o readonly=on datastore # zpool status pool: datastore state: ONLINE status: Some supported features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(5) for details. scan: scrub in progress since Sun May 10 00:24:01 2020 15,1T scanned at 0B/s, 15,1T issued at 0B/s, 15,1T total 1,44M repaired, 100,01% done, no estimated completion time config: NAMESTATE READ WRITE CKSUM datastore ONLINE 0 0 0 raidz2-0 ONLINE 0 0 0 sdb ONLINE 0 0 0 sdc ONLINE 0 0 0 sdd ONLINE 0 0 0 sde ONLINE 0 0 0 sdf ONLINE 0 0 0 sdg ONLINE 0 0 0 errors: No known data errors Running zdb it always stops ad 2,23T and then aborts. # zdb -e -bcsvL datastore Traversing all blocks to verify checksums ... 2.23T completed ( 149MB/s) estimated time remaining: 25hr 06min 25sec free(): invalid pointer Abort (SIGABRT) #zpool import datastore VERIFY3(c < SPA_MAXBLOCKSIZE >> SPA_MINBLOCKSHIFT) failed (36028797018963967 < 32768) PANIC at zio.c:293:zio_buf_alloc() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/1881810/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1881810] Re: ZFS import cause panic
Added dmesg output ** Attachment added: "dmesg log" https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/1881810/+attachment/5380764/+files/dm.log -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to zfs-linux in Ubuntu. https://bugs.launchpad.net/bugs/1881810 Title: ZFS import cause panic Status in zfs-linux package in Ubuntu: Triaged Bug description: 1) Ubuntu 20.04 LTS 2) apt policy zfsutils-linux zfsutils-linux: Installed: 0.8.3-1ubuntu12 Candidate:0.8.3-1ubuntu12 3) ZFS pool not possible to import, unless using read only. If trying to import normal way as write/read, casing panic message Expected to be able to import zpool without panic. Expected that scrub should not be able to run in read only. Expected that somehow be able to restore the zpool somehow. 4) # zpool status no pools available # zpool import pool: datastore id: 3190464655986727485 state: ONLINE status: Some supported features are not enabled on the pool. action: The pool can be imported using its name or numeric identifier, though some features will not be available without an explicit 'zpool upgrade'. config: datastore ONLINE raidz2-0 ONLINE sdb ONLINE sdc ONLINE sdd ONLINE sde ONLINE sdf ONLINE sdg ONLINE # zpool import -o readonly=on datastore # zpool status pool: datastore state: ONLINE status: Some supported features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(5) for details. scan: scrub in progress since Sun May 10 00:24:01 2020 15,1T scanned at 0B/s, 15,1T issued at 0B/s, 15,1T total 1,44M repaired, 100,01% done, no estimated completion time config: NAMESTATE READ WRITE CKSUM datastore ONLINE 0 0 0 raidz2-0 ONLINE 0 0 0 sdb ONLINE 0 0 0 sdc ONLINE 0 0 0 sdd ONLINE 0 0 0 sde ONLINE 0 0 0 sdf ONLINE 0 0 0 sdg ONLINE 0 0 0 errors: No known data errors Running zdb it always stops ad 2,23T and then aborts. # zdb -e -bcsvL datastore Traversing all blocks to verify checksums ... 2.23T completed ( 149MB/s) estimated time remaining: 25hr 06min 25sec free(): invalid pointer Abort (SIGABRT) #zpool import datastore VERIFY3(c < SPA_MAXBLOCKSIZE >> SPA_MINBLOCKSHIFT) failed (36028797018963967 < 32768) PANIC at zio.c:293:zio_buf_alloc() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/1881810/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
[Kernel-packages] [Bug 1881810] Re: ZFS import cause panic
Can you attach to the bug report the full output from dmesg, the full panic message is more instructive when trying to debug this. ** Changed in: zfs-linux (Ubuntu) Status: New => In Progress ** Changed in: zfs-linux (Ubuntu) Importance: Undecided => High ** Changed in: zfs-linux (Ubuntu) Assignee: (unassigned) => Colin Ian King (colin-king) ** Changed in: zfs-linux (Ubuntu) Status: In Progress => Triaged -- You received this bug notification because you are a member of Kernel Packages, which is subscribed to zfs-linux in Ubuntu. https://bugs.launchpad.net/bugs/1881810 Title: ZFS import cause panic Status in zfs-linux package in Ubuntu: Triaged Bug description: 1) Ubuntu 20.04 LTS 2) apt policy zfsutils-linux zfsutils-linux: Installed: 0.8.3-1ubuntu12 Candidate:0.8.3-1ubuntu12 3) ZFS pool not possible to import, unless using read only. If trying to import normal way as write/read, casing panic message Expected to be able to import zpool without panic. Expected that scrub should not be able to run in read only. Expected that somehow be able to restore the zpool somehow. 4) # zpool status no pools available # zpool import pool: datastore id: 3190464655986727485 state: ONLINE status: Some supported features are not enabled on the pool. action: The pool can be imported using its name or numeric identifier, though some features will not be available without an explicit 'zpool upgrade'. config: datastore ONLINE raidz2-0 ONLINE sdb ONLINE sdc ONLINE sdd ONLINE sde ONLINE sdf ONLINE sdg ONLINE # zpool import -o readonly=on datastore # zpool status pool: datastore state: ONLINE status: Some supported features are not enabled on the pool. The pool can still be used, but some features are unavailable. action: Enable all features using 'zpool upgrade'. Once this is done, the pool may no longer be accessible by software that does not support the features. See zpool-features(5) for details. scan: scrub in progress since Sun May 10 00:24:01 2020 15,1T scanned at 0B/s, 15,1T issued at 0B/s, 15,1T total 1,44M repaired, 100,01% done, no estimated completion time config: NAMESTATE READ WRITE CKSUM datastore ONLINE 0 0 0 raidz2-0 ONLINE 0 0 0 sdb ONLINE 0 0 0 sdc ONLINE 0 0 0 sdd ONLINE 0 0 0 sde ONLINE 0 0 0 sdf ONLINE 0 0 0 sdg ONLINE 0 0 0 errors: No known data errors Running zdb it always stops ad 2,23T and then aborts. # zdb -e -bcsvL datastore Traversing all blocks to verify checksums ... 2.23T completed ( 149MB/s) estimated time remaining: 25hr 06min 25sec free(): invalid pointer Abort (SIGABRT) #zpool import datastore VERIFY3(c < SPA_MAXBLOCKSIZE >> SPA_MINBLOCKSHIFT) failed (36028797018963967 < 32768) PANIC at zio.c:293:zio_buf_alloc() To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/1881810/+subscriptions -- Mailing list: https://launchpad.net/~kernel-packages Post to : kernel-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~kernel-packages More help : https://help.launchpad.net/ListHelp
