Re: zfsloader 9.0 BETA3 r225759 - i/o error - all block copies unavailable
On Oct 18, 2011, at 7:39 AM, Andriy Gapon wrote: on 18/10/2011 13:35 Henri Hennebert said the following: I upgrade another system to 9.0-RC1 and encounter the same problem, this time zfsloader do not run. After mv /mnt/boot /mnt/Boot mkdir /mnt/boot cd /mnt/Boot find . | cpio -pvdmu /mnt/boot FreeBSD boot OK [root@avoriaz zfsboottest]# ./zfsboottest /Boot/zfsloader /dev/ada0p2 /dev/ada1p2 ZFS: SPA version 28 pool: rpool config: NAME STATE rpool ONLINE mirror ONLINE ada0p2 ONLINE ada1p2 ONLINE ZFS: i/o error - all block copies unavailable can't lookup 10 minutes later: [root@avoriaz zfsboottest]# ./zfsboottest /Boot/zfsloader /dev/ada0p2 /dev/ada1p2|less ZFS: SPA version 28 pool: rpool config: NAME STATE rpool ONLINE mirror ONLINE ada0p2 ONLINE ada1p2 ONLINE blablabla it seems ok :-o and a other time: [root@avoriaz zfsboottest]# ./zfsboottest /Boot/zfsloader /dev/ada0p2 segmentation fault... Strange isn't it. I think that it would be smart to not do any filesystem modifications after the problem is detected / reproduced. Also, currently zfsboottest doesn't do much of a problem self-diagnostics, so using gdb or/and adding some printfs in the code are required to understand a nature of a problem. Like what kind of block gives an I/O error, if it actual reading that fails or checksum verification or etc, and so on. Running into the same issue with a post-RC1 kernel. 1. ZFS on root. 2. stable/9. 3. Here's my zpool status output: $ zpool status pool: sac state: ONLINE scan: none requested config: NAMESTATE READ WRITE CKSUM sac ONLINE 0 0 0 ada0p3ONLINE 0 0 0 errors: No known data errors pool: store state: ONLINE scan: none requested config: NAMESTATE READ WRITE CKSUM store ONLINE 0 0 0 mfid0p1 ONLINE 0 0 0 errors: No known data errors $ sudo ./zfsboottest /boot/zfsloader /dev/ada0p3 # Spits out the zpool status and a lot of other boot gobbledygook. If I use the new gptzfsboot, then my system becomes unbootable with the old kernel (it can't find the zpool). gpart bootcode -b /mnt2/boot/pmbr -p /mnt2/boot/gptzfsboot -i 1 ada0 It spits out I/O error - all block copies unavailable at boot; if I manually did boot kernel, then everything worked. So I figured it was an environmental issue. I changed /boot/kernel from a symlink to a kernel directory to a standard directory and things worked as expected. Your issue might be similar, but it would be nice if booting from symlinks was either fixed or enhanced to work properly with gptzfsboot. Thanks, -Garrett ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: zfsloader 9.0 BETA3 r225759 - i/o error - all block copies unavailable
On 10/06/2011 15:36, Andriy Gapon wrote: on 06/10/2011 15:30 Henri Hennebert said the following: The pool is a mirror: [root@morzine ~]# zpool status rpool pool: rpool state: ONLINE scan: scrub repaired 0 in 1h0m with 0 errors on Wed Aug 24 15:04:36 2011 config: NAMESTATE READ WRITE CKSUM rpool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 gptid/e915c6a0-fc72-11de-aa21-00e081706b68 ONLINE 0 0 0 gptid/eac8497d-fc72-11de-aa21-00e081706b68 ONLINE 0 0 0 errors: No known data errors and rpool/root is not compressed: [root@morzine ~]# zfs get compression rpool/root NAMEPROPERTY VALUE SOURCE rpool/root compression off inherited from rpool pool is v28 and filesystems are v5 No particular recipes for this environment, just a general suggestion. If you run into a situation like this again, please try to use tools/tools/zfsboottest to diagnose where exactly an error originates. I upgrade another system to 9.0-RC1 and encounter the same problem, this time zfsloader do not run. After mv /mnt/boot /mnt/Boot mkdir /mnt/boot cd /mnt/Boot find . | cpio -pvdmu /mnt/boot FreeBSD boot OK [root@avoriaz zfsboottest]# ./zfsboottest /Boot/zfsloader /dev/ada0p2 /dev/ada1p2 ZFS: SPA version 28 pool: rpool config: NAME STATE rpool ONLINE mirror ONLINE ada0p2 ONLINE ada1p2 ONLINE ZFS: i/o error - all block copies unavailable can't lookup 10 minutes later: [root@avoriaz zfsboottest]# ./zfsboottest /Boot/zfsloader /dev/ada0p2 /dev/ada1p2|less ZFS: SPA version 28 pool: rpool config: NAME STATE rpool ONLINE mirror ONLINE ada0p2 ONLINE ada1p2 ONLINE blablabla it seems ok :-o and a other time: [root@avoriaz zfsboottest]# ./zfsboottest /Boot/zfsloader /dev/ada0p2 segmentation fault... Strange isn't it. Henri ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: zfsloader 9.0 BETA3 r225759 - i/o error - all block copies unavailable
on 18/10/2011 13:35 Henri Hennebert said the following: I upgrade another system to 9.0-RC1 and encounter the same problem, this time zfsloader do not run. After mv /mnt/boot /mnt/Boot mkdir /mnt/boot cd /mnt/Boot find . | cpio -pvdmu /mnt/boot FreeBSD boot OK [root@avoriaz zfsboottest]# ./zfsboottest /Boot/zfsloader /dev/ada0p2 /dev/ada1p2 ZFS: SPA version 28 pool: rpool config: NAME STATE rpool ONLINE mirror ONLINE ada0p2 ONLINE ada1p2 ONLINE ZFS: i/o error - all block copies unavailable can't lookup 10 minutes later: [root@avoriaz zfsboottest]# ./zfsboottest /Boot/zfsloader /dev/ada0p2 /dev/ada1p2|less ZFS: SPA version 28 pool: rpool config: NAME STATE rpool ONLINE mirror ONLINE ada0p2 ONLINE ada1p2 ONLINE blablabla it seems ok :-o and a other time: [root@avoriaz zfsboottest]# ./zfsboottest /Boot/zfsloader /dev/ada0p2 segmentation fault... Strange isn't it. I think that it would be smart to not do any filesystem modifications after the problem is detected / reproduced. Also, currently zfsboottest doesn't do much of a problem self-diagnostics, so using gdb or/and adding some printfs in the code are required to understand a nature of a problem. Like what kind of block gives an I/O error, if it actual reading that fails or checksum verification or etc, and so on. -- Andriy Gapon ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: zfsloader 9.0 BETA3 r225759 - i/o error - all block copies unavailable
On 10/06/2011 16:20, Andriy Gapon wrote: on 06/10/2011 17:00 Henri Hennebert said the following: On 10/06/2011 15:36, Andriy Gapon wrote: on 06/10/2011 15:30 Henri Hennebert said the following: The pool is a mirror: [root@morzine ~]# zpool status rpool pool: rpool state: ONLINE scan: scrub repaired 0 in 1h0m with 0 errors on Wed Aug 24 15:04:36 2011 config: NAMESTATE READ WRITE CKSUM rpool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 gptid/e915c6a0-fc72-11de-aa21-00e081706b68 ONLINE 0 0 0 gptid/eac8497d-fc72-11de-aa21-00e081706b68 ONLINE 0 0 0 errors: No known data errors and rpool/root is not compressed: [root@morzine ~]# zfs get compression rpool/root NAMEPROPERTY VALUE SOURCE rpool/root compression off inherited from rpool pool is v28 and filesystems are v5 No particular recipes for this environment, just a general suggestion. If you run into a situation like this again, please try to use tools/tools/zfsboottest to diagnose where exactly an error originates. I try [ please note _M_enu.rc ]: [root@morzine ~]# /usr/obj/usr/src/tools/tools/zfsboottest/zfsboottest /boot/Menu.rc /dev/da0p2 /dev/da1p2 ZFS: SPA version 28 pool: rpool config: NAME STATE rpool ONLINE mirror ONLINE gptid/e915c6a0-fc72-11de-aa21-00e081706b68 ONLINE gptid/eac8497d-fc72-11de-aa21-00e081706b68 ONLINE \ Menu.rc \ $FreeBSD: head/sys/boot/forth/menu.rc 222417 2011-05-28 08:50:38Z julian $ \ \ Load required Forth modules include /boot/version.4th include /boot/brand.4th include /boot/menu.4th include /boot/menu-commands.4th include /boot/shortcuts.4th \ Screen prep clear \ clear the screen (see `screen.4th') print_version \ print version string (bottom-right; see `version.4th') draw-beastie \ draw freebsd mascot (on right; see `beastie.4th') draw-brand\ draw the FreeBSD title (top-left; see `brand.4th') menu-init \ initialize the menu area (see `menu.4th') \ Initialize main menu constructs (see `menu.4th') \ NOTE: To use the `ansi' variants, add `loader_color=1' to loader.conf(5) clip set menu_timeout_command=boot \ Display the main menu (see `menu.4th') menu-display [root@morzine ~] The line `ZFS: SPA version 28' come from my local patch: Index: sys/boot/zfs/zfsimpl.c === --- sys/boot/zfs/zfsimpl.c(revision 225759) +++ sys/boot/zfs/zfsimpl.c(working copy) @@ -63,6 +63,8 @@ STAILQ_INIT(zfs_vdevs); STAILQ_INIT(zfs_pools); +printf(ZFS: SPA version %u\n, (unsigned) SPA_VERSION); + zfs_temp_buf = malloc(TEMP_SIZE); zfs_temp_end = zfs_temp_buf + TEMP_SIZE; zfs_temp_ptr = zfs_temp_buf; Is it what you sugest ? Yes. And this report indicates that the boot code (built from your source tree) should be able to read that file. I do: mv /boot/Menu.rc /boot/menu.rc and reboot. The /boot/menu.rc can be read by zfsloader so I conclude that it was the directory entry of /boot/menu.rc thas has a problem in the first place. Next time it happen I will directly use zfsboottest before any update to the pool. Thank for your time! Henri ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
zfsloader 9.0 BETA3 r225759 - i/o error - all block copies unavailable
Hello all, I upgrade from 9.0-BETA2 to 9.0-BETA3 (r225759) and when booting from a zpool I get: ZFS: i/o error - all block copies unavailable can't open '/boot/menu.rc': no such file or directory. I pxe boot mfsbsd 8.2-RELEASE + zfs v28 then: mkdir /rpool zpool import -R /rpool rpool mount -t zfs rpool/boot /mnt mv /mnt/boot/menu.rc /mnt/boot/Menu.rc cp /mnt/boot/Menu.rc /mnt/boot/menu.rc umount /mnt shutdown -r now and the next boot run smoothly... PS. I have to boot from pxe because after the i/o error, the kernel was booted but encounter a double fault. I try 5 times with the same result. I think that on my configuration only the kernel without ACPI encouter this trap. Henri ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: zfsloader 9.0 BETA3 r225759 - i/o error - all block copies unavailable
on 06/10/2011 12:25 Henri Hennebert said the following: Hello all, I upgrade from 9.0-BETA2 to 9.0-BETA3 (r225759) and when booting from a zpool I get: ZFS: i/o error - all block copies unavailable can't open '/boot/menu.rc': no such file or directory. I pxe boot mfsbsd 8.2-RELEASE + zfs v28 then: mkdir /rpool zpool import -R /rpool rpool mount -t zfs rpool/boot /mnt mv /mnt/boot/menu.rc /mnt/boot/Menu.rc cp /mnt/boot/Menu.rc /mnt/boot/menu.rc umount /mnt shutdown -r now and the next boot run smoothly... Does your root fs have compression enabled? PS. I have to boot from pxe because after the i/o error, the kernel was booted but encounter a double fault. I try 5 times with the same result. I think that on my configuration only the kernel without ACPI encouter this trap. I think that you should try to gather and report more information about the panic (separately). -- Andriy Gapon ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: zfsloader 9.0 BETA3 r225759 - i/o error - all block copies unavailable
On 10/06/2011 12:44, Andriy Gapon wrote: on 06/10/2011 12:25 Henri Hennebert said the following: Hello all, I upgrade from 9.0-BETA2 to 9.0-BETA3 (r225759) and when booting from a zpool I get: ZFS: i/o error - all block copies unavailable can't open '/boot/menu.rc': no such file or directory. I pxe boot mfsbsd 8.2-RELEASE + zfs v28 then: mkdir /rpool zpool import -R /rpool rpool mount -t zfs rpool/boot /mnt mv /mnt/boot/menu.rc /mnt/boot/Menu.rc cp /mnt/boot/Menu.rc /mnt/boot/menu.rc umount /mnt shutdown -r now and the next boot run smoothly... Does your root fs have compression enabled? The pool is a mirror: [root@morzine ~]# zpool status rpool pool: rpool state: ONLINE scan: scrub repaired 0 in 1h0m with 0 errors on Wed Aug 24 15:04:36 2011 config: NAMESTATE READ WRITE CKSUM rpool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 gptid/e915c6a0-fc72-11de-aa21-00e081706b68 ONLINE 0 0 0 gptid/eac8497d-fc72-11de-aa21-00e081706b68 ONLINE 0 0 0 errors: No known data errors and rpool/root is not compressed: [root@morzine ~]# zfs get compression rpool/root NAMEPROPERTY VALUE SOURCE rpool/root compression off inherited from rpool pool is v28 and filesystems are v5 PS. I have to boot from pxe because after the i/o error, the kernel was booted but encounter a double fault. I try 5 times with the same result. I think that on my configuration only the kernel without ACPI encouter this trap. I think that you should try to gather and report more information about the panic (separately). I'll try this saterday ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: zfsloader 9.0 BETA3 r225759 - i/o error - all block copies unavailable
on 06/10/2011 15:30 Henri Hennebert said the following: The pool is a mirror: [root@morzine ~]# zpool status rpool pool: rpool state: ONLINE scan: scrub repaired 0 in 1h0m with 0 errors on Wed Aug 24 15:04:36 2011 config: NAMESTATE READ WRITE CKSUM rpool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 gptid/e915c6a0-fc72-11de-aa21-00e081706b68 ONLINE 0 0 0 gptid/eac8497d-fc72-11de-aa21-00e081706b68 ONLINE 0 0 0 errors: No known data errors and rpool/root is not compressed: [root@morzine ~]# zfs get compression rpool/root NAMEPROPERTY VALUE SOURCE rpool/root compression off inherited from rpool pool is v28 and filesystems are v5 No particular recipes for this environment, just a general suggestion. If you run into a situation like this again, please try to use tools/tools/zfsboottest to diagnose where exactly an error originates. -- Andriy Gapon ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: zfsloader 9.0 BETA3 r225759 - i/o error - all block copies unavailable
On 10/06/2011 15:36, Andriy Gapon wrote: on 06/10/2011 15:30 Henri Hennebert said the following: The pool is a mirror: [root@morzine ~]# zpool status rpool pool: rpool state: ONLINE scan: scrub repaired 0 in 1h0m with 0 errors on Wed Aug 24 15:04:36 2011 config: NAMESTATE READ WRITE CKSUM rpool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 gptid/e915c6a0-fc72-11de-aa21-00e081706b68 ONLINE 0 0 0 gptid/eac8497d-fc72-11de-aa21-00e081706b68 ONLINE 0 0 0 errors: No known data errors and rpool/root is not compressed: [root@morzine ~]# zfs get compression rpool/root NAMEPROPERTY VALUE SOURCE rpool/root compression off inherited from rpool pool is v28 and filesystems are v5 No particular recipes for this environment, just a general suggestion. If you run into a situation like this again, please try to use tools/tools/zfsboottest to diagnose where exactly an error originates. I try [ please note _M_enu.rc ]: [root@morzine ~]# /usr/obj/usr/src/tools/tools/zfsboottest/zfsboottest /boot/Menu.rc /dev/da0p2 /dev/da1p2 ZFS: SPA version 28 pool: rpool config: NAME STATE rpool ONLINE mirror ONLINE gptid/e915c6a0-fc72-11de-aa21-00e081706b68 ONLINE gptid/eac8497d-fc72-11de-aa21-00e081706b68 ONLINE \ Menu.rc \ $FreeBSD: head/sys/boot/forth/menu.rc 222417 2011-05-28 08:50:38Z julian $ \ \ Load required Forth modules include /boot/version.4th include /boot/brand.4th include /boot/menu.4th include /boot/menu-commands.4th include /boot/shortcuts.4th \ Screen prep clear \ clear the screen (see `screen.4th') print_version \ print version string (bottom-right; see `version.4th') draw-beastie \ draw freebsd mascot (on right; see `beastie.4th') draw-brand\ draw the FreeBSD title (top-left; see `brand.4th') menu-init \ initialize the menu area (see `menu.4th') \ Initialize main menu constructs (see `menu.4th') \ NOTE: To use the `ansi' variants, add `loader_color=1' to loader.conf(5) clip set menu_timeout_command=boot \ Display the main menu (see `menu.4th') menu-display [root@morzine ~] The line `ZFS: SPA version 28' come from my local patch: Index: sys/boot/zfs/zfsimpl.c === --- sys/boot/zfs/zfsimpl.c (revision 225759) +++ sys/boot/zfs/zfsimpl.c (working copy) @@ -63,6 +63,8 @@ STAILQ_INIT(zfs_vdevs); STAILQ_INIT(zfs_pools); + printf(ZFS: SPA version %u\n, (unsigned) SPA_VERSION); + zfs_temp_buf = malloc(TEMP_SIZE); zfs_temp_end = zfs_temp_buf + TEMP_SIZE; zfs_temp_ptr = zfs_temp_buf; Is it what you sugest ? ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: zfsloader 9.0 BETA3 r225759 - i/o error - all block copies unavailable
on 06/10/2011 17:00 Henri Hennebert said the following: On 10/06/2011 15:36, Andriy Gapon wrote: on 06/10/2011 15:30 Henri Hennebert said the following: The pool is a mirror: [root@morzine ~]# zpool status rpool pool: rpool state: ONLINE scan: scrub repaired 0 in 1h0m with 0 errors on Wed Aug 24 15:04:36 2011 config: NAMESTATE READ WRITE CKSUM rpool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 gptid/e915c6a0-fc72-11de-aa21-00e081706b68 ONLINE 0 0 0 gptid/eac8497d-fc72-11de-aa21-00e081706b68 ONLINE 0 0 0 errors: No known data errors and rpool/root is not compressed: [root@morzine ~]# zfs get compression rpool/root NAMEPROPERTY VALUE SOURCE rpool/root compression off inherited from rpool pool is v28 and filesystems are v5 No particular recipes for this environment, just a general suggestion. If you run into a situation like this again, please try to use tools/tools/zfsboottest to diagnose where exactly an error originates. I try [ please note _M_enu.rc ]: [root@morzine ~]# /usr/obj/usr/src/tools/tools/zfsboottest/zfsboottest /boot/Menu.rc /dev/da0p2 /dev/da1p2 ZFS: SPA version 28 pool: rpool config: NAME STATE rpool ONLINE mirror ONLINE gptid/e915c6a0-fc72-11de-aa21-00e081706b68 ONLINE gptid/eac8497d-fc72-11de-aa21-00e081706b68 ONLINE \ Menu.rc \ $FreeBSD: head/sys/boot/forth/menu.rc 222417 2011-05-28 08:50:38Z julian $ \ \ Load required Forth modules include /boot/version.4th include /boot/brand.4th include /boot/menu.4th include /boot/menu-commands.4th include /boot/shortcuts.4th \ Screen prep clear \ clear the screen (see `screen.4th') print_version \ print version string (bottom-right; see `version.4th') draw-beastie \ draw freebsd mascot (on right; see `beastie.4th') draw-brand\ draw the FreeBSD title (top-left; see `brand.4th') menu-init \ initialize the menu area (see `menu.4th') \ Initialize main menu constructs (see `menu.4th') \ NOTE: To use the `ansi' variants, add `loader_color=1' to loader.conf(5) clip set menu_timeout_command=boot \ Display the main menu (see `menu.4th') menu-display [root@morzine ~] The line `ZFS: SPA version 28' come from my local patch: Index: sys/boot/zfs/zfsimpl.c === --- sys/boot/zfs/zfsimpl.c(revision 225759) +++ sys/boot/zfs/zfsimpl.c(working copy) @@ -63,6 +63,8 @@ STAILQ_INIT(zfs_vdevs); STAILQ_INIT(zfs_pools); +printf(ZFS: SPA version %u\n, (unsigned) SPA_VERSION); + zfs_temp_buf = malloc(TEMP_SIZE); zfs_temp_end = zfs_temp_buf + TEMP_SIZE; zfs_temp_ptr = zfs_temp_buf; Is it what you sugest ? Yes. And this report indicates that the boot code (built from your source tree) should be able to read that file. -- Andriy Gapon ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
