Re: ZFS: i/o error - all block copies unavailable after upgrading to r225312

2011-09-16 Thread Andriy Gapon

I've discovered another issue with both the current code and the previous
versions of the patch.  So another version is at the same URL:
http://people.freebsd.org/~avg/zfs-boot-gang.diff

The problem was with gang blocks on a raidz vdev.  zio_read_gang would pass a
NULL bp to vdev_raidz_read, but that function doesn't expect that and really
needs a valid bp.

Pawel, I also understood the code in zio_read that does size rounding up based
on v_ashift.  That code is for vdev_raidz_read which needs the size to be
multiple of 1  v_ashift.  And, again, this would have posed a problem for
zio_read_gang, which always passed SPA_GANGBLOCKSIZE when reading a gang block
header.

So, a list of fixes and logical changes:
- correctly read gang header from raidz [*]
- decompress assembled gang block data if compressed
- verify checksum of a gang header
- verify checksum of assembled gang block data
- verify checksum of uber block

[*] - new in this version of patch.

Description of the code changes:
- remove offset parameter from zio_checksum_error
This parameter has only been used for the case of verifying checksum of a vdev
label.  The offset is now passed via DVA offset field in a made up bp pointing
to the label.
zio_checksum_error now gets all checksum parameters from the bp.

- zio_read_gang new uses an artificial bp to read a gang header via zio_read
This solves all the problems with gang blocks on raidz as zio_read already has
all the code to handle raidz correctly.

- zio_read performs size rounding only if v_read == vdev_raidz_read
This is to make the intention of the code more clear.  And also to slightly
optimize non-raidz cases with non-default ashift where an unnecessary
intermediate buffer would otherwise be used.

Some inline comments (marked with %):

Index: sys/cddl/boot/zfs/zfssubr.c
===
--- sys/cddl/boot/zfs/zfssubr.c (revision 225581)
+++ sys/cddl/boot/zfs/zfssubr.c (working copy)
@@ -181,14 +181,17 @@
 }

 static int
-zio_checksum_error(const blkptr_t *bp, void *data, uint64_t offset)
+zio_checksum_error(const blkptr_t *bp, void *data)
 {
-   unsigned int checksum = BP_IS_GANG(bp) ? ZIO_CHECKSUM_GANG_HEADER :
BP_GET_CHECKSUM(bp);
-   uint64_t size = BP_GET_PSIZE(bp);
+   uint64_t size;
+   unsigned int checksum;
zio_checksum_info_t *ci;
zio_cksum_t actual_cksum, expected_cksum, verifier;
int byteswap;

+   checksum = BP_GET_CHECKSUM(bp);
+   size = BP_GET_PSIZE(bp);
+
%
% checksum type and size are always taken from the bp.
% Note that BP_IS_GANG(bp) doesn't imply that the caller wants to verify
% the gang header (as was assumed before); the caller want want to verify
% the whole assembled data as well.
%
if (checksum = ZIO_CHECKSUM_FUNCTIONS)
return (EINVAL);
ci = zio_checksum_table[checksum];
@@ -206,7 +209,8 @@
if (checksum == ZIO_CHECKSUM_GANG_HEADER)
zio_checksum_gang_verifier(verifier, bp);
else if (checksum == ZIO_CHECKSUM_LABEL)
-   zio_checksum_label_verifier(verifier, offset);
+   zio_checksum_label_verifier(verifier,
+   DVA_GET_OFFSET(BP_IDENTITY(bp)));
%
% label offset is taken from the bp now.
%
else
verifier = bp-blk_cksum;

@@ -224,7 +228,6 @@
byteswap_uint64_array(expected_cksum,
sizeof (zio_cksum_t));
} else {
-   ASSERT(!BP_IS_GANG(bp));
%
% the assert is no longer valid as we pass a gang block bp
% when verifying checksum of assembled data
%
expected_cksum = bp-blk_cksum;
ci-ci_func[0](data, size, actual_cksum);
}
@@ -1248,7 +1251,7 @@
 raidz_checksum_verify(const blkptr_t *bp, void *data)
 {

-   return (zio_checksum_error(bp, data, 0));
+   return (zio_checksum_error(bp, data));
 }

 /*
Index: sys/boot/zfs/zfsimpl.c
===
--- sys/boot/zfs/zfsimpl.c  (revision 225581)
+++ sys/boot/zfs/zfsimpl.c  (working copy)
@@ -347,7 +347,7 @@
rc = vdev-v_phys_read(vdev, vdev-v_read_priv, offset, buf, psize);
if (rc)
return (rc);
-   if (bp  zio_checksum_error(bp, buf, offset))
+   if (bp  zio_checksum_error(bp, buf))
%
% the bp can still be NULL here: when raidz code reads blocks of data
% it doesn't pass any bp, because there is really no bp for that data.
% this should be the only case, in all other cases a valid bp must be provided.
%
return (EIO);

return (0);
@@ -800,6 +800,7 @@
BP_SET_PSIZE(bp, sizeof(vdev_phys_t));
BP_SET_CHECKSUM(bp, ZIO_CHECKSUM_LABEL);
BP_SET_COMPRESS(bp, ZIO_COMPRESS_OFF);
+   DVA_SET_OFFSET(BP_IDENTITY(bp), off);
%
% as described above, the offset is now passed via DVA offset field
%
   

Re: ZFS: i/o error - all block copies unavailable after upgrading to r225312

2011-09-13 Thread Andriy Gapon
on 07/09/2011 19:35 Andriy Gapon said the following:
 on 02/09/2011 16:35 Andriy Gapon said the following:
 Then:
 - obtain this patch http://people.freebsd.org/~avg/zfstest.head.diff
 - cd sys/boot/zfs
 - apply the patch to zfstest.c
 - cc -I. -I../../cddl/boot/zfs zfstest.c -o zfstest
 - run the resulting binary as root and provide your pool device(s) as
 parameter(s); e.g.:
   ./zfstest /dev/ada0p4
 
 Thanks to a lot of excellent testing, debugging and analysis from Sebastian 
 (which
 went behind the scenes) we now have this patch:
 http://people.freebsd.org/~avg/zfs-boot-gang.diff

I've updated the patch place.  The essence of the changes is the same, just done
in a slightly different fashion.  That should minimize the scope of the diff.
One extra change is that now the checksum is also verified for uberblock.

Pawel, can you please review it?

 The patch introduces the following changes:
 - checksum is now verified for gang header blocks
 - checksum is now verified for reconstituted data of whole gang blocks
   (previously it is verified only for individual gang member leaf blocks)
 - reconstituted data of a whole gang block is now decompressed if the gang 
 block
 is compressed
 
 The last change is _the_ change.
 
 If you use compression for a filesystem where your kernel resides and you get 
 a
 problem with booting, then please test this patch and report back.
 
 Many thanks to Sebastian!
 Additional heap of thanks to Doug Rabson who came up with the idea and
 implementation of zfstest.c!  This tool is of the immense help when debugging 
 an
 issue like this one.
 


-- 
Andriy Gapon
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: ZFS: i/o error - all block copies unavailable after upgrading to r225312

2011-09-11 Thread Andriy Gapon
on 10/09/2011 14:03 Peter Jeremy said the following:
 On 2011-Sep-10 12:46:50 +0300, Andriy Gapon a...@freebsd.org wrote:
 on 10/09/2011 11:07 Andriy Gapon said the following:
 Actually, removing either -mrtd _or_ -fno-unit-at-a-time produces the
 correct code.  Puzzled.
 
 The problem is reproducible with base gcc and gcc42, it is not
 reproducible with gcc45, gcc46 and clang.
 
 I was just checking gcc44  gcc46.  gcc44 inlines the entire function and I
 couldn't quickly find the offending code to see if the bug was there or
 not.  I agree you've triggered a gcc bug but I'm not sure of the correct
 approach to fix it.  I've tried a few trivial code transforms within
 vdev_read_phys() but haven't stumbled on one that avoids the problem.
 
 Since -mrtd changes the calling convention, it's a more intrusive change.
 I'm not sure if there's any simple way to alter CFLAGS for a single file
 (since we only want to alter the zfsboot.c compilation.

And I am actually wondering about -fno-unit-at-a-time option.
In my opinion this is an anti-optimization option and it can actually increase
a size of a final binary.  In fact, it looks like the option was introduced to
boot2 in r132870 in the year 2004, way before GCC 4.X switch, and it was
introduced to avoid some optimizations that produced broken code.
I wonder if there is any reason to keep using that option now.

At least the zfs boot code works fine without the option in my testing.

-- 
Andriy Gapon
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: ZFS: i/o error - all block copies unavailable after upgrading to r225312

2011-09-11 Thread Andriy Gapon
on 11/09/2011 11:31 Andriy Gapon said the following:
 And I am actually wondering about -fno-unit-at-a-time option.
 In my opinion this is an anti-optimization option and it can actually increase
 a size of a final binary.  In fact, it looks like the option was introduced to
 boot2 in r132870 in the year 2004, way before GCC 4.X switch, and it was
 introduced to avoid some optimizations that produced broken code.
 I wonder if there is any reason to keep using that option now.
 
 At least the zfs boot code works fine without the option in my testing.

And it looks like -fno-toplevel-reorder can be used instead of
-fno-unit-at-a-time if we really depend on preserving the order:
http://gcc.gnu.org/gcc-4.2/changes.html

-- 
Andriy Gapon
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: ZFS: i/o error - all block copies unavailable after upgrading to r225312

2011-09-11 Thread Peter Jeremy
On 2011-Sep-11 11:31:54 +0300, Andriy Gapon a...@freebsd.org wrote:
And I am actually wondering about -fno-unit-at-a-time option.
In my opinion this is an anti-optimization option and it can actually increase
a size of a final binary.  In fact, it looks like the option was introduced to
boot2 in r132870 in the year 2004, way before GCC 4.X switch, and it was
introduced to avoid some optimizations that produced broken code.
I wonder if there is any reason to keep using that option now.

In any case, size isn't an issue for any of gptboot, gptzfsboot or
zfsboot (unlike boot2).  For that matter, why do we need both
gptboot and gptzfsboot?  It would be more convenient to have a
single GPT bootstrap that handled both UFS  ZFS.

-- 
Peter Jeremy


pgp6XqTUV4Wxt.pgp
Description: PGP signature


Re: ZFS: i/o error - all block copies unavailable after upgrading to r225312

2011-09-11 Thread Nathan Whitehorn

On 09/11/11 13:31, Peter Jeremy wrote:

On 2011-Sep-11 11:31:54 +0300, Andriy Gapona...@freebsd.org  wrote:

And I am actually wondering about -fno-unit-at-a-time option.
In my opinion this is an anti-optimization option and it can actually increase
a size of a final binary.  In fact, it looks like the option was introduced to
boot2 in r132870 in the year 2004, way before GCC 4.X switch, and it was
introduced to avoid some optimizations that produced broken code.
I wonder if there is any reason to keep using that option now.


In any case, size isn't an issue for any of gptboot, gptzfsboot or
zfsboot (unlike boot2).  For that matter, why do we need both
gptboot and gptzfsboot?  It would be more convenient to have a
single GPT bootstrap that handled both UFS  ZFS.



It would be much more convenient, and also simplify adding installation 
onto ZFS partitions in the installer significantly.

-Nathan
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: ZFS: i/o error - all block copies unavailable after upgrading to r225312

2011-09-11 Thread Scot Hetzel
On Sun, Sep 11, 2011 at 6:31 AM, Peter Jeremy peterjer...@acm.org wrote:
 In any case, size isn't an issue for any of gptboot, gptzfsboot or
 zfsboot (unlike boot2).  For that matter, why do we need both
 gptboot and gptzfsboot?  It would be more convenient to have a
 single GPT bootstrap that handled both UFS  ZFS.

gptzfsboot contains CDDL licensed code.  This was done to allow users
to build/install FreeBSD without any CDDL licensed boot code.

Scot
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: ZFS: i/o error - all block copies unavailable after upgrading to r225312

2011-09-10 Thread Andriy Gapon
on 07/09/2011 19:35 Andriy Gapon said the following:
 Thanks to a lot of excellent testing, debugging and analysis from Sebastian 
 (which
 went behind the scenes) we now have this patch:
 http://people.freebsd.org/~avg/zfs-boot-gang.diff
 
 The patch introduces the following changes:
 - checksum is now verified for gang header blocks
 - checksum is now verified for reconstituted data of whole gang blocks
   (previously it is verified only for individual gang member leaf blocks)
 - reconstituted data of a whole gang block is now decompressed if the gang 
 block
 is compressed
 
 The last change is _the_ change.

I am now investigating what looks like a miscompilation of the code by *gcc*
after applying the patch.  It seems that -mrtd option is to blame.
I have found an older discussion about the -mrtd option causing trouble with 
clang:
http://lists.freebsd.org/pipermail/freebsd-current/2011-August/026263.html
There was a patch that made clang happy without disabling the flag, so I wonder
if I made some subtle mistake in my patch.  Or maybe it's better to disable mrtd
altogether for the zfs boot blocks, just to stay on the safe side.

Some technical details in the form of a diff with some superimposed comments:
--- /home/avg/tmp/vdev_read_phys-mrtd.s 2011-09-10 01:50:54.500620864 +0300
+++ /home/avg/tmp/vdev_read_phys-no-mrtd.s  2011-09-10 01:49:59.157701373 
+0300
@@ -29,16 +29,17 @@
... - in the code before this %edi gets assigned a pointer to a 
function
movl60(%ecx), %eax
movl%eax, 24(%esp)
movl%ecx, 20(%esp)
+   movl%edi, %ecx  - non-mrtd code saves the pointer
popl%ebx
popl%esi
popl%edi - %edi gets over-written with an unrelated value
popl%ebp
-   jmp *%edi  - mrtd code calls some garbage code
+   jmp *%ecx  - non-mrtd code calls the correct code
 .L601:
movl$5, %eax
popl%ebx
popl%esi
popl%edi
popl%ebp
-   ret $24
+   ret

The problem is in the patched vdev_read_phys function in zfsimpl.c.
Unpatched version of the function doesn't seem to be affected.

Any help/ideas will be greatly appreciated!

-- 
Andriy Gapon
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: ZFS: i/o error - all block copies unavailable after upgrading to r225312

2011-09-10 Thread Andriy Gapon
on 10/09/2011 10:32 Andriy Gapon said the following:
 I am now investigating what looks like a miscompilation of the code by *gcc*
 after applying the patch.  It seems that -mrtd option is to blame.
 I have found an older discussion about the -mrtd option causing trouble with 
 clang:
 http://lists.freebsd.org/pipermail/freebsd-current/2011-August/026263.html
 There was a patch that made clang happy without disabling the flag, so I 
 wonder
 if I made some subtle mistake in my patch.  Or maybe it's better to disable 
 mrtd
 altogether for the zfs boot blocks, just to stay on the safe side.

Actually, removing either -mrtd _or_ -fno-unit-at-a-time produces the correct
code.  Puzzled.

 Some technical details in the form of a diff with some superimposed comments:
 --- /home/avg/tmp/vdev_read_phys-mrtd.s   2011-09-10 01:50:54.500620864 
 +0300
 +++ /home/avg/tmp/vdev_read_phys-no-mrtd.s2011-09-10 01:49:59.157701373 
 +0300
 @@ -29,16 +29,17 @@
   ... - in the code before this %edi gets assigned a pointer to a 
 function
   movl60(%ecx), %eax
   movl%eax, 24(%esp)
   movl%ecx, 20(%esp)
 + movl%edi, %ecx  - non-mrtd code saves the pointer
   popl%ebx
   popl%esi
   popl%edi - %edi gets over-written with an unrelated value
   popl%ebp
 - jmp *%edi  - mrtd code calls some garbage code
 + jmp *%ecx  - non-mrtd code calls the correct code
  .L601:
   movl$5, %eax
   popl%ebx
   popl%esi
   popl%edi
   popl%ebp
 - ret $24
 + ret
 
 The problem is in the patched vdev_read_phys function in zfsimpl.c.
 Unpatched version of the function doesn't seem to be affected.
 
 Any help/ideas will be greatly appreciated!
 


-- 
Andriy Gapon
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: ZFS: i/o error - all block copies unavailable after upgrading to r225312

2011-09-10 Thread Andriy Gapon
on 10/09/2011 11:07 Andriy Gapon said the following:
 Actually, removing either -mrtd _or_ -fno-unit-at-a-time produces the correct
 code.  Puzzled.

The problem is reproducible with base gcc and gcc42, it is not reproducible with
gcc45, gcc46 and clang.

-- 
Andriy Gapon
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: ZFS: i/o error - all block copies unavailable after upgrading to r225312

2011-09-10 Thread Peter Jeremy
On 2011-Sep-10 12:46:50 +0300, Andriy Gapon a...@freebsd.org wrote:
on 10/09/2011 11:07 Andriy Gapon said the following:
 Actually, removing either -mrtd _or_ -fno-unit-at-a-time produces the correct
 code.  Puzzled.

The problem is reproducible with base gcc and gcc42, it is not reproducible
with gcc45, gcc46 and clang.

I was just checking gcc44  gcc46.  gcc44 inlines the entire function
and I couldn't quickly find the offending code to see if the bug was
there or not.  I agree you've triggered a gcc bug but I'm not sure of
the correct approach to fix it.  I've tried a few trivial code
transforms within vdev_read_phys() but haven't stumbled on one that
avoids the problem.

Since -mrtd changes the calling convention, it's a more intrusive
change.  I'm not sure if there's any simple way to alter CFLAGS for
a single file (since we only want to alter the zfsboot.c compilation.

-- 
Peter Jeremy


pgpmdN08zKeyD.pgp
Description: PGP signature


Re: ZFS: i/o error - all block copies unavailable after upgrading to r225312

2011-09-07 Thread Andriy Gapon
on 02/09/2011 16:35 Andriy Gapon said the following:
 Then:
 - obtain this patch http://people.freebsd.org/~avg/zfstest.head.diff
 - cd sys/boot/zfs
 - apply the patch to zfstest.c
 - cc -I. -I../../cddl/boot/zfs zfstest.c -o zfstest
 - run the resulting binary as root and provide your pool device(s) as
 parameter(s); e.g.:
   ./zfstest /dev/ada0p4

Thanks to a lot of excellent testing, debugging and analysis from Sebastian 
(which
went behind the scenes) we now have this patch:
http://people.freebsd.org/~avg/zfs-boot-gang.diff

The patch introduces the following changes:
- checksum is now verified for gang header blocks
- checksum is now verified for reconstituted data of whole gang blocks
  (previously it is verified only for individual gang member leaf blocks)
- reconstituted data of a whole gang block is now decompressed if the gang block
is compressed

The last change is _the_ change.

If you use compression for a filesystem where your kernel resides and you get a
problem with booting, then please test this patch and report back.

Many thanks to Sebastian!
Additional heap of thanks to Doug Rabson who came up with the idea and
implementation of zfstest.c!  This tool is of the immense help when debugging an
issue like this one.

-- 
Andriy Gapon
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: ZFS: i/o error - all block copies unavailable after upgrading to r225312

2011-09-07 Thread Peter Jeremy
On 2011-Sep-07 19:35:09 +0300, Andriy Gapon a...@freebsd.org wrote:
Thanks to a lot of excellent testing, debugging and analysis from Sebastian
(which went behind the scenes) we now have this patch:
http://people.freebsd.org/~avg/zfs-boot-gang.diff
...
If you use compression for a filesystem where your kernel resides and
you get a problem with booting, then please test this patch and
report back.

Since the problem isn't consistent (it can appear on some files but not
others), I modified zfstest to take a pathname, rather than have the
pathname hard-coded.  Using this, I found an example file that works
with the patch but not with the old ZFS code:

pjdesk% ./zfstest.old /boot/kernel.old/iwn6000fw.ko.symbols  /dev/ad0p3 |less
ZFS: i/o error - all block copies unavailable
can't lookup
pjdesk% ./zfstest.old /boot/kernel.old/iwn6000fw.ko.symbols  /dev/ad0p3 |less
ZFS: i/o error - all block copies unavailable
can't lookup
pjdesk% ./zfstest.old /boot/kernel.old/iwn6000fw.ko.symbols  /dev/ad0p3 |less
ZFS: i/o error - all block copies unavailable
can't lookup
pjdesk% ./zfstest /boot/kernel.old/iwn6000fw.ko.symbols  /dev/ad0p3 |less
^?ELF^B^A^A ^@^@^@^@^@^@^@^@^A^@^@^A^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@p
...
pjdesk% ./zfstest /boot/kernel.old/iwn6000fw.ko.symbols  /dev/ad0p3 | cmp - 
/boot/kernel.old/iwn6000fw.ko.symbols
pjdesk% ./zfstest /boot/kernel.old/iwn6000fw.ko.symbols  /dev/ad0p3 | cmp - 
/boot/kernel.old/iwn6000fw.ko.symbols
pjdesk% ./zfstest.old /boot/kernel.old/iwn6000fw.ko.symbols  /dev/ad0p3 | cmp - 
/boot/kernel.old/iwn6000fw.ko.symbols 
stdin /boot/kernel.old/iwn6000fw.ko.symbols differ: char 1, line 1
pjdesk% 

This was tested on an 8-STABLE system at r225392.  I've built new
bootblocks but not tested them yet - I will do that tomorrow.

-- 
Peter Jeremy


pgpVRdecjLxqB.pgp
Description: PGP signature


Re: ZFS: i/o error - all block copies unavailable after upgrading to r225312

2011-09-05 Thread Sebastian Chmielewski
On Fri, 02 Sep 2011 17:01:56 +0200
Henri Hennebert h...@restart.be wrote:

 I encounter the same problem. I reboot with mfsBSD and run zpool scrub
 without error.
 
 I reboot and get the same error - the error msg repeating 4 or 5 times.
 

Do you have lzjb compression in your pool?
My pool does use it (on some filesystems) and it's also almost full (95% used).
'zdb zroot' fails with 'assertion failed' message.

1089791116K64K64K64K  100.00  ZFS plain file


Traversing all blocks to verify checksums and verify nothing leaked ...
Assertion failed: (space_map_load(msp-ms_map, zdb_space_map_ops, SM_ALLOC, 
msp-ms_smo, spa-spa_meta_objset) == 0), file 
/work/opensource/OS/FreeBSD/current-gem/cddl/usr.sbin/zdb/../../../cddl/contrib/opensolaris/cmd/zdb/zdb.c,
 line 2115.
zsh: abort  zdb zroot

best regards,
-- 
Sebastian Chmielewski * jid:chmielss...@gmail.com * gg:3336919 * icq:224161389
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: ZFS: i/o error - all block copies unavailable after upgrading to r225312

2011-09-05 Thread Henri Hennebert

On 09/05/2011 19:51, Sebastian Chmielewski wrote:

On Fri, 02 Sep 2011 17:01:56 +0200
Henri Henneberth...@restart.be  wrote:


I encounter the same problem. I reboot with mfsBSD and run zpool scrub
without error.

I reboot and get the same error - the error msg repeating 4 or 5 times.



Do you have lzjb compression in your pool


Yes on some file systems but not on rpool/root


My pool does use it (on some filesystems) and it's also almost full (95% used).


I think it is not a problem, zfsloader must open the zfs file system in 
read-only. BUT it is AWFUL for io performance.



'zdb zroot' fails with 'assertion failed' message.

1089791116K64K64K64K  100.00  ZFS plain file


Traversing all blocks to verify checksums and verify nothing leaked ...
Assertion failed: (space_map_load(msp-ms_map,zdb_space_map_ops, 
SM_ALLOC,msp-ms_smo, spa-spa_meta_objset) == 0), file 
/work/opensource/OS/FreeBSD/current-gem/cddl/usr.sbin/zdb/../../../cddl/contrib/opensolaris/cmd/zdb/zdb.c,
 line 2115.
zsh: abort  zdb zroot


I try it on my active pool and no assertion error, only some

zdb_blkptr_cb: Got error 122 reading 180, 0, 0, 0  -- skipping

but on a live file system this seems normal.

Henri


best regards,


___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: ZFS: i/o error - all block copies unavailable after upgrading to r225312

2011-09-02 Thread Andriy Gapon
on 01/09/2011 23:36 Sebastian Chmielewski said the following:
 
 hi,
 I'm running FreeBSD-CURRENT r224522M installed on ZFS Root and GPT
 partitions and this setup is running fine. Today I've updated to r225312 and
 after rebooting I've got following error message from boot loader:
 
 ZFS: i/o error - all block copies unavailable
 
 Then I did zfs rollback using rescue disk to previous revision and I was 
 able to
 boot.
 Steps I did to update system:
 
 zfs snapshot 
 make buildworld
 make buildkernel
 make installkernel
 reboot
 can't boot
 zfs rollback using rescue disk
 boot to previous revision (r224522M).
 
 Are there any more steps required for this procedure to work?
 My previous updates from source on the same setup where successfull, this is
 the first time I've encountered this error message when upgrading from source.


Could you please try the following?

Repeat your upgrade procedure up to reboot.  Take all the necessary precautions,
so that you can recover your system again!

Then:
- obtain this patch http://people.freebsd.org/~avg/zfstest.head.diff
- cd sys/boot/zfs
- apply the patch to zfstest.c
- cc -I. -I../../cddl/boot/zfs zfstest.c -o zfstest
- run the resulting binary as root and provide your pool device(s) as
parameter(s); e.g.:
  ./zfstest /dev/ada0p4

If the program produces text of FreeBSD COPYRIGHT file as its output, then the 
ZFS
boot code can access your pool.  If the program works OK, but booting still 
fails
it may mean that you have stale boot blocks which you need to update before
rebooting.  Either installworld should do that or you can do make install in
sys/boot directory. Also you might need to update your freebsd-boot partition 
with
gpart command.

If the program fails, then please share its output and also try to analyze where
it fails by looking at the code and adding prints at the suspicious/crucial 
points.

 My setup:
 gpart show 
 =   34  250069613  ada0  GPT  (119G)
  34128 1  freebsd-boot  (64k)
 162   1886 5  bios-boot  (943k)
2048   16777216 2  !0fc63daf-8483-4772-8e79-3d69d8477de4  (8.0G)
16779264   33554432 3  freebsd-ufs  (16G)
50333696  199735951 4  freebsd-zfs  (95G)
 
 Partitions 3 and 4 are separate zfs pools (3 is geli encrypted and 4 is for
 operating system).  
 
 zfs list
 NAMEUSED  AVAIL  REFER  MOUNTPOINT
 home   10,4G  5,24G  10,4G  /home
 zroot  90,2G  3,27G  1,16G  legacy
 
 /etc/fstab
 # Device  Mountpoint  FStype  Options Dump
 Pass#
 zroot /   zfs rw,noatime  00
 /dev/acd0 /cdrom  cd9660  ro,noauto   00
 proc  /proc   procfs  rw  00


-- 
Andriy Gapon
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: ZFS: i/o error - all block copies unavailable after upgrading to r225312

2011-09-02 Thread Henri Hennebert
On 09/01/2011 22:36, Sebastian Chmielewski wrote:
 
 hi,
 I'm running FreeBSD-CURRENT r224522M installed on ZFS Root and GPT
 partitions and this setup is running fine. Today I've updated to r225312 and
 after rebooting I've got following error message from boot loader:
 
 ZFS: i/o error - all block copies unavailable

I encounter the same problem. I reboot with mfsBSD and run zpool scrub
without error.

I reboot and get the same error - the error msg repeating 4 or 5 times.

I reboot with  mfsBSD and I then do:

mkdir /rpool
zpool import -R /rpool rpool
mount -t zfs rpool/root /mnt
cd /mnt/boot
mv kernel kernel-error
mkdir kernel
cp kernel-error/* kernel

When I reboot all run smoothly :-o

strange indeed...

Henri


 
 Then I did zfs rollback using rescue disk to previous revision and I was 
 able to
 boot.
 Steps I did to update system:
 
 zfs snapshot 
 make buildworld
 make buildkernel
 make installkernel
 reboot
 can't boot
 zfs rollback using rescue disk
 boot to previous revision (r224522M).
 
 Are there any more steps required for this procedure to work?
 My previous updates from source on the same setup where successfull, this is
 the first time I've encountered this error message when upgrading from source.
 
 My setup:
 gpart show 
 =   34  250069613  ada0  GPT  (119G)
  34128 1  freebsd-boot  (64k)
 162   1886 5  bios-boot  (943k)
2048   16777216 2  !0fc63daf-8483-4772-8e79-3d69d8477de4  (8.0G)
16779264   33554432 3  freebsd-ufs  (16G)
50333696  199735951 4  freebsd-zfs  (95G)
 
 Partitions 3 and 4 are separate zfs pools (3 is geli encrypted and 4 is for
 operating system).  
 
 zfs list
 NAMEUSED  AVAIL  REFER  MOUNTPOINT
 home   10,4G  5,24G  10,4G  /home
 zroot  90,2G  3,27G  1,16G  legacy
 
 /etc/fstab
 # Device  Mountpoint  FStype  Options Dump
 Pass#
 zroot /   zfs rw,noatime  00
 /dev/acd0 /cdrom  cd9660  ro,noauto   00
 proc  /proc   procfs  rw  00
 
 best regards,

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


ZFS: i/o error - all block copies unavailable after upgrading to r225312

2011-09-01 Thread Sebastian Chmielewski

hi,
I'm running FreeBSD-CURRENT r224522M installed on ZFS Root and GPT
partitions and this setup is running fine. Today I've updated to r225312 and
after rebooting I've got following error message from boot loader:

ZFS: i/o error - all block copies unavailable

Then I did zfs rollback using rescue disk to previous revision and I was able 
to
boot.
Steps I did to update system:

zfs snapshot 
make buildworld
make buildkernel
make installkernel
reboot
can't boot
zfs rollback using rescue disk
boot to previous revision (r224522M).

Are there any more steps required for this procedure to work?
My previous updates from source on the same setup where successfull, this is
the first time I've encountered this error message when upgrading from source.

My setup:
gpart show 
=   34  250069613  ada0  GPT  (119G)
 34128 1  freebsd-boot  (64k)
162   1886 5  bios-boot  (943k)
   2048   16777216 2  !0fc63daf-8483-4772-8e79-3d69d8477de4  (8.0G)
   16779264   33554432 3  freebsd-ufs  (16G)
   50333696  199735951 4  freebsd-zfs  (95G)

Partitions 3 and 4 are separate zfs pools (3 is geli encrypted and 4 is for
operating system).  

zfs list
NAMEUSED  AVAIL  REFER  MOUNTPOINT
home   10,4G  5,24G  10,4G  /home
zroot  90,2G  3,27G  1,16G  legacy

/etc/fstab
# DeviceMountpoint  FStype  Options Dump
Pass#
zroot   /   zfs rw,noatime  0   0
/dev/acd0   /cdrom  cd9660  ro,noauto   0   0
proc/proc   procfs  rw  0   0

best regards,
-- 
Sebastian Chmielewski * jid:chmielss...@gmail.com * gg:3336919 * icq:224161389

___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org


Re: ZFS: i/o error - all block copies unavailable after upgrading to r225312

2011-09-01 Thread Sebastian Chmielewski
On Thu, 1 Sep 2011 22:36:46 +0200
Sebastian Chmielewski chmi...@o2.pl wrote:

 after rebooting I've got following error message from boot loader:
 
 ZFS: i/o error - all block copies unavailable
 
I'm very puzzled with this, in theory 'make installkernel' shouldn't break boot
process at this stage as loader was not changed.
I've created my setup using instructions from official wiki:
http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot
(using 8kB sector size) and patched ZFS loader from 
http://people.freebsd.org/~pjd/zfsboot/.
OS version was stable/8, upgraded later to -CURRENT with KMS patches.

I've run multiple upgrades from source since then (from revisions 223687 to 
224522) 
and never got any problems until now.

best regards,
-- 
Sebastian Chmielewski * jid:chmielss...@gmail.com * gg:3336919 * icq:224161389
___
freebsd-current@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org