ZFS: i/o error - all block copies unavailable on large disk number machines

2012-01-23 Thread Steven Hartland
We did a minor kernel update on a large storage machine here today which runs FreeBSD 8.2 and to our surprise it failed to boot at the loader with ZFS: i/o error - all block copies unavailable. After some digging we discovered that this was likely due to the fact that the BIOS only enumerates

Re: ZFS: i/o error - all block copies unavailable on large disk number machines

2012-01-23 Thread Chuck Swiger
On Jan 23, 2012, at 9:04 AM, Steven Hartland wrote: After some digging we discovered that this was likely due to the fact that the BIOS only enumerates the first 12 disks and this machine has more than that in the root zpool which was a striped raidz2 volume. This in turn means that the

Re: ZFS: i/o error - all block copies unavailable on large disk number machines

2012-01-23 Thread Matthew Seaman
On 23/01/2012 18:06, Chuck Swiger wrote: On Jan 23, 2012, at 9:04 AM, Steven Hartland wrote: After some digging we discovered that this was likely due to the fact that the BIOS only enumerates the first 12 disks and this machine has more than that in the root zpool which was a striped raidz2

Re: ZFS: i/o error - all block copies unavailable on large disk number machines

2012-01-23 Thread Steven Hartland
- Original Message - From: Chuck Swiger On Jan 23, 2012, at 9:04 AM, Steven Hartland wrote: After some digging we discovered that this was likely due to the fact that the BIOS only enumerates the first 12 disks and this machine has more than that in the root zpool which was a striped

Re: ZFS: i/o error - all block copies unavailable on large disk number machines

2012-01-23 Thread Steven Hartland
- Original Message - From: Matthew Seaman Even if you do split up your pool into vdevs using 8 drives, you will still run into the problem with zfs being unable to assemble the pool unless it sees all of the drives in it. Interesting that this only appeared as part of a minor kernel

Re: ZFS: i/o error - all block copies unavailable on large disk number machines

2012-01-23 Thread Matthew Seaman
On 23/01/2012 19:29, Steven Hartland wrote: Initially the zpool was just the first raidz2. Only after install was the second raidz2 added to increase capacity. So what I believe has happened is the new kernel when installed happens to have data be located on the second raidz2 which consists

Re: ZFS: i/o error - all block copies unavailable on large disk number machines

2012-01-23 Thread Olivier Smedts
Le lundi 23 janvier 2012, Matthew Seaman m.sea...@infracaninophile.co.uk a écrit : On 23/01/2012 19:29, Steven Hartland wrote: Initially the zpool was just the first raidz2. Only after install was the second raidz2 added to increase capacity. So what I believe has happened is the new kernel

Re: ZFS: i/o error - all block copies unavailable on large disk number machines

2012-01-23 Thread Freddie Cash
On Mon, Jan 23, 2012 at 11:21 AM, Steven Hartland kill...@multiplay.co.uk wrote: Not something I've seem made clear, but quite possibly. Even with 9 disks you could easily get this if the BIOS doesn't see all of said disks, be that initially or due to disks added to the machine. For reference

Re: ZFS: i/o error - all block copies unavailable on large disk number machines

2012-01-23 Thread Steven Hartland
- Original Message - From: Olivier Smedts In my case, I fixed it by having a separate /boot on some USB sticks -- this was only ever accessed to read the kernel, kernel modules and bootloader at boot time, so no worries over performance. Out of interest whats the procedure you used

Re: ZFS: i/o error - all block copies unavailable on large disk number machines

2012-01-23 Thread Matthew Seaman
On 23/01/2012 20:47, Steven Hartland wrote: In my case, I fixed it by having a separate /boot on some USB sticks -- this was only ever accessed to read the kernel, kernel modules and bootloader at boot time, so no worries over performance. Out of interest whats the procedure you used for