On 06/16/09 16:32, Jens Elkner wrote:
Hmmm,
just upgraded some servers to U7. Unfortunately one server's primary disk
died during the upgrade, so that luactivate was not able to activate the
s10u7 BE ("Unable to determine the configuration ..."). Since the rpool
is a 2-way mirror, the boot-device=/p...@1f,700000/s...@2/d...@1,0:a was
simply set to /p...@1f,700000/s...@2/d...@0,0:a and checked, whether the
machine still reboots unattended. As expected - no problem.
At the evening the faulty disk was replaced and the mirror resilvered via
'zpool replace rpool c1t1d0s0' (see below). Since there was no error and
everything stated to be healthy, the s10u7 BE was luactivated (no error
here message as well) and 'init 6'.
Unfortunately, now the server was gone and no known recipe helped to
revive it (I guess, LU damaged the zpool.cache?) :((((
Even if LU somehow damaged the zpool.cache, that wouldn't explain why an
import, while booted off the net, wouldn't work. LU would have had to
damage
the pool's label in some way.
I notice that when you booted the system from the net, you booted from an
Update 6 image. It's possible that the luupgrade to U7 upgraded the pool
version to one not understood by Update 6. One thing I suggest is booting
off the net from a U7 image and see if that allows you to import the pool.
The other suggestion I have is to remove the
/p...@1f,700000/s...@2/d...@1,0:a
device (the one that didn't appear to be able to boot at all) and try
booting
off just the other disk. There are some problems booting from one side
of a mirror when the disk on the other side is present, but somehow not
quite right. If that boot is successful, insert the other disk again (while
still booted, if your system supports hot-plugging), and then perform the
recovery procedure documented here:
http://www.solarisinternals.com/wiki/index.php/ZFS_Troubleshooting_Guide#Primary_Mirror_Disk_in_a_ZFS_Root_Pool_is_Unavailable_or_Fails
making sure you perform the "installboot" step to install a boot block.
If you can manage to boot off one side of the mirror, but your system
does not support hot-plugging, try doing a 'zpool detach' to remove
the other disk from the root mirror, then taking the system down, inserting
the replacement disk, rebooting, then re-attach the disk to the
mirror using "zpool attach", and then reinstall the boot block, as
shown in the above link.
If none of this works, I'll look into other reasons why a pool might
appear to have "corrupted data" and therefore be non-importable.
Lori
Any hints, how to get the rpool back?
Regards,
jel.
What has been tried 'til now:
########################################
{3} ok boot
Boot device: /p...@1f,700000/s...@2/d...@1,0:a File and args:
Bad magic number in disk label
Can't open disk label package
Can't open boot device
{3} ok
########################################
{3} ok boot /p...@1f,700000/s...@2/d...@0,0:a
Boot device: /p...@1f,700000/s...@2/d...@0,0:a File and args:
SunOS Release 5.10 Version Generic_138888-08 64-bit
Copyright 1983-2009 Sun Microsystems, Inc. All rights reserved.
Use is subject to license terms.
NOTICE:
spa_import_rootpool: error 22
Cannot mount root on /p...@1f,700000/s...@2/d...@0,0:a fstype zfs
panic[cpu3]/thread=180e000: vfs_mountroot: cannot mount root
000000000180b950 genunix:vfs_mountroot+358 (800, 200, 0, 1875c00,
189f800, 18ca000)
%l0-3: 00000000010ba000 00000000010ba208 000000000187bba8
%00000000011e8400
%l4-7: 00000000011e8400 00000000018cc400 0000000000000600
%0000000000000200
000000000180ba10 genunix:main+a0 (1815178, 180c000, 18397b0, 18c6800,
181b578, 1815000)
%l0-3: 0000000001015400 0000000000000001 0000000070002000
%0000000000000000
%l4-7: 000000000183ec00 0000000000000003 000000000180c000
%0000000000000000
skipping system dump - no dump device configured
rebooting...
SC Alert: Host System has Reset
########################################
{3} ok boot net -s
Boot device: /p...@1c,600000/netw...@2 File and args: -s
1000 Mbps FDX Link up
Timeout waiting for ARP/RARP packet
3a000 1000 Mbps FDX Link up
SunOS Release 5.10 Version Generic_137137-09 64-bit
Copyright 1983-2008 Sun Microsystems, Inc. All rights reserved.
Use is subject to license terms.
Hardware watchdog enabled
Booting to milestone "milestone/single-user:default".
Configuring devices.
Using RPC Bootparams for network configuration information.
Attempting to configure interface ce1...
Skipped interface ce1
Attempting to configure interface ce0...
Configured interface ce0
Requesting System Maintenance Mode
SINGLE USER MODE
# mount -F zfs /dev/dsk/c1t1d0s0 /mnt
cannot open '/dev/dsk/c1t1d0s0': invalid dataset name
# mount -F zfs /dev/dsk/c1t0d0s0 /mnt
cannot open '/dev/dsk/c1t0d0s0': invalid dataset name
# zpool import
pool: pool1
id: 5088500955966129017
state: ONLINE
action: The pool can be imported using its name or numeric identifier.
config:
pool1 ONLINE
mirror ONLINE
c1t2d0 ONLINE
c1t3d0 ONLINE
pool: rpool
id: 5910200402071733373
state: UNAVAIL
action: The pool cannot be imported due to damaged devices or data.
config:
rpool UNAVAIL insufficient replicas
mirror UNAVAIL corrupted data
c1t1d0s0 ONLINE
c1t0d0s0 ONLINE
# dd if=/dev/rdsk/c1t1d0s0 of=/tmp/bb bs=1b iseek=1 count=15
15+0 records in
15+0 records out
# dd if=/dev/rdsk/c1t1d0s0 of=/tmp/bb bs=1b iseek=1024 oseek=15 count=16
16+0 records in
16+0 records out
# cmp /tmp/bb /usr/platform/`uname -i`/lib/fs/zfs/bootblk
# echo $?
0
# dd if=/dev/rdsk/c1t0d0s0 of=/tmp/ab bs=1b iseek=1 count=15
15+0 records in
15+0 records out
# dd if=/dev/rdsk/c1t0d0s0 of=/tmp/ab bs=1b iseek=1024 oseek=15 count=16
16+0 records in
16+0 records out
# cmp /tmp/ab /usr/platform/`uname -i`/lib/fs/zfs/bootblk
# echo $?
0
#
pre-history:
============
admin.tpol ~ # zpool status -xv
pool: rpool
state: DEGRADED
status: One or more devices is currently being resilvered. The pool will
continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
scrub: resilver in progress for 0h28m, 98.10% done, 0h0m to go
config:
NAME STATE READ WRITE CKSUM
rpool DEGRADED 0 0 0
mirror DEGRADED 0 0 0
replacing DEGRADED 0 0 0
c1t1d0s0/old FAULTED 0 0 0 corrupted data
c1t1d0s0 ONLINE 0 0 0
c1t0d0s0 ONLINE 0 0 0
errors: No known data errors
admin.tpol ~ # zpool status -xv
all pools are healthy
admin.tpol ~ # zpool status
pool: pool1
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
pool1 ONLINE 0 0 0
mirror ONLINE 0 0 0
c1t2d0 ONLINE 0 0 0
c1t3d0 ONLINE 0 0 0
errors: No known data errors
pool: rpool
state: ONLINE
scrub: resilver completed after 0h28m with 0 errors on Tue Jun 16 19:13:57 2009
config:
NAME STATE READ WRITE CKSUM
rpool ONLINE 0 0 0
mirror ONLINE 0 0 0
c1t1d0s0 ONLINE 0 0 0
c1t0d0s0 ONLINE 0 0 0
errors: No known data errors
admin.tpol / # luactivate -l $ERR -o $OUT S10u7
...
In case of a failure while booting ...
mount -Fzfs /dev/dsk/c1t1d0s0 /mnt
...
Modifying boot archive service
Activation of boot environment <s10u7> successful.
8.32u 16.74s 1:25.15 29.4%
admin.tpol / # init 6
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
