On 1/29/2011 6:18 PM, Richard Elling wrote:
>
> On Jan 29, 2011, at 12:58 PM, Mike Tancsa wrote:
>
>> On 1/29/2011 12:57 PM, Richard Elling wrote:
>>>> 0(offsite)# zpool status
>>>> pool: tank1
>>>> state: UNAVAIL
>>>> status: One or more devices could not be opened. There are insufficient
>>>> replicas for the pool to continue functioning.
>>>> action: Attach the missing device and online it using 'zpool online'.
>>>> see: http://www.sun.com/msg/ZFS-8000-3C
>>>> scrub: none requested
>>>> config:
>>>>
>>>> NAME STATE READ WRITE CKSUM
>>>> tank1 UNAVAIL 0 0 0 insufficient replicas
>>>> raidz1 ONLINE 0 0 0
>>>> ad0 ONLINE 0 0 0
>>>> ad1 ONLINE 0 0 0
>>>> ad4 ONLINE 0 0 0
>>>> ad6 ONLINE 0 0 0
>>>> raidz1 ONLINE 0 0 0
>>>> ada4 ONLINE 0 0 0
>>>> ada5 ONLINE 0 0 0
>>>> ada6 ONLINE 0 0 0
>>>> ada7 ONLINE 0 0 0
>>>> raidz1 UNAVAIL 0 0 0 insufficient replicas
>>>> ada0 UNAVAIL 0 0 0 cannot open
>>>> ada1 UNAVAIL 0 0 0 cannot open
>>>> ada2 UNAVAIL 0 0 0 cannot open
>>>> ada3 UNAVAIL 0 0 0 cannot open
>>>> 0(offsite)#
>>>
>>> This is usually easily solved without data loss by making the
>>> disks available again. Can you read anything from the disks using
>>> any program?
>>
>> Thats the strange thing, the disks are readable. The drive cage just
>> reset a couple of times prior to the crash. But they seem OK now. Same
>> order as well.
>>
>> # camcontrol devlist
>> <WDC WD\021501FASR\25500W2B0 \200 0956> at scbus0 target 0 lun 0
>> (pass0,ada0)
>> <WDC WD\021501FASR\25500W2B0 \200 05.01D\0205> at scbus0 target 1 lun 0
>> (pass1,ada1)
>> <WDC WD\021501FASR\25500W2B0 \200 05.01D\0205> at scbus0 target 2 lun 0
>> (pass2,ada2)
>> <WDC WD\021501FASR\25500W2B0 \200 05.01D\0205> at scbus0 target 3 lun 0
>> (pass3,ada3)
>>
>>
>> # dd if=/dev/ada2 of=/dev/null count=20 bs=1024
>> 20+0 records in
>> 20+0 records out
>> 20480 bytes transferred in 0.001634 secs (12534561 bytes/sec)
>> 0(offsite)#
>
> The next step is to run "zdb -l" and look for all 4 labels. Something like:
> zdb -l /dev/ada2
>
> If all 4 labels exist for each drive and appear intact, then look more closely
> at how the OS locates the vdevs. If you can't solve the "UNAVAIL" problem,
> you won't be able to import the pool.
> -- richard
On 1/29/2011 10:13 PM, James R. Van Artsdalen wrote:
> On 1/28/2011 4:46 PM, Mike Tancsa wrote:
>>
>> I had just added another set of disks to my zfs array. It looks like the
>> drive cage with the new drives is faulty. I had added a couple of files
>> to the main pool, but not much. Is there any way to restore the pool
>> below ? I have a lot of files on ad0,1,4,6 and ada4,5,6,7 and perhaps
>> one file on the new drives in the bad cage.
>
> Get another enclosure and verify it works OK. Then move the disks from
> the suspect enclosure to the tested enclosure and try to import the pool.
>
> The problem may be cabling or the controller instead - you didn't
> specify how the disks were attached or which version of FreeBSD you're
> using.
>
First off thanks to all who responded on and offlist!
Good news (for me) it seems. New cage and all seems to be recognized
correctly. The history is
...
2010-04-22.14:27:38 zpool add tank1 raidz /dev/ada4 /dev/ada5 /dev/ada6
/dev/ada7
2010-06-11.13:49:33 zfs create tank1/argus-data
2010-06-11.13:49:41 zfs create tank1/argus-data/previous
2010-06-11.13:50:38 zfs set compression=off tank1/argus-data
2010-08-06.12:20:59 zpool replace tank1 ad1 ad1
2010-09-16.10:17:51 zpool upgrade -a
2011-01-28.11:45:43 zpool add tank1 raidz /dev/ada0 /dev/ada1 /dev/ada2
/dev/ada3
FreeBSD RELENG_8 from last week, 8G of RAM, amd64.
zpool status -v
pool: tank1
state: ONLINE
status: One or more devices has experienced an error resulting in data
corruption. Applications may be affected.
action: Restore the file in question if possible. Otherwise restore the
entire pool from backup.
see: http://www.sun.com/msg/ZFS-8000-8A
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
tank1 ONLINE 0 0 0
raidz1 ONLINE 0 0 0
ad0 ONLINE 0 0 0
ad1 ONLINE 0 0 0
ad4 ONLINE 0 0 0
ad6 ONLINE 0 0 0
raidz1 ONLINE 0 0 0
ada0 ONLINE 0 0 0
ada1 ONLINE 0 0 0
ada2 ONLINE 0 0 0
ada3 ONLINE 0 0 0
raidz1 ONLINE 0 0 0
ada5 ONLINE 0 0 0
ada8 ONLINE 0 0 0
ada7 ONLINE 0 0 0
ada6 ONLINE 0 0 0
errors: Permanent errors have been detected in the following files:
/tank1/argus-data/previous/argus-sites-radium.2011.01.28.16.00
tank1/argus-data:<0xc6>
/tank1/argus-data/argus-sites-radium
0(offsite)# zpool get all tank1
NAME PROPERTY VALUE SOURCE
tank1 size 14.5T -
tank1 used 7.56T -
tank1 available 6.94T -
tank1 capacity 52% -
tank1 altroot - default
tank1 health ONLINE -
tank1 guid 7336939736750289319 default
tank1 version 15 default
tank1 bootfs - default
tank1 delegation on default
tank1 autoreplace off default
tank1 cachefile - default
tank1 failmode wait default
tank1 listsnapshots on local
Do I just want to do a scrub ?
Unfortunately, http://www.sun.com/msg/ZFS-8000-8A gives a 503
zdb now shows
0(offsite)# zdb -l /dev/ada0
--------------------------------------------
LABEL 0
--------------------------------------------
version=15
name='tank1'
state=0
txg=44593174
pool_guid=7336939736750289319
hostid=3221266864
hostname='offsite.sentex.ca'
top_guid=6980939370923808328
guid=16144392433229115618
vdev_tree
type='raidz'
id=1
guid=6980939370923808328
nparity=1
metaslab_array=38
metaslab_shift=35
ashift=9
asize=4000799784960
is_log=0
children[0]
type='disk'
id=0
guid=16144392433229115618
path='/dev/ada4'
whole_disk=0
DTL=341
children[1]
type='disk'
id=1
guid=1210677308003674848
path='/dev/ada5'
whole_disk=0
DTL=340
children[2]
type='disk'
id=2
guid=2517076601231706249
path='/dev/ada6'
whole_disk=0
DTL=339
children[3]
type='disk'
id=3
guid=16621760039941477713
path='/dev/ada7'
whole_disk=0
DTL=338
--------------------------------------------
LABEL 1
--------------------------------------------
version=15
name='tank1'
state=0
txg=44592523
pool_guid=7336939736750289319
hostid=3221266864
hostname='offsite.sentex.ca'
top_guid=6980939370923808328
guid=16144392433229115618
vdev_tree
type='raidz'
id=1
guid=6980939370923808328
nparity=1
metaslab_array=38
metaslab_shift=35
ashift=9
asize=4000799784960
is_log=0
children[0]
type='disk'
id=0
guid=16144392433229115618
path='/dev/ada4'
whole_disk=0
DTL=341
children[1]
type='disk'
id=1
guid=1210677308003674848
path='/dev/ada5'
whole_disk=0
DTL=340
children[2]
type='disk'
id=2
guid=2517076601231706249
path='/dev/ada6'
whole_disk=0
DTL=339
children[3]
type='disk'
id=3
guid=16621760039941477713
path='/dev/ada7'
whole_disk=0
DTL=338
--------------------------------------------
LABEL 2
--------------------------------------------
version=15
name='tank1'
state=0
txg=44593174
pool_guid=7336939736750289319
hostid=3221266864
hostname='offsite.sentex.ca'
top_guid=6980939370923808328
guid=16144392433229115618
vdev_tree
type='raidz'
id=1
guid=6980939370923808328
nparity=1
metaslab_array=38
metaslab_shift=35
ashift=9
asize=4000799784960
is_log=0
children[0]
type='disk'
id=0
guid=16144392433229115618
path='/dev/ada4'
whole_disk=0
DTL=341
children[1]
type='disk'
id=1
guid=1210677308003674848
path='/dev/ada5'
whole_disk=0
DTL=340
children[2]
type='disk'
id=2
guid=2517076601231706249
path='/dev/ada6'
whole_disk=0
DTL=339
children[3]
type='disk'
id=3
guid=16621760039941477713
path='/dev/ada7'
whole_disk=0
DTL=338
--------------------------------------------
LABEL 3
--------------------------------------------
version=15
name='tank1'
state=0
txg=44592523
pool_guid=7336939736750289319
hostid=3221266864
hostname='offsite.sentex.ca'
top_guid=6980939370923808328
guid=16144392433229115618
vdev_tree
type='raidz'
id=1
guid=6980939370923808328
nparity=1
metaslab_array=38
metaslab_shift=35
ashift=9
asize=4000799784960
is_log=0
children[0]
type='disk'
id=0
guid=16144392433229115618
path='/dev/ada4'
whole_disk=0
DTL=341
children[1]
type='disk'
id=1
guid=1210677308003674848
path='/dev/ada5'
whole_disk=0
DTL=340
children[2]
type='disk'
id=2
guid=2517076601231706249
path='/dev/ada6'
whole_disk=0
DTL=339
children[3]
type='disk'
id=3
guid=16621760039941477713
path='/dev/ada7'
whole_disk=0
DTL=338
0(offsite)#
---Mike
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
