>>1.) did you move your drives around or change which controller each one was
>>connected to sometime after installing and setting up OpenSolaris?
>>If so, a pool export and re-import may be in order.
No I didn't. It was original setup.
2.) are you sure the drive is failing? Does the problem only affect this drive
or are other drives randomly affect too? If you've run 'zpool clear' and the
problem comes back, something is wrong but it could also be RAM, CPU,
motherboard, controller, or power supply problems. Smartmontools can read the
drive SMART data and device error logs...run it from an Ubuntu 10.04 Live CD
(sudo apt-get install
smartmontools) or from a PartedMagic Live CD if you have trouble getting
Smartmontools working on OpenSolaris with your hardware.
No I am not sure.I already cleared it with zpool clear.
This error is a last thing that worries me right now.
What I want to understand is why both iostat and format show that all drives in
the pool are Seagate which is not correct as two of them are WD.
zpool status tank shows 7 drives:
# zpool status tank
pool: tank
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
raidz2 ONLINE 0 0 0
c1t1d0 ONLINE 0 0 0
c2t0d0 ONLINE 0 0 0
c2t2d0 ONLINE 0 0 0
c2t3d0 ONLINE 0 0 0
c2t4d0 ONLINE 0 0 0
c2t5d0 ONLINE 0 0 0
c2t7d0 ONLINE 0 0 0
What I can't understand is why 'iostat -En' says that all these drives above
are Seagate ?
'iostat -En' does shows WD as well, but they show up at a bottom as sd11 and
sd12? Why ????? :)
See output below
-------------------------------------
# iostat -En
c3d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Model: GIGABYTE i-RAM Revision: Serial No: 33F100336D9CC24 Size: 2.15GB
<2146443264 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0
c1t0d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA Product: ST3500320AS Revision: SD15 Serial No:
Size: 500.11GB <500107862016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 56 Predictive Failure Analysis: 0
c1t1d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA Product: ST3500320AS Revision: SD15 Serial No:
Size: 500.11GB <500107862016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 66 Predictive Failure Analysis: 0
c0t0d0 Soft Errors: 0 Hard Errors: 198 Transport Errors: 0
Vendor: SONY Product: CD-ROM CDU5212 Revision: 5YS1 Serial No:
Size: 0.00GB <0 bytes>
Media Error: 0 Device Not Ready: 198 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c2t0d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA Product: ST3500320AS Revision: SD15 Serial No:
Size: 500.11GB <500107862016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c2t1d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA Product: ST3500320AS Revision: SD15 Serial No:
Size: 500.11GB <500107862016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c2t2d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA Product: ST3500320AS Revision: SD15 Serial No:
Size: 500.11GB <500107862016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c2t3d0 Soft Errors: 0 Hard Errors: 9 Transport Errors: 9
Vendor: ATA Product: ST3500320AS Revision: SD15 Serial No:
Size: 500.11GB <500107862016 bytes>
Media Error: 7 Device Not Ready: 0 No Device: 2 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c2t4d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA Product: ST3500320AS Revision: SD15 Serial No:
Size: 500.11GB <500107862016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c2t5d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA Product: ST3500320AS Revision: SD15 Serial No:
Size: 500.11GB <500107862016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c2t6d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA Product: ST3500320AS Revision: SD15 Serial No:
Size: 500.11GB <500107862016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c2t7d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA Product: ST3500320AS Revision: SD15 Serial No:
Size: 500.11GB <500107862016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
sd11 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA Product: WDC WD5001AALS-0 Revision: 1D05 Serial No:
Size: 500.11GB <500107862016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
sd12 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA Product: WDC WD5001AALS-0 Revision: 0K05 Serial No:
Size: 500.11GB <500107862016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
----------------------------------------------------
>>3.) on some systems I've found another version of the iostat command to be
>>more useful, particularly when iostat -En leaves the serial number field
>>empty or otherwise doesn't read the serial number correctly. Try
>>this:
' iostat -Eni ' indeed outputs Device ID on some of the drives,but I still
can't understand how it helps me to identify model of specific drive.
# iostat -Eni
c3d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Model: GIGABYTE i-RAM Revision: Device Id:
id1,c...@agigabyte_i-ram=33f100336d9cc244b01d
Size: 2.15GB <2146443264 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0
c1t0d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA Product: ST3500320AS Revision: SD15 Device Id:
id1,s...@ast3500320as=____________9qm34ybz
Size: 500.11GB <500107862016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 56 Predictive Failure Analysis: 0
c1t1d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA Product: ST3500320AS Revision: SD15 Device Id:
id1,s...@ast3500320as=____________9qm353d2
Size: 500.11GB <500107862016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 66 Predictive Failure Analysis: 0
c0t0d0 Soft Errors: 0 Hard Errors: 198 Transport Errors: 0
Vendor: SONY Product: CD-ROM CDU5212 Revision: 5YS1 Device Id:
Size: 0.00GB <0 bytes>
Media Error: 0 Device Not Ready: 198 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c2t0d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA Product: ST3500320AS Revision: SD15 Device Id:
id1,s...@n5000c5000b9f49ef
Size: 500.11GB <500107862016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c2t1d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA Product: ST3500320AS Revision: SD15 Device Id:
Size: 500.11GB <500107862016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c2t2d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA Product: ST3500320AS Revision: SD15 Device Id:
id1,s...@n5000c5000b9be7ab
Size: 500.11GB <500107862016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c2t3d0 Soft Errors: 0 Hard Errors: 9 Transport Errors: 9
Vendor: ATA Product: ST3500320AS Revision: SD15 Device Id:
id1,s...@n5000c5000b9fa1ab
Size: 500.11GB <500107862016 bytes>
Media Error: 7 Device Not Ready: 0 No Device: 2 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c2t4d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA Product: ST3500320AS Revision: SD15 Device Id:
id1,s...@n5000c5000b9fde5f
Size: 500.11GB <500107862016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c2t5d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA Product: ST3500320AS Revision: SD15 Device Id:
id1,s...@n5000c5000b9fae42
Size: 500.11GB <500107862016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c2t6d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA Product: ST3500320AS Revision: SD15 Device Id:
Size: 500.11GB <500107862016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
c2t7d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA Product: ST3500320AS Revision: SD15 Device Id:
id1,s...@n5000c5000b9f4eb0
Size: 500.11GB <500107862016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
sd11 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA Product: WDC WD5001AALS-0 Revision: 1D05 Device Id:
Size: 500.11GB <500107862016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
sd12 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
Vendor: ATA Product: WDC WD5001AALS-0 Revision: 0K05 Device Id:
Size: 500.11GB <500107862016 bytes>
Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
Illegal Request: 0 Predictive Failure Analysis: 0
-----Original Message-----
From: Haudy Kazemi [mailto:kaze0...@umn.edu]
Sent: Monday, July 19, 2010 11:07 AM
To: zfs-discuss@opensolaris.org
Cc: Cindy Swearingen; Yuri Homchuk
Subject: Re: [zfs-discuss] Help identify failed drive
A few things:
1.) did you move your drives around or change which controller each one was
connected to sometime after installing and setting up OpenSolaris?
If so, a pool export and re-import may be in order.
2.) are you sure the drive is failing? Does the problem only affect this drive
or are other drives randomly affect too? If you've run 'zpool clear' and the
problem comes back, something is wrong but it could also be RAM, CPU,
motherboard, controller, or power supply problems. Smartmontools can read the
drive SMART data and device error logs...run it from an Ubuntu 10.04 Live CD
(sudo apt-get install
smartmontools) or from a PartedMagic Live CD if you have trouble getting
Smartmontools working on OpenSolaris with your hardware.
3.) on some systems I've found another version of the iostat command to be more
useful, particularly when iostat -En leaves the serial number field empty or
otherwise doesn't read the serial number correctly. Try
this:
iostat -Eni
This should give you a list of drives showing their name in the cXtYdZsN
format, and their Device ID which may contain the drive serial numbers
concatenated with the model. Compare that list with your 'zpool status tank'
output, which in your case means looking for 'c2t3d0'. Once you find the
serial number, you can look at labels printed on your drives and verify which
one it is.
One tip: if your server case is hard to work in or it is otherwise difficult to
remove drives to read the serial numbers (lots of screws, cables in the way,
tight fits, etc.), create additional serial number labels for the drives and
stick them on the drive in a place you can read them without removing the drive
from the drive bay. This will make it easier to find a particular drive next
time you need to replace or upgrade one. This problem most significant on
hardware/OS combinations that don't provide a way to signal where a particular
drive is physically installed. (This includes a lot of whitebox and small
server hardware and OSes.)
iostat relevant man page entries:
http://docs.sun.com/app/docs/doc/816-5166/iostat-1m?l=en&n=1&a=view
<http://docs.sun.com/app/docs/doc/816-5166/iostat-1m?l=en&n=1&a=view>
-E
Display all device error statistics.
-i
In -E output, display the Device ID instead of the Serial No. The Device Id is
a unique identifier registered by a driver through ddi_devid_register(9F).
-n
Display names in descriptive format. For example, cXtYdZ, rmt/N,
server:/export/path.
By default, disks are identified by instance names such as ssd23 or md301.
Combining the -n option with the -x option causes disk names to display in the
cXtYdZsN format which is more easily associated with physical hardware
characteristics. The cXtYdZsN format is particularly useful in FibreChannel
(FC) environments where the FC World Wide Name appears in the t field.
Cindy Swearingen wrote:
> Hi--
>
> A google search of ST3500320AS turns up Seagate Barracuda drives.
>
> All 7 drives in the pool tank are ST3500320AS. The other two c1t0d0
> and c3d0 are unknown, but are not part of this pool.
>
> You can also use fmdump -eV to see how long c2t3d0 has had problems.
>
> Thanks,
>
> Cindy
>
> On 07/19/10 09:29, Yuri Homchuk wrote:
>> Thanks Cindy,
>>
>> But format shows exactly same thing:
>> All of them appear as Seagate, no WD at all...
>> How could it be ???
>>
>> # format
>> Searching for disks...done
>>
>>
>> AVAILABLE DISK SELECTIONS:
>> 0. c1t0d0 <DEFAULT cyl 60798 alt 2 hd 255 sec 63>
>> /p...@0,0/pci15d9,a...@5/d...@0,0
>> 1. c1t1d0 <ATA-ST3500320AS-SD15-465.76GB>
>> /p...@0,0/pci15d9,a...@5/d...@1,0
>> 2. c2t0d0 <ATA-ST3500320AS-SD15-465.76GB>
>> /p...@0,0/pci10de,3...@a/pci15d9,a...@0/s...@0,0
>> 3. c2t2d0 <ATA-ST3500320AS-SD15-465.76GB>
>> /p...@0,0/pci10de,3...@a/pci15d9,a...@0/s...@2,0
>> 4. c2t3d0 <ATA-ST3500320AS-SD15-465.76GB>
>> /p...@0,0/pci10de,3...@a/pci15d9,a...@0/s...@3,0
>> 5. c2t4d0 <ATA-ST3500320AS-SD15-465.76GB>
>> /p...@0,0/pci10de,3...@a/pci15d9,a...@0/s...@4,0
>> 6. c2t5d0 <ATA-ST3500320AS-SD15-465.76GB>
>> /p...@0,0/pci10de,3...@a/pci15d9,a...@0/s...@5,0
>> 7. c2t7d0 <ATA-ST3500320AS-SD15-465.76GB>
>> /p...@0,0/pci10de,3...@a/pci15d9,a...@0/s...@7,0
>> 8. c3d0 <GIGABYTE-100336D9CC244B01-0001-2.00GB>
>> /p...@1,0/pci1022,7...@2/pci-...@1/i...@1/c...@0,0
>> Specify disk (enter its number): ^C
>>
>>
>> Thanks again.
>>
>>
>> -----Original Message-----
>> From: Cindy Swearingen [mailto:cindy.swearin...@oracle.com] Sent:
>> Monday, July 19, 2010 9:16 AM
>> To: Yuri Homchuk
>> Cc: zfs-discuss@opensolaris.org
>> Subject: Re: [zfs-discuss] Help identify failed drive
>>
>> Hi--
>>
>> I don't know what's up with iostat -En but I think I remember a
>> problem where iostat does not correctly report drives running in
>> legacy IDE mode.
>>
>> You might use the format utility to identify these devices.
>>
>> Thanks,
>>
>> Cindy
>> On 07/18/10 14:15, Alxen4 wrote:
>>> This is a situation:
>>>
>>> I've got an error on one of the drives in 'zpool status' output:
>>>
>>> zpool status tank
>>>
>>> pool: tank
>>> state: ONLINE
>>> status: One or more devices has experienced an unrecoverable error. An
>>> attempt was made to correct the error. Applications are
>>> unaffected.
>>> action: Determine if the device needs to be replaced, and clear the
>>> errors
>>> using 'zpool clear' or replace the device with 'zpool replace'.
>>> see: http://www.sun.com/msg/ZFS-8000-9P
>>> scrub: none requested
>>> config:
>>>
>>> NAME STATE READ WRITE CKSUM
>>> tank ONLINE 0 0 0
>>> raidz2 ONLINE 0 0 0
>>> c1t1d0 ONLINE 0 0 0
>>> c2t0d0 ONLINE 0 0 0
>>> c2t2d0 ONLINE 0 0 0
>>> c2t3d0 ONLINE 1 0 0
>>> c2t4d0 ONLINE 0 0 0
>>> c2t5d0 ONLINE 0 0 0
>>> c2t7d0 ONLINE 0 0 0
>>>
>>> So I would like to replace 'c2t3d0'.
>>>
>>> I know for a fact the pool has 7 physical drives : 5 of Seagate and
>>> 2 of WD.
>>>
>>> I want to know if 'c2t3d0' Seagate or WD.
>>>
>>> If I run 'iostat -En' it shows that all c*t*d0 drives are Seagate
>>> and sd11/sd12 are WD.
>>>
>>> This totally confuses me...
>>> Why there are two different types of drives in iostat output :
>>> c*t*d0 and sd* ???
>>> How come all c*t*d0 appear as Seagate.I know for sure two of them
>>> are WD.
>>> Why WD drives appears as sd* and not as c*t*d0 ?
>>>
>>> Please help.
>>>
>>>
>>> ----------------------------------------------------------
>>>
>>> # iostat -En
>>>
>>>
>>> c1t1d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
>>> Vendor: ATA Product: ST3500320AS Revision: SD15 Serial No:
>>> Size: 500.11GB <500107862016 bytes>
>>> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
>>> Illegal Request: 54 Predictive Failure Analysis: 0
>>>
>>> c2t0d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
>>> Vendor: ATA Product: ST3500320AS Revision: SD15 Serial No:
>>> Size: 500.11GB <500107862016 bytes>
>>> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
>>> Illegal Request: 0 Predictive Failure Analysis: 0
>>>
>>> c2t1d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
>>> Vendor: ATA Product: ST3500320AS Revision: SD15 Serial No:
>>> Size: 500.11GB <500107862016 bytes>
>>> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
>>> Illegal Request: 0 Predictive Failure Analysis: 0
>>>
>>> c2t2d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
>>> Vendor: ATA Product: ST3500320AS Revision: SD15 Serial No:
>>> Size: 500.11GB <500107862016 bytes>
>>> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
>>> Illegal Request: 0 Predictive Failure Analysis: 0
>>>
>>> c2t3d0 Soft Errors: 0 Hard Errors: 9 Transport Errors: 9
>>> Vendor: ATA Product: ST3500320AS Revision: SD15 Serial No:
>>> Size: 500.11GB <500107862016 bytes>
>>> Media Error: 7 Device Not Ready: 0 No Device: 2 Recoverable: 0
>>> Illegal Request: 0 Predictive Failure Analysis: 0
>>>
>>> c2t4d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
>>> Vendor: ATA Product: ST3500320AS Revision: SD15 Serial No:
>>> Size: 500.11GB <500107862016 bytes>
>>> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
>>> Illegal Request: 0 Predictive Failure Analysis: 0
>>>
>>> c2t5d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
>>> Vendor: ATA Product: ST3500320AS Revision: SD15 Serial No:
>>> Size: 500.11GB <500107862016 bytes>
>>> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
>>> Illegal Request: 0 Predictive Failure Analysis: 0
>>>
>>> c2t6d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
>>> Vendor: ATA Product: ST3500320AS Revision: SD15 Serial No:
>>> Size: 500.11GB <500107862016 bytes>
>>> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
>>> Illegal Request: 0 Predictive Failure Analysis: 0
>>>
>>> c2t7d0 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
>>> Vendor: ATA Product: ST3500320AS Revision: SD15 Serial No:
>>> Size: 500.11GB <500107862016 bytes>
>>> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
>>> Illegal Request: 0 Predictive Failure Analysis: 0
>>>
>>> [b]sd11 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
>>> Vendor: ATA Product: WDC WD5001AALS-0 Revision: 1D05 Serial No:
>>> Size: 500.11GB <500107862016 bytes>
>>> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
>>> Illegal Request: 0 Predictive Failure Analysis: 0
>>>
>>> sd12 Soft Errors: 0 Hard Errors: 0 Transport Errors: 0
>>> Vendor: ATA Product: WDC WD5001AALS-0 Revision: 0K05 Serial No:
>>> Size: 500.11GB <500107862016 bytes>
>>> Media Error: 0 Device Not Ready: 0 No Device: 0 Recoverable: 0
>>> Illegal Request: 0 Predictive Failure Analysis: 0[/b]
>>>
>>>
>>>
>>>
>>>
>>> Thanks a lot.
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss@opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
