Wow Craig - thank you so much for that thorough response.
I am only using 1 vdev and I didn't realize two things:
1) that 50 GB on each of the 300s is essentially wasted. I thought it
would spread 300 GB of parity across all 6 disks, leaving me with 1350
GB of data space. Instead, you're saying 250 GB of parity is spread
across all 6 disks and an additional 150 GB is ignored, leaving me with,
as you said, 1250 GB data space.
2) I wasn't sure if zpool list described the total before or after
parity was subtracted (and extra space ignored). Thanks for clearing
that up.
However, I'm still a little confused how this adds up to 1.11T and 1.36T
for zfs list and zpool list, respectively (my box is under construction
atm so I can't capture the exact output).
To minimize the wasted space, can I create 1x250 and 1x50 GB partitions
on each of the 300 GB drives, then make them a new raidz vdev?
1st raidz 6x250 GB partitions: 1250 GB data space
2nd raidz 3x50 GB partitions: 100 GB data space
total: 1350 GB data space
Can I pool these together into one large pool? I'm trying to think it if
would be possible to lose data based on any one disk failure.
As my signature says, despite working for Sun I work on everything *but*
Solaris every day, so I really appreciate the guidance.
Just an FYI - I started with opensolaris but I needed to add a combo
IDE/SATA card to handle all my spare disks (mostly varying sizes) -
which wasn't recognized by opensolaris. I moved to Linux (which detects
the drives on the I/O card) with zfs-fuse (filesystem in userspace -
steps around CDDL/GPL incompatibility) but I found the features limited
and where it compiled on one distribution that had its own set of
unrelated issues, it failed to compile on another. Currently, I'm
installing OpenSolaris in a VirtualBox VM on a Linux host using raw disk
passthrough so I can use zfs with this I/O card. We'll see how it
goes :)
Thanks again,
-Brent
On Tue, 2009-03-17 at 18:02 -0700, Craig Cory wrote:
Brent,
Brent Wagner wrote:
Can someone point me to a document describing how available space in a
zfs is calculated or review the data below and tell me what I'm
missing?
Thanks in advance,
-Brent
===
I have a home project with 3x250 GB+3x300 GB in raidz, so I expect to
lose 1x300 GB to parity.
Total size:1650GB
Total size using 1024 to measure: ~1534 GB
Expected raidz zpool size after losing 300 GB to parity: ~1350 GB
Expected raidz zpool size using 1024 to measure: ~1255.5 GB
Actual zpool size: 1.36T
Single zfs on the pool - available size: 1.11T
I realize zfs is going to have some overhead but 250 GB seems a little
excessive...right? I thought maybe the zpool was showing all 6 disks and
the filesystem reflected the remaining space after discounting the
parity disk but that doesn't add up in a way that makes sense either
(see above). Can someone help explain these numbers?
Thanks,
-Brent
When you say 3x250 GB+3x300 GB in raidz do you mean:
1) # zpool create mypool raidz 250gb-1 250gb-2 250gb-3 300gb-1 \
300gb-2 300gb-3
or
2) # zpool create mypool raidz 250gb-1 250gb-2 250gb-3 \
raidz 300gb-1 300gb-2 300gb-3
As I understand it, #1 would waste the extra 50gb on each 300gb drive and give
you 1500gb usable space. 250gb of that (1/6th) would be parity, so 1250gb
data space.
#2 would make 2 vdevs of 750gb and 900gb totaling 1650gb space. Parity would
use 250gb from the 1st vdev and 300gb from the second; so 1100gb of data
space is available.
Either way, when you list raidz* pools with
# zpool list
you see the total physical space. When you list the filesystems with
# zfs list
you get the usable filesystem space, which is where the parity is implemented.
Here's an example with 250MB files and 300MB files:
For #1 scenario:
# zpool create -f mypool1 raidz /250d1 /250d2 /250d3 /300d1 /300d2 /300d3
# zpool list
NAMESIZEUSED AVAILCAP HEALTH ALTROOT
mypool11.44G145K 1.44G 0% ONLINE -
# zfs list
NAME USED AVAIL REFER MOUNTPOINT
mypool1 115K 1.16G 40.7K /mypool1
# zpool status
pool: mypool1
state: ONLINE
scrub: none requested
config:
NAMESTATE READ WRITE CKSUM
mypool1 ONLINE 0 0 0
raidz1ONLINE 0 0 0
/250d1 ONLINE 0 0 0
/250d2 ONLINE 0 0 0
/250d3 ONLINE 0 0 0
/300d1 ONLINE 0 0 0
/300d2 ONLINE 0 0 0
/300d3 ONLINE 0 0 0
--
And for #2:
# zpool create -f mypool2 raidz /250d1 /250d2 /250d3 raidz /300d1 /300d2
/300d3
# zpool list
NAMESIZEUSED AVAILCAP HEALTH ALTROOT
mypool2
