On Wed, Sep 28, 2011 at 8:21 AM, Edward Ned Harvey
> When a vdev resilvers, it will read each slab of data, in essentially time
> order, which is approximately random disk order, in order to reconstruct the
> data that must be written on the resilvering device. This creates two
> problems, (a) Since each disk must fetch a piece of each slab, the random
> access time of the vdev as a whole is approximately the random access time
> of the slowest individual device. So the more devices in the vdev, the
> worse the IOPS for the vdev... And (b) the more data slabs in the vdev, the
> more iterations of random IO operations must be completed.
> In other words, during resilvers, you're IOPS limited. If your pool is made
> of all SSD's, then problem (a) is basically nonexistent, since the random
> access time of all the devices are equal and essentially zero. Problem (b)
> isn't necessarily a problem... It's like, if somebody is giving you $1,000
> for free every month and then they suddenly drop down to only $500, you
> complain about what you've lost. ;-) (See below.)
If you regularly spend all of the given $1,000, then you're going to
complain hard when it suddenly drops to $500.
> So again: Not a problem if you're making your pool out of SSD's.
Big problem if your system is already using most of the available IOPS
during normal operation.
zfs-discuss mailing list