2012-05-16 22:21, bofh wrote:
There's something going on then.  I have 7x 3TB disk at home, in
raidz3, so about 12TB usable.  2.5TB actually used.  Scrubbing takes
about 2.5 hours.  I had done the resilvering as well, and that did not
take 15 hours/drive.

That is the critical moment ;)
The system we plan to upgrade runs nearly full for a couple of
years now, with some overflowing data spilling out to another
server or performant workstation.

This system has a lot of writes and rewrites (rolling backups,
VM images, home dirs and compilations, document storage and
so on) compounded by zfs-auto-snap services. Due to the fact
that there are many rewrites like this, fragmentation is a
substantial hit to scrub and resilver performance which both
need to walk the whole block-pointer tree which yields lots
of random reads of small blocks from all over the pool.

I began writing a letter with guesses/questions about that, but
it seems to be lengthy and may get published in a few days ;)

Additionally, if you're only replacing one drive at a time, you're
only resilvering 250GB at a time, regardless of the size of the new

That is true.

If you already have 45X 3TB drives waiting to go in, bite the bullet
and get that eSATA cage, since you want to re-do your zpools.  You can
reuse it for offsite backups in the future.

That is a good plan, except that the server's owners plan to
upgrade a dozen drives for now. Even this would triple the
current pool's size in a quarter of disk bays used. They only
have one for testing on hands and plan to buy a dozen more;
there are no 45 drives waiting. There is no spoon, either ;)

I will try to get them into making a (local/offsite) backup
box as well. Since a new server (from x4540 to a handmade
Supermicro) would likely be more performant and have more
RAM, I expect that this Thumper would be the backup box
for a new server, ultimately.

zfs-discuss mailing list

Reply via email to