On Wed, Jan 12, 2011 at 11:03:19PM -0400, Chris Forgeron wrote: > I've been testing out the v28 patch code for a month now, and I've yet to > report any real issues other than what is mentioned below. > > I'll detail some of the things I've tested, hopefully the stability of v28 in > FreeBSD will convince others to give it a try so the final release of v28 > will be as solid as possible. > > I've been using FreeBSD 9.0-CURRENT as of Dec 12th, and 8.2PRE as of Dec 16th > > What's worked well: > > - I've made and destroyed small raidz's (3-5 disks), large 26 disk raid-10's, > and a large 20 disk raid-50. > - I've upgraded from v15, zfs 4, no issues on the different arrays noted above > - I've confirmed that a v15 or v28 pool will import into Solaris 11 Express, > and vice versa, with the exception about dual log or cache devices noted > below. > - I've run many TB of data through the ZFS storage via benchmarks from my > VM's connected via NFS, to simple copies inside the same pool, or copies from > one pool to another. > - I've tested pretty much every compression level, and changing them as I > tweak my setup and try to find the best blend. > - I've added and subtracted many a log and cache device, some in failed > states from hot-removals, and the pools always stayed intact.
Thank you very much for all your testing, that's really a valuable
contribution. I'll be happy to work with you on tracking down the
bottleneck in ZFSv28.
> Issues:
>
> - Import of pools with multiple cache or log devices. (May be a very minor
> point)
>
> A v28 pool created in Solaris 11 Express with 2 or more log devices, or 2 or
> more cache devices won't import in FreeBSD 9. This also applies to a pool
> that is created in FreeBSD, is imported in Solaris to have the 2 log devices
> added there, then exported and attempted to be imported back in FreeBSD. No
> errors, zpool import just hangs forever. If I reboot into Solaris, import the
> pool, remove the dual devices, then reboot into FreeBSD, I can then import
> the pool without issue. A single cache, or log device will import just fine.
> Unfortunately I deleted my witness-enabled FreeBSD-9 drive, so I can't easily
> fire it back up to give more debug info. I'm hoping some kind soul will
> attempt this type of transaction and report more detail to the list.
>
> Note - I just decided to try adding 2 cache devices to a raidz pool in
> FreeBSD, export, and then importing, all without rebooting. That seems to
> work. BUT - As soon as you try to reboot FreeBSD with this pool staying
> active, it hangs on boot. Booting into Solaris, removing the 2 cache devices,
> then booting back into FreeBSD then works. Something is kept in memory
> between exporting then importing that allows this to work.
Unfortunately I'm unable to reproduce this. It works for me with 2 cache
and 2 log vdevs. I tried to reboot, etc. My test exactly looks like
this:
# zpool create tank raidz ada0 ada1
# zpool add tank cache ada0 ada1
# zpool export tank
# kldunload zfs
# zpool import tank
<works>
# reboot
<works>
> - Speed. (More of an issue, but what do we do?)
>
> Wow, it's much slower than Solaris 11 Express for transactions. I do
> understand that Solaris will have a slight advantage over any port of ZFS.
> All of my speed tests are made with a kernel without debug, and yes, these
> are -CURRENT and -PRE releases, but the speed difference is very large.
Before we go any further could you please confirm that you commented out
this line in sys/modules/zfs/Makefile:
CFLAGS+=-DDEBUG=1
This turns all kind of ZFS debugging and slows it down a lot, but for
the correctness testing is invaluable. This will be turned off once we
import ZFS into FreeBSD-CURRENT.
BTW. In my testing Solaris 11 Express is much, much slower than
FreeBSD/ZFSv28. And by much I mean two or more times in some tests.
I was wondering if they have some debug turned on in Express.
> At first, I thought it may be more of an issue with the ix0/Intel X520DA2
> 10Gbe drivers that I'm using, since the bulk of my tests are over NFS (I'm
> going to use this as a SAN via NFS, so I test in that environment).
>
> But - I did a raw cp command from one pool to another of several TB. I
> executed the same command under FreeBSD as I did under Solaris 11 Express.
> When executed in FreeBSD, the copy took 36 hours. With a fresh destination
> pool of the same settings/compression/etc under Solaris, the copy took 7.5
> hours.
When you turn off compression (because it turns all-zero blocks into
holes) you can test it by simply:
# dd if=/dev/zero of=/<zfs_fs>/zero bs=1m
--
Pawel Jakub Dawidek http://www.wheelsystems.com
p...@freebsd.org http://www.FreeBSD.org
FreeBSD committer Am I Evil? Yes, I Am!
pgprFLLYTe9F4.pgp
Description: PGP signature
