cross-posting to nfs-discuss
On Oct 20, 2009, at 10:35 AM, Gary Gogick wrote:
Heya all,
I'm working on testing ZFS with NFS, and I could use some guidance -
read speeds are a bit less than I expected.
Over a gig-e line, we're seeing ~30 MB/s reads on average - doesn't
seem to matter if we're doing large numbers of small files or small
numbers of large files, the speed seems to top out there. We've
disabled pre-fetching, which may be having some affect on read
speads, but proved necessary due to severe performance issues on
database reads with it enabled. (Reading from the DB with pre-
fetching enabled was taking 4-5 times as long than with it disabled.)
What is the performance when reading locally (eliminate NFS from the
equation)?
-- richard
Write speed seems to be fine. Testing is showing ~95 MB/s, which
seems pretty decent considering there's been no real network tuning
done.
The NFS server we're testing is a Sun x4500, configured with a
storage pool consisting of 20x 2-disk mirrors, using separate SSD
for logging. It's running the latest version of Nexenta Core.
(We've also got a second x4500 in with a raidZ2 config, running
OpenSolaris proper, showing the same issues with reads.)
We're using NFS v4 via TCP, serving various Linux clients (the
majority are CentOS 5.3). Connectivity is presently provided by a
single gigabit ethernet link; entirely conventional configuration
(no jumbo frames/etc).
Our workload is pretty read heavy; we're serving both website assets
and databases via NFS. The majority of files being served are small
(< 1MB). The databases are MySQL/InnoDB, with the data in separate
zfs filesystems with a record size of 16k. The website assets/etc.
are in zfs filesystems with the default record size. On the
database server side of things, we've disabled InnoDB's double write
buffer.
I'm wondering if there's any other tuning that'd be a good idea for
ZFS in this situation, or if there's some NFS tuning that should be
done when dealing specifically with ZFS. Any advice would be
greatly appreciated.
Thanks,
--
--------------------------------------------------------------------------------------------------
Gary Gogick
senior systems administrator | workhabit,inc.
// email: g...@workhabit.com | web: http://www.workhabit.com
// office: 866-workhabit | fax: 919-552-9690
--------------------------------------------------------------------------------------------------
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
