Heya all, I'm working on testing ZFS with NFS, and I could use some guidance - read speeds are a bit less than I expected.
Over a gig-e line, we're seeing ~30 MB/s reads on average - doesn't seem to matter if we're doing large numbers of small files or small numbers of large files, the speed seems to top out there. We've disabled pre-fetching, which may be having some affect on read speads, but proved necessary due to severe performance issues on database reads with it enabled. (Reading from the DB with pre-fetching enabled was taking 4-5 times as long than with it disabled.) Write speed seems to be fine. Testing is showing ~95 MB/s, which seems pretty decent considering there's been no real network tuning done. The NFS server we're testing is a Sun x4500, configured with a storage pool consisting of 20x 2-disk mirrors, using separate SSD for logging. It's running the latest version of Nexenta Core. (We've also got a second x4500 in with a raidZ2 config, running OpenSolaris proper, showing the same issues with reads.) We're using NFS v4 via TCP, serving various Linux clients (the majority are CentOS 5.3). Connectivity is presently provided by a single gigabit ethernet link; entirely conventional configuration (no jumbo frames/etc). Our workload is pretty read heavy; we're serving both website assets and databases via NFS. The majority of files being served are small (< 1MB). The databases are MySQL/InnoDB, with the data in separate zfs filesystems with a record size of 16k. The website assets/etc. are in zfs filesystems with the default record size. On the database server side of things, we've disabled InnoDB's double write buffer. I'm wondering if there's any other tuning that'd be a good idea for ZFS in this situation, or if there's some NFS tuning that should be done when dealing specifically with ZFS. Any advice would be greatly appreciated. Thanks, -- -------------------------------------------------------------------------------------------------- Gary Gogick senior systems administrator | workhabit,inc. // email: g...@workhabit.com | web: http://www.workhabit.com // office: 866-workhabit | fax: 919-552-9690 --------------------------------------------------------------------------------------------------
_______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
