cross-posting to nfs-discuss On Oct 20, 2009, at 10:35 AM, Gary Gogick wrote:
> Heya all, > > I'm working on testing ZFS with NFS, and I could use some guidance - > read speeds are a bit less than I expected. > > Over a gig-e line, we're seeing ~30 MB/s reads on average - doesn't > seem to matter if we're doing large numbers of small files or small > numbers of large files, the speed seems to top out there. We've > disabled pre-fetching, which may be having some affect on read > speads, but proved necessary due to severe performance issues on > database reads with it enabled. (Reading from the DB with pre- > fetching enabled was taking 4-5 times as long than with it disabled.) What is the performance when reading locally (eliminate NFS from the equation)? -- richard > > Write speed seems to be fine. Testing is showing ~95 MB/s, which > seems pretty decent considering there's been no real network tuning > done. > > The NFS server we're testing is a Sun x4500, configured with a > storage pool consisting of 20x 2-disk mirrors, using separate SSD > for logging. It's running the latest version of Nexenta Core. > (We've also got a second x4500 in with a raidZ2 config, running > OpenSolaris proper, showing the same issues with reads.) > > We're using NFS v4 via TCP, serving various Linux clients (the > majority are CentOS 5.3). Connectivity is presently provided by a > single gigabit ethernet link; entirely conventional configuration > (no jumbo frames/etc). > > Our workload is pretty read heavy; we're serving both website assets > and databases via NFS. The majority of files being served are small > (< 1MB). The databases are MySQL/InnoDB, with the data in separate > zfs filesystems with a record size of 16k. The website assets/etc. > are in zfs filesystems with the default record size. On the > database server side of things, we've disabled InnoDB's double write > buffer. > > I'm wondering if there's any other tuning that'd be a good idea for > ZFS in this situation, or if there's some NFS tuning that should be > done when dealing specifically with ZFS. Any advice would be > greatly appreciated. > > Thanks, > > -- > -------------------------------------------------------------------------------------------------- > Gary Gogick > senior systems administrator | workhabit,inc. > > // email: gary at workhabit.com | web: http://www.workhabit.com > // office: 866-workhabit | fax: 919-552-9690 > > -------------------------------------------------------------------------------------------------- > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
