Storage Discuss,
I've just finished building a large storage array. The array is comprised
of seven (7) shelves each with 24 disks. The disks (1TB SATA II) are
connected to a 3ware 9650SE in each shelf. None of the controllers have
BBU's.
I create 4 ZFS volumes (4.4TB usable) of six (6) disks per shelf. I export
them as iSCSI LU's with COMSTAR.
On the head, I mount all 28 LUN's making 4 RAIDZ's in a single ZFS. Each
RAIDZ is comprised of one LU of each shelf.
I then share the large ZFS with NFS. I have a Linux NFS client (NFSv3)
mount it and perform a bonnie run. The linux server has 4GB of RAM.
Here is the output:
[r...@qa-www-s106:~/bonnie-64-read-only]# ./Bonnie -d /limefs/ -s 8192
File '/limefs//Bonnie.3111', size: 8589934592
Writing with putc()...done
Rewriting...done
Writing intelligently...done
Reading with getc()...done
Reading intelligently...done
Seeker 1...Seeker 3...Seeker 2...start 'em...done...done...done...
-------Sequential Output-------- ---Sequential Input--
--Random--
-Per Char- --Block--- -Rewrite-- -Per Char- --Block---
--Seeks---
Machine GB M/sec %CPU M/sec %CPU M/sec %CPU M/sec %CPU M/sec %CPU /sec
%CPU
8 0.8 1.1 1.9 0.4 1.9 0.4 73.5 99.2 106.2 7.3 8
0.1
[r...@qa-www-s106:~/bonnie-64-read-only]#
As you see, the read performance is pretty good, but the write performance
is horrible.
Also, I keep seeing this in the Linux server's /var/log/messages:
Jun 30 22:09:57 qa-www-s106 kernel: nfs: server 172.17.96.200 not
responding, still trying
Jun 30 22:09:57 qa-www-s106 kernel: nfs: server 172.17.96.200 not
responding, still trying
Jun 30 22:09:57 qa-www-s106 kernel: nfs: server 172.17.96.200 not
responding, still trying
Jun 30 22:09:58 qa-www-s106 kernel: nfs: server 172.17.96.200 OK
Jun 30 22:09:58 qa-www-s106 kernel: nfs: server 172.17.96.200 OK
Jun 30 22:09:58 qa-www-s106 kernel: nfs: server 172.17.96.200 OK
>From some googling, I believe this has to do with cache flushes, but I'm not
certain. Can anyone shed light on this?
Best regards,
Michael Schenck
--
Michael Schenck - www.tek-ops.com
_______________________________________________
storage-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/storage-discuss
