On 03/26/2013 02:44 PM, Bob Friesenhahn wrote: > On Tue, 26 Mar 2013, Sašo Kiselkov wrote: >> >> Once I gave it bit more thought, I realized tmpfs *should* be faster, >> since it doesn't traverse the block device/SCSI interface and instead >> intercepts calls pretty high up the VFS stack. Nonetheless, I suspect >> the tmpfs implementation isn't really designed for multi-GB/s throughput >> (it's a filesystem for /tmp FFS, it's supposed to hold a couple of kB of >> data anyway). > > Sašo, you are continuing to ignore that the simple dd to tmpfs test > turned in abysmal results on this quad Xeon E5 system as compared to the > many other systems tested. > > This seems to be a problem with the system, or the way Illumos runs on it. > > I had not heard of Illumos running on a quad Xeon E5 system before now. > It is doubtful that Illumos has been seriously tweaked/tuned for > particular newer hardware since the days of Sun. Most efforts seem on > the level of keeping things running properly without the considerable > resources required for performance tuning/testing. Maybe there are > significant issues to be resolved.
Well, seeing as his throughput to a 2x6-drive raidz2 seems totally expected, we are seeing a significant inconsistency here: 1) local ZFS throughput: 750 MB/s (OK) 2) COMSTAR over FC to ZFS-backed LUN: 200 MB/s (low?) 3) tmpfs throughput: <low> Please note that tmpfs and COMSTAR have *nothing* in common here (besides running on the same machine). Before going on to analyze performance pathologies, let's make at least sure the effect we're seeing here is real. To the simple question of "what kind of FC?" I've yet to hear an answer. As I said before, 200MB/s in writes on 4GbFC is something I've seen in reality before - in fact, I can reproduce it right now, at will: root@iptv-cfg-master:~# uname -a SunOS iptv-cfg-master 5.11 omnios-33fdde4 i86pc i386 i86pc Solaris root@iptv-cfg-master:~# zfs create -o compress=off rpool/test root@iptv-cfg-master:~# dd if=/dev/zero of=/rpool/test/ttt bs=1M count=2048 2048+0 records in 2048+0 records out 2147483648 bytes (2.1 GB) copied, 9.93533 s, 216 MB/s It's entirely possible tmpfs has problems on 4-socket, 64-core Xeon E5 systems, but resolving them need not help FC/COMSTAR throughput. First we need to establish that what we're looking at here has a common source. Cheers, -- Saso ------------------------------------------- illumos-discuss Archives: https://www.listbox.com/member/archive/182180/=now RSS Feed: https://www.listbox.com/member/archive/rss/182180/21175430-2e6923be Modify Your Subscription: https://www.listbox.com/member/?member_id=21175430&id_secret=21175430-6a77cda4 Powered by Listbox: http://www.listbox.com
