On 2012-11-08 4:43, Edward Ned Harvey
(opensolarisisdeadlongliveopensolaris) wrote: 

> When I said
performance was abysmal, I meant, if you dig right down and pressure the
system for throughput to disk, you've got a Linux or Windows VM isnide
of ESX, which is writing to a virtual disk, which ESX is then wrapping
up inside NFS and TCP, talking on the virtual LAN to the ZFS server,
which unwraps the TCP and NFS, pushes it all through the ZFS/Zpool
layer, writing back to the virtual disk that ESX gave it, which is
itself a layer on top of Ext3, before it finally hits disk. Based purely
on CPU and memory throughput, my VM guests were seeing a max throughput
of around 2-3 Gbit/sec. That's not *horrible* abysmal. But it's bad to
be CPU/memory/bus limited if you can just eliminate all those extra
layers, and do the virtualization directly isnide a system that supports


I have some experience with Virtualisation, mainly using Xen
and KVM, but not much. I am just wondering why you export the ZFS system
through NFS? 

I have had much better results (albeit spending more time
setting up) using iSCSI. I found that performance was much better, I
believe because a layer was being cut out of the loop. Rather than the
hypervisor having to emulate block storage from files on NFS, the block
storage is exposed directly from Solaris (in my case) through iSCSI, and
passed through the virtual LAN to the other guests. The hypervisor then
sees nothing but ethernet packets. 

This was a few years ago and I
can't remember the numbers, but CPU consumption was drastically reduced
and performance of the guests was increased significantly. 

The only
downside was a slightly more complicated setup for the guests, but not
by enough to sacrifice the performance benefits. 

It is possible that
this is not the case in ESXi, or that more modern hypervisors deal with
it more efficiently. That's why I'm asking the question :) 
zfs-discuss mailing list
        • Re: ... Edward Ned Harvey (opensolarisisdeadlongliveopensolaris)
          • ... Dan Swartzendruber
            • ... Edward Ned Harvey (opensolarisisdeadlongliveopensolaris)
            • ... Dan Swartzendruber
            • ... Dan Swartzendruber
            • ... Dan Swartzendruber
            • ... Geoff Nordli
            • ... Dan Swartzendruber
            • ... Edward Ned Harvey (opensolarisisdeadlongliveopensolaris)
          • ... Dan Swartzendruber
      • Re: [zfs... Karl Wagner
        • Re: ... Edward Ned Harvey (opensolarisisdeadlongliveopensolaris)
          • ... Karl Wagner
            • ... Edward Ned Harvey (opensolarisisdeadlongliveopensolaris)
            • ... Jim Klimov
      • Re: [zfs... Dan Swartzendruber
  • Re: [zfs-discuss]... Jim Klimov

Reply via email to