This discussion is very timely, but I don't think we're done yet. I've
been working on using NexentaStor with Sun's DVI stack. The demo I've
been playing with glues SunRays to VirtualBox instances using ZFS zvols
over iSCSI for the boot image, with all the associated ZFS
snapshot/clone goodness we all love so well.
The supported config for the ZFS storage server is Solaris 10u7 or 10u8.
When I eventually got VDI going with NexentaStor (my value add), I found
that some operations which only took 10 minutes with Solaris 10u8 were
taking over an hour with NexentaStor. Using pfiles I found that
iscsitgtd has the zvol open O_SYNC.
My hope is that COMSTAR is a lot more intelligent, and that it does
indeed support DKIOCFLUSHWRITECACHE. However, if your iSCSI client
expects all writes to be flushed synchronously, all the debate we've
seen on this list about the new wcd=false option for rdsk zvols is moot
(as using the option, when it is available, could result in data loss).
When you do iSCSI to other big brand storage appliances, you generally
have the benefit of NVRAM cacheing. As we all know, the same can be
achieved with ZFS and an SSD "Logzilla". I didn't have one at hand, and
I didn't think of disabling the ZIL (although some have reported that
this only seems to help ZFS hosted files, not zvols). Instead, since I
didn't mind losing my data, for the same of experiment, I added a TMPFS
"Logzilla" ...
# mkfile 4g /tmp/zilla
# zpool add vdipool log /tmp/zilla
WARNING: DON'T TRY THIS ON ZPOOLS YOU CARE ABOUT! However, for this
purposes of my experiment, it worked a treat, proving to me that an SSD
"Logzilla" was the way ahead.
I think a lot of the angst in this thread is because "it used to work"
(i.e. we used to get great iSCSI performance from zvols). But then Sun
fixed a glaring bug (i.e. that zvols were unsafe for synchronous writes)
and our world fell apart.
Whilst the latest bug fixes put the world to rights again with respect
to correctness, it may be that some of our performance workaround are
still unsafe (i.e. if my iSCSI client assumes all writes are
synchronised to nonvolatile storage, I'd better be pretty sure of the
failure modes before I work around that).
Right now, it seems like an SSD "Logzilla" is needed if you want
correctness and performance.
Phil Harman
Harman Holistix - focusing on the detail and the big picture
Our holistic services include: performance health checks, system tuning,
DTrace training, coding advice, developer assassinations
http://blogs.sun.com/pgdh (mothballed)
http://harmanholistix.com/mt (current)
http://linkedin.com/in/philharman
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
