2012-09-30 19:29, Mathieu Simon wrote:
I thought the ZFS mirror on separately hosted iscsi targets was going to be a
great idea ... it turns out it's only great at destroying your pool...
iSCSI is not a shared storage and if you write to the same disk/lun from 2
initiators without "something in between" like DRBD (Linux), AVS, or
HAST (FreeBSD)
things are really expected to cause problems or call other evil dragons. :-)
Well, if you read carefully, Edward did not use this as shared
storage (yet), he made two separate raid10 pools, each one
using two local and two remote sides of the mirrors. So as long
as each host only imports the same one pool, there is no shared
storage and cluster-unaware-zfs problem. Just one machine has
an up-to-date replica of another's storage in case of, say, more
fatal problems where an admin would online the pool halves on
the surviving machine.
Compared to replication with zfs-send|zfs-recv, such mirroring
does not protect against admin/user errors (i.e. deletion of
needed datasets), but does (should) provide instant replication.
Also, I can't really comment on "iSCSI is not a shared storage",
but I would expect it to be, and to allow several clients to
read and write to a common block device shared by the iSCSI
server(target). *How* these clients don't run into conflicts,
i.e. by using a cluster-aware FS on top of this block device,
is a problem for a different layer of the storage stack - the
FS layer.
2012-09-30 2:44, Richard Elling wrote:
I would expect a little more plug-n-play-ish intelligence here. When
the remote iscsi device disappears, it seems to me, it should be
handled a little more similarly to somebody yanking out the USB
external disk.
iSCSI devices are not removable disks.
Given remote connectability, they actually should be, no?
On local disks you have the HBA reporting the link up-state,
on USB and such as well. Over network this catering for some
occasional unavailability is not only natural, but expected.
Rather than retrying and increasing the error count, the running
system should be alerted this disk is going offline, and handle that
situation more gracefully ... Well, that's just my expectation
anyway. I only care right now because it's causing problems.
Given the constraints of IP, how do you propose that an iSCSI initiator
will discover that a remote LU is down?
How about the target device (server) sending a message to the
initiators when the service is going to be administratively
down (i.e. during shutdown of the stmf daemon), and notifying
them when the service is going up (perhaps, remembering the
connected clients at the moment of shutdown, as NFS server
servers seem to do)?
I understand that this could be stapled on externally with
some clusterware or scripts of choice, but putting this into
the protocol also seems like a useful idea.
This would not be a magical silver bullet useful for any
life situation (i.e. when networking gear breaks or loses
power), but in case where shutdown is administrative and
comms work when you expect them to, such approach can help.
My 2c,
//Jim Klimov
-------------------------------------------
illumos-discuss
Archives: https://www.listbox.com/member/archive/182180/=now
RSS Feed: https://www.listbox.com/member/archive/rss/182180/21175430-2e6923be
Modify Your Subscription:
https://www.listbox.com/member/?member_id=21175430&id_secret=21175430-6a77cda4
Powered by Listbox: http://www.listbox.com
