> From: zfs-discuss-boun...@opensolaris.org [mailto:zfs-discuss-
> boun...@opensolaris.org] On Behalf Of Edward Ned Harvey
> it doesn't work right - It turns out, iscsi
> devices (And I presume SAS devices) are not removable storage.  That
> means, if the device goes offline and comes back online again, it doesn't just
> gracefully resilver and move on without any problems, it's in a perpetual
> state of IO error, device unreadable.  

I am revisiting this issue today.  I've tried everything I can think of to 
recreate this issue, and haven't been able to do it.  I have certainly 
encountered some bad behaviors - which I'll expound upon momentarily - but they 
all seem to be addressable, fixable, logical problems, and none of them result 
in a supposedly good pool (as reported in zpool status) returning scsi IO 
errors or halting the system.  The most likely explanation right now, for the 
bad behavior I saw before, perpetual IO error even after restoring connection, 
is that I screwed something up in my iscsi config the first time.

Herein lie the new problems:

If I don't export the pool before rebooting, then either the iscsi target or 
initiator is shutdown before the filesystems are unmounted.  So the system 
spews all sorts of error messages while trying to go down, but it eventually 
succeeds.  It's somewhat important to know if it was the target or initiator 
that went down first - If it was the target, then only the local disks became 
inaccessible, but if it was the intiiator, then both the local and remote disks 
became inaccessible.  I don't know yet.

Upon reboot, the pool fails to import, so the svc:/system/filesystem/local 
service fails, and comes up in maintenance mode.  The whole world is a mess, 
you have to login at physical text console to export the pool, and reboot.  But 
it comes up cleanly the second time.

These sorts of problems seem like they should be solvable by introducing some 
service manifest dependencies...  But there's no way to make it a 
generalization for the distribution as a whole (illumos/openindiana/oracle).  
It's just something that should be solvable on a case-by-case basis.

If you are going to be an initiator only, then it makes sense for 
svc:/network/iscsi/initiator to be required by svc:/system/filesystem/local 

If you are going to be a target only, then it makes sense for 
svc:/system/filesystem/local to be required by svc:/network/iscsi/target

If you are going to be a target & initiator, then you could get yourself into a 
deadlock situation.  Make the filesystem depend on the initiator, and make the 
initiator depend on the target, and make the target depend on the filesystem.  

But we can break that cycle easy enough in a lot of situations - If you're 
doing as I'm doing, where the only targets are raw devices (not zvols) then it 
should be ok to make the filesystem depend on the initiator, which depends on 
the target, and the target doesn't depend on anything.

If you're both a target and an initiator, but all of your targets are zvols 
that you export to other systems (you're not nesting a filesystem in a zvol of 
your own, are you?) then it's ok to let the target needs filesystem and 
filesystem needs initiator, but initiator doesn't need anything.

So in my case, I'm sharing raw disks, I'm going to try and make filessytem 
needs initiator, initiator needs target, and target doesn't need anything.

Haven't tried yet ... Hopefully google will help accelerate me figuring out how 
to do that.

zfs-discuss mailing list

Reply via email to