Blake,
Thx for the link. He's talking about doing the little lab
experiment I was doing some months back where multiple nodes share
their free space to form a storage pool that exists, erm,
'holographically' on the net. Too bad nobody else is in on the
conversation -- I think it's a cool idea.
Holographically in the sense that it's always there and always
assured to be perfectly accurate (given good level of redundancy of
your RAID setup), but the vibrancy (speed) of the picture (data) is
directly proportional to your connectivity with the member hosts.
Would be interesting to discuss this in a scatter-gather context and a
p2p context as well.
Ah, might as well post this & see if there's any interest..
Sorry for the blatant abuse of the term holographic in a fublic porum.
jake
On Nov 15, 2007 11:26 AM, Blake <[EMAIL PROTECTED]> wrote:
> this is the whole thread :)
>
>
>
> Forwarded conversation
> Subject: [storage-discuss] ZFS and clustering - wrong place?
> ------------------------
>
> From: Ross <[EMAIL PROTECTED]>
> Date: Nov 6, 2007 10:40 AM
> To: [email protected]
>
>
> I'd posted this under ZFS, but I guess this is the better place.
>
> I'm just starting to learn about Solaris, ZFS, etc... It's amazing me how
> much is possible, but it's just shy of what I'd really, really like to see.
>
> I can see there's a fair amount of interest in ZFS and clustering, and it
> seems Sun are actively looking into this, but I'm wondering if that's the
> right place to do it?
>
> Now I may be missing something obvious here, but it seems to me that for
> really reliable clustering of data you need to be dealing with it at a
> higher layer, effectively where iSCSI sits. Instead of making ZFS cluster
> aware, wouldn't it be easier to add support for things like mirroring,
> striping (even raid) to the iSCSI protocol?
>
> That way you get to use ZFS locally with all the benefits that entails
> (guaranteed data integrity, etc), and you also have a protocol somewhere in
> the network layer that guarantees data integrity to the client (confirming
> writes at multiple locations, painless failover, etc...). Essentially doing
> for iSCSI what ZFS did for disk.
>
> You'd need support for this in the iSCSI target as it would seem make sense
> to store the configuration of the cluster on every target. That way the
> client can connect to any target and read the information on how it is to
> connect.
>
> But once that's done, your SAN speed is only limited by the internal speed
> of your switch. If you need fast performance, add half a dozen devices and
> stripe data across them. If you need reliability mirror them. If you want
> both, use a raid approach. Who needs expensive fibre channel when you can
> just stripe a pile of cheap iSCSI devices?
>
> It would make disaster recovery and HA a piece of cake. For any network
> like ourselves with a couple of offices and a fast link between them (any
> university campus would fit that model), you just have two completely
> independent servers and configure the clients to stream data to them both.
> No messy configuration of clustered servers, and support for multicast on
> the network means you don't even have to slow your clients down.
>
> The iSCSI target would probably need to integrate with the file system to
> cope with disasters. You'd need an intelligent way to re-synchronise
> machines when they came back online, but that shouldn't be too difficult
> with ZFS.
>
> I reckon you could turn Solaris & ZFS into the basis for one of the most
> flexible SAN solutions out there.
>
> What do you all think? Am I off my rocker or would an approach like this
> work? Would love to hear some feedback on the idea.
>
>
> This message posted from opensolaris.org
> _______________________________________________
> storage-discuss mailing list
> [email protected]
> http://mail.opensolaris.org/mailman/listinfo/storage-discuss
> ----------
> From: Ross <[EMAIL PROTECTED]>
> Date: Nov 7, 2007 3:15 AM
> To: [email protected]
>
>
> Hmm... just found an article from 2003 talking about almost exactly the same
> idea. If it's this obvious and this simple, how come nobody (apart from
> Lefthand) seems to be doing it?
>
> http://www.byteandswitch.com/document.asp?doc_id=34249&print=true
>
> ----------
> From: Ross <[EMAIL PROTECTED]>
> Date: Nov 8, 2007 6:22 AM
> To: [email protected]
>
>
> Just had another thought about this. One of the benefits of RAID-Z is meant
> to be the low calculation overhead. Could the ZFS code be ported straight
> into the iSCSI client? Also, if the iSCSI client knows how to save the data
> to multiple places, do you actually need anything configured on the iSCSI
> server end?
>
> Ok, for disaster recovery and rebuilding purposes it would be helpful to
> have the storage device able to take care of that. But it's not strictly
> necessary for this to work.
>
> In addition to the obvious network benefit of reducing redundant data
> transmitted over the wire, you're also distributing the raid calculations
> across your servers. So you don't need one powerful dedicated raid
> controller, and you don't need dedicated NIC's for the storage network
> (although those would still be beneficial for rebuilding).
>
> By putting it in the iSCSI client, you also remove the requirement for all
> the devices to be identical, making disaster recovery a lot easier if you
> ever have old equipment fail. No need to source obsolete parts, you can
> simply drop any new iSCSI box in it's place.
>
> Surely it can't be this simple?
>
> ----------
> From: Ross <[EMAIL PROTECTED] >
> Date: Nov 9, 2007 8:10 AM
> To: [email protected]
>
>
> Hmm... thinking about this more (and bear in mind I'm having to guess about
> internal workings here as I'm very new to ZFS, SAN's, clustering and iSCSI),
> if you use iSCSI for the clustering, you've got the problem of what's
> actually being saved to each system, and how do you track it.
>
> You'd need to be implementing a file system for this to work wouldn't you?
> So in essence, you'd be repeating effort for a lot of the stuff in ZFS.
>
> So then I thought about shoehorning iSCSI support inside of ZFS, but I think
> I've read that's already possible - you can use an iSCSI target as a raw
> disk for ZFS?
>
> If that's the case, you can actually distribute ZFS already across multiple
> systems, using striping, mirroring, even raid-z2. The only problem is you
> have a single point of failure in the ZFS host.
>
> Which then makes sense of the current plans to implement clustering support
> for ZFS. If you can cluster that, and you can cluster iSCSI targets, you
> then have a highly available ZFS service where the data can also be stored
> on a distributed network of servers.
>
> Going a stage further, if you consider the individual servers providing the
> iSCSI targets for this. They can also be running ZFS internally. So you're
> using two layers of ZFS - one guarding against disk failure, and another
> guarding against host / controller / network failure. And because it's ZFS,
> you're also doing checksummed data integrity checking on all your network
> traffic.
>
> SWEET!
>
> I think this also means you could hot swop or retire servers really easily.
> Just use zpool replace on the network ZFS and all of a sudden retiring an
> old server and bringing in a new one is a piece of cake.
>
> At the most basic level I wonder if you'll be able to implement this with
> just two servers (would handy as we're probably buying two thumpers
> shortly). Run ZFS on each sharing it out over an iSCSI target. Then also
> run HA-ZFS on each box (can I do that at the same time?), and have that
> create a mirrored set of the two iSCSI targets. Then use HA-iSCSI and
> (fingers crossed) HA-CIFS to make that available on your network (in our
> case to windows servers and clients).
>
> If that works, voila! Instant, cheap, easy to manage, and superbly
> realiable storage for any network you care to imagine. And it's scalable
> and expandable too. Either by upgrading the individual drives, by replacing
> the thumpers with bigger ones (lol, let me dream) and using zpool replace,
> or by adding extra servers with zpool add.
>
> Now *that's* what I call an enterprise file system (and affordable for small
> companies too!)
>
>
>
_______________________________________________
storage-discuss mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/storage-discuss
