Edward Ned Harvey wrote: > I have a bunch of compute servers. They all have local disks mounted as > /scratch to use for computation scratch space. This ensures maximum > performance on all systems, and no competition for a shared resource > during crunch time. At present, all of their /scratch directories are > local, separate and distinct. I think it would be awesome if /scratch > looked the same on all systems. Does anyone know of a way to “unify” > this storage, without compromising performance? Of course, if some > files reside on server A, and they are requested from server B, then the > files must go across the network, but I don’t want the files to go > across the network unless they are requested. And yet, if you do > something like “ls /scratch” you would ideally get the same results > regardless of which machine you’re on. > > > > Due to the nature of heavy runtime IO (read, seek, write, repeat…) it’s > not well suited to NFS or any network filesystem… Due to the nature of > many systems all doing the same thing at the same time, it’s not well > suited to a SAN using shared disks… > > > > I looked at gfs (the cluster filesystem) – but – it seems gfs assumes a > shared disk (like a san) in which case there is competition for a shared > resource. > > > > I looked at gfs (the google filesystem) – but – it seems they constantly > push all the data across the network, which is good for redundancy and > mostly-just-read operations, and not good for heavy computation IO. > > > > Not sure what else I should look at. Any ideas?
I'll say this once more, in case it got lost in the volume before. You should definitely have a look at GlusterFS. It's free with commercial support available (they even have try-and-buy 30 day support packages). It's *really* fast given our 1GE testing. It's a little rough around some edges, but it's bones are very good. They're actively working on getting a feature-replete software suite. It does pretty much what your requirements say, plus a little bit more. With local affinity turned on, if you write data on machine X to the global FS, it will try to store it on the local disk so your read of the same data will be serviced locally. It also can do block or file level mirroring to other hosts for redundancy. It's definitely worth taking a look for scratch space. We're about to deploy it on some of our smaller clusters after fairly successful sandbox testing. _______________________________________________ Tech mailing list [email protected] http://lopsa.org/cgi-bin/mailman/listinfo/tech This list provided by the League of Professional System Administrators http://lopsa.org/
