I have a bunch of compute servers. They all have local disks mounted as /scratch to use for computation scratch space. This ensures maximum performance on all systems, and no competition for a shared resource during crunch time. At present, all of their /scratch directories are local, separate and distinct. I think it would be awesome if /scratch looked the same on all systems. Does anyone know of a way to "unify" this storage, without compromising performance? Of course, if some files reside on server A, and they are requested from server B, then the files must go across the network, but I don't want the files to go across the network unless they are requested. And yet, if you do something like "ls /scratch" you would ideally get the same results regardless of which machine you're on.
Due to the nature of heavy runtime IO (read, seek, write, repeat.) it's not well suited to NFS or any network filesystem. Due to the nature of many systems all doing the same thing at the same time, it's not well suited to a SAN using shared disks. I looked at gfs (the cluster filesystem) - but - it seems gfs assumes a shared disk (like a san) in which case there is competition for a shared resource. I looked at gfs (the google filesystem) - but - it seems they constantly push all the data across the network, which is good for redundancy and mostly-just-read operations, and not good for heavy computation IO. Not sure what else I should look at. Any ideas? TIA.
_______________________________________________ Tech mailing list [email protected] http://lopsa.org/cgi-bin/mailman/listinfo/tech This list provided by the League of Professional System Administrators http://lopsa.org/
