On Wed, Jul 1, 2009 at 6:53 PM, Edward Ned Harvey <[email protected]>wrote:
> The goal I'm trying to accomplish - it's expected that some amount of > network traffic is required, but it should be minimal. If a 1G file is > created on some machine, then a few packets should fly across the network, > just to say the file exists. But 1G should not go anywhere. I'm not sure I understand your goal then. There's no FS I know of that will do what you're asking. Your options are: local disk (ext3, xfs), shared disk (iSCSI, fiber channel) running GFS (global FS, not google FS), network file system (NFS) or distributed file system (lustre, GPFS, AFS). Anything beyond local disk will require communication over the network. With GFS, you'll mostly be speaking with the lock manager over the network. So that would accomplish your goal of "write locally and only send limited amount of data over the network". However, GFS isn't one of the shining stars in distributed/parallel processing or HPC (high performance computing). I think you're saying - it writes across more than one machine, which would > slow down the write operation Actually, writing across multiple hosts speeds it up. Much along the lines of a RAID 0 striping pattern, the data is spread across multiple destinations. > I've heard of lustre, but never seen it or read anything about it before. > The Supercomputing Top 500 was recently released. Lustre is in use on two thirds of the top 500, and 7 out of the top 10 supercomputers. It's designed for massively parallel access at incredibly high speeds. It's certainly not a swiss army knife, but your requirements of "fast" and "visible to all nodes" would suggest that it might do what you want. How many client computers are you talking about, though? -John
_______________________________________________ Tech mailing list [email protected] http://lopsa.org/cgi-bin/mailman/listinfo/tech This list provided by the League of Professional System Administrators http://lopsa.org/
