On Wed, 1 Jul 2009, John Reddy wrote:
On Wed, Jul 1, 2009 at 6:53 PM, Edward Ned Harvey <[email protected]>wrote:
The goal I'm trying to accomplish - it's expected that some amount of
network traffic is required, but it should be minimal. If a 1G file is
created on some machine, then a few packets should fly across the network,
just to say the file exists. But 1G should not go anywhere.
I'm not sure I understand your goal then. There's no FS I know of that will
do what you're asking.
Your options are: local disk (ext3, xfs), shared disk (iSCSI, fiber channel)
running GFS (global FS, not google FS), network file system (NFS) or
distributed file system (lustre, GPFS, AFS).
Anything beyond local disk will require communication over the network.
With GFS, you'll mostly be speaking with the lock manager over the network.
So that would accomplish your goal of "write locally and only send limited
amount of data over the network". However, GFS isn't one of the shining
stars in distributed/parallel processing or HPC (high performance
computing).
I'm not sure that what he wants does exist (and neither is he, that's why
he's asking)
what he's wanting is a cluster filesystem where any system can read data
that lives on any drive, but when he creates a file it should be created
on the local drive.
the idea is to give local-drive performance for anything created by this
box, while falling back to NFS-like performance if you access files
created on a different box.
he's not planning to have a high-bandwidth network in place to share this
data.
I think you're saying - it writes across more than one machine, which would
slow down the write operation
Actually, writing across multiple hosts speeds it up. Much along the lines
of a RAID 0 striping pattern, the data is spread across multiple
destinations.
that assumes that your network is faster than your local disks
David Lang
_______________________________________________
Tech mailing list
[email protected]
http://lopsa.org/cgi-bin/mailman/listinfo/tech
This list provided by the League of Professional System Administrators
http://lopsa.org/
_______________________________________________
Tech mailing list
[email protected]
http://lopsa.org/cgi-bin/mailman/listinfo/tech
This list provided by the League of Professional System Administrators
http://lopsa.org/
