On 05/27/2011 07:12 AM, Jon Tegner wrote:
A general question, suppose I have a parallel application, using mpi,
where really fast access to the file system is critical.
Would it be stupid to consider a ram disk based setup? Say a 36 port QDR
Ram disks won't work directly, due to lack of locking in tmpfs. You
could create a tmpfs, then create a file that fills this up, then a
loopback device pointing to that file, then build a file system atop
that, and mount it. And then mount gluster atop that.
Needless to say, all these layers significantly decrease performance and
introduce inefficiencies.
infiniband with half of the ports connected to computational nodes and
the other half to gluster nodes?
There may be other options, but the options are not going to be
cheap/inexpensive. How fast, and by fast do you mean bandwidth and/or
latency (e.g. streaming bandwidth or random IOPs)? What does your IO
profile look like?
You can get nodes that stream 4.6+ GB/s read, and 3.6+ GB/s writes for
single readers/writers to single files. For MPI jobs with single
readers/writers, this is good. For very large IO jobs where you need
10's of GB/s, you probably need a more specific design to your problem.
Regards,
Joe
--
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics, Inc.
email: [email protected]
web : http://scalableinformatics.com
http://scalableinformatics.com/sicluster
phone: +1 734 786 8423 x121
fax : +1 866 888 3112
cell : +1 734 612 4615
_______________________________________________
Gluster-users mailing list
[email protected]
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
