On 05/27/2011 04:31 PM, Joe Landman wrote:
On 05/27/2011 07:12 AM, Jon Tegner wrote:
A general question, suppose I have a parallel application, using mpi,
where really fast access to the file system is critical.
Would it be stupid to consider a ram disk based setup? Say a 36 port QDR
Ram disks won't work directly, due to lack of locking in tmpfs. You
could create a tmpfs, then create a file that fills this up, then a
loopback device pointing to that file, then build a file system atop
that, and mount it. And then mount gluster atop that.
Needless to say, all these layers significantly decrease performance
and introduce inefficiencies.
infiniband with half of the ports connected to computational nodes and
the other half to gluster nodes?
There may be other options, but the options are not going to be
cheap/inexpensive. How fast, and by fast do you mean bandwidth and/or
latency (e.g. streaming bandwidth or random IOPs)? What does your IO
profile look like?
You can get nodes that stream 4.6+ GB/s read, and 3.6+ GB/s writes for
single readers/writers to single files. For MPI jobs with single
readers/writers, this is good. For very large IO jobs where you need
10's of GB/s, you probably need a more specific design to your problem.
Regards,
Joe
Thanks!
Would you say that the inefficiencies related to the ram disk would
remove all advantages of using ram instead of hard drives (or could it
still be worth a try)?
As for speed, I would think that latency would be the most critical -
but I don't really know, it its the code of a colleague of mine (I was
trying to come up with a replacement of his SMP-machine which is getting
old).
Regards,
/jon
_______________________________________________
Gluster-users mailing list
[email protected]
http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
