Hi, Dustin.
We use GlusterFS for homes and shared data directories in our cluster.
We're relatively light users (5-10 users, maybe a dozen jobs per day).
My general GlusterFS recommendations (nothing to do with SLURM) would be:
- If you want speed, use MANY physical nodes (ie 5-10 2U machines
with 20-30TB of disks)
- When you use mirrored volumes, the client is responsible for
mirroring (effectively halves bandwidth to send copies to both mirrors)
- Gigabit Ethernet will be "slow" (even if you do link aggregation)
because individual TCP flows cannot exceed 1Gbps
- ... but with many clients and many servers you will see more
return on aggregated 1GbE links as channel capacity can be 2Gbps (2 *
1GbE), or more
- Use CentOS 6 or RHEL 6; GlusterFS doesn't really care what you
use... but it's a Red Hat product and it just works *so* well with
RHEL-based distros
Hope that helps.
Alan
On 06/14/2013 01:50 AM, Dustin Rice wrote:
> Hello folks, just wanted to ask if anyone out there is using GlusterFS
> on thier SLURM cluster. I'm looking to implement it myself and am
> interested if there have been any great successes (or failures) with it.
>
> Thanks!
>
--
Alan Orth
[email protected]
http://alaninkenya.org
http://mjanja.co.ke
"I have always wished for my computer to be as easy to use as my telephone; my
wish has come true because I can no longer figure out how to use my telephone."
-Bjarne Stroustrup, inventor of C++
