On 04/06/2013 11:50 AM, Daniel Mons wrote:
On 5 April 2013 14:23, <[email protected]> wrote:
Nice read indeed! Question regarding raid. In an hpc-environment we use
gluster on raid10 as a primary file system, and on raid6 on the back up one.
Didn't test, but went for raid10 cause I thoght it would be advantageous
from a performance perspective - can one say something general ragarding
this?
Regards,
/jon
I answer the same question from a forum user in the original link:
http://forums.overclockers.com.au/showpost.php?p=15234327&postcount=14
GlusterFS's main bottleneck is rarely local storage. I'm running 16
spindles per node in a RAID6+1S setup, with intelligent SSD
caching/reordering for small random IOPS (SSDs are bypassed for large
sequential reads and writes). The performance of the local disk is
not the problem. Gluster's single biggest bottleneck (and this is
common to many clustered file systems) is file lookup over the network
for uncached content, and especially negative lookup. These are
several orders of magnitude slower than the storage, and increasing
the storage IOPS won't help things much at all.
I'm currently losing 3 disks per 16 (~19%) to per-node redundancy.
Moving to RAID10 would bump that to 50% space loss for no real world
net gain, meaning I'd get less storage for the same dollar outlay, and
no measurable speed increase.
What I'd be interested in seeing is someone testing Infiniband/RDMA
compared to Fibre 10GbE on the same storage bricks to see if the
latency reduction has a positive effect on the file lookup speed of
GlusterFS. If anyone has a link to that sort of test, please post it
in-thread.
-Dan
Thanks!
Our primary system is using infiniband, and gluster 3.2.6-1, with ext4.
Our backup filsystem uses gluster 3.3.1-1, with gigabit and xfs. I'm
planning to extend this (the backup one) to use infiniband. Then we
could test it out (bearing in mind that one system is using ext4 and the
other is using xfs). However, seen some information that rdma is not
supported on 3.3.x. Would it be waste of effort to upgrade to infiniband
in this scenario?
Regards,
/jon
_______________________________________________
Gluster-users mailing list
[email protected]
http://supercolony.gluster.org/mailman/listinfo/gluster-users
