Yes, I used the GlusterFS plugin. Gluster version is - 3.3 beta 2.
For the Volumes Distributed-mirroring volume: Using 4 server and 2(brick)x2(replica) configuration Stripe-mirroring volume : Using 4 Server and 4(stripe count) x 2 (repica) configuration For the Map/reduce system I user 6 server ( 4 is the brick server and other 2 is for just map/reduce ) I checked your source file, but I can’t find any clue for the Performance degradation in Merging Stage. ( I think it is connected with writing) Actaully, In writing test, Gluster was quite good. So I’m little confused right now. Regards Andrew From: [email protected] [mailto:[email protected]] On Behalf Of Venky Shankar Sent: Thursday, October 20, 2011 1:35 AM To: andrew; [email protected] Subject: Re: [Gluster-users] gluster map/reduce performance.. Hi there, Appreciate if you could share the following info with us: * Are you using GlusterFS hadoop plugin (which is here http://download.gluster.com/pub/gluster/glusterfs/qa-releases/3.3-beta-2/glusterfs-hadoop-0.20.2-0.1.x86_64.rpm and is still in beta) or are you using GlusterFS as an additional layer below Hadoop's FileSystem (HDFS) ? The latter is basically configuring Hadoop to use GlusterFS mount point (e.g. FUSE mount) as the data directory for Hadoop's DFS. Let us know your setup (including GlusterFS version) to debug further. Thanks, -Venky ________________________________ From: [email protected] [[email protected]] on behalf of andrew [[email protected]] Sent: Wednesday, October 19, 2011 6:15 PM To: [email protected] Subject: [Gluster-users] gluster map/reduce performance.. Hi, all, i try to check the performance of Map/Reduce of Gluster File system. Mapper side speed is quite good and it is sometimes faster than hadoop's map job. But in the Reduce Side job is much slower than hadoop. i analyze the result and i found the primary reason of slow speed is bad performance in Merging stage. Would you have any suggestion for this issue FYI check the blog http://storage4com.blogspot.com/ thanks.
_______________________________________________ Gluster-users mailing list [email protected] http://gluster.org/cgi-bin/mailman/listinfo/gluster-users
