Could you please also capture fop profiling output using the following link please: http://gluster.org/community/documentation/index.php/Gluster_3.2:_Running_Gluster_Volume_Profile_Command
That should give what fops are executing and what are their latencies which will be helpful. Pranith ----- Original Message ----- > From: "Jay Vyas" <[email protected]> > To: "Paul Robert Marino" <[email protected]> > Cc: "[email protected]" <[email protected]> > Sent: Wednesday, January 29, 2014 6:07:14 AM > Subject: Re: [Gluster-users] gluster CPU cycles at 100% > > the workload : copying 21,000 1KB files.... small files, into gluster. > The CPU is spiked the whole time and it takes a long time to do the copying. > > There are some other moving parts, but Im pretty sure gluster shouldnt spike > so high for just 21 MB of data, right? > > there might be an error in my numbers above, so dont lose sleep - i still am > trying to reproduce this "bug"... if thats what it is. > > > On Tue, Jan 28, 2014 at 7:11 PM, Paul Robert Marino < [email protected] > > wrote: > > > Well what do you mean by %100 CPU do you mean just 1 core or multiple also > what's your IO wait percentage on your gluster servers. > > Its not uncommon for gluster to spike up to use more than a whole core. > > You may simply be exhausting the effective write cache on the gluster servers > in which case unless you are striping your data across many raids you may > just be hitting the limits of your hardware. > > You may also have client(s) that are asymetricly connecting to only one or a > small subset of servers due to a connectivity issue which would cause that > one node or group of nodes to bear the brunt of replicating the entire > workload via constant self heal. > > To help you I would rather know more about your configuration before giving > advice. > > > > > -- Sent from my HP Pre3 > > > On Jan 28, 2014 14:08, Harshavardhana < [email protected] > wrote: > > You should take a 'gluster' statedump of the process - and open a > bugzilla for analysis with logs. > > On Tue, Jan 28, 2014 at 10:30 AM, Jay Vyas < [email protected] > wrote: > > Hi folks : > > > > Im running mahout on top of gluster using the GlusterFileSystem hadoop > > plugin. > > > > It works well, but Im noticing that glusterfsd is at 100% CPU, and a copy > > action seems to be stalling after a long workload. > > > > Any ways to rescue the system? or should i restart it ? > > > > -- > > Jay Vyas > > http://jayunit100.blogspot.com > > > > _______________________________________________ > > Gluster-users mailing list > > [email protected] > > http://supercolony.gluster.org/mailman/listinfo/gluster-users > > > > -- > Religious confuse piety with mere ritual, the virtuous confuse > regulation with outcomes > _______________________________________________ > Gluster-users mailing list > [email protected] > http://supercolony.gluster.org/mailman/listinfo/gluster-users > > > > -- > Jay Vyas > http://jayunit100.blogspot.com > > _______________________________________________ > Gluster-users mailing list > [email protected] > http://supercolony.gluster.org/mailman/listinfo/gluster-users _______________________________________________ Gluster-users mailing list [email protected] http://supercolony.gluster.org/mailman/listinfo/gluster-users
