[
https://issues.apache.org/jira/browse/MAPREDUCE-3831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13202431#comment-13202431
]
Karam Singh commented on MAPREDUCE-3831:
----------------------------------------
Settings for For RM scalability comparison :-
GridMix settings were for both Hadoop-0.20.204 and Hadoop-0.23 :
{code}
gridmix.client.submit.threads=10
gridmix.submit.multiplier=0.01
gridmix.client.pending.queue.depth=100
gridmix.job.type=SLEEPJOB
gridmix.sleep.maptask-only=false
gridmix.sleep.max-map-time=300000
gridmix.sleep.max-reduce-tim=300000
gridmix.sleep.fake-locations=3
gridmix.distributed-cache-emulation.enable=falseemulation.enable>
gridmix.compression-emulation.enable=false
{code}
>From JobHistroy Parsing and GridMix client found that :
Runtime (seconds): 2473
GridMix Simulation Time Spent: 41mins 8sec
Workflow End: 2046 (From histroy parsing)
While looking at GridMix log and JobHistory files :
1. Found that according to gm client last was completed was
{code}
12/02/07 08:32:26 INFO gridmix.JobMonitor: GRIDMIX000029
job_1328600848949_1182) success.
Whereas the
EndTime of Job is : 1328602818684 Tue, 07 Feb 2012 08:20:18
{code}
Which means somehow GridMix got Job completion event 12 minutes after
the the actual job got completed .
2. Similarly acc. to JobHistory last Jo completed was :
{code}
job_1328600848949_1162: 1328603121882 Tue, 07 Feb 2012 08:25:21
Whereas according to GridMix client log:
12/02/07 08:32:08 INFO gridmix.JobMonitor: GRIDMIX000029
(job_1328600848949_1162) success
{code}
Which again means GridMix got job completion event nearly 7 minutes
after the actual job got finished.
Whereas this problem does not exists with Hadoop-0.20.204
Seems that, Somehow in Hadoop-0.23, GridMix is getting job completion events
long after the actual job getting completed
> RM scalability runtime is worse than 0.20.204 by 14.2%
> ------------------------------------------------------
>
> Key: MAPREDUCE-3831
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-3831
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: benchmarks, resourcemanager
> Affects Versions: 0.23.1
> Reporter: Vinay Kumar Thota
>
> RM scalability runtime is worse than 0.20.204 by 14.2%
> Overall runtime against Fred in a 350 nodes cluster is 2155 secs.
> Overall runtime against .23 in 350 nodes cluster is 2462 secs.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
