[
https://issues.apache.org/jira/browse/MAPREDUCE-5465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14503966#comment-14503966
]
Ray Chiang commented on MAPREDUCE-5465:
---------------------------------------
+1 (nonbinding) for version 9 patch.
- All failed Jenkins unit tests pass in my tree.
- Version 9 generates a clean state diagram. Version 8 had a state with no
transitions to it, while the latest has clean transitions from
SUCCESS_FINISHING_CONTAINER to either SUCCESS_CONTAINER_CLEANUP or
KILL_CONTAINER_CLEANUP.
- Running with a modified WordCount job, I get the following results:
+ The non-patched version (running with depth=6) generates a profile.out of
69818 bytes and does not contain the SITES or CPU SAMPLES information.
+ The patched version (running with depth=6) generates a profile.out of
11348778 bytes and contains both the SITES and CPU SAMPLES information.
+ The patched version (running with depth=100) generates a profile.out of
379301524 bytes and contains both the SITES and CPU SAMPLES information.
Similar result with depth=10000.
> Container killed before hprof dumps profile.out
> -----------------------------------------------
>
> Key: MAPREDUCE-5465
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5465
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mr-am, mrv2
> Reporter: Radim Kolar
> Assignee: Ming Ma
> Attachments: MAPREDUCE-5465-2.patch, MAPREDUCE-5465-3.patch,
> MAPREDUCE-5465-4.patch, MAPREDUCE-5465-5.patch, MAPREDUCE-5465-6.patch,
> MAPREDUCE-5465-7.patch, MAPREDUCE-5465-8.patch, MAPREDUCE-5465-9.patch,
> MAPREDUCE-5465.patch
>
>
> If there is profiling enabled for mapper or reducer then hprof dumps
> profile.out at process exit. It is dumped after task signaled to AM that work
> is finished.
> AM kills container with finished work without waiting for hprof to finish
> dumps. If hprof is dumping larger outputs (such as with depth=4 while depth=3
> works) , it could not finish dump in time before being killed making entire
> dump unusable because cpu and heap stats are missing.
> There needs to be better delay before container is killed if profiling is
> enabled.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
