[
https://issues.apache.org/jira/browse/TEZ-776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rajesh Balamohan updated TEZ-776:
---------------------------------
Attachment: With_Patch_AM_hotspots.png
With_Patch_AM_profile.png
Attaching profiler output of AM to understand expensive methods with the patch.
"With_Patch_AM_profile.png" lists the set of expensive methods. Out of the AM
CPU spent, ~40% was spent on TezEvent.write/serializeEvent in
TezHeartbeatResponse code path. IMO, this wouldn't change irrespective of the
patch.
In the set of hotspot methods ("With_Patch_AM_hotspots.png),
Edge.maybeAddTezEventForDestinationTask() comes up, but just with 1% & 3
microseconds per call.
> Reduce AM mem usage caused by storing TezEvents
> -----------------------------------------------
>
> Key: TEZ-776
> URL: https://issues.apache.org/jira/browse/TEZ-776
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Siddharth Seth
> Assignee: Bikas Saha
> Attachments: TEZ-776.ondemand.1.patch, TEZ-776.ondemand.2.patch,
> TEZ-776.ondemand.3.patch, TEZ-776.ondemand.4.patch, TEZ-776.ondemand.5.patch,
> TEZ-776.ondemand.patch, With_Patch_AM_hotspots.png,
> With_Patch_AM_profile.png, Without_patch_AM_CPU_Usage.png,
> events-problem-solutions.txt, with_patch_jmc_output_of_AM.png,
> without_patch_jmc_output_of_AM.png
>
>
> This is open ended at the moment.
> A fair chunk of the AM heap is taken up by TezEvents (specifically
> DataMovementEvents - 64 bytes per event).
> Depending on the connection pattern - this puts limits on the number of tasks
> that can be processed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
