[
https://issues.apache.org/jira/browse/TEZ-776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362632#comment-14362632
]
Bikas Saha commented on TEZ-776:
--------------------------------
Sorry I take that back. Combining on-demand and current routing is ugly and
adds complexity that will potentially make other code changes in the affected
files tricky. Given the fact that the APIs are all optional and dont need to be
implemented (even across releases), none of the existing API's have been
deprecated and the feature will be off by default, I think we can make progress
on this jira and get the changes in for users currently struggling with large
jobs (who can choose to turn this on in their deployments and use built-in
implementations). Going forward, newer/better implementations can be proposed,
evaluated and incorporated as it always happens in the natural progression of a
project.
> Reduce AM mem usage caused by storing TezEvents
> -----------------------------------------------
>
> Key: TEZ-776
> URL: https://issues.apache.org/jira/browse/TEZ-776
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Siddharth Seth
> Assignee: Bikas Saha
> Attachments: TEZ-776.ondemand.1.patch, TEZ-776.ondemand.2.patch,
> TEZ-776.ondemand.3.patch, TEZ-776.ondemand.4.patch, TEZ-776.ondemand.5.patch,
> TEZ-776.ondemand.6.patch, TEZ-776.ondemand.patch, With_Patch_AM_hotspots.png,
> With_Patch_AM_profile.png, Without_patch_AM_CPU_Usage.png,
> events-problem-solutions.txt, with_patch_jmc_output_of_AM.png,
> without_patch_jmc_output_of_AM.png
>
>
> This is open ended at the moment.
> A fair chunk of the AM heap is taken up by TezEvents (specifically
> DataMovementEvents - 64 bytes per event).
> Depending on the connection pattern - this puts limits on the number of tasks
> that can be processed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
