[
https://issues.apache.org/jira/browse/TEZ-776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14522714#comment-14522714
]
Bikas Saha commented on TEZ-776:
--------------------------------
bq. That's still making more changes which belong to other jiras, and warrant a
review in themselves.
Not sure why this seems to be so. Let me try to clarify. Between patches A and
B the proposed new API changed in this form.
{code}+ public @Nullable Collection<DataMovementEvent>
routeCompositeDataMovementEventToDestination(
+ CompositeDataMovementEvent event, int sourceTaskIndex, int
destinationTaskIndex)
{code} to {code}+ public @Nullable EventRouteMetadata
routeCompositeDataMovementEventToDestination(
+ int sourceTaskIndex, int destinationTaskIndex)
{code}
Since this API is being introduced in this jira for the first time, isn't now
the right time to put in the best form possible and review it here instead of
in a different jira? Other than this, the rest of the changes are essentially
the same. There is no new functionality implemented. So I am struggling to
understand your concerns about increasing scope or the changes belonging
somewhere else. Perhaps you did not see patch B properly. I hope this code
snippet and explanation helps clarify things.
[~rajesh.balamohan], [~hitesh] Since you guys have been looking at the patches
and have seen version A and have reviewed version B, please let me know if I am
missing something.
I am consciously trying to not increase the scope of this jira and have left
many improvements for follow ups. E.g. separating root input event routing,
composite event routing, event obsoletion etc.
> Reduce AM mem usage caused by storing TezEvents
> -----------------------------------------------
>
> Key: TEZ-776
> URL: https://issues.apache.org/jira/browse/TEZ-776
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Siddharth Seth
> Assignee: Bikas Saha
> Attachments: TEZ-776.1.patch, TEZ-776.2.patch, TEZ-776.3.patch,
> TEZ-776.4.patch, TEZ-776.5.patch, TEZ-776.6.A.patch, TEZ-776.6.B.patch,
> TEZ-776.7.patch, TEZ-776.ondemand.1.patch, TEZ-776.ondemand.2.patch,
> TEZ-776.ondemand.3.patch, TEZ-776.ondemand.4.patch, TEZ-776.ondemand.5.patch,
> TEZ-776.ondemand.6.patch, TEZ-776.ondemand.7.patch, TEZ-776.ondemand.patch,
> With_Patch_AM_hotspots.png, With_Patch_AM_profile.png,
> Without_patch_AM_CPU_Usage.png, events-problem-solutions.txt,
> with_patch_jmc_output_of_AM.png, without_patch_jmc_output_of_AM.png
>
>
> This is open ended at the moment.
> A fair chunk of the AM heap is taken up by TezEvents (specifically
> DataMovementEvents - 64 bytes per event).
> Depending on the connection pattern - this puts limits on the number of tasks
> that can be processed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
