[
https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ming Ma updated YARN-221:
-------------------------
Attachment: YARN-221-6.patch
[~xgong] and others, here is the draft patch based on the new design. Besides
the above discussions,
* If the application specifies an invalid log aggregation policy class, the
current implementation will fallback to the default policy instead of the
failing the application. Alternative approach is to have NM fail the
application instead.
* For each new application, a new policy object will be created and used only
by that application. This should be ok from memory footprint as well as runtime
perf point of view. Alternative approach is to have applications share the same
policy object if they use the same policy class and same policy parameters.
> NM should provide a way for AM to tell it not to aggregate logs.
> ----------------------------------------------------------------
>
> Key: YARN-221
> URL: https://issues.apache.org/jira/browse/YARN-221
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: log-aggregation, nodemanager
> Reporter: Robert Joseph Evans
> Assignee: Ming Ma
> Attachments: YARN-221-6.patch, YARN-221-trunk-v1.patch,
> YARN-221-trunk-v2.patch, YARN-221-trunk-v3.patch, YARN-221-trunk-v4.patch,
> YARN-221-trunk-v5.patch
>
>
> The NodeManager should provide a way for an AM to tell it that either the
> logs should not be aggregated, that they should be aggregated with a high
> priority, or that they should be aggregated but with a lower priority. The
> AM should be able to do this in the ContainerLaunch context to provide a
> default value, but should also be able to update the value when the container
> is released.
> This would allow for the NM to not aggregate logs in some cases, and avoid
> connection to the NN at all.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
