[
https://issues.apache.org/jira/browse/YARN-221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Trezzo updated YARN-221:
------------------------------
Attachment: YARN-221-trunk-v1.patch
Attached is a patch that provides the ability to configure log aggregation on a
per container basis. All changes are at the yarn level. The main changes this
patch makes are as follows:
1. Addition of a new set of methods to the ContainerLaunchContext that lets a
yarn client set the log aggregation policy for that container.
2. A new set of log aggregation policies listed in the
ContainerLogAggregationPolicy enum.
3. Modifications to the LogAggregationService and associated code paths to
allow for per container configuration.
4. Addition of new unit tests and modification to existing tests to incorporate
changes.
I am going to follow this patch with another patch in YARN-85 that will make
the necessary changes at the MapReduce level to provide per job configuration
of log aggregation.
Also, which repository should I list for yarn patches on reviews.apache.org? I
see hdfs/common/mapreduce, but no yarn.
Feedback/+1's would be much appreciated. [~revans2] [~sandyr]
Thanks!
> NM should provide a way for AM to tell it not to aggregate logs.
> ----------------------------------------------------------------
>
> Key: YARN-221
> URL: https://issues.apache.org/jira/browse/YARN-221
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: nodemanager
> Reporter: Robert Joseph Evans
> Attachments: YARN-221-trunk-v1.patch
>
>
> The NodeManager should provide a way for an AM to tell it that either the
> logs should not be aggregated, that they should be aggregated with a high
> priority, or that they should be aggregated but with a lower priority. The
> AM should be able to do this in the ContainerLaunch context to provide a
> default value, but should also be able to update the value when the container
> is released.
> This would allow for the NM to not aggregate logs in some cases, and avoid
> connection to the NN at all.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
