[
https://issues.apache.org/jira/browse/MAPREDUCE-6052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191561#comment-14191561
]
Junping Du commented on MAPREDUCE-6052:
---------------------------------------
Thanks for review and comments, [~zjshen]!
bq. The method has javadoc. It's good to add one line for "conf".
Nice catch, fix it in v4 patch.
bq. Unnecessary import in JobSubmitter
I don't see any. Only some deprecated. Would you double check and point it out
here?
bq. It seems that ClientDistributedCacheManager blah blah doesn't need to be
moved. We can invoke addLog4jToDistributedCache inside copyAndConfigureFiles.
That's a good proposal. Update in v4 patch.
bq. Return null too? Otherwise, mapreduce.job.log4j-properties-file = "" will
result in exception?
No. mapreduce.job.log4j-properties-file is default to be empty. However, we
already check this property is not empty before calling to this method, the
method here should make sure the path is valid for local and adding with
complete scheme.
bq. The comment is staled? ... It seem not to be necessary because
submitJobDir should have been created in copyAndConfigureFiles
Good points. Update both.
BTW, would you please comment how the patch works if you've conducted
end-to-end local test?
I tried the patch in my local cluster setup, steps are like this:
1. adding a local log4j.properties file path to the property of
"mapreduce.job.log4j-properties-file" in mapred-site.xml
2. submit a mr job. You can find that this file get deliver to each NM's
localized directory, and works to change the log format for each container of
this job.
> Support overriding log4j.properties per job
> -------------------------------------------
>
> Key: MAPREDUCE-6052
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6052
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 2.5.0
> Reporter: Junping Du
> Assignee: Junping Du
> Attachments: MAPREDUCE-6052-v2.patch, MAPREDUCE-6052-v3.patch,
> MAPREDUCE-6052.patch
>
>
> For current MR application, the "log4j.configuration" is hard coded to
> container-log4j.properties within each node. We still need flexibility to
> override it per job like what we do in MRV1.
> {code}
> public static void addLog4jSystemProperties(
> String logLevel, long logSize, int numBackups, List<String> vargs) {
> vargs.add("-Dlog4j.configuration=container-log4j.properties");
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
