[
https://issues.apache.org/jira/browse/TEZ-3359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15392422#comment-15392422
]
Hitesh Shah commented on TEZ-3359:
----------------------------------
Comments:
{code}
// Copy historyLogLevel from tezConf into dagConf if its not overridden in
dagConf.
1053 String logLevel =
this.dagConf.get(TezConfiguration.TEZ_HISTORY_LOGLEVEL);
1054 if (logLevel == null) {
1055 logLevel = tezConf.get(TezConfiguration.TEZ_HISTORY_LOGLEVEL);
1056 }
if (logLevel != null) {
1058 if (!HistoryLogLevel.validateLogLevel(logLevel)) {
1059 throw new IllegalArgumentException("Config: " +
TezConfiguration.TEZ_HISTORY_LOGLEVEL +
1060 " is set to invalid value: " + logLevel);
1061 }
1062 PlanKeyValuePair.Builder kvp = PlanKeyValuePair.newBuilder();
1063 kvp.setKey(TezConfiguration.TEZ_HISTORY_LOGLEVEL);
1064 kvp.setValue(tezConf.get(TezConfiguration.TEZ_HISTORY_LOGLEVEL));
1065 confProtoBuilder.addConfKeyValues(kvp);
1066 }
{code}
- why is the AM log level overriding the dag conf provided level? line 1064?
- Please enhance unit tests to catch this bug.
Patch 05 seems incomplete.
> Add granular log levels for HistoryLoggingService.
> --------------------------------------------------
>
> Key: TEZ-3359
> URL: https://issues.apache.org/jira/browse/TEZ-3359
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Harish Jaiprakash
> Assignee: Harish Jaiprakash
> Attachments: TEZ-3359.01.patch, TEZ-3359.02.patch, TEZ-3359.03.patch,
> TEZ-3359.04.patch, TEZ-3359.05.patch
>
>
> We publish too many events to ATS, this increases the file size for ATS.
> Reduce data size logged into ATS by:
> * Having a more granular control over the loglevel, disable task level logs,
> or all logs and so on.
> * Disable logging counters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
