[
https://issues.apache.org/jira/browse/TEZ-3359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15392327#comment-15392327
]
Hitesh Shah commented on TEZ-3359:
----------------------------------
Patch mostly looks good.
Comments:
{code}
if (!this.dagConf.containsKey(TezConfiguration.TEZ_HISTORY_LOGLEVEL) &&
1054 tezConf.get(TezConfiguration.TEZ_HISTORY_LOGLEVEL) != null) {
1055 if (!HistoryLogLevel.validateLogLevel(tezConf)) {
1056 throw new IllegalArgumentException("Config: " +
TezConfiguration.TEZ_HISTORY_LOGLEVEL +
1057 " is set to invalid value: " +
tezConf.get(TezConfiguration.TEZ_HISTORY_LOGLEVEL));
1058 }
{code}
- there is still a hole in checks if someone invokes DAG::setConf(string,
string) with the key as history log level.
- the simpler check will be to just validate the value if set in dag conf and
if not set, validate the value set in the config object ( assume default val if
not present obviously ).
{code}
private static HistoryLogLevel getLogLevel(Configuration conf, HistoryLogLevel
defaultValue,
50 HistoryLogLevel invalidValue) {
{code}
- maybe less code re-use will help here to reduce confusing code esp. in the
use of "invalidValue". Lets keep the validation code separate and then the
getLogLevel() should never catch IllegalArg? Does that make sense / agree?
> Add granular log levels for HistoryLoggingService.
> --------------------------------------------------
>
> Key: TEZ-3359
> URL: https://issues.apache.org/jira/browse/TEZ-3359
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Harish Jaiprakash
> Assignee: Harish Jaiprakash
> Attachments: TEZ-3359.01.patch, TEZ-3359.02.patch, TEZ-3359.03.patch
>
>
> We publish too many events to ATS, this increases the file size for ATS.
> Reduce data size logged into ATS by:
> * Having a more granular control over the loglevel, disable task level logs,
> or all logs and so on.
> * Disable logging counters.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
