[
https://issues.apache.org/jira/browse/TEZ-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14737237#comment-14737237
]
Siddharth Seth commented on TEZ-2774:
-------------------------------------
Scenarios make all the difference. If speculation is the concern, some of these
messages can be enabled only when speculation is turned on - instead of turning
them on for all running jobs.
Task completions are already available via the history events.
The task scheduler logs for min-held and prewarm are alredy at INFO level -
requested by other reviewers as well, for the same reason.
The input spec log is already present in the TaskSpec log.
The intent is to reduce the size of the AM log - which otherwise has a lot of
duplicated information, and becomes difficult to read. Keeping all log lines
wholesale because they've been helpful in the past means we'll keep these later
as well. Instead of keeping all of them, it's better to look at where else this
information is available, how relevant it is, and whether it can be obtained
without excessive logging.
The task request to the scheduler - I don't really see how this helps anything.
Logging the equivalent of the CLC once per vertex should give us enough
information on what tasks are requesting.
So once again, please look at what is available after the changes go in, and
suggest potential improvements - with scenarios in mind - rather than trying to
keep all log lines with duplicated infromation.
> Reduce logging in the AM, and parts of the runtime
> --------------------------------------------------
>
> Key: TEZ-2774
> URL: https://issues.apache.org/jira/browse/TEZ-2774
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Siddharth Seth
> Assignee: Siddharth Seth
> Attachments: TEZ-2774.1.txt, TEZ-2774.2.txt
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
