Hitesh Shah commented on YARN-3154:

This should likely work. Main thing should be to retain current behavior of 
aggregating all logs is retained. 

Some questions: 
  - how should an app handle stdout and stderr i.e. rotating of these files?
  - what regex format is supported? Will the NMClient do a sanity check to see 
if the format is valid?
Considering the std rolling log appender which uses .N suffixes, will rolling 
aggregation ever work in practice? How should we educate users on using 
"correct" appender/appender configuration which does the rotation in an atomic 
and immutable manner? 

> Should not upload partial logs for MR jobs or other "short-running' 
> applications 
> ---------------------------------------------------------------------------------
>                 Key: YARN-3154
>                 URL: https://issues.apache.org/jira/browse/YARN-3154
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, resourcemanager
>            Reporter: Xuan Gong
>            Assignee: Xuan Gong
>            Priority: Blocker
>         Attachments: YARN-3154.1.patch, YARN-3154.2.patch
> Currently, if we are running a MR job, and we do not set the log interval 
> properly, we will have their partial logs uploaded and then removed from the 
> local filesystem which is not right.
> We only upload the partial logs for LRS applications.

This message was sent by Atlassian JIRA

Reply via email to