Zhijie Shen commented on YARN-2748:

[~varun_saxena], thanks for the patch. I mostly agree on your points. Given 
{{Log Root Dir/sub-dir1/sub-dir2/.../xxxx.log}}, we can use the relative path 
{{sub-dir1/sub-dir2/.../xxxx.log}} to uniquely identify a log.

One question is if we want to support a single-level sub folder or recursive 
sub folders. The choice will make the solution significantly different. The use 
case of the single-level sub folder is to put the all rolling log files of one 
particular log in a single folder. I'd like to hear more from [~sumitmohanty] 
about the use case he met before.

> Upload logs in the sub-folders under the local log dir when aggregating logs
> ----------------------------------------------------------------------------
>                 Key: YARN-2748
>                 URL: https://issues.apache.org/jira/browse/YARN-2748
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: log-aggregation
>    Affects Versions: 2.6.0
>            Reporter: Zhijie Shen
>            Assignee: Varun Saxena
>             Fix For: 2.7.0
>         Attachments: YARN-2748.001.patch
> YARN-2734 has a temporal fix to skip sub folders to avoid exception. Ideally, 
> if the app is creating a sub folder and putting its rolling logs there, we 
> need to upload these logs as well.

This message was sent by Atlassian JIRA

Reply via email to