[
https://issues.apache.org/jira/browse/YARN-2724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinod Kumar Vavilapalli updated YARN-2724:
------------------------------------------
Target Version/s: (was: 2.5.1)
bq. As the log aggregation is done by NM user, giving it the permissions to
access the generated log file should fix this issue.
Agreed. I guess the problem that YARN should address is to surface the issue
with aggregation to the end-user - right now it's not clear what really
happened.
> If an unreadable file is encountered during log aggregation then aggregated
> file in HDFS badly formed
> -----------------------------------------------------------------------------------------------------
>
> Key: YARN-2724
> URL: https://issues.apache.org/jira/browse/YARN-2724
> Project: Hadoop YARN
> Issue Type: Bug
> Components: log-aggregation
> Affects Versions: 2.5.1
> Reporter: Sumit Mohanty
> Assignee: Xuan Gong
>
> Look into the log output snippet. It looks like there is an issue during
> aggregation when an unreadable file is encountered. Likely, this results in
> bad encoding.
> {noformat}
> LogType: command-13.json
> LogLength: 13934
> Log Contents:
> Error aggregating log file. Log file :
> /grid/0/yarn/log/application_1413865041660_0002/container_1413865041660_0002_01_000004/command-13.json/grid/0/yarn/log/application_1413865041660_0002/container_1413865041660_0002_01_000004/command-13.json
> (Permission denied)command-3.json13983Error aggregating log file. Log file :
> /grid/0/yarn/log/application_1413865041660_0002/container_1413865041660_0002_01_000004/command-3.json/grid/0/yarn/log/application_1413865041660_0002/contaierrors-13.txt0660_0002_01_000004/command-3.json
> (Permission denied)
>
> errors-3.txt0gc.log-20141021044514484052014-10-21T04:45:12.046+0000: 5.134:
> [GC2014-10-21T04:45:12.046+0000: 5.134: [ParNew: 163840K->15575K(184320K),
> 0.0488700 secs] 163840K->15575K(1028096K), 0.0492510 secs] [Times: user=0.06
> sys=0.01, real=0.05 secs]
> 2014-10-21T04:45:14.939+0000: 8.027: [GC2014-10-21T04:45:14.939+0000: 8.027:
> [ParNew: 179415K->11865K(184320K), 0.0941310 secs] 179415K->17228K(1028096K),
> 0.0943140 secs] [Times: user=0.13 sys=0.04, real=0.09 secs]
> 2014-10-21T04:46:42.099+0000: 95.187: [GC2014-10-21T04:46:42.099+0000:
> 95.187: [ParNew: 175705K->12802K(184320K), 0.0466420 secs]
> 181068K->18164K(1028096K), 0.0468490 secs] [Times: user=0.06 sys=0.00,
> real=0.04 secs]
> {noformat}
> Specifically, look at the text after the exception text. There should be two
> more entries for log files but none exist. This is likely due to the fact
> that command-13.json is expected to be of length 13934 but its is not as the
> file was never read.
> I think, it should have been
> {noformat}
> LogType: command-13.json
> LogLength: <Length of the exception text>
> Log Contents:
> Error aggregating log file. Log file :
> /grid/0/yarn/log/application_1413865041660_0002/container_1413865041660_0002_01_000004/command-13.json/grid/0/yarn/log/application_1413865041660_0002/container_1413865041660_0002_01_000004/command-13.json
> (Permission denied)command-3.json13983Error aggregating log file. Log file :
> /grid/0/yarn/log/application_1413865041660_0002/container_1413865041660_0002_01_000004/command-3.json/grid/0/yarn/log/application_1413865041660_0002/contaierrors-13.txt0660_0002_01_000004/command-3.json
> (Permission denied)
> {noformat}
> {noformat}
> LogType: errors-3.txt
> LogLength:0
> Log Contents:
> {noformat}
> {noformat}
> LogType:gc.log
> LogLength:???
> Log Contents:
> ......-20141021044514484052014-10-21T04:45:12.046+0000: 5.134:
> [GC2014-10-21T04:45:12.046+0000: 5.134: [ParNew: 163840K- .......
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
