Rohith Sharma K S created YARN-1005:
---------------------------------------
Summary: Log aggregators should check for FSDataOutputStream close
before renaming to aggregated file.
Key: YARN-1005
URL: https://issues.apache.org/jira/browse/YARN-1005
Project: Hadoop YARN
Issue Type: Bug
Components: nodemanager
Affects Versions: 2.1.0-beta, 2.0.5-alpha
Reporter: Rohith Sharma K S
If AggregatedLogFormat.LogWriter.closeWriter() is interuppted, then
"remoteNodeTmpLogFileForApp" is renamed to "remoteNodeLogFileForApp" file. This
renamed file does not contain valid aggregated logs. There can be situation
renamed file can be not in BCFile format.
This cause issue while viewing from JobHistoryServer web page.
{noformat}
2013-07-27 18:51:14,787 ERROR org.apache.hadoop.yarn.webapp.View: Error getting
logs for job_1374918614757_0002
java.io.IOException: Not a valid BCFile.
at
org.apache.hadoop.io.file.tfile.BCFile$Magic.readAndVerify(BCFile.java:927)
at org.apache.hadoop.io.file.tfile.BCFile$Reader.<init>(BCFile.java:628)
at org.apache.hadoop.io.file.tfile.TFile$Reader.<init>(TFile.java:804)
at
org.apache.hadoop.yarn.logaggregation.AggregatedLogFormat$LogReader.<init>(AggregatedLogFormat.java:337)
at
org.apache.hadoop.yarn.webapp.log.AggregatedLogsBlock.render(AggregatedLogsBlock.java:89)
at
org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:64)
at
org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:74)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
