Zhijie Shen created YARN-2733:
---------------------------------
Summary: Aggregated log file may be corrupted if IOException
happens at copying
Key: YARN-2733
URL: https://issues.apache.org/jira/browse/YARN-2733
Project: Hadoop YARN
Issue Type: Improvement
Components: log-aggregation
Reporter: Zhijie Shen
In LogValue.write:
{code}
while ((len = in.read(buf)) != -1) {
//If buffer contents within fileLength, write
if (len < bytesLeft) {
out.write(buf, 0, len);
bytesLeft-=len;
}
//else only write contents within fileLength, then exit early
else {
out.write(buf, 0, (int)bytesLeft);
break;
}
}
{code}
If IOException happens at any iteration, copying is going to be stopped, and
exception message will be appended to the end. It is very likely that the total
copied content length is smaller than the file length which has been written
into the aggregated file in advance. Later on, when the reader uses this length
to parse the content, error is going to happen because of reading more bytes
than actually being written.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
