[
https://issues.apache.org/jira/browse/YARN-4818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15196297#comment-15196297
]
Jason Lowe commented on YARN-4818:
----------------------------------
Is there a recent change in 2.8 that broke this? I haven't seen it in 2.7, yet
it too is using the buffer to copy the bytes in a similar manner and I haven't
heard of any reports from our users about truncated logs. The FileInputStream
claim in the description above isn't mentioned in the javadocs, and was I not
able to replicate it with a simple cat-like program:
{code}
import java.io.File;
import java.io.FileInputStream;
class cat {
public static void main(String[] args) throws Exception {
byte[] buf = new byte[65536];
File f = new File(args[0]);
FileInputStream in = new FileInputStream(f);
int len;
while ((len = in.read(buf)) != -1) {
System.out.write(buf, 0, len);
}
}
}
{code}
I'm able to use the code above to successfully copy files of arbitrary sizes
with JDK7 on Linux. Could this be a platform-specific issue?
> AggregatedLogFormat.LogValue.write() incorrectly truncates files
> ----------------------------------------------------------------
>
> Key: YARN-4818
> URL: https://issues.apache.org/jira/browse/YARN-4818
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 2.8.0
> Reporter: Brook Zhou
> Assignee: Brook Zhou
> Labels: log-aggregation
> Fix For: 2.8.0
>
> Attachments: YARN-4818-v0.patch
>
>
> AggregatedLogFormat.LogValue.write() currently has a bug where it only writes
> in blocks of the buffer size (65535). This is because
> FileInputStream.read(byte[] buf) returns -1 if there are less than buf.length
> bytes remaining. In cases where the file size is not an exact multiple of
> 65535 bytes, the remaining bytes are truncated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
