John Doe created HADOOP-15425: --------------------------------- Summary: CopyFilesMapper.doCopyFile hangs with misconfigured sizeBuf Key: HADOOP-15425 URL: https://issues.apache.org/jira/browse/HADOOP-15425 Project: Hadoop Common Issue Type: Bug Components: tools Affects Versions: 2.5.0 Reporter: John Doe
When the sizeBuf is configured to be 0, the for loop in DistCpV1$CopyFilesMapper.doCopyFile function hangs endlessly. This is because when the buf.size (i.e., sizeBuf) is 0, the bytesRead will always be 0 by invoking bytesRead=in.read(buf). Here is the code snippet. {code:java} sizeBuf = job.getInt("copy.buf.size", 128 * 1024); //when copy.buf.size = 0 buffer = new byte[sizeBuf]; private long doCopyFile(FileStatus srcstat, Path tmpfile, Path absdst, Reporter reporter) throws IOException { FSDataInputStream in = null; FSDataOutputStream out = null; long bytesCopied = 0L; try { Path srcPath = srcstat.getPath(); // open src file in = srcPath.getFileSystem(job).open(srcPath); reporter.incrCounter(Counter.BYTESEXPECTED, srcstat.getLen()); // open tmp file out = create(tmpfile, reporter, srcstat); LOG.info("Copying file " + srcPath + " of size " + srcstat.getLen() + " bytes..."); // copy file for(int bytesRead; (bytesRead = in.read(buffer)) >= 0; ) { out.write(buffer, 0, bytesRead); bytesCopied += bytesRead; reporter.setStatus(... ); } } finally { checkAndClose(in); checkAndClose(out); } return bytesCopied; } {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org