[
https://issues.apache.org/jira/browse/HDFS-10543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16410436#comment-16410436
]
Hudson commented on HDFS-10543:
-------------------------------
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13869 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/13869/])
HDFS-10543: libhdfs++: hdfsRead stops at block boundary. Contributed by
(james.clampffer: rev a0c7aee3b64d6d4532dfc4f744aabfd46ae7ae42)
* (edit)
hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/fs/filehandle.cc
Revert HDFS-10543 patch due to correctness issues it caused. Done by
(james.clampffer: rev d75c835696ac09eef173c4a90d65e5e300edf06a)
* (edit)
hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/libhdfspp/lib/fs/filehandle.cc
> hdfsRead read stops at block boundary
> -------------------------------------
>
> Key: HDFS-10543
> URL: https://issues.apache.org/jira/browse/HDFS-10543
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs-client
> Reporter: Xiaowei Zhu
> Assignee: James Clampffer
> Priority: Major
> Fix For: HDFS-8707
>
> Attachments: HDFS-10543.HDFS-8707.000.patch,
> HDFS-10543.HDFS-8707.001.patch, HDFS-10543.HDFS-8707.002.patch,
> HDFS-10543.HDFS-8707.003.patch, HDFS-10543.HDFS-8707.004.patch
>
>
> Reproducer:
> char *buf2 = new char[file_info->mSize];
> memset(buf2, 0, (size_t)file_info->mSize);
> int ret = hdfsRead(fs, file, buf2, file_info->mSize);
> delete [] buf2;
> if(ret != file_info->mSize) {
> std::stringstream ss;
> ss << "tried to read " << file_info->mSize << " bytes. but read " <<
> ret << " bytes";
> ReportError(ss.str());
> hdfsCloseFile(fs, file);
> continue;
> }
> When it runs with a file ~1.4GB large, it will return an error like "tried to
> read 1468888890 bytes. but read 134217728 bytes". The HDFS cluster it runs
> against has a block size of 134217728 bytes. So it seems hdfsRead will stop
> at a block boundary. Looks like a regression. We should add retry to continue
> reading cross blocks in case of files w/ multiple blocks.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
