This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.1 by this push:
new ba47a4a9c5f [fix](hdfs) remove cached file handle when read fails
(#54988)
ba47a4a9c5f is described below
commit ba47a4a9c5ff5946eec1564d033b2ae93b4cee83
Author: Yongqiang YANG <[email protected]>
AuthorDate: Fri Sep 12 09:44:19 2025 +0800
[fix](hdfs) remove cached file handle when read fails (#54988)
### What problem does this PR solve?
pick #54926
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
Co-authored-by: Yongqiang YANG <[email protected]>
---
be/src/io/fs/hdfs_file_reader.cpp | 15 ++++++++++++---
be/src/io/fs/hdfs_file_reader.h | 3 ++-
2 files changed, 14 insertions(+), 4 deletions(-)
diff --git a/be/src/io/fs/hdfs_file_reader.cpp
b/be/src/io/fs/hdfs_file_reader.cpp
index ac75e2e722b..ffcb1fbea13 100644
--- a/be/src/io/fs/hdfs_file_reader.cpp
+++ b/be/src/io/fs/hdfs_file_reader.cpp
@@ -82,9 +82,18 @@ Status HdfsFileReader::close() {
return Status::OK();
}
-#ifdef USE_HADOOP_HDFS
Status HdfsFileReader::read_at_impl(size_t offset, Slice result, size_t*
bytes_read,
const IOContext* io_ctx) {
+ auto status = do_read_at_impl(offset, result, bytes_read, io_ctx);
+ if (!status.ok()) {
+ _accessor.destroy();
+ }
+ return status;
+}
+
+#ifdef USE_HADOOP_HDFS
+Status HdfsFileReader::do_read_at_impl(size_t offset, Slice result, size_t*
bytes_read,
+ const IOContext* io_ctx) {
DCHECK(!closed());
if (offset > _handle->file_size()) {
return Status::IOError("offset exceeds file size(offset: {}, file
size: {}, path: {})",
@@ -130,8 +139,8 @@ Status HdfsFileReader::read_at_impl(size_t offset, Slice
result, size_t* bytes_r
#else
// The hedged read only support hdfsPread().
// TODO: rethink here to see if there are some difference between hdfsPread()
and hdfsRead()
-Status HdfsFileReader::read_at_impl(size_t offset, Slice result, size_t*
bytes_read,
- const IOContext* io_ctx) {
+Status HdfsFileReader::do_read_at_impl(size_t offset, Slice result, size_t*
bytes_read,
+ const IOContext* io_ctx) {
DCHECK(!closed());
if (offset > _handle->file_size()) {
return Status::IOError("offset exceeds file size(offset: {}, file
size: {}, path: {})",
diff --git a/be/src/io/fs/hdfs_file_reader.h b/be/src/io/fs/hdfs_file_reader.h
index 0f4a3f14019..0fb77446eb5 100644
--- a/be/src/io/fs/hdfs_file_reader.h
+++ b/be/src/io/fs/hdfs_file_reader.h
@@ -55,7 +55,8 @@ public:
protected:
Status read_at_impl(size_t offset, Slice result, size_t* bytes_read,
const IOContext* io_ctx) override;
-
+ Status do_read_at_impl(size_t offset, Slice result, size_t* bytes_read,
+ const IOContext* io_ctx);
void _collect_profile_before_close() override;
private:
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
