Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/11874
to look at the new patch set (#3).
Change subject: IMPALA-7738: Implement timeouts for HDFS open calls
......................................................................
IMPALA-7738: Implement timeouts for HDFS open calls
This is part 1 of a push to add timeouts for all HDFS operations.
It adds timeouts for opening an HDFS file handle.
It introduces a new HdfsMonitor which implements timeouts by
constructing a task (subclasses of the new HdfsMonitoredOp)
and submitting it to a thread pool. The original caller waits
for completion of the task, and it times out based on a new
startup parameter hdfs_operation_timeout_sec, which defaults
to 5 minutes.
Testing:
1. Hand testing of timeout for a single operation
2. Hand testing of timeout for failing to submit to the thread
pool when all threads are busy.
3. Core ASAN, exhaustive release
TODO: I'm working to add an automated test.
Change-Id: Ia14403ca5f3f19c6d5f61b9ab2306b0ad3267454
---
M be/src/runtime/io/CMakeLists.txt
M be/src/runtime/io/disk-io-mgr.cc
M be/src/runtime/io/disk-io-mgr.h
M be/src/runtime/io/handle-cache.h
M be/src/runtime/io/handle-cache.inline.h
M be/src/runtime/io/hdfs-file-reader.cc
M be/src/runtime/io/hdfs-file-reader.h
A be/src/runtime/io/hdfs-monitored-ops.cc
A be/src/runtime/io/hdfs-monitored-ops.h
M be/src/util/thread-pool.h
10 files changed, 363 insertions(+), 81 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/74/11874/3
--
To view, visit http://gerrit.cloudera.org:8080/11874
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Ia14403ca5f3f19c6d5f61b9ab2306b0ad3267454
Gerrit-Change-Number: 11874
Gerrit-PatchSet: 3
Gerrit-Owner: Joe McDonnell <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
