Hello Riza Suminto, David Rorke, Joe McDonnell, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19228
to look at the new patch set (#4).
Change subject: IMPALA-11704: (Addendum) fix crash on open for HDFS cache
......................................................................
IMPALA-11704: (Addendum) fix crash on open for HDFS cache
When trying to read from HDFS cache, ReadFromCache calls
FileReader::Open(false) to force the file to open. The prior commit for
IMPALA-11704 didn't allow for that case when using a data cache, as the
data cache check would always happen. This resulted in a crash calling
CachedFile as exclusive_hdfs_fh_ was nullptr. Tests only catch this when
reading from HDFS cache with data cache enabled.
Replaces explicit arguments to override FileReader behavior with a flag
to communicate whether FileReader supports delayed open. Then the caller
can choose whether to call Open before read. Also simplifies calls to
ReadFromPos as it already has a pointer to ScanRange and can check
whether file handle caching is enabled directly. The Open call in
DoInternalRead uses a slightly wider net by only checking UseDataCache.
If the data cache is unavailable or a miss the file will then be opened.
Adds a select from tpch.nation to the query for test_data_cache.py as
something that triggers checking the HDFS cache.
Change-Id: I741488d6195e586917de220a39090895886a2dc5
---
M be/src/runtime/io/cache-reader-test-stub.h
M be/src/runtime/io/file-reader.h
M be/src/runtime/io/hdfs-file-reader.cc
M be/src/runtime/io/hdfs-file-reader.h
M be/src/runtime/io/local-file-reader.cc
M be/src/runtime/io/local-file-reader.h
M be/src/runtime/io/request-ranges.h
M be/src/runtime/io/scan-range.cc
M testdata/workloads/functional-query/queries/QueryTest/data-cache.test
9 files changed, 39 insertions(+), 41 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/28/19228/4
--
To view, visit http://gerrit.cloudera.org:8080/19228
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I741488d6195e586917de220a39090895886a2dc5
Gerrit-Change-Number: 19228
Gerrit-PatchSet: 4
Gerrit-Owner: Michael Smith <[email protected]>
Gerrit-Reviewer: David Rorke <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Joe McDonnell <[email protected]>
Gerrit-Reviewer: Michael Smith <[email protected]>
Gerrit-Reviewer: Riza Suminto <[email protected]>
