Michael Smith has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19228 )
Change subject: IMPALA-11704: (Addendum) fix crash on open for HDFS cache ...................................................................... IMPALA-11704: (Addendum) fix crash on open for HDFS cache When trying to read from HDFS cache, ReadFromCache calls FileReader::Open(false) to force the file to open. The prior commit for IMPALA-11704 didn't allow for that case when using a data cache, as the data cache check would always happen. This resulted in a crash calling CachedFile as exclusive_hdfs_fh_ was nullptr. Tests only catch this when reading from HDFS cache with data cache enabled. Replaces explicit arguments to override FileReader behavior with a flag to communicate whether FileReader supports delayed open. Then the caller can choose whether to call Open before read. Also simplifies calls to ReadFromPos as it already has a pointer to ScanRange and can check whether file handle caching is enabled directly. The Open call in DoInternalRead uses a slightly wider net by only checking UseDataCache. If the data cache is unavailable or a miss the file will then be opened. Adds a select from tpch.nation to the query for test_data_cache.py as something that triggers checking the HDFS cache. Change-Id: I741488d6195e586917de220a39090895886a2dc5 Reviewed-on: http://gerrit.cloudera.org:8080/19228 Reviewed-by: Joe McDonnell <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/src/runtime/io/cache-reader-test-stub.h M be/src/runtime/io/file-reader.h M be/src/runtime/io/hdfs-file-reader.cc M be/src/runtime/io/hdfs-file-reader.h M be/src/runtime/io/local-file-reader.cc M be/src/runtime/io/local-file-reader.h M be/src/runtime/io/request-ranges.h M be/src/runtime/io/scan-range.cc M testdata/workloads/functional-query/queries/QueryTest/data-cache.test 9 files changed, 39 insertions(+), 41 deletions(-) Approvals: Joe McDonnell: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/19228 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I741488d6195e586917de220a39090895886a2dc5 Gerrit-Change-Number: 19228 Gerrit-PatchSet: 5 Gerrit-Owner: Michael Smith <[email protected]> Gerrit-Reviewer: David Rorke <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Joe McDonnell <[email protected]> Gerrit-Reviewer: Michael Smith <[email protected]> Gerrit-Reviewer: Riza Suminto <[email protected]>
