Michael Smith has uploaded this change for review. ( http://gerrit.cloudera.org:8080/19204
Change subject: IMPALA-11704: Delay hdfsOpenFile with data cache ...................................................................... IMPALA-11704: Delay hdfsOpenFile with data cache Delays hdfsOpenFile until after data cache lookup if using a data cache. IMPALA-10147 implemented this, but only when using the file handle cache. This patch adds an additional check in case file handle caching is disabled. In networked environments, hdfsOpenFile can take significant time, as observed in a TPC-DS run of q90 where TotalRawHdfsOpenFileTime represented a majority of time spent for HDFS_SCAN_NODE. This patch brings that time to 0 with a primed data cache. Change-Id: I9429a41fb16de27ccb57730203f95559df0dbfb6 --- M be/src/runtime/io/cache-reader-test-stub.h M be/src/runtime/io/file-reader.h M be/src/runtime/io/hdfs-file-reader.cc M be/src/runtime/io/hdfs-file-reader.h M be/src/runtime/io/local-file-reader.cc M be/src/runtime/io/local-file-reader.h M be/src/runtime/io/request-ranges.h M be/src/runtime/io/scan-range.cc M tests/common/skip.py M tests/custom_cluster/test_data_cache.py 10 files changed, 46 insertions(+), 14 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/19204/1 -- To view, visit http://gerrit.cloudera.org:8080/19204 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I9429a41fb16de27ccb57730203f95559df0dbfb6 Gerrit-Change-Number: 19204 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Smith <[email protected]>
