Hello Riza Suminto, David Rorke, Joe McDonnell, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/19204
to look at the new patch set (#2).
Change subject: IMPALA-11704: Delay hdfsOpenFile with data cache
......................................................................
IMPALA-11704: Delay hdfsOpenFile with data cache
Delays hdfsOpenFile until after data cache lookup if using a data cache.
IMPALA-10147 implemented this, but only when using the file handle
cache. This patch adds an additional check in case file handle caching
is disabled.
In networked environments, hdfsOpenFile can take significant time, as
observed in a TPC-DS run of q90 where TotalRawHdfsOpenFileTime
represented a majority of time spent for HDFS_SCAN_NODE. This patch
brings that time to 0 with a primed data cache.
Change-Id: I9429a41fb16de27ccb57730203f95559df0dbfb6
---
M be/src/runtime/io/cache-reader-test-stub.h
M be/src/runtime/io/file-reader.h
M be/src/runtime/io/hdfs-file-reader.cc
M be/src/runtime/io/hdfs-file-reader.h
M be/src/runtime/io/local-file-reader.cc
M be/src/runtime/io/local-file-reader.h
M be/src/runtime/io/request-ranges.h
M be/src/runtime/io/scan-range.cc
M tests/common/skip.py
M tests/custom_cluster/test_data_cache.py
10 files changed, 53 insertions(+), 14 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/04/19204/2
--
To view, visit http://gerrit.cloudera.org:8080/19204
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I9429a41fb16de27ccb57730203f95559df0dbfb6
Gerrit-Change-Number: 19204
Gerrit-PatchSet: 2
Gerrit-Owner: Michael Smith <[email protected]>
Gerrit-Reviewer: David Rorke <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Joe McDonnell <[email protected]>
Gerrit-Reviewer: Michael Smith <[email protected]>
Gerrit-Reviewer: Riza Suminto <[email protected]>
