Michael Smith has uploaded this change for review. ( http://gerrit.cloudera.org:8080/18841
Change subject: IMPALA-10213: Add test for local vs remote scheduling ...................................................................... IMPALA-10213: Add test for local vs remote scheduling Impala already supports locality-aware scheduling with Ozone because it returns location data on partitions. That data doesn't include specific storage ids in getStorageIds, so we skip a warning that will always trigger on Ozone. Updates Ozone to add implicit rules mapping localhost -> 127.0.0.1 for local development. HDFS translates localhost to 127.0.0.1 for host names in its location data, which Impala will identify as colocated with executors in the dev environment. Ozone doesn't, and the default Impala hostname is the machine hostname - not localhost - so without this change all HDFS access in the minicluster is local but all Ozone access is remote. Adds a test to verify local vs remote assignment by using custom clusters with hostnames that either do or don't match storage hostnames. Change-Id: I4e5606528404c3d4fd164c03dec8315345be5f6d --- M be/src/runtime/io/disk-io-mgr.cc M be/src/scheduling/executor-group.cc M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java M tests/common/skip.py A tests/custom_cluster/test_scheduler_locality.py M tests/metadata/test_stats_extrapolation.py 6 files changed, 71 insertions(+), 6 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/41/18841/1 -- To view, visit http://gerrit.cloudera.org:8080/18841 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I4e5606528404c3d4fd164c03dec8315345be5f6d Gerrit-Change-Number: 18841 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Smith <[email protected]>
