Bikramjeet Vig has uploaded this change for review. ( http://gerrit.cloudera.org:8080/16385
Change subject: IMPALA-9792: Implement splitting kudu scan ranges for greater parallelism ...................................................................... IMPALA-9792: Implement splitting kudu scan ranges for greater parallelism This patch adds the ability to split kudu scan token via the provided kudu java API. This helps increase parallelism during scanning resulting in more efficient use with higher mt_dop. This however comes at an added cost of an RPC to tablet server per partition so it is only set by default for mt_dop > 1. A query option "TARGETED_KUDU_SCAN_RANGE_LENGTH" has been added to set the scan range length used in this implementation. Limitation: - The scan range length sent to kudu is just a hint and does not guarantee that the token will be split at that limit. - A slow tablet server which can already slow down scanning during execution can now also potentially slow down planning. Testing: - Added e2e tests Change-Id: Ia02fd94cc1d13c61bc6cb0765dd2cbe90e9a5ce8 --- M be/src/service/query-options-test.cc M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ImpalaInternalService.thrift M common/thrift/ImpalaService.thrift M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java M tests/query_test/test_kudu.py 7 files changed, 109 insertions(+), 3 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/85/16385/1 -- To view, visit http://gerrit.cloudera.org:8080/16385 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ia02fd94cc1d13c61bc6cb0765dd2cbe90e9a5ce8 Gerrit-Change-Number: 16385 Gerrit-PatchSet: 1 Gerrit-Owner: Bikramjeet Vig <[email protected]>
