Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16385 )
Change subject: IMPALA-9792: Add ability to split kudu scan ranges ...................................................................... IMPALA-9792: Add ability to split kudu scan ranges This patch adds the ability to split kudu scan token via the provided kudu java API. A query option "TARGETED_KUDU_SCAN_RANGE_LENGTH" has been added to set the scan range length used in this implementation. Potential benefit: This helps increase parallelism during scanning which can result in more efficient use of CPU with higher mt_dop. Limitation: - The scan range length sent to kudu is just a hint and does not guarantee that the token will be split at that limit. - Comes at an added cost of an RPC to tablet server per token in order to split it. A slow tablet server which can already slow down scanning during execution can now also potentially slow down planning. - Also adds the cost of an RPC per token to open a new scanner for it on the kudu side. Therefore, scanning many smaller split tokens can slow down scanning and we can also lose benefits of scanning a single large token sequentially with a single scanner. Testing: - Added an e2e test Change-Id: Ia02fd94cc1d13c61bc6cb0765dd2cbe90e9a5ce8 Reviewed-on: http://gerrit.cloudera.org:8080/16385 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/src/service/query-options-test.cc M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ImpalaInternalService.thrift M common/thrift/ImpalaService.thrift M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java M tests/query_test/test_kudu.py 7 files changed, 86 insertions(+), 3 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/16385 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Ia02fd94cc1d13c61bc6cb0765dd2cbe90e9a5ce8 Gerrit-Change-Number: 16385 Gerrit-PatchSet: 11 Gerrit-Owner: Bikramjeet Vig <[email protected]> Gerrit-Reviewer: Bikramjeet Vig <[email protected]> Gerrit-Reviewer: Grant Henke <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Qifan Chen <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]>
