Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/16120 )
Change subject: IMPALA-9903: Reduce Kudu openTable calls per query ...................................................................... IMPALA-9903: Reduce Kudu openTable calls per query This patch reduces the number of Kudu openTable calls for the lifetime of a query by storing the KuduTable object in the Analyzer GlobalState and using it in the KuduScanNode. It does not cache the KuduTable object longer than a single query, does not impact DDL statements, and does not introduce the need to invalidate metadata when interacting with Kudu tables. Additionally, this patch adjusts the backend scanner to use the KuduTable instance from the KuduScanner instead of using openTable to get a new instance. Reducing the number of openTable calls is important because each call results in a GetTableSchema RPC to the remote leader Kudu master. With very high rates of queries against Kudu tables this can overload the master leading to degraded query performance. In manual testing this patched reduced the Kudu GetTableSchema RPC calls to the master from 5 per query to 1 per query. Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63 Reviewed-on: http://gerrit.cloudera.org:8080/16120 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M be/src/exec/kudu-scan-node-base.cc M be/src/exec/kudu-scan-node-base.h M be/src/exec/kudu-scanner.cc M bin/impala-config.sh M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java M fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java 8 files changed, 86 insertions(+), 32 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/16120 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63 Gerrit-Change-Number: 16120 Gerrit-PatchSet: 12 Gerrit-Owner: Grant Henke <[email protected]> Gerrit-Reviewer: Grant Henke <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Qifan Chen <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Reviewer: Vihang Karajgaonkar <[email protected]>
