Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/16120 )
Change subject: IMPALA-9903: Reduce Kudu openTable calls per query ...................................................................... Patch Set 2: (3 comments) http://gerrit.cloudera.org:8080/#/c/16120/2/fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java File fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java: http://gerrit.cloudera.org:8080/#/c/16120/2/fe/src/main/java/org/apache/impala/catalog/FeKuduTable.java@166 PS2, Line 166: public static TResultSet getPartitions(FeKuduTable table) These are methods that implement the show partitions DDL, so we don't need to worry about caching these - I think to maintain the current semantics we need to open the table once per DDL anyway. I.e. I don't think we want to change the behaviour of these methods. Maybe we can be a comment to be explicit that we want to fetch the latest version of the Kudu table. http://gerrit.cloudera.org:8080/#/c/16120/2/fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java File fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java: http://gerrit.cloudera.org:8080/#/c/16120/2/fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java@56 PS2, Line 56: private org.apache.kudu.client.KuduTable kuduTable_; Caching it in LocalTable makes sense since it's per-query anyway. So this part is fine - caching it here means we can open it once per query then re-use it. http://gerrit.cloudera.org:8080/#/c/16120/2/fe/src/main/java/org/apache/impala/planner/KuduScanNode.java File fe/src/main/java/org/apache/impala/planner/KuduScanNode.java: http://gerrit.cloudera.org:8080/#/c/16120/2/fe/src/main/java/org/apache/impala/planner/KuduScanNode.java@135 PS2, Line 135: org.apache.kudu.client.KuduTable rpcTable = kuduTable_.getKuduTable(); I think this invocation should go via 'analyzer' to retrieve the per-query cached version of the table. That would work for both catalog implementations. -- To view, visit http://gerrit.cloudera.org:8080/16120 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Iec12a5be9b30e19a123142af5453a91bd4300b63 Gerrit-Change-Number: 16120 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke <[email protected]> Gerrit-Reviewer: Grant Henke <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Reviewer: Vihang Karajgaonkar <[email protected]> Gerrit-Comment-Date: Fri, 17 Jul 2020 17:33:51 +0000 Gerrit-HasComments: Yes
