Hello Matthew Jacobs, I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/6792 to look at the new patch set (#4). Change subject: IMPALA-5167: Reduce the number of Kudu clients created ...................................................................... IMPALA-5167: Reduce the number of Kudu clients created Creating Kudu clients is very expensive as each will fetch metadata from the Kudu master, so we should minimize the number of Kudu clients that get created. This patch stores a map from Kudu master addressed to Kudu clients in the ExecEnv to be used across the BE for all queries, and a map in KuduUtil to be used acress the FE and catalog for all queries. This relies on changes on the Kudu side that clear non-covered range entries from the client's cache on table open (fdc022fe6231af20e307012d98c35b16cbfa7b33 and d07ecd6ded01201c912d2e336611a6a941f48d98) Testing: - Ran a stress test locally: scan of a Kudu table, 100 concurrent queries, load on the Kudu master was reduced signficantly, from ~25% cpu to ~4% cpu. TODO: perf testing on a cluster. - Ran the Kudu e2e tests. - Manually ran a test with concurrent INSERTs and 'ALTER TABLE ADD PARTITION' (which is affected by the Kudu side change mentiond above) and verified correctness. Change-Id: I6b0c12a256c33e8ef32315b3736cae2dea2ae705 --- M be/src/exec/kudu-scan-node-base.cc M be/src/exec/kudu-table-sink.cc M be/src/exec/kudu-table-sink.h M be/src/exprs/kudu-partition-expr.cc M be/src/runtime/exec-env.cc M be/src/runtime/exec-env.h M be/src/runtime/query-state.cc M be/src/runtime/query-state.h M fe/src/main/java/org/apache/impala/catalog/KuduTable.java M fe/src/main/java/org/apache/impala/planner/KuduScanNode.java M fe/src/main/java/org/apache/impala/service/KuduCatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/KuduUtil.java 12 files changed, 114 insertions(+), 91 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/6792/4 -- To view, visit http://gerrit.cloudera.org:8080/6792 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I6b0c12a256c33e8ef32315b3736cae2dea2ae705 Gerrit-PatchSet: 4 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Thomas Tauber-Marshall <tmarsh...@cloudera.com> Gerrit-Reviewer: Marcel Kornacker <mar...@cloudera.com> Gerrit-Reviewer: Matthew Jacobs <m...@cloudera.com> Gerrit-Reviewer: Thomas Tauber-Marshall <tmarsh...@cloudera.com>