Thomas Tauber-Marshall has uploaded a new change for review. http://gerrit.cloudera.org:8080/6792
Change subject: IMPALA-5167: Use a single Kudu client across the BE ...................................................................... IMPALA-5167: Use a single Kudu client across the BE Creating Kudu clients is very expensive as each will fetch metadata from the Kudu master, so we should minimize the number of Kudu clients that get created. This patch stores a single Kudu client in the ExecEnv to be used across the BE for all queries. Testing: - Ran a stress test locally: scan of a Kudu table, 100 concurrent queries, load on the Kudu master was reduced by about half, from ~25% cpu to ~12% cpu, and latency was reduced by ~18% (from ~13s for 10,000 queries to complete down to ~11s). TODO: perf testing on a cluster. Change-Id: I6b0c12a256c33e8ef32315b3736cae2dea2ae705 --- M be/src/exec/kudu-scan-node-base.cc M be/src/exec/kudu-table-sink.cc M be/src/exec/kudu-table-sink.h M be/src/exprs/kudu-partition-expr.cc M be/src/runtime/exec-env.cc M be/src/runtime/exec-env.h M be/src/runtime/query-state.cc M be/src/runtime/query-state.h M bin/impala-config.sh 9 files changed, 68 insertions(+), 59 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/92/6792/1 -- To view, visit http://gerrit.cloudera.org:8080/6792 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I6b0c12a256c33e8ef32315b3736cae2dea2ae705 Gerrit-PatchSet: 1 Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-Owner: Thomas Tauber-Marshall <[email protected]>
