[Impala-ASF-CR] IMPALA-4671: (part-1) Copy kudu::ServicePool into Impala namespace
Sailesh Mukil has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/8471 ) Change subject: IMPALA-4671: (part-1) Copy kudu::ServicePool into Impala namespace .. IMPALA-4671: (part-1) Copy kudu::ServicePool into Impala namespace *** THIS PATCH IS NOT FOR REVIEW. IT IS JUST UPLOADED TO MAKE THE REVIEW OF PART-2 EASIER IN GERRIT BY SHOWING ONLY THE DIFFS OF THAT PATCH AGAINST THIS PATCH. MORE EXPLANATION BELOW*** The KuduRPC subsystem uses kudu::ServicePool to service all incoming RPCs. Since this lives inside the Kudu codebase, all instrumentation of RPCs flowing through the KuduRPC subsystem is not visible to Impala, as Kudu does not export this instrumentation, but only maintains it internally within kudu::ServicePool. In order to reliably view the instrumentation of all RPCs flowing through KRPC, one option is to modify kudu::ServicePool to take their instrumentation and display it on our webpages, which is very invasive. A second option is to have a parallel implementation of kudu::ServicePool which for all purposes behaves the same way, but instead has this extra code that displays this instrumentation in our webpages. This patch is Part 1 of the second option. In this patch, we simply copy the kudu::ServicePool code into be/src/rpc/. In the next patch (Part 2), we rename it to ImpalaServicePool, and we do the instrumentation such that it shows up on the webpage. We split it up this way into 2 parts so that it will be easy to review. Change-Id: I5bb927d4726d4fe418251b9734a4e232810fef69 --- A be/src/rpc/impala-service-pool.cc A be/src/rpc/impala-service-pool.h 2 files changed, 317 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/71/8471/3 -- To view, visit http://gerrit.cloudera.org:8080/8471 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I5bb927d4726d4fe418251b9734a4e232810fef69 Gerrit-Change-Number: 8471 Gerrit-PatchSet: 3 Gerrit-Owner: Sailesh Mukil
[Impala-ASF-CR] IMPALA-4671: (part-2) Replace kudu::ServicePool with one that uses Impala threads
Sailesh Mukil has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/8472 ) Change subject: IMPALA-4671: (part-2) Replace kudu::ServicePool with one that uses Impala threads .. IMPALA-4671: (part-2) Replace kudu::ServicePool with one that uses Impala threads The KuduRPC subsystem uses kudu::ServicePool to service all incoming RPCs. Since this lives inside the Kudu codebase, all instrumentation of RPCs flowing through the KuduRPC subsystem is not visible to Impala, as Kudu does not export this instrumentation, but only maintains it internally within kudu::ServicePool. In order to reliably view the instrumentation of all RPCs flowing through KRPC, one option is to modify kudu::ServicePool to take their instrumentation and display it on our webpages, which is very invasive. A second option is to have a parallel implementation of kudu::ServicePool which for all purposes behaves the same way, but instead has this extra code that displays this instrumentation in our webpages. This patch is Part 2 of the second option. In Part 1, we simply copied the code from kudu::ServicePool into the Impala namespace. In this patch, we rename the redundant kudu::ServicePool (that was copied into be/src/rpc/) to ImpalaServicePool. We also expose the instrumentaion of the ImpalaServicePool as JSON. This patch also enables the tracking of the RpcMgr via the Webserver. The RpcMgr calls into the active ImpalaServicePool objects and retrieves their instrumentation as JSON and makes it available in the Impala webpage. Finally, the threads in use by the service pool will also be visible in the /threadz page. A screenshot of the new additions to the /rpcz page is here: https://ibb.co/nciLDG This patch is partially based on an abandoned patch by Henry Robinson. Change-Id: Ia3e0fa9ae22c4803b0cea736eb35e333e2671448 --- M be/src/rpc/CMakeLists.txt M be/src/rpc/impala-service-pool.cc M be/src/rpc/impala-service-pool.h M be/src/rpc/rpc-mgr.cc M be/src/rpc/rpc-mgr.h M be/src/rpc/rpc-trace.cc M be/src/rpc/rpc-trace.h M be/src/service/impalad-main.cc M www/rpcz.tmpl 9 files changed, 411 insertions(+), 139 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/72/8472/3 -- To view, visit http://gerrit.cloudera.org:8080/8472 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia3e0fa9ae22c4803b0cea736eb35e333e2671448 Gerrit-Change-Number: 8472 Gerrit-PatchSet: 3 Gerrit-Owner: Sailesh Mukil
[Impala-ASF-CR] IMPALA-6105: Clarify argument order in single node perf run
Jim Apple has uploaded this change for review. (
http://gerrit.cloudera.org:8080/8470
Change subject: IMPALA-6105: Clarify argument order in single_node_perf_run
..
IMPALA-6105: Clarify argument order in single_node_perf_run
single_node_perf_run.py uses git_hash_A vs. git_hash_B, distinguish
them by their position in the command-line
arguments. single_node_perf_run.py calls report_benchmark_results.py,
which uses the "reference vs. input", distinguished by their
command-line flags. The output of report_benchmark_results.py uses
"{empty string} vs Base".
In the long run, I think it would be better to fix all three to use
the same terminology, but this comment hopefully adds clarity.
Change-Id: Ib236ce7e83dc193ef1382f630ce58759a639
---
M bin/single_node_perf_run.py
1 file changed, 22 insertions(+), 2 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/70/8470/1
--
To view, visit http://gerrit.cloudera.org:8080/8470
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ib236ce7e83dc193ef1382f630ce58759a639
Gerrit-Change-Number: 8470
Gerrit-PatchSet: 1
Gerrit-Owner: Jim Apple
