Hello Tianyi Wang, Philip Zeyliger, Impala Public Jenkins, Vuk Ercegovac, I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/11561 to look at the new patch set (#4). Change subject: WIP IMPALA-7626: Throttle catalog partial RPC requests ...................................................................... WIP IMPALA-7626: Throttle catalog partial RPC requests With more coordinators running in local catalog mode, the expected RPC traffic on the Catalog server is higher compared to the non-local-catalog mode. Each such RPC is handled in its own thread and consumes some non-trivial CPU for serializing and deserializing the metadata. With this change, the maximum number of threads performing the actual work are capped to a certain limit at any point and the remaining requests (if any) are blocked until the current requests are serviced. Adds the following parameters for controlling this behavior. --catalog_partial_rpc_max_parallel_runs --catalog_partial_rpc_queue_timeout_s --catalog_partial_rpc_queue_timeout_s controls the timeout for queued requests. Added some basic supportability to examine the queue length via metrics. Marking this as WIP since this probably needs more testing. Suggestions welcome. Change-Id: I11f77a16cfa38ada42d8b7c859850198ea7dd142 --- M be/src/catalog/catalog-server.cc M be/src/catalog/catalog-server.h M be/src/catalog/catalog.cc M be/src/catalog/catalog.h M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M common/thrift/JniCatalog.thrift M common/thrift/metrics.json M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/service/BackendConfig.java M fe/src/main/java/org/apache/impala/service/JniCatalog.java 11 files changed, 135 insertions(+), 8 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/61/11561/4 -- To view, visit http://gerrit.cloudera.org:8080/11561 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I11f77a16cfa38ada42d8b7c859850198ea7dd142 Gerrit-Change-Number: 11561 Gerrit-PatchSet: 4 Gerrit-Owner: Bharath Vissapragada <bhara...@cloudera.com> Gerrit-Reviewer: Bharath Vissapragada <bhara...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Philip Zeyliger <phi...@cloudera.com> Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com> Gerrit-Reviewer: Vuk Ercegovac <vercego...@cloudera.com>