[
https://issues.apache.org/jira/browse/IMPALA-9824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17140829#comment-17140829
]
Sahil Takiar commented on IMPALA-9824:
--------------------------------------
[~vihangk1] sorry for the delayed response. I'm not super familiar with these
tests. AFAICT there are 4 places where we create a {{MetaStoreClientPool}}:
{code:java}
stakiar @ stakiar-desktop -bash ~/Impala 2020-06-19 13:30:21 master
[51] → grep -iIR 'new MetaStoreClientPool' fe/src/main/
fe/src/main/java/org/apache/impala/service/Frontend.java:
metaStoreClientPool_ = new MetaStoreClientPool(1, 0);
fe/src/main/java/org/apache/impala/catalog/Catalog.java: this(new
MetaStoreClientPool(0, 0));
fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java:
new MetaStoreClientPool(INITIAL_META_STORE_CLIENT_POOL_SIZE,
fe/src/main/java/org/apache/impala/catalog/local/DirectMetaProvider.java:
msClientPool_ = new MetaStoreClientPool(cfg.num_metadata_loading_threads, {code}
It seems each one can grow up to a max size of 32 connections. I think the
usage in Frontend.java is only initialized for coordinators and the rest are
all initialized in the catalog?
Right, so after IMPALA-9375 the usage in DirectMetaProvider will go away, and
we will have the usage in CatalogMetaProvider and Catalog remaining? Is there a
bound on how many connections the Catalog and CatalogMetaProvider end up using,
or can they both potentially create 32 connections depending on the load?
Linking [https://gerrit.cloudera.org/#/c/16030/] for reference
> MetastoreClientPool should be singleton
> ---------------------------------------
>
> Key: IMPALA-9824
> URL: https://issues.apache.org/jira/browse/IMPALA-9824
> Project: IMPALA
> Issue Type: Improvement
> Components: Catalog
> Reporter: Vihang Karajgaonkar
> Assignee: Vihang Karajgaonkar
> Priority: Minor
> Fix For: Not Applicable
>
>
> Currently, the MetastoreClientPool is instantiated at multiple places in the
> code and it would be good to refactor the code to make it a singleton. Each
> MetastoreClientPool creates multiple clients to HMS and unnecessary creation
> of multiple pools could cause problems on HMS side.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
