Bharath Vissapragada has posted comments on this change. Change subject: IMPALA-2347: Reuse metastore client connections in Catalog ......................................................................
Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/3381/2//COMMIT_MSG Commit Message: Line 13: this patch increases the default intial metastore pool size to 10 from > typo: initial. Why did you decide to increase this? I have seen connection - Like I mentioned in the other comment, earlier every call to a getClient() will create a new MetastoreClient. This patch can ease the behavior a little by re-using the connections as the release() call now puts them in the pool and further getClients() will reuse these connections. - Regarding the increase in initial pool size, I did so as I feel it is very less after looking at how often we use getClient() calls (CatalogOp calls). http://gerrit.cloudera.org:8080/#/c/3381/2/fe/src/main/java/com/cloudera/impala/catalog/MetaStoreClientPool.java File fe/src/main/java/com/cloudera/impala/catalog/MetaStoreClientPool.java: Line 76: public void release() { > Some time I saw connection to be in the hundreds which means this may not b It could be due to two things, 1) Like you mentioned, we may not be calling release() properly which is a leak, (please raise a jira if you see code where release() isn't being called.) 2) We don't set a limit on the max no. of connections. If you see the behavior prior to this patch, we just create a new MetaStoreClient() for every getClient() call. This under heavy load can appear as a lot of connections. -- To view, visit http://gerrit.cloudera.org:8080/3381 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: comment Gerrit-Change-Id: I517c0e1efef2584cd8d34017b33574f2ad69bd52 Gerrit-PatchSet: 2 Gerrit-Project: Impala Gerrit-Branch: cdh5-trunk Gerrit-Owner: Bharath Vissapragada <[email protected]> Gerrit-Reviewer: Bharath Vissapragada <[email protected]> Gerrit-Reviewer: Dimitris Tsirogiannis <[email protected]> Gerrit-Reviewer: Huaisi Xu <[email protected]> Gerrit-HasComments: Yes
