[
https://issues.apache.org/jira/browse/HIVE-20511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mahesh kumar behera updated HIVE-20511:
---------------------------------------
Attachment: HIVE-20511.04.patch
> REPL DUMP is leaking metastore connections
> ------------------------------------------
>
> Key: HIVE-20511
> URL: https://issues.apache.org/jira/browse/HIVE-20511
> Project: Hive
> Issue Type: Bug
> Components: repl
> Affects Versions: 4.0.0
> Reporter: mahesh kumar behera
> Assignee: mahesh kumar behera
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-20511.01.patch, HIVE-20511.02.patch,
> HIVE-20511.03.patch, HIVE-20511.04.patch
>
>
> With remote metastore, REPL DUMP leaking connections. Each repl dump task is
> leaking one connection due to the usage of stale hive object.
> {code}
> 18/09/04 16:01:46 INFO ReplState: REPL::EVENT_DUMP:
> {"dbName":"*","eventId":"566","eventType":"EVENT_COMMIT_TXN","eventsDumpProgress":"1/0","dumpTime":1536076906}
> 18/09/04 16:01:46 INFO events.AbstractEventHandler: Processing#567 OPEN_TXN
> message :
> {"txnIds":null,"timestamp":1536076905,"fromTxnId":269,"toTxnId":269,"server":"thrift://metastore-service.warehouse-1536062326-s74h.svc.cluster.local:9083","servicePrincipal":""}
> 18/09/04 16:01:46 INFO ReplState: REPL::EVENT_DUMP:
> {"dbName":"*","eventId":"567","eventType":"EVENT_OPEN_TXN","eventsDumpProgress":"2/0","dumpTime":1536076906}
> 18/09/04 16:01:46 INFO metastore.HiveMetaStoreClient: Trying to connect to
> metastore with URI
> thrift://metastore-service.warehouse-1536062326-s74h.svc.cluster.local:9083
> 18/09/04 16:01:46 INFO metastore.HiveMetaStoreClient: Opened a connection to
> metastore, current connections: 471
> 18/09/04 16:01:46 INFO metastore.HiveMetaStoreClient: Connected to metastore.
> 18/09/04 16:01:46 INFO metastore.RetryingMetaStoreClient:
> RetryingMetaStoreClient proxy=class
> org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient ugi=hive
> (auth:SIMPLE) retries=24 delay=5 lifetime=0
> 18/09/04 16:01:46 INFO ReplState: REPL::END:
> {"dbName":"*","dumpType":"INCREMENTAL","actualNumEvents":2,"dumpEndTime":1536076906,"dumpDir":"/user/hive/repl/e45bde27-74dc-45cd-9823-400a8fc1aea3","lastReplId":"567"}
> 18/09/04 16:01:46 INFO repl.ReplDumpTask: Done dumping events, preparing to
> return /user/hive/repl/e45bde27-74dc-45cd-9823-400a8fc1aea3,567
> 18/09/04 16:01:46 INFO ql.Driver: Completed executing
> command(queryId=hive_20180904160145_30f9570a-44e0-4f3b-b961-1906d3972fc4);
> Time taken: 0.585 seconds
> OK
> 18/09/04 16:01:46 INFO ql.Driver: OK
> 18/09/04 16:01:46 INFO lockmgr.DbTxnManager: Stopped heartbeat for query:
> hive_20180904160145_30f9570a-44e0-4f3b-b961-1906d3972fc4
> 18/09/04 16:01:46 INFO metastore.HiveMetaStoreClient: Trying to connect to
> metastore with URI
> thrift://metastore-service.warehouse-1536062326-s74h.svc.cluster.local:9083
> 18/09/04 16:01:46 INFO metastore.HiveMetaStoreClient: Opened a connection to
> metastore, current connections: 472
> 18/09/04 16:01:46 INFO metastore.HiveMetaStoreClient: Connected to metastore.
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
