[
https://issues.apache.org/jira/browse/SENTRY-1630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16021815#comment-16021815
]
Sergio Peña commented on SENTRY-1630:
-------------------------------------
This issue is related to concurrency not handled very well on the local
metastore. This only affects unit tests because is where we use local
metastore.
The issue is already solved on Hive 1.3.0 and 2.0.
Found on HIVE-6893 but fixed on HIVE-10956
[~akolb] another reason to bump our hive version.
> out of sequence error in HMSFollower
> ------------------------------------
>
> Key: SENTRY-1630
> URL: https://issues.apache.org/jira/browse/SENTRY-1630
> Project: Sentry
> Issue Type: Sub-task
> Components: Hdfs Plugin
> Affects Versions: sentry-ha-redesign
> Reporter: Hao Hao
> Assignee: Sergio Peña
> Priority: Blocker
> Fix For: sentry-ha-redesign
>
>
> When HMSFollower exits in an abnormal way, for all the following connection
> would encounter out of sequence and SocketTimeoutException: Read timed out.
> Looking at HIVE-6893, it seems to relate to leakage connection problem.
> {noformat}2017-02-15 19:03:42,822 ERROR
> org.apache.sentry.hdfs.FullUpdateInitializer: Task did not complete
> successfully after 0 tries. Exception got:
> org.apache.thrift.TApplicationException: get_database failed: out of sequence
> response
> 2017-02-15 19:03:42,827 ERROR org.apache.sentry.hdfs.FullUpdateInitializer:
> Task did not complete successfully after 0 tries. Exception got:
> MetaException(message:Got exception:
> org.apache.thrift.transport.TTransportException null)
> 2017-02-15 19:03:43,803 INFO hive.metastore: Closed a connection to
> metastore, current connections: 0
> 2017-02-15 19:03:43,803 ERROR org.apache.sentry.service.thrift.HMSFollower:
> Exception occurred persisting Hive full snapshot into DB
> java.lang.RuntimeException: org.apache.thrift.TApplicationException:
> get_database failed: out of sequence response
> at
> org.apache.sentry.hdfs.FullUpdateInitializer.createInitialUpdate(FullUpdateInitializer.java:324)
> at
> org.apache.sentry.service.thrift.HMSFollower.fetchFullUpdate(HMSFollower.java:343)
> at
> org.apache.sentry.service.thrift.HMSFollower.run(HMSFollower.java:244)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.thrift.TApplicationException: get_database failed: out
> of sequence response
> at
> org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:84)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.recv_get_database(ThriftHiveMetastore.java:662)
> at
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.get_database(ThriftHiveMetastore.java:649)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.getDatabase(HiveMetaStoreClient.java:1213)
> at
> org.apache.sentry.hdfs.FullUpdateInitializer$DbTask.doTask(FullUpdateInitializer.java:256)
> at
> org.apache.sentry.hdfs.FullUpdateInitializer$BaseTask$RetryStrategy.exec(FullUpdateInitializer.java:110)
> at
> org.apache.sentry.hdfs.FullUpdateInitializer$BaseTask.call(FullUpdateInitializer.java:150)
> at
> org.apache.sentry.hdfs.FullUpdateInitializer$BaseTask.call(FullUpdateInitializer.java:79)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> ... 3 more
> 2017-02-15 19:03:43,849 INFO org.apache.sentry.service.thrift.HMSFollower:
> Making a kerberos connection to HMS{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
