[
https://issues.apache.org/jira/browse/HIVE-21022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16716996#comment-16716996
]
Ashutosh Bapat commented on HIVE-21022:
---------------------------------------
The test TestRemoteHiveMetaStoreZK failed again. The fix isn't enough.
Looking more at the logs I see
The metastore server was started at 2018-12-10 19:15:27.663
[INFO ] 2018-12-10 19:15:27.663 [main] MetaStoreTestUtils - MetaStore Thrift
Server started on port: 33599 with warehouse dir:
file:/home/hiveptest/104.155.141.195-hiveptest-0/apache-github-source-source/standalone-metastore/metastore-server/target/warehouse/33599
with jdbcUrl:
jdbc:derby:;databaseName=/home/hiveptest/104.155.141.195-hiveptest-0/apache-github-source-source/standalone-metastore/metastore-server/target/tmp/junit_metastore_db_33599;create=true
Starting MetaStore Server on port 33599
The last line comes from TestRemoteHiveMetaStore.setUp() method
{quote}port =
MetaStoreTestUtils.startMetaStoreWithRetry(HadoopThriftAuthBridge.getBridge(),
conf);
System.out.println("Starting MetaStore Server on port " + port);
isServerStarted = true;
// This is default case with setugi off for both client and server
client = createClient();
{quote}
The next thing the code does is to start the client. So the client will start
immediately. But we see
[INFO ] 2018-12-10 19:15:27.715 [MetaStoreThread-33599] ZooKeeperHiveHelper -
Created the root name space:
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK on ZooKeeper
[ERROR] 2018-12-10 19:15:27.719 [main] MetaStoreUtils - Got exception:
org.apache.zookeeper.KeeperException$NoNodeException KeeperErrorCode = NoNode
for /org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK
org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode
for /org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStoreZK
The root namespace was created about 100 milliseconds after the metastore
server started whereas the failure comes only 4 milliseconds after the root
namespace was created. It looks highly likely that the client contacted the
zookeeper before the root namespace was created and hence the failure.
The right fix here would be to wait for the metastore server to add itself to
the ZooKeeper before returning from MetaStoreTestUtils.startMetaStoreWithRetry.
That can be done in MetaStoreTestUtils.loopUntilHMSReady.
> Fix remote metastore tests which use ZooKeeper
> ----------------------------------------------
>
> Key: HIVE-21022
> URL: https://issues.apache.org/jira/browse/HIVE-21022
> Project: Hive
> Issue Type: Bug
> Components: Tests
> Affects Versions: 4.0.0
> Reporter: Ashutosh Bapat
> Assignee: Ashutosh Bapat
> Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21022.01, HIVE-21022.01, HIVE-21022.01,
> HIVE-21022.02.patch
>
>
> Per [~vgarg]'s comment on HIVE-20794 at
> https://issues.apache.org/jira/browse/HIVE-20794?focusedCommentId=16714093&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16714093,
> the remote metatstore tests using ZooKeeper are flaky. They are failing with
> error "Got exception: org.apache.zookeeper.KeeperException$NoNodeException
> KeeperErrorCode = NoNode for /hs2mszktest".
> Both of these tests are using the same root namespace and hence the reason
> for this failure could be that the root namespace becomes unavailable to one
> test when the other drops it. The drop seems to be happening automatically
> through TestingServer code.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
