[
https://issues.apache.org/jira/browse/IMPALA-13421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Smith closed IMPALA-13421.
----------------------------------
Resolution: Duplicate
> HiveMetastore failed to launch in ozone build
> ---------------------------------------------
>
> Key: IMPALA-13421
> URL: https://issues.apache.org/jira/browse/IMPALA-13421
> Project: IMPALA
> Issue Type: Bug
> Reporter: Quanlong Huang
> Assignee: Michael Smith
> Priority: Critical
>
> Hive Metastore failed to start in an ozone build.
> {noformat}
> 12:33:04 Starting ozone (Web UI - http://localhost:9874)
> 12:33:09 The kms cluster is running
> 12:33:37 The ozone cluster is running
> 12:33:37 Creating Ozone volume/bucket
> 12:33:37 Ozone encryption enabled for impala/test-warehouse
> 12:33:37 24/10/03 12:33:39 WARN util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> 12:33:39 24/10/03 12:33:40 INFO rpc.RpcClient: Creating Volume: impala, with
> jenkins as owner and space quota set to -1 bytes, counts quota set to -1
> 12:33:40 VOLUME_ALREADY_EXISTS Volume already exists
> 12:33:40 24/10/03 12:33:41 WARN util.NativeCodeLoader: Unable to load
> native-hadoop library for your platform... using builtin-java classes where
> applicable
> 12:33:41 24/10/03 12:33:41 INFO rpc.RpcClient: Creating Bucket:
> impala/test-warehouse, with server-side default bucket layout, jenkins as
> owner, Versioning false, Storage Type set to DISK and Encryption set to true,
> Replication Type set to server-side default replication type, Namespace Quota
> set to -1, Space Quota set to -1
> 12:33:41 BUCKET_ALREADY_EXISTS Bucket already exist
> 12:33:42 Stopping kudu
> 12:33:42 Starting kudu (Web UI - http://localhost:8051)
> 12:33:49 The kudu cluster is running
> 12:34:04 --> Starting Hive Metastore Service
> 12:34:04 No handlers could be found for logger "thrift.transport.TSocket"
> 12:34:04 Waiting for the Metastore at localhost:9083...
> 12:34:48 Waiting for the Metastore at localhost:9083...
> 12:34:48 Waiting for the Metastore at localhost:9083...
> ...
> 12:39:05 Waiting for the Metastore at localhost:9083...
> 12:39:05 Metastore service failed to start within 300.0 seconds.
> 12:39:05 ERROR in
> /data/jenkins/workspace/impala-asf-master-core-ozone/repos/Impala/testdata/bin/run-hive-server.sh
> at line 166: ${CLUSTER_BIN}/wait-for-metastore.py
> --transport=${METASTORE_TRANSPORT}
> 12:39:05 Generated:
> /data/jenkins/workspace/impala-asf-master-core-ozone/repos/Impala/logs/extra_junit_xml_logs/generate_junitxml.buildall.run-hive-server.20241003_19_39_05.xml
> 12:39:05 ERROR in
> /data/jenkins/workspace/impala-asf-master-core-ozone/repos/Impala/testdata/bin/run-all.sh
> at line 91: tee ${IMPALA_CLUSTER_LOGS_DIR}/run-hive-server.log
> 12:39:05 Generated:
> /data/jenkins/workspace/impala-asf-master-core-ozone/repos/Impala/logs/extra_junit_xml_logs/generate_junitxml.buildall.run-all.20241003_19_39_05.xml
> 12:39:05 + echo 'buildall.sh ' -format '-snapshot_file
> /data/jenkins/workspace/impala-asf-master-core-ozone/testdata/test-warehouse-SNAPSHOT/test-warehouse-cdh7-1339-SNAPSHOT.tar.gz'
> '-metastore_snapshot_file
> /data/jenkins/workspace/impala-asf-master-core-ozone/testdata/hive_metastore_snapshot/hive_impala_dump_cdh7-1339.txt
> failed.'
> 12:39:05 buildall.sh -format -snapshot_file
> /data/jenkins/workspace/impala-asf-master-core-ozone/testdata/test-warehouse-SNAPSHOT/test-warehouse-cdh7-1339-SNAPSHOT.tar.gz
> -metastore_snapshot_file
> /data/jenkins/workspace/impala-asf-master-core-ozone/testdata/hive_metastore_snapshot/hive_impala_dump_cdh7-1339.txt
> failed{noformat}
> Looking into HMS logs, it failed to connect to the Ozone Manager:
> {noformat}
> 2024-10-03T12:34:08,927 INFO [main] metastore.HiveMetaStore: Starting hive
> metastore on port 9083
> 2024-10-03T12:34:08,935 INFO [main] metastore.AuthFactory: Using
> authentication NOSASL with kerberos authentication disabled
> 2024-10-03T12:34:11,908 INFO [main] retry.RetryInvocationHandler:
> com.google.protobuf.ServiceException: java.net.ConnectException: Call From
> impala-ec2-centos79-m6i-4xlarge-xldisk-0058.vpc.cloudera.com/127.0.0.1 to
> localhost:9862 failed on connection exception: java.net.ConnectException:
> Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused, while invoking
> $Proxy31.submitRequest over nodeId=null,nodeAddress=localhost:9862 after 1
> failover attempts. Trying to failover after sleeping for 4000ms.
> ...
> 2024-10-03T12:38:41,926 INFO [main] retry.RetryInvocationHandler:
> com.google.protobuf.ServiceException: java.net.ConnectException: Call From
> impala-ec2-centos79-m6i-4xlarge-xldisk-0058.vpc.cloudera.com/127.0.0.1 to
> localhost:9862 failed on connection exception: java.net.ConnectException:
> Connection refused; For more details see:
> http://wiki.apache.org/hadoop/ConnectionRefused, while invoking
> $Proxy31.submitRequest over nodeId=null,nodeAddress=localhost:9862 after 16
> failover attempts. Trying to failover after sleeping for 34000ms.
> 2024-10-03T12:39:05,886 INFO [shutdown-hook-0] metastore.HiveMetaStore:
> Shutting down hive metastore.{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
