[
https://issues.apache.org/jira/browse/IMPALA-9097?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17094624#comment-17094624
]
ASF subversion and git services commented on IMPALA-9097:
---------------------------------------------------------
Commit 75a6d7b2bba66825efb3a37f14c9447e64ea584f in impala's branch
refs/heads/master from Joe McDonnell
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=75a6d7b ]
IMPALA-9097: Don't require minicluster for backend tests
Currently, many backend tests require a running minicluster,
because they initialize a Frontend object that requires a
connection to the Hive Metastore. If the minicluster is not
running or if cluster configurations are missing (i.e.
bin/create-test-configurations.sh needs to run), the backend
tests will fail. The docker based tests always hit this,
because they run the backend tests without a minicluster.
The HMS dependency comes from the Frontend's MetaStoreClientPool,
which is unnecesary for backend tests. This modifies the
code so that it does not initialize this for backend tests,
and thus backend tests pass without a running minicluster.
Testing:
- Ran backend tests without a running minicluster
Change-Id: I8f1b1385853fb23df28d24d38761237e6e5c97a7
Reviewed-on: http://gerrit.cloudera.org:8080/15641
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Some backend tests fail if the Hive Metastore is not running
> ------------------------------------------------------------
>
> Key: IMPALA-9097
> URL: https://issues.apache.org/jira/browse/IMPALA-9097
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 3.4.0
> Reporter: Joe McDonnell
> Priority: Major
>
> In our docker-based tests (i.e. docker/test_with_docker.py), we run the
> backend tests without starting a minicluster. This is now failing due to a
> new dependency on the Hive Metastore. This applies to a bunch of tests, which
> all fail with this error:
> {noformat}
> F0917 00:37:47.849447 7660 frontend.cc:134] IllegalStateException:
> java.lang.RuntimeException: Unable to instantiate
> org.apache.hadoop.hive.metastore.HiveMetaStoreClientF0917
> 00:37:47.849447 7660 frontend.cc:134] IllegalStateException:
> java.lang.RuntimeException: Unable to instantiate
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient
> CAUSED BY: RuntimeException: Unable to instantiate
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient
> CAUSED BY: InvocationTargetException: null
> CAUSED BY: MetaException: Could not connect to meta store using any of the
> URIs provided. Most recent failure:
> org.apache.thrift.transport.TTransportException: java.net.ConnectException:
> Connection refused (Connection refused)
> at org.apache.thrift.transport.TSocket.open(TSocket.java:226)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:545)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:303)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1773)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:80)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:130)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:101)
> at
> org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:94)
> at
> org.apache.impala.catalog.MetaStoreClientPool$MetaStoreClient.<init>(MetaStoreClientPool.java:99)
> at
> org.apache.impala.catalog.MetaStoreClientPool$MetaStoreClient.<init>(MetaStoreClientPool.java:78)
> at
> org.apache.impala.catalog.MetaStoreClientPool.initClients(MetaStoreClientPool.java:174)
> at
> org.apache.impala.catalog.MetaStoreClientPool.<init>(MetaStoreClientPool.java:163)
> at
> org.apache.impala.catalog.MetaStoreClientPool.<init>(MetaStoreClientPool.java:155)
> at org.apache.impala.service.Frontend.<init>(Frontend.java:301)
> at org.apache.impala.service.Frontend.<init>(Frontend.java:270)
> at
> org.apache.impala.service.JniFrontend.<init>(JniFrontend.java:141){noformat}
> The list of tests that currently fail in this configuration are:
>
> {noformat}
> 2019-10-25 14:21:36.059346 The following tests FAILED:
> 2019-10-25 14:21:36.059378 3 - llvm-codegen-test (Child aborted)
> 2019-10-25 14:21:36.059404 8 - hash-table-test (Failed)
> 2019-10-25 14:21:36.059436 11 - row-batch-list-test (Child aborted)
> 2019-10-25 14:21:36.059468 19 - hdfs-parquet-scanner-test (Failed)
> 2019-10-25 14:21:36.059491 20 - expr-test (Failed)
> 2019-10-25 14:21:36.059521 21 - expr-codegen-test (Child aborted)
> 2019-10-25 14:21:36.059551 28 - data-stream-test (Child aborted)
> 2019-10-25 14:21:36.059586 39 - buffered-tuple-stream-test (Child aborted)
> 2019-10-25 14:21:36.059613 41 - tmp-file-mgr-test (Failed)
> 2019-10-25 14:21:36.059644 42 - row-batch-serialize-test (Failed)
> 2019-10-25 14:21:36.059673 43 - row-batch-test (Child aborted)
> 2019-10-25 14:21:36.059710 44 - collection-value-builder-test (Child
> aborted)
> 2019-10-25 14:21:36.059741 45 - runtime-state-test (Child aborted)
> 2019-10-25 14:21:36.059774 46 - buffer-allocator-test (Child aborted)
> 2019-10-25 14:21:36.059804 47 - buffer-pool-test (Child aborted)
> 2019-10-25 14:21:36.059829 48 - free-list-test (Failed)
> 2019-10-25 14:21:36.059863 49 - reservation-tracker-test (Child aborted)
> 2019-10-25 14:21:36.059890 50 - suballocator-test (Failed)
> 2019-10-25 14:21:36.059917 51 - disk-io-mgr-test (Failed)
> 2019-10-25 14:21:36.059946 52 - data-cache-test (Child aborted)
> 2019-10-25 14:21:36.059977 53 - admission-controller-test (Failed)
> 2019-10-25 14:21:36.060015 59 - session-expiry-test (Child aborted)
> 2019-10-25 14:21:36.060043 67 - rpc-mgr-test (Child aborted)
> 2019-10-25 14:21:36.060076 68 - rpc-mgr-kerberized-test (Child aborted)
> 2019-10-25 14:21:36.060102 77 - bloom-filter-test (Failed)
> 2019-10-25 14:21:36.060127 85 - hdfs-util-test (Failed)
> 2019-10-25 14:21:36.060154 91 - min-max-filter-test (Failed){noformat}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
