[
https://issues.apache.org/jira/browse/IMPALA-9049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17944884#comment-17944884
]
Quanlong Huang commented on IMPALA-9049:
----------------------------------------
The test is simple:
{code:python}
@UniqueDatabase.parametrize(sync_ddl=True)
def test_sync_ddl_drop(self, unique_database):
"""Verifies the catalog gets updated properly when dropping objects with
sync_ddl
enabled"""
self.client.set_configuration({'sync_ddl': 1})
# Drop the database immediately after creation (within a statestore
heartbeat) and
# verify the catalog gets updated properly.
self.client.execute("drop database {0}".format(unique_database))
assert unique_database not in self.all_db_names()
# Re-create database to make unique_database teardown succeed.
self._create_db(unique_database) {code}
[https://github.com/apache/impala/blob/c5a0ec8bdffd958984c0f6d34e0fcb8acc138457/tests/metadata/test_ddl.py#L440-L450]
In Standard Error, the last statement is "show databases". It failed so the
next create database statement is not executed. Thus resulting in
unique_database teardown failed. The original failure of "show databases" is
covered up. Do you have any ideas to avoid this? [~rizaon]
In the impalad logs, I can see why the "show databases" failed:
{noformat}
I0415 16:29:49.199643 53798 Frontend.java:2385]
114cc8ac31f082e1:c369d1c100000000] Analyzing query: show databases db: default
...
I0415 16:29:49.207096 53798 CatalogdMetaProvider.java:1684]
114cc8ac31f082e1:c369d1c100000000] Invalidated objects in cache: [list of
database names, HMS_METADATA for DB test_hms_event_sync_f2086869_4]
...
I0415 16:29:49.207396 53798 jni-util.cc:321] 114cc8ac31f082e1:c369d1c100000000]
org.apache.impala.catalog.local.InconsistentMetadataFetchException: Fetching
DATABASE failed: DB_NOT_FOUND. Could not find TCatalogObject(type:DATABASE,
catalog_version:0, db:TDatabase(db_name:test_hms_event_sync_f2086869_4))
at
org.apache.impala.catalog.local.CatalogdMetaProvider.sendRequest(CatalogdMetaProvider.java:489)
at
org.apache.impala.catalog.local.CatalogdMetaProvider.access$100(CatalogdMetaProvider.java:208)
at
org.apache.impala.catalog.local.CatalogdMetaProvider$2.call(CatalogdMetaProvider.java:716)
at
org.apache.impala.catalog.local.CatalogdMetaProvider$2.call(CatalogdMetaProvider.java:711)
at
org.apache.impala.catalog.local.CatalogdMetaProvider.loadWithCaching(CatalogdMetaProvider.java:586)
at
org.apache.impala.catalog.local.CatalogdMetaProvider.loadDb(CatalogdMetaProvider.java:708)
at
org.apache.impala.catalog.local.LocalDb.getMetaStoreDb(LocalDb.java:101)
at org.apache.impala.catalog.local.LocalDb.toThrift(LocalDb.java:318)
at org.apache.impala.service.JniFrontend.getDbs(JniFrontend.java:386)
{noformat}
> TestDdlStatements.test_sync_ddl_drop is flaky
> ---------------------------------------------
>
> Key: IMPALA-9049
> URL: https://issues.apache.org/jira/browse/IMPALA-9049
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Affects Versions: Impala 3.4.0
> Reporter: Yongzhi Chen
> Assignee: Quanlong Huang
> Priority: Major
> Labels: broken-build, flaky
>
> It failed precommit tests for unrelated change:
> https://jenkins.impala.io/job/ubuntu-16.04-dockerised-tests/1405/testReport/metadata.test_ddl/TestDdlStatements/test_sync_ddl_drop_protocol__beeswax___exec_option____sync_ddl___0___batch_size___0___num_nodes___0___disable_codegen_rows_threshold___0___disable_codegen___False___abort_on_error___1___exec_single_node_rows_threshold___0____table_format__text_none_unique_database0_/
> Error Message
> test setup failure
> Stacktrace
> conftest.py:319: in cleanup
> {'sync_ddl': sync_ddl})
> common/impala_test_suite.py:790: in wrapper
> return function(*args, **kwargs)
> common/impala_test_suite.py:798: in execute_query_expect_success
> result = cls.__execute_query(impalad_client, query, query_options, user)
> common/impala_test_suite.py:893: in __execute_query
> return impalad_client.execute(query, user=user)
> common/impala_connection.py:205: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:187: in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:362: in __execute_query
> handle = self.execute_query_async(query_string, user=user)
> beeswax/impala_beeswax.py:356: in execute_query_async
> handle = self.__do_rpc(lambda: self.imp_service.query(query,))
> beeswax/impala_beeswax.py:519: in __do_rpc
> raise ImpalaBeeswaxException(self.__build_error_message(b), b)
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> E INNER EXCEPTION: <class 'beeswaxd.ttypes.BeeswaxException'>
> E MESSAGE: AnalysisException: Database does not exist:
> test_sync_ddl_drop_eb3f13e1
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
