[
https://issues.apache.org/jira/browse/IMPALA-9665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Tamas resolved IMPALA-9665.
--------------------------------
Fix Version/s: Impala 4.0
Resolution: Fixed
> Database not found errors in query_test.test_insert (TestInsertQueries)
> -----------------------------------------------------------------------
>
> Key: IMPALA-9665
> URL: https://issues.apache.org/jira/browse/IMPALA-9665
> Project: IMPALA
> Issue Type: Bug
> Reporter: Andrew Sherman
> Assignee: Adam Tamas
> Priority: Critical
> Fix For: Impala 4.0
>
>
> The query_test/test_insert.py test uses the unique_database fixture to create
> a database.
> The tests fail when they try to use the database
> Example failure:
> {code}
> .
> query_test/test_insert.py::TestInsertQueries::()::test_insert_large_string[compression_codec:
> none | protocol: beeswax | exec_option: {'sync_ddl': 1, 'batch_size': 1,
> 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': True,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> parquet/none]
> .
> query_test/test_insert.py::TestInsertQueries::()::test_insert[compression_codec:
> none | protocol: beeswax | exec_option: {'sync_ddl': 0, 'batch_size': 0,
> 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': True,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> text/none]
> F
> query_test/test_insert.py::TestInsertQueries::()::test_insert[compression_codec:
> snappy | protocol: beeswax | exec_option: {'sync_ddl': 1, 'batch_size': 0,
> 'num_nodes': 0, 'disable_codegen_rows_threshold': 0, 'disable_codegen': True,
> 'abort_on_error': 1, 'exec_single_node_rows_threshold': 0} | table_format:
> parquet/none]
> query_test/test_insert.py:136: in test_insert
> multiple_impalad=vector.get_value('exec_option')['sync_ddl'] == 1)
> common/impala_test_suite.py:567: in run_test_case
> table_format_info, use_db, pytest.config.option.scale_factor)
> common/impala_test_suite.py:778: in change_database
> impala_client.execute(query)
> common/impala_connection.py:205: in execute
> return self.__beeswax_client.execute(sql_stmt, user=user)
> beeswax/impala_beeswax.py:187: in execute
> handle = self.__execute_query(query_string.strip(), user=user)
> beeswax/impala_beeswax.py:363: in __execute_query
> handle = self.execute_query_async(query_string, user=user)
> beeswax/impala_beeswax.py:357: in execute_query_async
> handle = self.__do_rpc(lambda: self.imp_service.query(query,))
> beeswax/impala_beeswax.py:520: in __do_rpc
> raise ImpalaBeeswaxException(self.__build_error_message(b), b)
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> E INNER EXCEPTION: <class 'beeswaxd.ttypes.BeeswaxException'>
> E MESSAGE: AnalysisException: Database does not exist:
> test_insert_ecc90474
> {code}
> The server logs seem to show that the sequence is
> {code}
> DROP DATABASE IF EXISTS test_insert_ecc90474
> CREATE DATABASE test_insert_ecc90474
> USE test_insert_ecc90474
> DROP DATABASE test_insert_ecc90474
> USE test_insert_ecc90474
> {code}
> The server logs show:
> {code}
> I0416 07:46:52.364746 15419 impala-beeswax-server.cc:54] query(): query=DROP
> DATABASE IF EXISTS `test_insert_ecc90474` CASCADE
> 01: query (string) = "DROP DATABASE IF EXISTS `test_insert_ecc90474`
> CASCADE",
> I0416 07:46:52.368064 15419 Frontend.java:1490]
> 2348c561d46371c5:c8a2c29b00000000] Analyzing query: DROP DATABASE IF EXISTS
> `test_insert_ecc90474` CASCADE db: default
> I0416 07:46:57.091251 15419 impala-beeswax-server.cc:54] query():
> query=CREATE DATABASE `test_insert_ecc90474`
> 01: query (string) = "CREATE DATABASE `test_insert_ecc90474`",
> I0416 07:46:57.095499 15419 Frontend.java:1490]
> 84415a0cf8d9f7c9:878434bc00000000] Analyzing query: CREATE DATABASE
> `test_insert_ecc90474` db: default
> I0416 07:46:57.105612 15419 ImpaladCatalog.java:209]
> 84415a0cf8d9f7c9:878434bc00000000] Adding: DATABASE:test_insert_ecc90474
> version: 44117 size: 193
> I0416 07:46:57.111361 25769 impala-beeswax-server.cc:54] query(): query=use
> test_insert_ecc90474
> 01: query (string) = "use test_insert_ecc90474",
> I0416 07:46:57.143752 25769 Frontend.java:1490]
> df4f5c820a08f50f:71a027de00000000] Analyzing query: use test_insert_ecc90474
> db: default
> I0416 07:46:57.485566 15419 impala-beeswax-server.cc:54] query(): query=DROP
> DATABASE `test_insert_ecc90474` CASCADE
> 01: query (string) = "DROP DATABASE `test_insert_ecc90474` CASCADE",
> I0416 07:46:57.490993 15419 Frontend.java:1490]
> 8f4c03dd31cc7d2b:03bf91cf00000000] Analyzing query: DROP DATABASE
> `test_insert_ecc90474` CASCADE db: default
> I0416 07:46:57.503159 15419 ImpaladCatalog.java:209]
> 8f4c03dd31cc7d2b:03bf91cf00000000] Deleting: DATABASE:test_insert_ecc90474
> version: 44118 size: 193
> I0416 07:47:01.090381 19101 ImpaladCatalog.java:209] Adding:
> DATABASE:test_insert_ecc90474 version: 44117 size: 186
> I0416 07:47:03.091727 19101 ImpaladCatalog.java:209] Deleting:
> DATABASE:test_insert_ecc90474 version: 44118 size: 186
> [second log file]
> I0416 07:46:57.148188 25771 impala-beeswax-server.cc:54] query(): query=use
> test_insert_ecc90474
> 01: query (string) = "use test_insert_ecc90474",
> I0416 07:46:57.166188 25771 Frontend.java:1490]
> f8417aebe9247626:834c219f00000000] Analyzing query: use test_insert_ecc90474
> db: default
> I0416 07:46:57.166695 25771 jni-util.cc:288]
> f8417aebe9247626:834c219f00000000]
> org.apache.impala.common.AnalysisException: Database does not exist:
> test_insert_ecc90474
> I0416 07:46:57.293211 25771 status.cc:129] f8417aebe9247626:834c219f00000000]
> AnalysisException: Database does not exist: test_insert_ecc90474
> I0416 07:47:00.009994 19067 ImpaladCatalog.java:209] Adding:
> DATABASE:test_insert_ecc90474 version: 44117 size: 186
> I0416 07:47:02.010959 19068 ImpaladCatalog.java:209] Deleting:
> DATABASE:test_insert_ecc90474 version: 44118 size: 186
> {code}
> Catalog shows db being created and deleted
> {code}
> I0416 07:46:57.110661 18776 catalog-server.cc:746] Collected update:
> 1:DATABASE:test_insert_ecc90474, version=44117, original size=186, compressed
> size=150
> I0416 07:46:57.111083 18776 catalog-server.cc:746] Collected update:
> 1:CATALOG_SERVICE_ID, version=44117, original size=60, compressed size=58
> I0416 07:46:59.111246 18799 catalog-server.cc:351] A catalog update with 2
> entries is assembled. Catalog version: 44117 Last sent catalog version: 44116
> I0416 07:46:59.113987 13108 HdfsTable.java:1004] Reloading metadata for table
> definition and all partition(s) of
> test_mt_dop_insert_bc9f3730.table_with_header_insert (TRUNCATE)
> I0416 07:46:59.119468 13108 Table.java:419] Load Valid Write Id List Done.
> Time taken: 1.514us
> I0416 07:46:59.120395 13108 HdfsTable.java:700] Loaded file and block
> metadata for test_mt_dop_insert_bc9f3730.table_with_header_insert partitions:
> . Time taken: 213.608us
> I0416 07:46:59.120442 13108 HdfsTable.java:1035] Incrementally loaded table
> metadata for: test_mt_dop_insert_bc9f3730.table_with_header_insert
> I0416 07:46:59.120779 18776 catalog-server.cc:746] Collected update:
> 1:TABLE:test_mt_dop_insert_bc9f3730.table_with_header_insert, version=44119,
> original size=1352, compressed size=822
> I0416 07:46:59.121081 18776 catalog-server.cc:746] Collected deletion:
> 1:DATABASE:test_insert_ecc90474, version=44118, original size=186, compressed
> size=150
> {code}
> HMS logs show db being created and deleted
> {code}
> 2020-04-16T07:46:57,096 INFO [pool-8-thread-175] metastore.HiveMetaStore:
> 151: source:127.0.0.1 create_database: Database(name:test_insert_ecc90474,
> description:null, locationUri:null, parameters:null, ownerName:jenkins,
> ownerType:USER)
> 2020-04-16T07:46:57,096 INFO [pool-8-thread-175] HiveMetaStore.audit:
> ugi=jenkins ip=127.0.0.1 cmd=source:127.0.0.1 create_database:
> Database(name:test_insert_ecc90474, description:null, locationUri:null,
> parameters:null, ownerName:jenkins, ownerType:USER)
> 2020-04-16T07:46:57,096 WARN [pool-8-thread-175] metastore.ObjectStore:
> Failed to get database test_insert_ecc90474, returning NoSuchObjectException
> 2020-04-16T07:46:57,098 INFO [pool-8-thread-175] common.FileUtils: Creating
> directory if it doesn't exist:
> hdfs://localhost:20500/test-warehouse/test_insert_ecc90474.db
> 2020-04-16T07:46:57,104 INFO [pool-8-thread-175] metastore.HiveMetaStore:
> 151: source:127.0.0.1 get_database: test_insert_ecc90474
> 2020-04-16T07:46:57,104 INFO [pool-8-thread-175] HiveMetaStore.audit:
> ugi=jenkins ip=127.0.0.1 cmd=source:127.0.0.1 get_database:
> test_insert_ecc90474
> 2020-04-16T07:46:57,491 INFO [pool-8-thread-176] metastore.HiveMetaStore:
> 150: source:127.0.0.1 get_database: test_insert_ecc90474
> 2020-04-16T07:46:57,491 INFO [pool-8-thread-176] HiveMetaStore.audit:
> ugi=jenkins ip=127.0.0.1 cmd=source:127.0.0.1 get_database:
> test_insert_ecc90474
> 2020-04-16T07:46:57,492 INFO [pool-8-thread-176] metastore.HiveMetaStore:
> 150: source:127.0.0.1 get_all_tables: db=test_insert_ecc90474
> 2020-04-16T07:46:57,492 INFO [pool-8-thread-176] HiveMetaStore.audit:
> ugi=jenkins ip=127.0.0.1 cmd=source:127.0.0.1 get_all_tables:
> db=test_insert_ecc90474
> 2020-04-16T07:46:57,493 INFO [pool-8-thread-176] metastore.HiveMetaStore:
> 150: source:127.0.0.1 get_multi_table : db=test_insert_ecc90474 tbls=
> 2020-04-16T07:46:57,493 INFO [pool-8-thread-176] HiveMetaStore.audit:
> ugi=jenkins ip=127.0.0.1 cmd=source:127.0.0.1 get_multi_table :
> db=test_insert_ecc90474 tbls=
> 2020-04-16T07:46:57,493 INFO [pool-8-thread-176] metastore.HiveMetaStore:
> 150: source:127.0.0.1 drop_database: test_insert_ecc90474
> 2020-04-16T07:46:57,493 INFO [pool-8-thread-176] HiveMetaStore.audit:
> ugi=jenkins ip=127.0.0.1 cmd=source:127.0.0.1 drop_database:
> test_insert_ecc90474
> 2020-04-16T07:46:57,495 INFO [pool-8-thread-176] metastore.HiveMetaStore:
> 150: source:127.0.0.1 get_functions: db=test_insert_ecc90474 pat=*
> 2020-04-16T07:46:57,495 INFO [pool-8-thread-176] HiveMetaStore.audit:
> ugi=jenkins ip=127.0.0.1 cmd=source:127.0.0.1 get_functions:
> db=test_insert_ecc90474 pat=*
> 2020-04-16T07:46:57,495 INFO [pool-8-thread-176] metastore.ObjectStore:
> Dropping database test_insert_ecc90474 along with all tables
> 2020-04-16T07:46:57,502 INFO [pool-8-thread-176] fs.TrashPolicyDefault:
> Moved: 'hdfs://localhost:20500/test-warehouse/test_insert_ecc90474.db' to
> trash at:
> hdfs://localhost:20500/user/jenkins/.Trash/Current/test-warehouse/test_insert_ecc90474.db
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
