Quanlong Huang created IMPALA-11307:
---------------------------------------
Summary: Loading snapshot testdata failed
Key: IMPALA-11307
URL: https://issues.apache.org/jira/browse/IMPALA-11307
Project: IMPALA
Issue Type: Bug
Reporter: Quanlong Huang
Assignee: Quanlong Huang
I tried to load a snapshot of testdata and encountered this failure:
{code}
Loading TPC-H data (logging to
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/logs/data_loading/load-tpch.log)...
Loading TPC-DS data (logging to
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/logs/data_loading/load-tpcds.log)...
FAILED (Took: 4 min 52 sec)
'load-data functional-query exhaustive' failed. Tail of log:
21:30:39 load-functional-query-exhaustive-hive-generated-seq-bzip-record.sql
21:30:39 load-functional-query-exhaustive-hive-generated-seq-snap-block.sql
21:30:39 load-functional-query-exhaustive-hive-generated-seq-snap-record.sql
...
21:32:22 Error executing impala SQL:
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-seq-snap-record.sql
See:
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-seq-snap-record.sql.log
21:32:22 Beginning execution of impala SQL on localhost:
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/logs/data_loading/sql/functional/create-functional-query-exhaustive-impala-generated-rc-gzip-block.sql
Background task Loading functional-query data (pid 8863) failed.
Loading workload 'tpch' using exploration strategy 'core' OK (Took: 6 min 25
sec)
Loading workload 'tpcds' using exploration strategy 'core' OK (Took: 11 min
42 sec)
ERROR in
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/testdata/bin/create-load-data.sh
at line 85: fi
Generated:
/data/jenkins/workspace/impala-private-parameterized/repos/Impala/logs/extra_junit_xml_logs/generate_junitxml.buildall.create-load-data.20220520_04_39_12.xml
{code}
Looking into the error log file, the failure is
{code}
ERROR: DROP TABLE IF EXISTS
functional_seq_record_snap.insert_only_transactional_table
Traceback (most recent call last):
File
"/data/jenkins/workspace/impala-private-parameterized/repos/Impala/bin/load-data.py",
line 188, in exec_impala_query_from_file
result = impala_client.execute(query)
File
"/data/jenkins/workspace/impala-private-parameterized/repos/Impala/tests/beeswax/impala_beeswax.py",
line 189, in execute
handle = self.__execute_query(query_string.strip(), user=user)
File
"/data/jenkins/workspace/impala-private-parameterized/repos/Impala/tests/beeswax/impala_beeswax.py",
line 367, in __execute_query
self.wait_for_finished(handle)
File
"/data/jenkins/workspace/impala-private-parameterized/repos/Impala/tests/beeswax/impala_beeswax.py",
line 388, in wait_for_finished
raise ImpalaBeeswaxException("Query aborted:" + error_log, None)
ImpalaBeeswaxException: ImpalaBeeswaxException:
Query aborted:ImpalaRuntimeException: Error making 'dropTable' RPC to Hive
Metastore:
CAUSED BY: MetaException: Cannot drop table as it is used in the following
materialized views [functional_seq_record_snap.materialized_view]
{code}
The DROP TABLE statement is executed before each CREATE TABLE statment, e.g.
{code:sql}
DROP TABLE IF EXISTS functional_seq_record_snap.insert_only_transactional_table;
CREATE TABLE IF NOT EXISTS
functional_seq_record_snap.insert_only_transactional_table (
col1 int
)
STORED AS SEQUENCEFILE
LOCATION
'/test-warehouse/managed/functional_seq_record_snap.db/insert_only_transactional_table_seq_record_snap'
TBLPROPERTIES (
'transactional_properties' = 'insert_only',
'transactional' = 'true'
);
{code}
The MV is supposed to be created later in
load-functional-query-exhaustive-hive-generated-seq-snap-record.sql:
{code:sql}
DROP TABLE IF EXISTS functional_seq_record_snap.materialized_view;
-- The create materialized view command is moved down so that the database's
-- managed directory has been created. Otherwise the command would fail. This
-- is a bug in Hive.
CREATE MATERIALIZED VIEW IF NOT EXISTS
functional_seq_record_snap.materialized_view
AS SELECT * FROM functional_seq_record_snap.insert_only_transactional_table;
{code}
As I'm loading the snapshot which is done by loading HMS snapshot and then
loading the HDFS snapshot, I think those CREATE/DROP statements should be
skipped.
CC [~amansinha]
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
