Joe McDonnell created IMPALA-9092:
-------------------------------------
Summary: Fix "show create table" tests on USE_CDP_HIVE=true to
account for HIVE=22158
Key: IMPALA-9092
URL: https://issues.apache.org/jira/browse/IMPALA-9092
Project: IMPALA
Issue Type: Bug
Components: Frontend
Affects Versions: Impala 3.4.0
Reporter: Joe McDonnell
Hive changed behavior with HIVE-22158 so that only transactional tables are
considered managed and all other are considered external. This means that a
regular "create table" will result in an external table with table properties
of 'TRANSLATED_TO_EXTERNAL'='TRUE', 'external.table.purge'='TRUE'. This breaks
our tests that rely on "show create table", because the table is newly external
and has extra table properties. For example:
{noformat}
query_test/test_kudu.py:842: in test_primary_key_and_distribution
db=cursor.conn.db_name, kudu_addr=KUDU_MASTER_HOSTS))
query_test/test_kudu.py:824: in assert_show_create_equals
assert cursor.fetchall()[0][0] == \
E assert "CREATE EXTER...='localhost')" == "CREATE TABLE ...='localhost')"
E - CREATE EXTERNAL TABLE testshowcreatetable_15312_ggn1hk.nvbpxfuxze
E ? ---------
E + CREATE TABLE testshowcreatetable_15312_ggn1hk.nvbpxfuxze (
E ? ++
E + c INT NOT NULL ENCODING AUTO_ENCODING COMPRESSION DEFAULT_COMPRESSION,
E + PRIMARY KEY (c)
E + )
E + PARTITION BY HASH (c) PARTITIONS 3
E STORED AS KUDU
E - TBLPROPERTIES ('TRANSLATED_TO_EXTERNAL'='TRUE',
'external.table.purge'='TRUE', 'kudu.master_addresses'='localhost')
E + TBLPROPERTIES ('kudu.master_addresses'='localhost'){noformat}
We need to decide on the right behavior for "show create table" and update the
tests.
For Kudu tables, tables with TRANSLATED_TO_EXTERNAL=true and
external.table.purge=TRUE should be equivalent to a non-external Kudu table,
and we can just detect this case and generate the same SQL as before.
Other cases may need new logic. I think it makes sense to also address other
tests due to MANAGED vs EXTERNAL distinction or extra table properties with
this JIRA. Here is a list of tests that seem to have this problem:
{noformat}
metadata/test_ddl.py TestDdlStatements.test_create_alter_tbl_properties
metadata/test_show_create_table.py *
query_test/test_kudu.py TestShowCreateTable*
org.apache.impala.catalog.CatalogTest.testCreateTableMetadata
org.apache.impala.catalog.local.LocalCatalogTest.testKuduTable{noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
