[
https://issues.apache.org/jira/browse/IMPALA-9093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16961032#comment-16961032
]
Csaba Ringhofer commented on IMPALA-9093:
-----------------------------------------
I think that this is a bigger issue than removing some tests: we are dropping
the feature of supporting upgraded tables.
I see 3 affected use cases:
1. creating a _new_ non-ACID managed table with Impala and upgrading it to
ACID - this became impossible, as we cannot create non-ACID managed tables
2. do the same in Hive - did it also became impossible to create "upgraded"
tables in newer versions of Hive?
3. upgrade a table created with older version of Hive or Impala - how does the
translation layer handle this? interpreting these "used to be managed" tables
as external tables?
I have no problem with dropping the support for upgraded tables, but it would
be good to know whether Impala can "meet" upgraded tables from use case 2. and
3. If it has to handle them, then we shouldn't leave this scenario untested.
If we do not want to lose the test coverage for upgraded tables completely,
then we could create such tables by copying files to the root of ACID tables
(that have no base directory yet).
> Fix ACID upgrade tests to account for HIVE-22158 (table translation)
> --------------------------------------------------------------------
>
> Key: IMPALA-9093
> URL: https://issues.apache.org/jira/browse/IMPALA-9093
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 3.4.0
> Reporter: Joe McDonnell
> Assignee: Gabor Kaszab
> Priority: Blocker
>
> There are tests that create a normal managed table and upgrade that table to
> a transactional table. For example, in test_acid.py, we run acid.test, which
> has:
> {noformat}
> create table upgraded_table (x int);
> insert into upgraded_table values (1);
> # Upgrade to the table to insert only acid when there are already values in
> it.
> alter table upgraded_table set tblproperties
> ('transactional' = 'true', 'transactional_properties' = 'insert_only');
> insert into upgraded_table values (2);
> insert into upgraded_table values (3);{noformat}
> With HIVE-22158, the create table is now translated to an external table, and
> this now fails with:
> {noformat}
> E ImpalaBeeswaxException: ImpalaBeeswaxException:
> E INNER EXCEPTION: <class 'beeswaxd.ttypes.BeeswaxException'>
> E MESSAGE: ImpalaRuntimeException: Error making 'alter_table' RPC to Hive
> Metastore:
> E CAUSED BY: MetaException: test_acid_basic_5d04240b.upgraded_table cannot
> be declared transactional because it's an external table{noformat}
> If external tables can't be upgraded and all managed tables are now external,
> then this test case is invalid and can be removed. We should make sure that
> this is how it is supposed to work.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
