[
https://issues.apache.org/jira/browse/IMPALA-9071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16960202#comment-16960202
]
ASF subversion and git services commented on IMPALA-9071:
---------------------------------------------------------
Commit 0f70ade0d78a9eb3afafdfd1a3e36cc8a5563cb4 in impala's branch
refs/heads/master from stiga-huang
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=0f70ade ]
IMPALA-9071: Fix wrong table path of transaction table created by CTAS
The previous patch of IMPALA-9071 assumes that all tables created by
CTAS statement are non transactional table. This is wrong since CTAS
statement can also specify tblproperties so can create transactional
table.
This patch fixs the hard coded external checking. Instead, we judge on
whether the table is transactional. If not, it will be translated to
external table by HMS.
Tests:
- Add coverage for creating transactional tables by CTAS.
Change-Id: I4b585216e33e4f7962b19ae2351165288691eaf2
Reviewed-on: http://gerrit.cloudera.org:8080/14546
Reviewed-by: Joe McDonnell <[email protected]>
Reviewed-by: Zoltan Borok-Nagy <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> When metastore.warehouse.dir != metastore.warehouse.external.dir, Impala
> writes to the wrong location for external tables
> -------------------------------------------------------------------------------------------------------------------------
>
> Key: IMPALA-9071
> URL: https://issues.apache.org/jira/browse/IMPALA-9071
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Affects Versions: Impala 3.4.0
> Reporter: Joe McDonnell
> Assignee: Quanlong Huang
> Priority: Blocker
> Labels: broken-build
>
> Hive introduced a translation layer that can convert a normal table to an
> external table. When doing so without a specified location, the translated
> external table uses metastore.warehouse.external.dir as the location rather
> than metastore.warehouse.dir. Impala does not know about this distinction, so
> it writes to the location it thinks the table should be (under
> metastore.warehouse.dir). This means I can do the following:
> {noformat}
> [localhost:21000] joetest> select count(*) from functional.alltypes;
> Query: select count(*) from functional.alltypes
> Query submitted at: 2019-10-19 13:08:24 (Coordinator:
> http://joemcdonnell:25000)
> Query progress can be monitored at:
> http://joemcdonnell:25000/query_plan?query_id=68434b05e2badd50:a18a2e3000000000
> +----------+
> | count(*) |
> +----------+
> | 7300 |
> +----------+
> Fetched 1 row(s) in 0.14s
> [localhost:21000] joetest> create table testtable as select * from
> functional.alltypes;
> Query: create table testtable as select * from functional.alltypes
> Query submitted at: 2019-10-19 13:08:36 (Coordinator:
> http://joemcdonnell:25000)
> Query progress can be monitored at:
> http://joemcdonnell:25000/query_plan?query_id=794b92fb68f36ab0:910d036400000000
> +----------------------+
> | summary |
> +----------------------+
> | Inserted 7300 row(s) |
> +----------------------+
> Fetched 1 row(s) in 0.50s
> [localhost:21000] joetest> select count(*) from testtable;
> Query: select count(*) from testtable
> Query submitted at: 2019-10-19 13:08:43 (Coordinator:
> http://joemcdonnell:25000)
> Query progress can be monitored at:
> http://joemcdonnell:25000/query_plan?query_id=66423abf016e65af:8362460900000000
> +----------+
> | count(*) |
> +----------+
> | 0 |
> +----------+
> Fetched 1 row(s) in 0.13s
> {noformat}
> We inserted 7300 rows, but we can't select them back because they were
> written to the wrong location.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
