[
https://issues.apache.org/jira/browse/HIVE-23968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HIVE-23968:
----------------------------------
Labels: pull-request-available (was: )
> CTAS with TBLPROPERTIES ('transactional'='false') does not entertain
> translated table location
> ----------------------------------------------------------------------------------------------
>
> Key: HIVE-23968
> URL: https://issues.apache.org/jira/browse/HIVE-23968
> Project: Hive
> Issue Type: Bug
> Components: Hive
> Affects Versions: 4.0.0
> Reporter: Rajkumar Singh
> Assignee: Rajkumar Singh
> Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> HMS translation layer convert the table to external based on the
> transactional property set to false but MoveTask does not entertain the
> translated table location and move the data to the managed table location;
> steps to repro:
> {code:java}
> create table nontxnal TBLPROPERTIES ('transactional'='false') as select *
> from abc;
> {code}
> select query on table return nothing t but the source table has data in it.
> {code:java}
> select * from nontxnal;
> +--------------+
> | nontxnal.id |
> +--------------+
> +--------------+
> {code}
> --show create table
> {code:java}
> CREATE EXTERNAL TABLE `nontxnal`( |
> | `id` int) |
> | ROW FORMAT SERDE |
> | 'org.apache.hadoop.hive.ql.io.orc.OrcSerde' |
> | STORED AS INPUTFORMAT |
> | 'org.apache.hadoop.hive.ql.io.orc.OrcInputFormat' |
> | OUTPUTFORMAT |
> | 'org.apache.hadoop.hive.ql.io.orc.OrcOutputFormat' |
> | LOCATION |
> | 'hdfs://hostname:8020/warehouse/tablespace/external/hive/nontxnal' |
> | TBLPROPERTIES ( |
> | 'TRANSLATED_TO_EXTERNAL'='TRUE', |
> | 'bucketing_version'='2', |
> | 'external.table.purge'='TRUE', |
> | 'transient_lastDdlTime'='1596215634') |
> {code}
> table data is moved to the managed location:
> ```
> dfs -ls -R hdfs://hostname:8020/warehouse/tablespace/managed/hive/nontxnal
> . . . . . . . . . . . . . . . . . . . . . . .> ;
> +----------------------------------------------------+
> | DFS Output |
> +----------------------------------------------------+
> | -rw-rw----+ 3 hive hadoop 201 2020-07-31 17:05
> hdfs://hostname:8020/warehouse/tablespace/managed/hive/nontxnal/000000_0 |
> +----------------------------------------------------+
> ```
> The problem seems to be here
> isExternal evaluates to false since the statement is missing external
> https://github.com/apache/hive/blob/d4bfd2ea1ee797f53227f447749cbc97803cd5dc/ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java#L446
> and location return to the managed location
> https://github.com/apache/hive/blob/d4bfd2ea1ee797f53227f447749cbc97803cd5dc/ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java#L455
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
