[
https://issues.apache.org/jira/browse/HIVE-21164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17059963#comment-17059963
]
Gopal Vijayaraghavan edited comment on HIVE-21164 at 3/16/20, 5:30 AM:
-----------------------------------------------------------------------
bq. it deals with inserting data to existing tables and contains no code
specific to execution engines.
Actually, that might not be true because of the file commit handling code which
is specific to each execution engine.
You can try replacing the "insert overwrite" with a simple "insert" to explore
that theory.
was (Author: gopalv):
bq. it deals with inserting data to existing tables and contains no code
specific to execution engines.
Actually, that might not be true because of the file commit handling code which
is specific to each execution engine.
> ACID: explore how we can avoid a move step during inserts/compaction
> --------------------------------------------------------------------
>
> Key: HIVE-21164
> URL: https://issues.apache.org/jira/browse/HIVE-21164
> Project: Hive
> Issue Type: Bug
> Components: Transactions
> Affects Versions: 3.1.1
> Reporter: Vaibhav Gumashta
> Assignee: Marta Kuczora
> Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21164.1.patch, HIVE-21164.10.patch,
> HIVE-21164.11.patch, HIVE-21164.11.patch, HIVE-21164.12.patch,
> HIVE-21164.13.patch, HIVE-21164.14.patch, HIVE-21164.14.patch,
> HIVE-21164.15.patch, HIVE-21164.16.patch, HIVE-21164.17.patch,
> HIVE-21164.18.patch, HIVE-21164.19.patch, HIVE-21164.2.patch,
> HIVE-21164.20.patch, HIVE-21164.21.patch, HIVE-21164.22.patch,
> HIVE-21164.3.patch, HIVE-21164.4.patch, HIVE-21164.5.patch,
> HIVE-21164.6.patch, HIVE-21164.7.patch, HIVE-21164.8.patch, HIVE-21164.9.patch
>
>
> Currently, we write compacted data to a temporary location and then move the
> files to a final location, which is an expensive operation on some cloud file
> systems. Since HIVE-20823 is already in, it can control the visibility of
> compacted data for the readers. Therefore, we can perhaps avoid writing data
> to a temporary location and directly write compacted data to the intended
> final path.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
