[
https://issues.apache.org/jira/browse/HIVE-21164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16882596#comment-16882596
]
Vaibhav Gumashta commented on HIVE-21164:
-----------------------------------------
[~sankarh] Wanted to get your feedback on the replication tests: the change in
this patch makes an acid insert write directly to the final destination as
compared to writing to a temporary location and promoting it at the end of job
completion. As a result, this also removes dependency of an insert on the
MoveTask operation. Do the above failed replication tests somehow depend on the
old logic (I'm still examining the root cause, but thought I'll get your
opinion as well)?
> ACID: explore how we can avoid a move step during inserts/compaction
> --------------------------------------------------------------------
>
> Key: HIVE-21164
> URL: https://issues.apache.org/jira/browse/HIVE-21164
> Project: Hive
> Issue Type: Bug
> Components: Transactions
> Affects Versions: 3.1.1
> Reporter: Vaibhav Gumashta
> Assignee: Vaibhav Gumashta
> Priority: Major
> Attachments: HIVE-21164.1.patch, HIVE-21164.2.patch,
> HIVE-21164.3.patch, HIVE-21164.4.patch, HIVE-21164.5.patch, HIVE-21164.6.patch
>
>
> Currently, we write compacted data to a temporary location and then move the
> files to a final location, which is an expensive operation on some cloud file
> systems. Since HIVE-20823 is already in, it can control the visibility of
> compacted data for the readers. Therefore, we can perhaps avoid writing data
> to a temporary location and directly write compacted data to the intended
> final path.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
