[
https://issues.apache.org/jira/browse/HIVE-20581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16677333#comment-16677333
]
Sergey Shelukhin edited comment on HIVE-20581 at 11/6/18 9:58 PM:
------------------------------------------------------------------
MM tables were implemented separately with an explicit goal of avoiding all
move operations. Actually there should be no move operations at all for queries
writing into MM tables (except for compactions). Writing directly to
destination requires some metadata manipulations to make files visible.
ACID didn't have that requirement so it operates similarly to regular Hive
tables.
I think this JIRA is to extend the MM functionality of not moving stuff to ACID
tables.
was (Author: sershe):
MM tables were implemented separately with an explicit goal of avoiding all
move operations. Actually there should be no move operations at all for queries
involving MM tables (except for compactions). Writing directly to destination
requires some metadata manipulations to make files visible.
ACID didn't have that requirement so it operates similarly to regular Hive
tables.
I think this JIRA is to extend the MM functionality of not moving stuff to ACID
tables.
> Eliminate rename() from full CRUD transactional tables
> ------------------------------------------------------
>
> Key: HIVE-20581
> URL: https://issues.apache.org/jira/browse/HIVE-20581
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
> Reporter: Eugene Koifman
> Assignee: Emily lozano d1751740 551913586
> Priority: Major
>
> The {{MoveTask}} in a query writing to full CRUD transactional table still
> performs a {{FileSystem.rename()}}. Full CRUD should follow the insert-only
> transactional table implementation and write directly to delta_x_x in the
> partition dir. If the txn fails, this delta will be marked aborted and will
> not be read.
> There are several places that rely on this rename. For example, support for
> {{Insert ... select ... Union All ... Select }} which creates multiple dirs,
> 1 for each leg of the union.
> Others?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
