[
https://issues.apache.org/jira/browse/HIVE-14535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15593112#comment-15593112
]
Sergey Shelukhin commented on HIVE-14535:
-----------------------------------------
Just to add to [~gopalv] 's response - the "rest" of the MM table support,
namely the commit mechanic in metastore, is what makes it safe to write
directly to the table without moves/copies, in the presence of task
failures/retries/speculative execution, catastrophic query failures (when
there's noone left to clean up), and also considering reads parallel with
in-flight writes.
There has to be some way to tell apart the committed files from uncommitted.
My initial plan was to store file names in metastore for every file that
MoveTask would have moved, but the ID approach is much more efficient for
commit and DB storage requirements.
> add micromanaged tables to Hive (metastore keeps track of the files)
> --------------------------------------------------------------------
>
> Key: HIVE-14535
> URL: https://issues.apache.org/jira/browse/HIVE-14535
> Project: Hive
> Issue Type: Improvement
> Reporter: Sergey Shelukhin
> Assignee: Sergey Shelukhin
>
> Design doc:
> https://docs.google.com/document/d/1b3t1RywfyRb73-cdvkEzJUyOiekWwkMHdiQ-42zCllY
> Feel free to comment.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
