[
https://issues.apache.org/jira/browse/HIVE-20435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16738774#comment-16738774
]
Eugene Koifman edited comment on HIVE-20435 at 1/9/19 11:30 PM:
----------------------------------------------------------------
addressed in HIVE-21052
was (Author: ekoifman):
address in HIVE-21052
> Failed Dynamic Partition Insert into insert only table may loose transaction
> metadata
> -------------------------------------------------------------------------------------
>
> Key: HIVE-20435
> URL: https://issues.apache.org/jira/browse/HIVE-20435
> Project: Hive
> Issue Type: Bug
> Components: Transactions
> Affects Versions: 3.0.0
> Reporter: Eugene Koifman
> Assignee: Eugene Koifman
> Priority: Critical
> Fix For: 4.0.0
>
>
> {{TxnHandler.enqueueLockWithRetry()}} has an optimization where it doesn't
> writ to {{TXN_COMPONENTS}} if the write is a dynamic partition insert because
> it expects to write to this table from {{addDynamicPartitions()}}.
> For insert-only, transactional tables, we create the target dir and start
> writing to it before {{addDynamicPartitions()}} is called. So if a txn is
> aborted, we may have a delta dir in the partition but no corresponding entry
> in {{TXN_COMPONENTS}}. This means {{TxnStore.cleanEmptyAbortedTxns()}} may
> clean up {{TXNS}} entry for the aborted transaction before Compactor removes
> this delta dir, at which point it looks like committed data.
> Streaming API V2 with dynamic partition mode also has this problem.
> Full CRUD are currently immune to this since they rely on "move" operation in
> MoveTask but longer term they should follow the same model as insert-only
> tables.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
