Eugene Koifman resolved HIVE-20435.
       Resolution: Won't Fix
    Fix Version/s: 4.0.0
     Release Note: n/a

address in HIVE-21052

> Failed Dynamic Partition Insert into insert only table may loose transaction 
> metadata
> -------------------------------------------------------------------------------------
>                 Key: HIVE-20435
>                 URL: https://issues.apache.org/jira/browse/HIVE-20435
>             Project: Hive
>          Issue Type: Bug
>          Components: Transactions
>    Affects Versions: 3.0.0
>            Reporter: Eugene Koifman
>            Assignee: Eugene Koifman
>            Priority: Critical
>             Fix For: 4.0.0
> {{TxnHandler.enqueueLockWithRetry()}} has an optimization where it doesn't 
> writ to {{TXN_COMPONENTS}} if the write is a dynamic partition insert because 
> it expects to write to this table from {{addDynamicPartitions()}}.
> For insert-only, transactional tables, we create the target dir and start 
> writing to it before {{addDynamicPartitions()}} is called. So if a txn is 
> aborted, we may have a delta dir in the partition but no corresponding entry 
> in {{TXN_COMPONENTS}}. This means {{TxnStore.cleanEmptyAbortedTxns()}} may 
> clean up {{TXNS}} entry for the aborted transaction before Compactor removes 
> this delta dir, at which point it looks like committed data.
> Streaming API V2 with dynamic partition mode also has this problem.
> Full CRUD are currently immune to this since they rely on "move" operation in 
> MoveTask but longer term they should follow the same model as insert-only 
> tables.

This message was sent by Atlassian JIRA

Reply via email to