[ https://issues.apache.org/jira/browse/HIVE-16669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405648#comment-16405648 ]
Eugene Koifman commented on HIVE-16669: --------------------------------------- if compactor runs in a txn, it must write to min_history. Suppose txnid:7 is A, txnid:70 is Compactor. 71 starts and sees 7 as A and 70 as O. If 70 makes 7 empty we need to make sure {{cleanEmptyAbortedTxns()}} doesn't remove 7's entry from TXNS since files produced by 70 are not visible to 71 and if 70 reads older files, it will treat 7's data as committed (if {{cleanEmptyAbortedTxns()}} runs). > Fine tune Compaction to take advantage of Acid 2.0 > -------------------------------------------------- > > Key: HIVE-16669 > URL: https://issues.apache.org/jira/browse/HIVE-16669 > Project: Hive > Issue Type: Bug > Components: Transactions > Reporter: Eugene Koifman > Assignee: Eugene Koifman > Priority: Critical > Attachments: HIVE-16669.wip.patch > > > * There is little point using 2.0 vectorized reader since there is no > operator pipeline in compaction > * If minor compaction just concats delete_delta files together, then the 2 > stage compaction should always ensure that we have a limited number of Orc > readers to do the merging and current OrcRawRecordMerger should be fine > * ... -- This message was sent by Atlassian JIRA (v7.6.3#76005)