[ https://issues.apache.org/jira/browse/HIVE-18589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350984#comment-16350984 ]
Gopal V commented on HIVE-18589: -------------------------------- LGTM - +1 Not entirely happy with the metafile idea, but this can be revisited as HIVE-18320 makes more progress. > java.io.IOException: Not enough history available > ------------------------------------------------- > > Key: HIVE-18589 > URL: https://issues.apache.org/jira/browse/HIVE-18589 > Project: Hive > Issue Type: Bug > Components: Transactions > Affects Versions: 3.0.0 > Reporter: Eugene Koifman > Assignee: Eugene Koifman > Priority: Critical > Attachments: HIVE-18589.01.patch, HIVE-18589.02.patch > > > txnid:17 starts reading T2/P1 > txnid:20 does insert overwrite T1/P1, creates base_20, commits. txnid:17 > still running > txnid:21 stars reading T1/P1. It's ValidTxnList will txnid:17 as open. > before Insert overwrite was supported, only the compactor could produce > base_20 by running major compaction. Major compaction erases history and so > a reader with txnid:17 open, can't use base_20. > Normally, the Cleaner is smart enough to not clean pre-compaction files if > it's possible that there is a reader that requires them. There is a safety > check that creates "Not enough history.." error if it finds that the current > reader can't properly execute based on the files available. > > with the introduction of IOW on acid tables, there is another way to produce > a base. The difference is that here, the base has no history by definition > and so the same check is not needed but is triggered in the scenario above. > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)