[
https://issues.apache.org/jira/browse/HIVE-8368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alan Gates updated HIVE-8368:
-----------------------------
Attachment: HIVE-8368.patch
This patch adds a new isDelete method to AcidInputFormat.RawReader. The
compactor then calls isDelete when getting records from RawReader and then
throws them out if it is doing a major compaction.
A note on adding isDelete. I think it would be better to change RawReader.next
to not return deleted records when the option compact is set to true. Two
things prevented me from doing this. One, while I think no one ever needs
deletes in the compact case I might be wrong. Two, the loop that does the
merge and returns the next record in RawReader.next is complicated code, and
changing it looked very error prone.
> compactor is improperly writing delete records in base file
> -----------------------------------------------------------
>
> Key: HIVE-8368
> URL: https://issues.apache.org/jira/browse/HIVE-8368
> Project: Hive
> Issue Type: Bug
> Components: Transactions
> Affects Versions: 0.14.0
> Reporter: Alan Gates
> Assignee: Alan Gates
> Priority: Critical
> Fix For: 0.14.0
>
> Attachments: HIVE-8368.patch
>
>
> When the compactor reads records from the base and deltas, it is not properly
> dropping delete records. This leads to oversized base files, and possibly to
> wrong query results.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
