[
https://issues.apache.org/jira/browse/HIVE-25521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Panagiotis Garefalakis updated HIVE-25521:
------------------------------------------
Fix Version/s: 4.0.0
> Data corruption when concatenating files with different compressions in same
> table/partition
> --------------------------------------------------------------------------------------------
>
> Key: HIVE-25521
> URL: https://issues.apache.org/jira/browse/HIVE-25521
> Project: Hive
> Issue Type: Bug
> Reporter: Harish JP
> Assignee: Harish JP
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Currently if files of different compressions are in same directory then
> concatenate can fail and cause data corruption. This happens because file can
> be moved by one task as incompatible file and the other tasks will fail after
> this.
>
> This issue is addressed in this Jira by only processing a file in one task
> where offset 0 is process and ignoring the the file in all other tasks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
