[
https://issues.apache.org/jira/browse/IMPALA-13759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on IMPALA-13759 started by Csaba Ringhofer.
------------------------------------------------
> Hive ACID table base folder identification procedure is inconsistent with Hive
> ------------------------------------------------------------------------------
>
> Key: IMPALA-13759
> URL: https://issues.apache.org/jira/browse/IMPALA-13759
> Project: IMPALA
> Issue Type: Bug
> Components: Frontend
> Reporter: Peter Rozsa
> Assignee: Csaba Ringhofer
> Priority: Critical
> Labels: ACID
>
> Impala's base folder identification uses a different approach to decide
> whether a base folder is feasible for reading or not in the sense of open
> writeIds. This could cause read inconsistencies with Hive, as Hive reads the
> base folder even if there's an open writeId before a newer base writeId.
> Impala's validation:
> [https://github.com/apache/impala/blob/b8f4034754b691a4790e502af214935486aa3ced/fe/src/main/java/org/apache/impala/util/AcidUtils.java#L261]
> Hive's validation:
> [https://github.com/apache/hive/blob/0759352ddddc793c0e717c460f0e08eb3f14c1e9/ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java#L1774-L1797]
> PR that changed the behavior:
> [https://github.com/apache/hive/commit/8ee3497f87f81fa84ee1023e891dc54087c2cd5e]
>
> Also, it's worth mentioning whether the described situation is considered
> valid in the first place from Hive's side.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
