[
https://issues.apache.org/jira/browse/IMPALA-8406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tim Armstrong resolved IMPALA-8406.
-----------------------------------
Fix Version/s: Impala 4.0
Assignee: Quanlong Huang
Resolution: Fixed
> Failed REFRESH can partially modify table without bumping version number
> ------------------------------------------------------------------------
>
> Key: IMPALA-8406
> URL: https://issues.apache.org/jira/browse/IMPALA-8406
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Affects Versions: Impala 3.2.0
> Reporter: Todd Lipcon
> Assignee: Quanlong Huang
> Priority: Major
> Fix For: Impala 4.0
>
>
> Currently, various incremental operations in the catalogd modify Table
> objects in place, including REFRESH, which modifies each partition. In this
> case, if one partition fails to refresh (eg due to incorrect partitions or
> some other file access problem), other partitions can still be modified,
> either because they were modified first (in a non-parallel operation) or
> modified in parallel (for REFRESH).
> In this case, the REFRESH operation will throw an Exception back to the user,
> but in fact it has modified the catalog entry. The version number, however,
> is not bumped, which breaks some invariants of the catalog that an object
> doesn't change without changing version numbers.
> This also produces some unexpected behavior such as:
> - SHOW FILES IN t;
> - REFRESH t; -- gets a failure
> - SHOW FILES in t; -- see the same result as originally
> - ALTER TABLE t SET UNCACHED; -- bumps the version number due to unrelated
> operation
> - SHOW FILES IN t; -- the set of files has changed due to the earlier
> partially-complete REFRESH
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
