Todd Lipcon created IMPALA-8406:
-----------------------------------
Summary: Failed REFRESH can partially modify table without bumping
version number
Key: IMPALA-8406
URL: https://issues.apache.org/jira/browse/IMPALA-8406
Project: IMPALA
Issue Type: Bug
Components: Catalog
Affects Versions: Impala 3.2.0
Reporter: Todd Lipcon
Currently, various incremental operations in the catalogd modify Table objects
in place, including REFRESH, which modifies each partition. In this case, if
one partition fails to refresh (eg due to incorrect partitions or some other
file access problem), other partitions can still be modified, either because
they were modified first (in a non-parallel operation) or modified in parallel
(for REFRESH).
In this case, the REFRESH operation will throw an Exception back to the user,
but in fact it has modified the catalog entry. The version number, however, is
not bumped, which breaks some invariants of the catalog that an object doesn't
change without changing version numbers.
This also produces some unexpected behavior such as:
- SHOW FILES IN t;
- REFRESH t; -- gets a failure
- SHOW FILES in t; -- see the same result as originally
- ALTER TABLE t SET UNCACHED; -- bumps the version number due to unrelated
operation
- SHOW FILES IN t; -- the set of files has changed due to the earlier
partially-complete REFRESH
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
