[
https://issues.apache.org/jira/browse/IMPALA-8406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16814951#comment-16814951
]
Todd Lipcon commented on IMPALA-8406:
-------------------------------------
Another oddity is the message we have today which says "Failed to load file
metadata for 1 paths for table u_todd.test. Table's file metadata could be
partially loaded. Check the Catalog server log for more details."
The "metadata could be partially loaded" sounds like you'll be able to query
the table except for the partitions that had an error, which is sort-of-true
after a REFRESH, but this same error message is currently used on an initial
load of a table, in which case the table won't be queryable at all.
> Failed REFRESH can partially modify table without bumping version number
> ------------------------------------------------------------------------
>
> Key: IMPALA-8406
> URL: https://issues.apache.org/jira/browse/IMPALA-8406
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog
> Affects Versions: Impala 3.2.0
> Reporter: Todd Lipcon
> Priority: Major
>
> Currently, various incremental operations in the catalogd modify Table
> objects in place, including REFRESH, which modifies each partition. In this
> case, if one partition fails to refresh (eg due to incorrect partitions or
> some other file access problem), other partitions can still be modified,
> either because they were modified first (in a non-parallel operation) or
> modified in parallel (for REFRESH).
> In this case, the REFRESH operation will throw an Exception back to the user,
> but in fact it has modified the catalog entry. The version number, however,
> is not bumped, which breaks some invariants of the catalog that an object
> doesn't change without changing version numbers.
> This also produces some unexpected behavior such as:
> - SHOW FILES IN t;
> - REFRESH t; -- gets a failure
> - SHOW FILES in t; -- see the same result as originally
> - ALTER TABLE t SET UNCACHED; -- bumps the version number due to unrelated
> operation
> - SHOW FILES IN t; -- the set of files has changed due to the earlier
> partially-complete REFRESH
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
