[
https://issues.apache.org/jira/browse/IMPALA-13120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17860737#comment-17860737
]
ASF subversion and git services commented on IMPALA-13120:
----------------------------------------------------------
Commit ab4e62d3c3dd623a8b5ad896641db07782cbb939 in impala's branch
refs/heads/master from Venu Reddy
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=ab4e62d3c ]
IMPALA-13120: Load failed table without need for manual invalidate
If the metastore is down when the table load is triggered, catalogd
updates a new version of incomplete table with cause as
TableLoadingException. On coordinator/impalad, StmtMetadataLoader
loadTables that has been waiting for table load to complete,
considers the table as loaded. Then, during the analyzer’s table
resolve step, for the incomplete table, TableLoadingException
(i.e., Could not connect to meta store, failed to load metadata for
table and running invalidate metadata for table may resolve this
problem) is thrown.
Henceforth, any query on the table doesn’t trigger the load since
the table is incomplete with TableLoadingException cause. Even though
metastore is UP later at some time, queries continue to throw
the same exception. It is both misleading and also not required to
manually invalidate all such tables.
Note: Incomplete table with cause is considered as loaded.
This patch tries again to load the table that has previously failed
due to metastore connection error(i.e., a recoverable error), when a
query involving the table is fired. Idea is to keep track of table
object present in db that requires load. On successful/failed load,
table object in db is updated. Therefore the tracked table object
reference can be compared to table object in db to detect the
completion of load.
Testing:
- Added end-to-end tests
Change-Id: Ia882fdd865ef716351be7f1eaf203a9fb04c1c15
Reviewed-on: http://gerrit.cloudera.org:8080/21478
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Failed table loads are not tried to load again even though hive metastore is
> UP
> -------------------------------------------------------------------------------
>
> Key: IMPALA-13120
> URL: https://issues.apache.org/jira/browse/IMPALA-13120
> Project: IMPALA
> Issue Type: Improvement
> Reporter: Venugopal Reddy K
> Priority: Major
>
> *Description:*
> If the metastore is down at the time when the table load is triggered,
> catalogd creates a new IncompleteTable instance with
> cause=TableLoadingException and updates catalog with a new version. And on
> coordinator/impalad, StmtMetadataLoader loadTables() that has been waiting
> for table load to complete, considers table as loaded/failed load. Then
> during the analyzer’s table resolve step, if the table is incomplete,
> TableLoadingException is thrown to user.
> Note: IncompleteTable with cause not being null is considered as loaded.
> *Henceforth, queries on the table doesn’t trigger the table load(at
> StmtMetadataLoader) since the table is IncompleteTable with non-null
> cause(i.e.,TableLoadingException). Even though metastore is UP later at some
> time, queries continue to fail with same TableLoadingException:*
> {{CAUSED BY: TableLoadingException: Failed to load metadata for table:
> default.t1. Running 'invalidate metadata default.t1' may resolve this
> problem.}}
> {{CAUSED BY: MetaException: Could not connect to meta store using any of the
> URIs provided. Most recent failure:
> org.apache.thrift.transport.TTransportException: java.net.ConnectException:
> Connection refused (Connection refused)}}
> *At present, explicit Invalidate metadata is the only way to recover table
> from this state.* {*}Queries executed after metastore is up should succeed
> without the need for explicit invalidate metadata{*}{*}{{*}}
> *Steps to Reproduce:*
> # create a table from hive and insert some data into it.
> # Bring down the hive metastore process
> # Run a query on impala that triggers the table load. Query fails with
> TableLoadingException.
> # Bring up the hive metastore process
> # Run the query on impala again. It still fails with same
> TableLoadingException.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
