[
https://issues.apache.org/jira/browse/IMPALA-8434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16842424#comment-16842424
]
Xiaomeng Zhang commented on IMPALA-8434:
----------------------------------------
The root cause is in
[https://github.com/apache/impala/blob/master/fe/src/main/java/org/apache/impala/catalog/ImpaladCatalog.java#L395]
{code:java}
private void addDb(TDatabase thriftDb, long catalogVersion) {
Db existingDb = getDb(thriftDb.getDb_name());
if (existingDb == null ||
existingDb.getCatalogVersion() < catalogVersion) {
Db newDb = Db.fromTDatabase(thriftDb);
newDb.setCatalogVersion(catalogVersion);
addDb(newDb);
if (existingDb != null) {
CatalogObjectVersionSet.INSTANCE.updateVersions(
existingDb.getCatalogVersion(), catalogVersion);
CatalogObjectVersionSet.INSTANCE.removeAll(existingDb.getTables());
CatalogObjectVersionSet.INSTANCE.removeAll(
existingDb.getFunctions(null, new PatternMatcher()));
} else {
CatalogObjectVersionSet.INSTANCE.addVersion(catalogVersion);
}
}
}
{code}
AlterDb will trigger method addDb to replace existing Db with thriftDb. But
thriftDb doesn't have table and function inside. So after alterDb, impalad lose
table and function info, require "invalidate metadata".
I tried to fix it by not removing table & function version, adding table &
function back into new created Db. But this change cause "invalidate metadata"
hanging, as "invalidate metadata" also triggers method addDb, to replace
current Db to incomplete one. And it has to remove old table & function version
to let min catalog version increase to catalog's latest update version
[https://github.com/apache/impala/blob/master/be/src/service/impala-server.cc#L1691]
So the best way maybe add table& function only for alterDb command, but from
the passing info, I can't find any parameter can be used as alterDb type
identification. Any help?
> Alter Db leads to functions missing unless run "refresh functions"
> -------------------------------------------------------------------
>
> Key: IMPALA-8434
> URL: https://issues.apache.org/jira/browse/IMPALA-8434
> Project: IMPALA
> Issue Type: Bug
> Reporter: Xiaomeng Zhang
> Assignee: Xiaomeng Zhang
> Priority: Critical
>
> I was testing on master branch. In a database with java and native functions.
> Run queries below, all functions are missing after alter db until run
> "refresh functions" in db.
> {code:java}
> [localhost:21000] xm> show functions;
> Query: show functions
> +-------------+------------------------+-------------+---------------+
> | return type | signature | binary type | is persistent |
> +-------------+------------------------+-------------+---------------+
> | STRING | add10impala(STRING) | JAVA | true |
> | STRING | add10udf(STRING) | JAVA | true |
> | INT | add2(INT, INT) | NATIVE | true |
> | INT | add2xm(INT, INT) | NATIVE | true |
> | INT | addtwomaster(INT, INT) | NATIVE | true |
> +-------------+------------------------+-------------+---------------+
> Fetched 5 row(s) in 0.01s
> [localhost:21000] xm> alter database xm set owner user impala218;
> Query: alter database xm set owner user impala218
> +-------------------+
> | summary |
> +-------------------+
> | Updated database. |
> +-------------------+
> Fetched 1 row(s) in 0.59s
> [localhost:21000] xm> show functions;
> Query: show functions
> Fetched 0 row(s) in 0.01s
> [localhost:21000] xm> refresh functions xm;
> Query: refresh functions xm
> Query submitted at: 2019-04-18 14:19:00 (Coordinator:
> http://xiaomeng-OptiPlex-9020:25000)
> Query progress can be monitored at:
> http://xiaomeng-OptiPlex-9020:25000/query_plan?query_id=fa40cdffde223550:df2a6cc000000000
> Fetched 0 row(s) in 0.08s
> [localhost:21000] xm> show functions;
> Query: show functions
> +-------------+------------------------+-------------+---------------+
> | return type | signature | binary type | is persistent |
> +-------------+------------------------+-------------+---------------+
> | STRING | add10impala(STRING) | JAVA | true |
> | STRING | add10udf(STRING) | JAVA | true |
> | INT | add2(INT, INT) | NATIVE | true |
> | INT | add2xm(INT, INT) | NATIVE | true |
> | INT | addtwomaster(INT, INT) | NATIVE | true |
> +-------------+------------------------+-------------+---------------+
> Fetched 5 row(s) in 0.00s
> {code}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
