[
https://issues.apache.org/jira/browse/IMPALA-10663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17504772#comment-17504772
]
Quanlong Huang commented on IMPALA-10663:
-----------------------------------------
Hi [~boroknagyz] , is the issue resolved now?
> Coordinator might observe stale metadata in local catalog mode
> --------------------------------------------------------------
>
> Key: IMPALA-10663
> URL: https://issues.apache.org/jira/browse/IMPALA-10663
> Project: IMPALA
> Issue Type: Bug
> Components: Catalog, Frontend
> Reporter: Zoltán Borók-Nagy
> Priority: Major
>
> Coordinator might observe stale metadata in local catalog mode, even if the
> DDL operation was executed on the same coordinator.
> The issue is reproducible in the dev environment using local catalog mode:
> {noformat}
> bin/start-impala-cluster.py --impalad_args --use_local_catalog=true
> --catalogd_args --catalog_topic_mode=minimal --catalogd_args
> --hms_event_polling_interval_s=1
> {noformat}
> {noformat}
> CREATE TABLE doub (d double);
> {noformat}
> Renaming the table back and forth and querying from it reproduces the issue:
> bin/impala-shell.sh -q "alter table doub rename to doub_tmp; select * from
> doub_tmp; alter table doub_tmp rename to doub; select * from doub; alter
> table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp
> rename to doub; select * from doub; alter table doub rename to doub_tmp;
> select * from doub_tmp;alter table doub_tmp rename to doub; select * from
> doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table
> doub_tmp rename to doub; select * from doub; alter table doub rename to
> doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select
> * from doub; alter table doub rename to doub_tmp; select * from
> doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table
> doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename
> to doub; select * from doub; alter table doub rename to doub_tmp; select *
> from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter
> table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp
> rename to doub; select * from doub; alter table doub rename to doub_tmp;
> select * from doub_tmp;alter table doub_tmp rename to doub; select * from
> doub; alter table doub rename to doub_tmp; select * from doub_tmp;alter table
> doub_tmp rename to doub; select * from doub; alter table doub rename to
> doub_tmp; select * from doub_tmp;alter table doub_tmp rename to doub; select
> * from doub; alter table doub rename to doub_tmp; select * from
> doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter table
> doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp rename
> to doub; select * from doub; alter table doub rename to doub_tmp; select *
> from doub_tmp;alter table doub_tmp rename to doub; select * from doub; alter
> table doub rename to doub_tmp; select * from doub_tmp;alter table doub_tmp
> rename to doub; select * from doub; alter table doub rename to doub_tmp;
> select * from doub_tmp;alter table doub_tmp rename to doub; select * from
> doub;"
> {noformat}
> ...
> ... then, after a while ...
> ...
> Query: alter table doub rename to doub_tmp
> +--------------------------+
> | summary |
> +--------------------------+
> | Renaming was successful. |
> +--------------------------+
> Fetched 1 row(s) in 0.02s
> Query: select * from doub_tmp
> Query submitted at: 2021-04-20 14:49:52 (Coordinator: http://localhost:25000)
> ERROR: AnalysisException: Could not resolve table reference: 'doub_tmp'
> Could not execute command: select * from doub_tmp
> {noformat}
> Then, if I execute the same SELECT a bit later:
> {noformat}
> $ bin/impala-shell.sh -q "select * from doub_tmp"
> Fetched 0 row(s) in 0.12s
> {noformat}
> I think if the DDL operation succeeds, then the following operations on the
> same coordinator should see its effect.
> The current behavior makes tests flaky. The workaround is to set
> sync_ddl=true, but it makes these operations very slow.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
