[
https://issues.apache.org/jira/browse/IMPALA-9913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17149588#comment-17149588
]
ASF subversion and git services commented on IMPALA-9913:
---------------------------------------------------------
Commit e4a3e9a35eeb3f2791f4f80e76ca35f9a048f13b in impala's branch
refs/heads/master from Vihang Karajgaonkar
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=e4a3e9a ]
IMPALA-9913: Use table id to match the table in drop table event
This patch modifies the matching logic of the table while
processing a DROP_TABLE event. The EventsProcessor currently relies
on createTime of the table to determine if the table in the catalog
matches with the table in the event. It is possible that if there
is a time-skew between the different HMS instances and a same table
is dropped and recreated, the creation time of the table which
is present in the catalog is less than or equals to the creationTime
of a table which has been dropped. In such case, there is a false
negative during the matching and the DROP_TABLE event is skipped.
HIVE-20556 exposes a id field of the table which can be used to
uniquely identify a table. A table which is dropped and recreated
with the same name has a different id. This patch makes use of this
id to match the table in catalog with the table from the HMS event.
Note that the DROP_DATABASE event continues to use createTime for
matching. We should use a database id for it when it is available.
I have created HIVE-23785 for tracking that on the HMS side.
Testing:
1. Ran Events processor tests.
Change-Id: Id6a80bbf5757e46318af1b57911fc127d7dd1f01
Reviewed-on: http://gerrit.cloudera.org:8080/16129
Reviewed-by: Anurag Mantripragada <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Use table id to detect uniqueness of table for drop table event
> ---------------------------------------------------------------
>
> Key: IMPALA-9913
> URL: https://issues.apache.org/jira/browse/IMPALA-9913
> Project: IMPALA
> Issue Type: Improvement
> Reporter: Vihang Karajgaonkar
> Assignee: Vihang Karajgaonkar
> Priority: Minor
>
> When a DROP_TABLE event is received by the EventsProcessor, it needs to make
> a determination of whether the table in the cache is same as the table in the
> event. Without a unique id from the HMS it uses {{createTime}} field of the
> HMS table object to determine if the table matches with the table in the
> event.
> Since then HMS API was enhanced to include an {{id}} field which is
> guaranteed to unique accross all the tables (even the past deleted tables).
> Instead of the using createTime we should use id field to do the matching.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
