Sourabh Goyal created IMPALA-10926:
--------------------------------------
Summary: Sync db/table to latest event id for ddl operations in
catalog megastore server
Key: IMPALA-10926
URL: https://issues.apache.org/jira/browse/IMPALA-10926
Project: IMPALA
Issue Type: Task
Reporter: Sourabh Goyal
Assignee: Sourabh Goyal
Whenever DDL operations like add/remove partitions, alter table etc are
performed from catalog HMS endpoints, we will do the following:
# Acquire lock on db/table
# Perform HMS operation
# For db/table in step 1, fetch all events from HMS
# Apply those events on the db/table in catalogd cache
# Set the last processed event id for db/table
In addition to the above, the following needs to be handled as well:
# Event processor should skip processing an event on a db/table if the
db/table is already synced till that event id.
# If last synced event id in table/db is less than the event being processed
by event processor, it should set the last synced db/table event id after
successfully processing the event.
# Full table refresh should set the last event processed.
cc - [~kishendas] [~vihangk1]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
