[
https://issues.apache.org/jira/browse/HIVE-21886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
mahesh kumar behera updated HIVE-21886:
---------------------------------------
Attachment: HIVE-21886.02.patch
> REPL - With table list - Handle rename events during replace policy
> -------------------------------------------------------------------
>
> Key: HIVE-21886
> URL: https://issues.apache.org/jira/browse/HIVE-21886
> Project: Hive
> Issue Type: Sub-task
> Components: repl
> Reporter: mahesh kumar behera
> Assignee: mahesh kumar behera
> Priority: Major
> Labels: DR, Replication, pull-request-available
> Attachments: HIVE-21886.01.patch, HIVE-21886.02.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> If some rename events are found to be dumped and replayed while replace
> policy is getting executed, it needs to take care of the policy inclusion in
> both the policy for each table name.
> 1. Create a list of tables to be bootstrapped.
> 2. During handling of alter table, if the alter type is rename
> 1. If the old table name is present in the list of table to be
> bootstrapped, remove it.
> 2. If the new table name, matches the new policy, add it to the list
> of tables to be bootstrapped.
> 3. During handling of drop table
> 1. if the table is in the list of tables to be bootstrapped, then
> remove it and ignore the event.
> 4. During other event handling
> 1. if the table is there in the list of tables to be bootstrapped,
> then ignore the event.
>
> Rename handling during replace policy
> # Old name not matching old policy – The old table will not be there at the
> target cluster. The table will not be returned by get-all-table.
> ## Old name is not matching new policy
> ### New name not matching old policy
> #### New name not matching new policy
> ***** Ignore the event, no need to do anything.
> #### New name matching new policy
> ***** The table will be returned by get-all-table. Replace policy handler
> will bootstrap this table as its matching new policy and not matching old
> policy.
> ***** All the future events will be ignored as part of check added by
> replace policy handling.
> ***** All the event with old table name will anyways be ignored as the old
> name is not matching the new policy.
> ### New name matching old policy
> #### New name not matching new policy
> ***** As the new name is not matching the new policy, the table need not be
> replicated.
> ***** As the old name is not matching the new policy, the rename events will
> be ignored.
> ***** So nothing to be done for this scenario.
> #### New name matching new policy
> ***** As the new name is matching both old and new policy, replace handler
> will not bootstrap the table.
> ***** Add the table to the list of tables to be bootstrapped.
> ***** Ignore all the events with new name.
> ***** If there is a drop event for the table (with new name), then remove
> the table from the the list of table to be bootstrapped.
> ***** In case of rename event (double rename)
> ****** If the new name satisfies the table pattern, then add the new name to
> the list of tables to be bootstrapped and remove the old name from the list
> of tables to be bootstrapped.
> ****** If the new name does not satisfies then just removed the table name
> from the list of tables to be bootstrapped.
> ## Old name is matching new policy – As per replace policy handler, which
> checks based on old table, the table should be bootstrapped and event should
> be ignored. But rename handler should decide based on new name.The old table
> name will not be returned by get-all-table, so replace handler will not d
> anything for the old table.
> ### New name not matching old policy
> #### New name not matching new policy
> ***** As the old table is not there at target and new name is not matching
> new policy. Ignore the event.
> ***** No need to add the table to the list of tables to be bootstrapped.
> ***** All the subsequent events will be ignored as the new name is not
> matching the new policy.
> #### New name matching new policy
> ***** As the new name is not matching old policy but matching new policy,
> the table will be bootstrapped by replace policy handler. So rename event
> need not add this table to list of table to be bootstrapped.
> ***** All the future events will be ignored by replace policy handler.
> ***** For rename event (double rename)
> ****** If there is a rename, the table (with intermittent new name) will not
> be present and thus replace handler will not bootstrap the table.
> ****** So if the new name (the latest one) is matching the new policy, then
> add it to the list of table to be bootstrapped.
> ****** And If the new name (the latest one) is not matching the new policy,
> then just ignore the event as the intermittent new name would not have added
> to the list of table to be bootstrapped.
> ### New name matching old policy
> #### New name not matching new policy
> ***** Dump the event. The table will be dropped by repl load at the target.
> #### New name matching new policy
> ***** Replace handler will not bootstrap this table as the new name is
> matching both policies.
> ***** As old name is not matching the old policy, the table will not be
> there at target. The rename event should add the new name to the list of
> table to be bootstrapped.
> ***** Subsequent events with new table name should be ignored.
> ***** Drop events should not be ignored as if the table is present during
> bootstrapped, then its a new table and thus should be dropped.
> ***** In case of rename event (double rename)
> ****** If the new name satisfies the table pattern, then add the new name to
> the list of tables to be bootstrapped and remove the old name from the list
> of tables to be bootstrapped.
> ****** If the new name does not satisfies then just removed the table name
> from the list of tables to be bootstrapped.
> # Old name is matching old policy – The old table will be there at the
> target cluster. The table will not be returned by get-all-table. Repl load
> should delete the old table as it is not matching the new policy.
> ## Old name is not matching new policy
> ### New name not matching old policy
> #### New name not matching new policy
> ***** Nothing to be done. Ignore the event.
> #### New name matching new policy
> ***** As the new name is not matching old policy but matching new policy,
> the table will be bootstrapped by replace policy handler. So rename event
> need not add this table to list of table to be bootstrapped.
> ***** All the future events will be ignored by replace policy handler.
> ***** For rename event (double rename)
> ****** If there is a rename, the table (with intermittent new name) will not
> be present and thus replace handler will not bootstrap the table.
> ****** So if the new name (the latest one) is matching the new policy, then
> add it to the list of table to be bootstrapped.
> ****** And If the new name (the latest one) is not matching the new policy,
> then just ignore the event as the intermittent new name would not have added
> to the list of table to be bootstrapped.
> ### New name matching old policy
> #### New name not matching new policy
> ***** Table with new name will be dropped by repl load
> ***** Along with other event, ignore the rename event also.
> #### New name matching new policy
> ***** As the new name is matching both old and new policy, replace handler
> will not bootstrap the table.
> ***** Add the table to the list of tables to be bootstrapped.
> ***** Ignore all the events with new name.
> ***** If there is a drop event for the table (with new name), then remove
> the table from the the list of table to be bootstrapped.
> ***** In case of rename event (double rename)
> ****** If the new name satisfies the table pattern, then add the new name to
> the list of tables to be bootstrapped and remove the old name from the list
> of tables to be bootstrapped.
> ****** If the new name does not satisfies then just removed the table name
> from the list of tables to be bootstrapped.
> ## Old name is matching new policy
> ### New name not matching old policy
> #### New name not matching new policy
> ***** The old table needs to be dropped at target. Ignore this event, as the
> old table is not matching the new policy, it will be dropped by repl load.
> #### New name matching new policy
> ***** Allow the event to dump and replayed at target.
> ***** Allow further events to be handled as usual.
> ***** In case of rename event (double rename)
> ****** If the latest new name is matching the new policy, then keep it as is
> it. Let rename event replayed at target.
> ****** If the latest new name is not matching the new policy, then change
> the rename event to drop event.
> ### New name matching old policy
> #### New name not matching new policy
> ##### Nothing to be done.
> #### New name matching new policy
> ##### Add the table name to the list of tables to be bootstrapped.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
