[jira] [Updated] (HIVE-17183) Disable rename operations during bootstrap dump
[ https://issues.apache.org/jira/browse/HIVE-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-17183: - Resolution: Fixed Status: Resolved (was: Patch Available) Patch committed to master. Thanks for the patch [~sankarh] and for the review [~anishek]! > Disable rename operations during bootstrap dump > --- > > Key: HIVE-17183 > URL: https://issues.apache.org/jira/browse/HIVE-17183 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Fix For: 3.0.0 > > Attachments: HIVE-17183.01.patch, HIVE-17183.02.patch, > HIVE-17183.03.patch, HIVE-17183.04.patch, HIVE-17183.05.patch, > HIVE-17183.06.patch > > > Currently, bootstrap dump shall lead to data loss when any rename happens > while dump in progress. > *Scenario:* > - Fetch table names (T1 and T2) > - Dump table T1 > - Rename table T2 to T3 generates RENAME event > - Dump table T2 is noop as table doesn’t exist. > - In target after load, it only have T1. > - Apply RENAME event will fail as T2 doesn’t exist in target. > This feature can be supported in next phase development as it need proper > design to keep track of renamed tables/partitions. > So, for time being, we shall disable rename operations when bootstrap dump in > progress to avoid any inconsistent state. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17183) Disable rename operations during bootstrap dump
[ https://issues.apache.org/jira/browse/HIVE-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-17183: Status: Patch Available (was: Open) > Disable rename operations during bootstrap dump > --- > > Key: HIVE-17183 > URL: https://issues.apache.org/jira/browse/HIVE-17183 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Fix For: 3.0.0 > > Attachments: HIVE-17183.01.patch, HIVE-17183.02.patch, > HIVE-17183.03.patch, HIVE-17183.04.patch, HIVE-17183.05.patch, > HIVE-17183.06.patch > > > Currently, bootstrap dump shall lead to data loss when any rename happens > while dump in progress. > *Scenario:* > - Fetch table names (T1 and T2) > - Dump table T1 > - Rename table T2 to T3 generates RENAME event > - Dump table T2 is noop as table doesn’t exist. > - In target after load, it only have T1. > - Apply RENAME event will fail as T2 doesn’t exist in target. > This feature can be supported in next phase development as it need proper > design to keep track of renamed tables/partitions. > So, for time being, we shall disable rename operations when bootstrap dump in > progress to avoid any inconsistent state. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17183) Disable rename operations during bootstrap dump
[ https://issues.apache.org/jira/browse/HIVE-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-17183: Attachment: HIVE-17183.06.patch Added 06.patch with fixes for Anishek's comments. Request [~anishek] to review the new patch. > Disable rename operations during bootstrap dump > --- > > Key: HIVE-17183 > URL: https://issues.apache.org/jira/browse/HIVE-17183 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Fix For: 3.0.0 > > Attachments: HIVE-17183.01.patch, HIVE-17183.02.patch, > HIVE-17183.03.patch, HIVE-17183.04.patch, HIVE-17183.05.patch, > HIVE-17183.06.patch > > > Currently, bootstrap dump shall lead to data loss when any rename happens > while dump in progress. > *Scenario:* > - Fetch table names (T1 and T2) > - Dump table T1 > - Rename table T2 to T3 generates RENAME event > - Dump table T2 is noop as table doesn’t exist. > - In target after load, it only have T1. > - Apply RENAME event will fail as T2 doesn’t exist in target. > This feature can be supported in next phase development as it need proper > design to keep track of renamed tables/partitions. > So, for time being, we shall disable rename operations when bootstrap dump in > progress to avoid any inconsistent state. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17183) Disable rename operations during bootstrap dump
[ https://issues.apache.org/jira/browse/HIVE-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-17183: Status: Open (was: Patch Available) > Disable rename operations during bootstrap dump > --- > > Key: HIVE-17183 > URL: https://issues.apache.org/jira/browse/HIVE-17183 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Fix For: 3.0.0 > > Attachments: HIVE-17183.01.patch, HIVE-17183.02.patch, > HIVE-17183.03.patch, HIVE-17183.04.patch, HIVE-17183.05.patch > > > Currently, bootstrap dump shall lead to data loss when any rename happens > while dump in progress. > *Scenario:* > - Fetch table names (T1 and T2) > - Dump table T1 > - Rename table T2 to T3 generates RENAME event > - Dump table T2 is noop as table doesn’t exist. > - In target after load, it only have T1. > - Apply RENAME event will fail as T2 doesn’t exist in target. > This feature can be supported in next phase development as it need proper > design to keep track of renamed tables/partitions. > So, for time being, we shall disable rename operations when bootstrap dump in > progress to avoid any inconsistent state. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17183) Disable rename operations during bootstrap dump
[ https://issues.apache.org/jira/browse/HIVE-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-17183: Status: Patch Available (was: Open) > Disable rename operations during bootstrap dump > --- > > Key: HIVE-17183 > URL: https://issues.apache.org/jira/browse/HIVE-17183 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Fix For: 3.0.0 > > Attachments: HIVE-17183.01.patch, HIVE-17183.02.patch, > HIVE-17183.03.patch, HIVE-17183.04.patch, HIVE-17183.05.patch > > > Currently, bootstrap dump shall lead to data loss when any rename happens > while dump in progress. > *Scenario:* > - Fetch table names (T1 and T2) > - Dump table T1 > - Rename table T2 to T3 generates RENAME event > - Dump table T2 is noop as table doesn’t exist. > - In target after load, it only have T1. > - Apply RENAME event will fail as T2 doesn’t exist in target. > This feature can be supported in next phase development as it need proper > design to keep track of renamed tables/partitions. > So, for time being, we shall disable rename operations when bootstrap dump in > progress to avoid any inconsistent state. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17183) Disable rename operations during bootstrap dump
[ https://issues.apache.org/jira/browse/HIVE-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-17183: Status: Open (was: Patch Available) > Disable rename operations during bootstrap dump > --- > > Key: HIVE-17183 > URL: https://issues.apache.org/jira/browse/HIVE-17183 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Fix For: 3.0.0 > > Attachments: HIVE-17183.01.patch, HIVE-17183.02.patch, > HIVE-17183.03.patch, HIVE-17183.04.patch, HIVE-17183.05.patch > > > Currently, bootstrap dump shall lead to data loss when any rename happens > while dump in progress. > *Scenario:* > - Fetch table names (T1 and T2) > - Dump table T1 > - Rename table T2 to T3 generates RENAME event > - Dump table T2 is noop as table doesn’t exist. > - In target after load, it only have T1. > - Apply RENAME event will fail as T2 doesn’t exist in target. > This feature can be supported in next phase development as it need proper > design to keep track of renamed tables/partitions. > So, for time being, we shall disable rename operations when bootstrap dump in > progress to avoid any inconsistent state. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17183) Disable rename operations during bootstrap dump
[ https://issues.apache.org/jira/browse/HIVE-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-17183: Attachment: (was: HIVE-17183.05.patch) > Disable rename operations during bootstrap dump > --- > > Key: HIVE-17183 > URL: https://issues.apache.org/jira/browse/HIVE-17183 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Fix For: 3.0.0 > > Attachments: HIVE-17183.01.patch, HIVE-17183.02.patch, > HIVE-17183.03.patch, HIVE-17183.04.patch, HIVE-17183.05.patch > > > Currently, bootstrap dump shall lead to data loss when any rename happens > while dump in progress. > *Scenario:* > - Fetch table names (T1 and T2) > - Dump table T1 > - Rename table T2 to T3 generates RENAME event > - Dump table T2 is noop as table doesn’t exist. > - In target after load, it only have T1. > - Apply RENAME event will fail as T2 doesn’t exist in target. > This feature can be supported in next phase development as it need proper > design to keep track of renamed tables/partitions. > So, for time being, we shall disable rename operations when bootstrap dump in > progress to avoid any inconsistent state. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17183) Disable rename operations during bootstrap dump
[ https://issues.apache.org/jira/browse/HIVE-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-17183: Attachment: HIVE-17183.05.patch > Disable rename operations during bootstrap dump > --- > > Key: HIVE-17183 > URL: https://issues.apache.org/jira/browse/HIVE-17183 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Fix For: 3.0.0 > > Attachments: HIVE-17183.01.patch, HIVE-17183.02.patch, > HIVE-17183.03.patch, HIVE-17183.04.patch, HIVE-17183.05.patch > > > Currently, bootstrap dump shall lead to data loss when any rename happens > while dump in progress. > *Scenario:* > - Fetch table names (T1 and T2) > - Dump table T1 > - Rename table T2 to T3 generates RENAME event > - Dump table T2 is noop as table doesn’t exist. > - In target after load, it only have T1. > - Apply RENAME event will fail as T2 doesn’t exist in target. > This feature can be supported in next phase development as it need proper > design to keep track of renamed tables/partitions. > So, for time being, we shall disable rename operations when bootstrap dump in > progress to avoid any inconsistent state. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17183) Disable rename operations during bootstrap dump
[ https://issues.apache.org/jira/browse/HIVE-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-17183: Status: Patch Available (was: Open) > Disable rename operations during bootstrap dump > --- > > Key: HIVE-17183 > URL: https://issues.apache.org/jira/browse/HIVE-17183 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Fix For: 3.0.0 > > Attachments: HIVE-17183.01.patch, HIVE-17183.02.patch, > HIVE-17183.03.patch, HIVE-17183.04.patch, HIVE-17183.05.patch > > > Currently, bootstrap dump shall lead to data loss when any rename happens > while dump in progress. > *Scenario:* > - Fetch table names (T1 and T2) > - Dump table T1 > - Rename table T2 to T3 generates RENAME event > - Dump table T2 is noop as table doesn’t exist. > - In target after load, it only have T1. > - Apply RENAME event will fail as T2 doesn’t exist in target. > This feature can be supported in next phase development as it need proper > design to keep track of renamed tables/partitions. > So, for time being, we shall disable rename operations when bootstrap dump in > progress to avoid any inconsistent state. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17183) Disable rename operations during bootstrap dump
[ https://issues.apache.org/jira/browse/HIVE-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-17183: Attachment: (was: HIVE-17183.05.patch) > Disable rename operations during bootstrap dump > --- > > Key: HIVE-17183 > URL: https://issues.apache.org/jira/browse/HIVE-17183 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Fix For: 3.0.0 > > Attachments: HIVE-17183.01.patch, HIVE-17183.02.patch, > HIVE-17183.03.patch, HIVE-17183.04.patch, HIVE-17183.05.patch > > > Currently, bootstrap dump shall lead to data loss when any rename happens > while dump in progress. > *Scenario:* > - Fetch table names (T1 and T2) > - Dump table T1 > - Rename table T2 to T3 generates RENAME event > - Dump table T2 is noop as table doesn’t exist. > - In target after load, it only have T1. > - Apply RENAME event will fail as T2 doesn’t exist in target. > This feature can be supported in next phase development as it need proper > design to keep track of renamed tables/partitions. > So, for time being, we shall disable rename operations when bootstrap dump in > progress to avoid any inconsistent state. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17183) Disable rename operations during bootstrap dump
[ https://issues.apache.org/jira/browse/HIVE-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-17183: Attachment: HIVE-17183.05.patch > Disable rename operations during bootstrap dump > --- > > Key: HIVE-17183 > URL: https://issues.apache.org/jira/browse/HIVE-17183 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Fix For: 3.0.0 > > Attachments: HIVE-17183.01.patch, HIVE-17183.02.patch, > HIVE-17183.03.patch, HIVE-17183.04.patch, HIVE-17183.05.patch > > > Currently, bootstrap dump shall lead to data loss when any rename happens > while dump in progress. > *Scenario:* > - Fetch table names (T1 and T2) > - Dump table T1 > - Rename table T2 to T3 generates RENAME event > - Dump table T2 is noop as table doesn’t exist. > - In target after load, it only have T1. > - Apply RENAME event will fail as T2 doesn’t exist in target. > This feature can be supported in next phase development as it need proper > design to keep track of renamed tables/partitions. > So, for time being, we shall disable rename operations when bootstrap dump in > progress to avoid any inconsistent state. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17183) Disable rename operations during bootstrap dump
[ https://issues.apache.org/jira/browse/HIVE-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-17183: Attachment: HIVE-17183.05.patch Added 05.patch after rebasing with master. > Disable rename operations during bootstrap dump > --- > > Key: HIVE-17183 > URL: https://issues.apache.org/jira/browse/HIVE-17183 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Fix For: 3.0.0 > > Attachments: HIVE-17183.01.patch, HIVE-17183.02.patch, > HIVE-17183.03.patch, HIVE-17183.04.patch, HIVE-17183.05.patch > > > Currently, bootstrap dump shall lead to data loss when any rename happens > while dump in progress. > *Scenario:* > - Fetch table names (T1 and T2) > - Dump table T1 > - Rename table T2 to T3 generates RENAME event > - Dump table T2 is noop as table doesn’t exist. > - In target after load, it only have T1. > - Apply RENAME event will fail as T2 doesn’t exist in target. > This feature can be supported in next phase development as it need proper > design to keep track of renamed tables/partitions. > So, for time being, we shall disable rename operations when bootstrap dump in > progress to avoid any inconsistent state. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17183) Disable rename operations during bootstrap dump
[ https://issues.apache.org/jira/browse/HIVE-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-17183: Status: Open (was: Patch Available) > Disable rename operations during bootstrap dump > --- > > Key: HIVE-17183 > URL: https://issues.apache.org/jira/browse/HIVE-17183 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Fix For: 3.0.0 > > Attachments: HIVE-17183.01.patch, HIVE-17183.02.patch, > HIVE-17183.03.patch, HIVE-17183.04.patch > > > Currently, bootstrap dump shall lead to data loss when any rename happens > while dump in progress. > *Scenario:* > - Fetch table names (T1 and T2) > - Dump table T1 > - Rename table T2 to T3 generates RENAME event > - Dump table T2 is noop as table doesn’t exist. > - In target after load, it only have T1. > - Apply RENAME event will fail as T2 doesn’t exist in target. > This feature can be supported in next phase development as it need proper > design to keep track of renamed tables/partitions. > So, for time being, we shall disable rename operations when bootstrap dump in > progress to avoid any inconsistent state. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17183) Disable rename operations during bootstrap dump
[ https://issues.apache.org/jira/browse/HIVE-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-17183: Status: Patch Available (was: Open) > Disable rename operations during bootstrap dump > --- > > Key: HIVE-17183 > URL: https://issues.apache.org/jira/browse/HIVE-17183 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Fix For: 3.0.0 > > Attachments: HIVE-17183.01.patch, HIVE-17183.02.patch, > HIVE-17183.03.patch, HIVE-17183.04.patch > > > Currently, bootstrap dump shall lead to data loss when any rename happens > while dump in progress. > *Scenario:* > - Fetch table names (T1 and T2) > - Dump table T1 > - Rename table T2 to T3 generates RENAME event > - Dump table T2 is noop as table doesn’t exist. > - In target after load, it only have T1. > - Apply RENAME event will fail as T2 doesn’t exist in target. > This feature can be supported in next phase development as it need proper > design to keep track of renamed tables/partitions. > So, for time being, we shall disable rename operations when bootstrap dump in > progress to avoid any inconsistent state. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17183) Disable rename operations during bootstrap dump
[ https://issues.apache.org/jira/browse/HIVE-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-17183: Attachment: HIVE-17183.04.patch Added 04.patch with below updates. - Added UUID to the key to differentiate multiple bootstrap dumps in parallel for scenario primary -> replica_1 and primary->replica_2. - Removed all the dump state entries in the database properties before dumping the database object for bootstrap dump. This is to avoid the case where rename always blocked in replica if the dump state property is replicated. Request [~anishek]/[~thejas] to please review the patch. > Disable rename operations during bootstrap dump > --- > > Key: HIVE-17183 > URL: https://issues.apache.org/jira/browse/HIVE-17183 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Fix For: 3.0.0 > > Attachments: HIVE-17183.01.patch, HIVE-17183.02.patch, > HIVE-17183.03.patch, HIVE-17183.04.patch > > > Currently, bootstrap dump shall lead to data loss when any rename happens > while dump in progress. > *Scenario:* > - Fetch table names (T1 and T2) > - Dump table T1 > - Rename table T2 to T3 generates RENAME event > - Dump table T2 is noop as table doesn’t exist. > - In target after load, it only have T1. > - Apply RENAME event will fail as T2 doesn’t exist in target. > This feature can be supported in next phase development as it need proper > design to keep track of renamed tables/partitions. > So, for time being, we shall disable rename operations when bootstrap dump in > progress to avoid any inconsistent state. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17183) Disable rename operations during bootstrap dump
[ https://issues.apache.org/jira/browse/HIVE-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-17183: Status: Open (was: Patch Available) > Disable rename operations during bootstrap dump > --- > > Key: HIVE-17183 > URL: https://issues.apache.org/jira/browse/HIVE-17183 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Fix For: 3.0.0 > > Attachments: HIVE-17183.01.patch, HIVE-17183.02.patch, > HIVE-17183.03.patch > > > Currently, bootstrap dump shall lead to data loss when any rename happens > while dump in progress. > *Scenario:* > - Fetch table names (T1 and T2) > - Dump table T1 > - Rename table T2 to T3 generates RENAME event > - Dump table T2 is noop as table doesn’t exist. > - In target after load, it only have T1. > - Apply RENAME event will fail as T2 doesn’t exist in target. > This feature can be supported in next phase development as it need proper > design to keep track of renamed tables/partitions. > So, for time being, we shall disable rename operations when bootstrap dump in > progress to avoid any inconsistent state. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17183) Disable rename operations during bootstrap dump
[ https://issues.apache.org/jira/browse/HIVE-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-17183: Description: Currently, bootstrap dump shall lead to data loss when any rename happens while dump in progress. *Scenario:* - Fetch table names (T1 and T2) - Dump table T1 - Rename table T2 to T3 generates RENAME event - Dump table T2 is noop as table doesn’t exist. - In target after load, it only have T1. - Apply RENAME event will fail as T2 doesn’t exist in target. This feature can be supported in next phase development as it need proper design to keep track of renamed tables/partitions. So, for time being, we shall disable rename operations when bootstrap dump in progress to avoid any inconsistent state. was: Currently, bootstrap dump shall lead to data loss when any rename happens while dump in progress. This feature can be supported in next phase development as it need proper design to keep track of renamed tables/partitions. So, for time being, we shall disable rename operations when bootstrap dump in progress to avoid any inconsistent state. > Disable rename operations during bootstrap dump > --- > > Key: HIVE-17183 > URL: https://issues.apache.org/jira/browse/HIVE-17183 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Fix For: 3.0.0 > > Attachments: HIVE-17183.01.patch, HIVE-17183.02.patch, > HIVE-17183.03.patch > > > Currently, bootstrap dump shall lead to data loss when any rename happens > while dump in progress. > *Scenario:* > - Fetch table names (T1 and T2) > - Dump table T1 > - Rename table T2 to T3 generates RENAME event > - Dump table T2 is noop as table doesn’t exist. > - In target after load, it only have T1. > - Apply RENAME event will fail as T2 doesn’t exist in target. > This feature can be supported in next phase development as it need proper > design to keep track of renamed tables/partitions. > So, for time being, we shall disable rename operations when bootstrap dump in > progress to avoid any inconsistent state. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17183) Disable rename operations during bootstrap dump
[ https://issues.apache.org/jira/browse/HIVE-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-17183: Status: Patch Available (was: Open) > Disable rename operations during bootstrap dump > --- > > Key: HIVE-17183 > URL: https://issues.apache.org/jira/browse/HIVE-17183 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Fix For: 3.0.0 > > Attachments: HIVE-17183.01.patch, HIVE-17183.02.patch, > HIVE-17183.03.patch > > > Currently, bootstrap dump shall lead to data loss when any rename happens > while dump in progress. This feature can be supported in next phase > development as it need proper design to keep track of renamed > tables/partitions. > So, for time being, we shall disable rename operations when bootstrap dump in > progress to avoid any inconsistent state. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17183) Disable rename operations during bootstrap dump
[ https://issues.apache.org/jira/browse/HIVE-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-17183: Attachment: HIVE-17183.03.patch > Disable rename operations during bootstrap dump > --- > > Key: HIVE-17183 > URL: https://issues.apache.org/jira/browse/HIVE-17183 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Fix For: 3.0.0 > > Attachments: HIVE-17183.01.patch, HIVE-17183.02.patch, > HIVE-17183.03.patch > > > Currently, bootstrap dump shall lead to data loss when any rename happens > while dump in progress. This feature can be supported in next phase > development as it need proper design to keep track of renamed > tables/partitions. > So, for time being, we shall disable rename operations when bootstrap dump in > progress to avoid any inconsistent state. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17183) Disable rename operations during bootstrap dump
[ https://issues.apache.org/jira/browse/HIVE-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-17183: Attachment: (was: HIVE-17183.03.patch) > Disable rename operations during bootstrap dump > --- > > Key: HIVE-17183 > URL: https://issues.apache.org/jira/browse/HIVE-17183 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Fix For: 3.0.0 > > Attachments: HIVE-17183.01.patch, HIVE-17183.02.patch > > > Currently, bootstrap dump shall lead to data loss when any rename happens > while dump in progress. This feature can be supported in next phase > development as it need proper design to keep track of renamed > tables/partitions. > So, for time being, we shall disable rename operations when bootstrap dump in > progress to avoid any inconsistent state. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17183) Disable rename operations during bootstrap dump
[ https://issues.apache.org/jira/browse/HIVE-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-17183: Status: Open (was: Patch Available) > Disable rename operations during bootstrap dump > --- > > Key: HIVE-17183 > URL: https://issues.apache.org/jira/browse/HIVE-17183 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Fix For: 3.0.0 > > Attachments: HIVE-17183.01.patch, HIVE-17183.02.patch, > HIVE-17183.03.patch > > > Currently, bootstrap dump shall lead to data loss when any rename happens > while dump in progress. This feature can be supported in next phase > development as it need proper design to keep track of renamed > tables/partitions. > So, for time being, we shall disable rename operations when bootstrap dump in > progress to avoid any inconsistent state. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17183) Disable rename operations during bootstrap dump
[ https://issues.apache.org/jira/browse/HIVE-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-17183: Status: Patch Available (was: Open) > Disable rename operations during bootstrap dump > --- > > Key: HIVE-17183 > URL: https://issues.apache.org/jira/browse/HIVE-17183 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Fix For: 3.0.0 > > Attachments: HIVE-17183.01.patch, HIVE-17183.02.patch, > HIVE-17183.03.patch > > > Currently, bootstrap dump shall lead to data loss when any rename happens > while dump in progress. This feature can be supported in next phase > development as it need proper design to keep track of renamed > tables/partitions. > So, for time being, we shall disable rename operations when bootstrap dump in > progress to avoid any inconsistent state. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17183) Disable rename operations during bootstrap dump
[ https://issues.apache.org/jira/browse/HIVE-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-17183: Attachment: HIVE-17183.03.patch Added 03.patch with fixes for Anishek's comments. > Disable rename operations during bootstrap dump > --- > > Key: HIVE-17183 > URL: https://issues.apache.org/jira/browse/HIVE-17183 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Fix For: 3.0.0 > > Attachments: HIVE-17183.01.patch, HIVE-17183.02.patch, > HIVE-17183.03.patch > > > Currently, bootstrap dump shall lead to data loss when any rename happens > while dump in progress. This feature can be supported in next phase > development as it need proper design to keep track of renamed > tables/partitions. > So, for time being, we shall disable rename operations when bootstrap dump in > progress to avoid any inconsistent state. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17183) Disable rename operations during bootstrap dump
[ https://issues.apache.org/jira/browse/HIVE-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-17183: Status: Open (was: Patch Available) > Disable rename operations during bootstrap dump > --- > > Key: HIVE-17183 > URL: https://issues.apache.org/jira/browse/HIVE-17183 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Fix For: 3.0.0 > > Attachments: HIVE-17183.01.patch, HIVE-17183.02.patch > > > Currently, bootstrap dump shall lead to data loss when any rename happens > while dump in progress. This feature can be supported in next phase > development as it need proper design to keep track of renamed > tables/partitions. > So, for time being, we shall disable rename operations when bootstrap dump in > progress to avoid any inconsistent state. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17183) Disable rename operations during bootstrap dump
[ https://issues.apache.org/jira/browse/HIVE-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-17183: Status: Patch Available (was: Open) > Disable rename operations during bootstrap dump > --- > > Key: HIVE-17183 > URL: https://issues.apache.org/jira/browse/HIVE-17183 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Fix For: 3.0.0 > > Attachments: HIVE-17183.01.patch, HIVE-17183.02.patch > > > Currently, bootstrap dump shall lead to data loss when any rename happens > while dump in progress. This feature can be supported in next phase > development as it need proper design to keep track of renamed > tables/partitions. > So, for time being, we shall disable rename operations when bootstrap dump in > progress to avoid any inconsistent state. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17183) Disable rename operations during bootstrap dump
[ https://issues.apache.org/jira/browse/HIVE-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-17183: Attachment: HIVE-17183.02.patch Added 02.patch after rebasing with master > Disable rename operations during bootstrap dump > --- > > Key: HIVE-17183 > URL: https://issues.apache.org/jira/browse/HIVE-17183 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Fix For: 3.0.0 > > Attachments: HIVE-17183.01.patch, HIVE-17183.02.patch > > > Currently, bootstrap dump shall lead to data loss when any rename happens > while dump in progress. This feature can be supported in next phase > development as it need proper design to keep track of renamed > tables/partitions. > So, for time being, we shall disable rename operations when bootstrap dump in > progress to avoid any inconsistent state. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17183) Disable rename operations during bootstrap dump
[ https://issues.apache.org/jira/browse/HIVE-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-17183: Status: Open (was: Patch Available) > Disable rename operations during bootstrap dump > --- > > Key: HIVE-17183 > URL: https://issues.apache.org/jira/browse/HIVE-17183 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Fix For: 3.0.0 > > Attachments: HIVE-17183.01.patch > > > Currently, bootstrap dump shall lead to data loss when any rename happens > while dump in progress. This feature can be supported in next phase > development as it need proper design to keep track of renamed > tables/partitions. > So, for time being, we shall disable rename operations when bootstrap dump in > progress to avoid any inconsistent state. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17183) Disable rename operations during bootstrap dump
[ https://issues.apache.org/jira/browse/HIVE-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-17183: Status: Patch Available (was: Open) > Disable rename operations during bootstrap dump > --- > > Key: HIVE-17183 > URL: https://issues.apache.org/jira/browse/HIVE-17183 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Fix For: 3.0.0 > > Attachments: HIVE-17183.01.patch > > > Currently, bootstrap dump shall lead to data loss when any rename happens > while dump in progress. This feature can be supported in next phase > development as it need proper design to keep track of renamed > tables/partitions. > So, for time being, we shall disable rename operations when bootstrap dump in > progress to avoid any inconsistent state. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17183) Disable rename operations during bootstrap dump
[ https://issues.apache.org/jira/browse/HIVE-17183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sankar Hariappan updated HIVE-17183: Attachment: HIVE-17183.01.patch Added 01.patch with below changes. - Added new parameter (bootstrap.dump.state) to database object. - Set the value to "active" when bootstrap dump in progress and "idle" if not. - Added a test case to verify concurrent rename partition/table when bootstrap dump in progress. Request [~anishek]/[~daijy]/[~thejas] to please review the patch! > Disable rename operations during bootstrap dump > --- > > Key: HIVE-17183 > URL: https://issues.apache.org/jira/browse/HIVE-17183 > Project: Hive > Issue Type: Sub-task > Components: repl >Affects Versions: 2.1.0 >Reporter: Sankar Hariappan >Assignee: Sankar Hariappan > Labels: DR, replication > Fix For: 3.0.0 > > Attachments: HIVE-17183.01.patch > > > Currently, bootstrap dump shall lead to data loss when any rename happens > while dump in progress. This feature can be supported in next phase > development as it need proper design to keep track of renamed > tables/partitions. > So, for time being, we shall disable rename operations when bootstrap dump in > progress to avoid any inconsistent state. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
