[
https://issues.apache.org/jira/browse/HIVE-21731?focusedWorklogId=243422&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-243422
]
ASF GitHub Bot logged work on HIVE-21731:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 16/May/19 15:52
Start Date: 16/May/19 15:52
Worklog Time Spent: 10m
Work Description: sankarh commented on pull request #628: HIVE-21731 :
Hive import fails, post upgrade of source 3.0 cluster, to a target 4.0 cluster
with strict managed table set to true.
URL: https://github.com/apache/hive/pull/628#discussion_r284758759
##########
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplicationWithTableMigration.java
##########
@@ -504,4 +507,56 @@ public void dynamicallyConvertExternalToManagedTable()
throws Throwable {
.runFailure("alter table t1 set tblproperties('EXTERNAL'='false')")
.runFailure("alter table t2 set
tblproperties('EXTERNAL'='false')");
}
+
+ @Test
+ public void testMigrationWithUpgrade() throws Throwable {
+ WarehouseInstance.Tuple tuple = primary.run("use " + primaryDbName)
+ .run("create table tacid (id int) clustered by(id) into 3 buckets
stored as orc ")
+ .run("create table texternal (id int) ")
+ .run("insert into texternal values (1)")
+ .dump(primaryDbName, null);
+ replica.load(replicatedDbName, tuple.dumpLocation)
+ .run("use " + replicatedDbName)
+ .run("repl status " + replicatedDbName)
+ .verifyResult(tuple.lastReplicationId)
+ .run("select count(*) from tacid")
+ .verifyResult("0")
+ .run("select id from texternal")
+ .verifyResult("1");
+
+ assertTrue(isFullAcidTable(replica.getTable(replicatedDbName, "tacid")));
+
assertFalse(MetaStoreUtils.isExternalTable(replica.getTable(replicatedDbName,
"texternal")));
+
+ // forcefully (setting db property) alter the table type. For acid table,
set the bootstrap acid table to true. For
Review comment:
Shall explicitly mention that this is a mock up for
HiveStrictManagedMigration tool which is used during upgrade to migrate the
tables based on migration rules.
We simulate the scenario by explicitly allowing alter commands to change
table types.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 243422)
Time Spent: 40m (was: 0.5h)
> Hive import fails, post upgrade of source 3.0 cluster, to a target 4.0
> cluster with strict managed table set to true.
> ---------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-21731
> URL: https://issues.apache.org/jira/browse/HIVE-21731
> Project: Hive
> Issue Type: Bug
> Reporter: mahesh kumar behera
> Assignee: mahesh kumar behera
> Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-21731.01.patch, HIVE-21731.02.patch
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> The scenario is
> # Replication policy is set with hive 3.0 source cluster (strict managed
> table set to false) and hive 4.0 target cluster with strict managed table set
> true.
> # User upgrades the 3.0 source cluster to 4.0 cluster using upgrade tool.
> # The upgrade converts all managed tables to acid tables.
> # In the next repl dump, user sets hive .repl .dump .include .acid .tables
> and hive .repl .bootstrap. acid. tables set true triggering bootstrap of
> newly converted ACID tables.
> # As the old tables are non-txn tables, dump is not filtering the events
> even tough bootstrap acid table is set to true. This is causing the repl load
> to fail as the write id is not set in the table object.
> # If we ignore the event replay, the bootstrap is failing with dump
> directory mismatch error.
> The fix should be
> # Ignore dumping the alter table event if bootstrap acid table is set true
> and the alter is converting a non-acid table to acid table.
> # In case of bootstrap during incremental load, ignore the dump directory
> property set in table object.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
