ayushtkn commented on a change in pull request #1954:
URL: https://github.com/apache/hive/pull/1954#discussion_r581666766
##########
File path:
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplicationScenariosExternalTables.java
##########
@@ -1375,6 +1376,205 @@ public void differentCatalogIncrementalReplication()
throws Throwable {
primary.run("drop database if exists " + sparkDbName + " cascade");
}
+ @Test
+ public void testDatabaseLevelCopyLazy() throws Throwable {
+ testDatabaseLevelCopy(true);
+ }
+
+ @Test
+ public void testDatabaseLevelCopyAtSource() throws Throwable {
+ testDatabaseLevelCopy(false);
+ }
+
+ public void testDatabaseLevelCopy(boolean isAtTarget) throws Throwable {
+ Path externalTableLocation =
+ new Path("/" + testName.getMethodName() + "/" + primaryDbName + "/" +
"a/");
+ DistributedFileSystem fs = primary.miniDFSCluster.getFileSystem();
+ fs.mkdirs(externalTableLocation, new FsPermission("777"));
+
+ Path externalTablePartitionLocation =
+ new Path("/" + testName.getMethodName() + "/" + primaryDbName + "/" +
"part/");
+ fs.mkdirs(externalTableLocation, new FsPermission("777"));
+
+ List<String> withClause = Arrays.asList(
+ "'distcp.options.update'='','hive.repl.external.warehouse.single.copy"
+ + ".task'='true'",
+ "'" + HiveConf.ConfVars.REPL_RUN_DATA_COPY_TASKS_ON_TARGET.varname
+ + "'='" + isAtTarget + "'");
+
+ // Create a table within the warehouse location, one outside and one with
+ // a partition outside the default location.
+ WarehouseInstance.Tuple tuple =
+ primary.run("use " + primaryDbName)
+ .run("create external table a (i int, j int) "
+ + "row format delimited fields terminated by ',' "
+ + "location '" + externalTableLocation.toUri() + "'")
+ .run("insert into a values(1,2)")
+ .run("create external table b (id int)")
+ .run("insert into b values(5)")
+ .run("create external table c (place string) partitioned by
(country "
+ + "string)")
+ .run("insert into table c partition(country='india') values "
+ + "('bangalore')")
+ .run("ALTER TABLE c ADD PARTITION (country='france') LOCATION '"
+ + externalTablePartitionLocation.toString() + "'")
+ .run("insert into c partition(country='france') values('paris')")
+ .dump(primaryDbName, withClause);
+
+ // Do a load and verify all the data is there.
+ replica.load(replicatedDbName, primaryDbName, withClause)
+ .run("use " + replicatedDbName)
+ .run("select i from a where j=2")
+ .verifyResult("1")
+ .run("select id from b")
+ .verifyResult("5")
+ .run("select place from c where country='india'")
+ .verifyResult("bangalore")
+ .run("select place from c where country='france'")
+ .verifyResult("paris");
+
+ // Check the task copied post bootstrap, It should have the database loc,
+ // the table 'a' since that is outside of the default location, and the
+ // 'c', since its partition is out of the default location.
+ assertExternalFileInfo(Arrays.asList(primaryDbName.toLowerCase(), "a",
"c"),
Review comment:
That would be the partition location, the name used is the table name,
That logic was already there
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
