[
https://issues.apache.org/jira/browse/HIVE-24484?focusedWorklogId=770105&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-770105
]
ASF GitHub Bot logged work on HIVE-24484:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 13/May/22 10:04
Start Date: 13/May/22 10:04
Worklog Time Spent: 10m
Work Description: ayushtkn commented on code in PR #3279:
URL: https://github.com/apache/hive/pull/3279#discussion_r872214761
##########
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplicationOnHDFSEncryptedZones.java:
##########
@@ -123,57 +122,24 @@ public void
targetAndSourceHaveDifferentEncryptionZoneKeys() throws Throwable {
put(HiveConf.ConfVars.REPLDIR.varname, primary.repldDir);
}}, "test_key123");
- List<String> dumpWithClause = Arrays.asList(
- "'hive.repl.add.raw.reserved.namespace'='true'",
- "'" + HiveConf.ConfVars.REPL_EXTERNAL_TABLE_BASE_DIR.varname +
"'='"
- + replica.externalTableWarehouseRoot + "'",
- "'distcp.options.skipcrccheck'=''",
- "'" + HiveConf.ConfVars.HIVE_SERVER2_ENABLE_DOAS.varname +
"'='false'",
- "'" + HiveConf.ConfVars.HIVE_DISTCP_DOAS_USER.varname + "'='"
- + UserGroupInformation.getCurrentUser().getUserName()
+"'");
- WarehouseInstance.Tuple tuple =
- primary.run("use " + primaryDbName)
- .run("create table encrypted_table (id int, value string)")
- .run("insert into table encrypted_table values
(1,'value1')")
- .run("insert into table encrypted_table values
(2,'value2')")
- .dump(primaryDbName, dumpWithClause);
-
- replica
- .run("repl load " + primaryDbName + " into " + replicatedDbName
- + " with('hive.repl.add.raw.reserved.namespace'='true', "
- + "'hive.repl.replica.external.table.base.dir'='" +
replica.externalTableWarehouseRoot + "', "
- + "'hive.exec.copyfile.maxsize'='0',
'distcp.options.skipcrccheck'='')")
- .run("use " + replicatedDbName)
- .run("repl status " + replicatedDbName)
- .verifyResult(tuple.lastReplicationId);
-
- try {
- replica
- .run("select value from encrypted_table")
- .verifyResults(new String[] { "value1", "value2" });
- Assert.fail("Src EZKey shouldn't be present on target");
- } catch (IOException e) {
- Assert.assertTrue(e.getCause().getMessage().contains("KeyVersion name
'test_key@0' does not exist"));
- }
-
//read should pass without raw-byte distcp
- dumpWithClause = Arrays.asList( "'" +
HiveConf.ConfVars.REPL_EXTERNAL_TABLE_BASE_DIR.varname + "'='"
+ List<String> dumpWithClause = Arrays.asList( "'" +
HiveConf.ConfVars.REPL_EXTERNAL_TABLE_BASE_DIR.varname + "'='"
+ replica.externalTableWarehouseRoot + "'");
- tuple = primary.run("use " + primaryDbName)
+ WarehouseInstance.Tuple tuple =
+ primary.run("use " + primaryDbName)
.run("create external table encrypted_table2 (id int, value
string)")
.run("insert into table encrypted_table2 values (1,'value1')")
.run("insert into table encrypted_table2 values (2,'value2')")
.dump(primaryDbName, dumpWithClause);
replica
- .run("repl load " + primaryDbName + " into " + replicatedDbName
- + " with('hive.repl.replica.external.table.base.dir'='" +
replica.externalTableWarehouseRoot + "', "
- + "'hive.exec.copyfile.maxsize'='0',
'distcp.options.skipcrccheck'='')")
- .run("use " + replicatedDbName)
- .run("repl status " + replicatedDbName)
- .verifyResult(tuple.lastReplicationId)
Review Comment:
DistCp itself fails, It is running with hive.repl.add.raw.reserved.namespace
and you can't copy if the key is not present on target cluster. Earlier I
converted this to a failure case test, but then the next iteration fails which
is without hive.repl.add.raw.reserved.namespace because the last load wasn't
successful, so I kept the success case
##########
itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplicationOnHDFSEncryptedZones.java:
##########
@@ -123,57 +122,24 @@ public void
targetAndSourceHaveDifferentEncryptionZoneKeys() throws Throwable {
put(HiveConf.ConfVars.REPLDIR.varname, primary.repldDir);
}}, "test_key123");
- List<String> dumpWithClause = Arrays.asList(
Review Comment:
Same as above:
DistCp itself fails, It is running with hive.repl.add.raw.reserved.namespace
and you can't copy if the key is not present on target cluster. Earlier I
converted this to a failure case test, but then the next iteration fails which
is without hive.repl.add.raw.reserved.namespace because the last load wasn't
successful, so I kept the success case
I am not sure how it was working before, but with todat raw type the key
should be there, earlier this test was also single instance, while fixing it
became 2 instances....
Issue Time Tracking
-------------------
Worklog Id: (was: 770105)
Time Spent: 9.55h (was: 9h 23m)
> Upgrade Hadoop to 3.3.1
> -----------------------
>
> Key: HIVE-24484
> URL: https://issues.apache.org/jira/browse/HIVE-24484
> Project: Hive
> Issue Type: Improvement
> Reporter: David Mollitor
> Assignee: David Mollitor
> Priority: Major
> Labels: pull-request-available
> Time Spent: 9.55h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
