nsivabalan commented on a change in pull request #1978:
URL: https://github.com/apache/hudi/pull/1978#discussion_r484820440
##########
File path:
hudi-client/src/test/java/org/apache/hudi/index/hbase/TestHBaseIndex.java
##########
@@ -156,6 +160,53 @@ public void testSimpleTagLocationAndUpdate() throws
Exception {
}
}
+ @Test
+ public void testTagLocationAndPartitionPathUpdate() throws Exception {
+ final String newCommitTime = "001";
+ final int numRecords = 10;
+ final String oldPartitionPath = "1970/01/01";
+
+ // init old index to hbase
+ List<HoodieRecord> records = dataGen.generateInserts(newCommitTime,
numRecords);
+ JavaRDD<HoodieRecord> writeRecords = jsc().parallelize(records, 1);
+ HoodieWriteConfig config = getConfig(true);
+ HBaseIndex index = new HBaseIndex(getConfig(true));
+
+ try (HoodieWriteClient writeClient = getHoodieWriteClient(config);) {
+ // allowed path change test
+ metaClient = HoodieTableMetaClient.reload(metaClient);
+ HoodieTable hoodieTable = HoodieTable.create(metaClient, config,
hadoopConf);
+
+ JavaRDD<HoodieRecord> records1 = index.tagLocation(writeRecords, jsc(),
hoodieTable);
+ assertEquals(0, records1.filter(record ->
record.isCurrentLocationKnown()).count());
+ writeClient.startCommitWithTime(newCommitTime);
+ JavaRDD<WriteStatus> writeStatues = writeClient.upsert(writeRecords,
newCommitTime);
+ writeClient.commit(newCommitTime, writeStatues);
+ assertNoWriteErrors(writeStatues.collect());
+ // mock old partition data
+ Table table =
utility.getConnection().getTable(TableName.valueOf(TABLE_NAME));
Review comment:
sorry, I missed to take a closer look at the test last time. instead of
fiddling w/ table directly, can we test using index apis only.
- generate records with pp_1 and insert
- update records with pp_2 and call index.tagLocation.
You should be able to test both paths (false and true for the interested
config) with this.
true path:
Verify that index.tagLocation returns 2 * #records, and half of them are
referring to delete records in pp_1 and remaining records are pointing to pp_2
false path:
Verify that index.tagLocation return 1 * #records, and all of them are
pointing to pp_1.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
