nsivabalan commented on a change in pull request #1091: [HUDI-389] Fixing Index
look up to return right partitions for a given key along with fileId with
Global Bloom
URL: https://github.com/apache/incubator-hudi/pull/1091#discussion_r358963953
##########
File path:
hudi-client/src/test/java/org/apache/hudi/TestHoodieClientOnCopyOnWriteStorage.java
##########
@@ -382,6 +383,50 @@ public void testDeletes() throws Exception {
HoodieWriteClient::upsert, true, 50, 150, 2);
}
+ /**
+ * Test update of a record to different partition with Global Index.
+ */
+ @Test
+ public void testUpsertToDiffPartitionGlobalIndex() throws Exception {
+ HoodieWriteClient client =
getHoodieWriteClient(getConfig(IndexType.GLOBAL_BLOOM), false);
+ /**
+ * Write 1 (inserts and deletes) Write actual 200 insert records and
ignore 100 delete records
+ */
+ String initCommitTime = "000";
+ String newCommitTime = "001";
+ List<HoodieRecord> inserts1 = dataGen.generateInserts(newCommitTime, 10);
+
+ // Write 1 (only inserts)
+ client.startCommitWithTime(newCommitTime);
+ JavaRDD<HoodieRecord> writeRecords = jsc.parallelize(inserts1, 1);
+
+ JavaRDD<WriteStatus> result = client.insert(writeRecords, newCommitTime);
+ List<WriteStatus> statuses = result.collect();
+ assertNoWriteErrors(statuses);
+
+ // check the partition metadata is written out
+ assertPartitionMetadata(HoodieTestDataGenerator.DEFAULT_PARTITION_PATHS,
fs);
+
+ /**
+ * Write 2 updates
+ */
+ String prevCommitTime = newCommitTime;
+ newCommitTime = "004";
+
+ // Write 1 (only inserts)
+ client.startCommitWithTime(newCommitTime);
+
+ List<HoodieRecord> updates1 =
dataGen.generateUpdatesWithDiffPartition(newCommitTime, inserts1);
+ JavaRDD<HoodieRecord> updateRecords = jsc.parallelize(updates1, 1);
+
+ JavaRDD<WriteStatus> result1 = client.upsert(updateRecords, newCommitTime);
+ List<WriteStatus> statuses1 = result1.collect();
+ assertNoWriteErrors(statuses1);
+
+ // check the partition metadata is written out
+ assertPartitionMetadata(HoodieTestDataGenerator.DEFAULT_PARTITION_PATHS,
fs);
Review comment:
Most tests in this class does not verify records per se, just total records
at each step. I will fix and do the same here.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
