manojpec commented on a change in pull request #3825:
URL: https://github.com/apache/hudi/pull/3825#discussion_r732970044
##########
File path:
hudi-client/hudi-client-common/src/test/java/org/apache/hudi/common/testutils/HoodieMetadataTestTable.java
##########
@@ -56,11 +60,52 @@ public static HoodieTestTable of(HoodieTableMetaClient
metaClient, HoodieTableMe
return new HoodieMetadataTestTable(metaClient.getBasePath(),
metaClient.getRawFs(), metaClient, writer);
}
+ /**
+ * Add commits to the requested partitions and update metadata table.
+ *
+ * @param commitTime - Commit time for the operation
+ * @param operationType - Operation type
+ * @param newPartitionsToAdd - New partitions to add for the operation
+ * @param partitions - List of partitions for this operation
+ * @param filesPerPartition - Total file count to create per partition
+ * @param bootstrap - Whether bootstrapping needed for the
operation
+ * @param createInflightCommit - Whether in flight commit needed for the
operation
+ * @return Commit metadata for the commit operation performed.
+ * @throws Exception
+ */
@Override
public HoodieCommitMetadata doWriteOperation(String commitTime,
WriteOperationType operationType,
List<String>
newPartitionsToAdd, List<String> partitions,
- int filesPerPartition, boolean
bootstrap, boolean createInflightCommit) throws Exception {
- HoodieCommitMetadata commitMetadata = super.doWriteOperation(commitTime,
operationType, newPartitionsToAdd, partitions, filesPerPartition, bootstrap,
createInflightCommit);
+ int filesPerPartition, boolean
bootstrap,
+ boolean createInflightCommit)
throws Exception {
+ if (partitions.isEmpty()) {
+ partitions = Collections.singletonList(EMPTY_STRING);
+ }
+ Map<String, List<Pair<String, Integer>>> partitionToFilesNameLengthMap =
getPartitionFiles(partitions,
+ filesPerPartition);
+ return this.doWriteOperation(commitTime, operationType,
newPartitionsToAdd, partitionToFilesNameLengthMap,
Review comment:
The derived HoodieMetadataTestTable has overridden two of
doWriteOperation() methods. There are tests which are using both of these
versions. `super.doWriteOperation()` does proxies the call to few more hops and
ultimately calls back the other doWriteOperation(), which is again overridden
in HoodieMetadataTestTable. This ended up making two
HoodieTableMetadataWriter::update() call. We need to avoid back and forth
hopping between HoodieMetadataTestTable and HoodieTestTable for the same
method. So, did the current model.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
