nsivabalan commented on a change in pull request #4840:
URL: https://github.com/apache/hudi/pull/4840#discussion_r811108570
##########
File path:
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/table/action/commit/BaseSparkCommitActionExecutor.java
##########
@@ -156,19 +156,21 @@ private void initKeyGenIfNeeded(boolean
populateMetaFields) {
LOG.info("RDD PreppedRecords was persisted at: " +
inputRecordsRDD.getStorageLevel());
}
- WorkloadProfile profile = null;
+ WorkloadProfile workloadProfile = null;
if (isWorkloadProfileNeeded()) {
context.setJobStatus(this.getClass().getSimpleName(), "Building workload
profile");
- profile = new WorkloadProfile(buildProfile(inputRecordsRDD),
operationType);
- LOG.info("Workload profile :" + profile);
- saveWorkloadProfileMetadataToInflight(profile, instantTime);
+ workloadProfile = new WorkloadProfile(buildProfile(inputRecordsRDD),
operationType);
+ LOG.info("Input workload profile :" + workloadProfile);
}
// handle records update with clustering
JavaRDD<HoodieRecord<T>> inputRecordsRDDWithClusteringUpdate =
clusteringHandleUpdate(inputRecordsRDD);
// partition using the insert partitioner
- final Partitioner partitioner = getPartitioner(profile);
+ final Partitioner partitioner = getPartitioner(workloadProfile);
+ if (isWorkloadProfileNeeded()) {
+ saveWorkloadProfileMetadataToInflight(workloadProfile, instantTime);
Review comment:
if exception is thrown, the write will fail right anyways. I am not sure
I get your question
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
