hudi-agent commented on code in PR #20224:
URL: https://github.com/apache/hudi/pull/20224#discussion_r4239761496
##########
hudi-utilities/src/main/java/org/apache/hudi/utilities/streamer/StreamSync.java:
##########
@@ -710,6 +710,7 @@ Pair<InputBatch, Boolean>
fetchNextBatchFromSource(Option<Checkpoint> resumeChec
InputBatch inputBatchForWriter = null; // row writer
boolean reconcileSchema =
props.getBoolean(DataSourceWriteOptions.RECONCILE_SCHEMA().key());
if (transformer.isPresent()) {
+ hoodieSparkContext.setJobStatus(this.getClass().getSimpleName(),
"Transforming source batch: " + cfg.targetTableName);
Review Comment:
🤖 nit: could you move this `setJobStatus` call below the existing
"Transformation is needed..." comment so that comment stays attached to the
transform logic it describes?
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag
quality.</i></sub>
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/commit/BaseWriteHelper.java:
##########
@@ -64,6 +64,9 @@ public HoodieWriteMetadata<O> write(String instantTime,
try {
HoodieTimer sourceReadAndIndexTimer = HoodieTimer.start();
// De-dupe/merge if needed
+ if (shouldCombine) {
Review Comment:
🤖 nit: would it make sense to put the job status inside
`combineOnCondition`/`deduplicateRecords` instead of a separate `if
(shouldCombine)` here, so the condition isn't checked twice?
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag
quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
