nsivabalan commented on code in PR #13005:
URL: https://github.com/apache/hudi/pull/13005#discussion_r2106338188
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java:
##########
@@ -410,6 +410,19 @@ public void bootstrap(Option<Map<String, String>>
extraMetadata) {
*/
public abstract O upsertPreppedRecords(I preppedRecords, final String
instantTime);
+ /**
+ * Upserts the given prepared records into the Hoodie table, at the supplied
instantTime.
+ * <p>
+ * This implementation requires that the input records are already tagged,
and de-duped if needed.
+ *
+ * @param preppedRecords Prepared HoodieRecords to upsert
+ * @param instantTime Instant time of the commit
+ * @return Collection of WriteStatus to inspect errors and counts
+ */
+ public O upsertPreppedRecords(I preppedRecords, final String instantTime,
Option<List<Pair<String, String>>> partitionFileIdPairsHolderOpt) {
Review Comment:
hey @danny0405 : its computed on the driver and there is no point in making
it lazy.
We anyways have tried to make the entire write to metadata in a streaming
manner to be part of the same RDD stage boundary as data table writes.
So, this is the only one that can help achieve that.
and I don't see any value in making it lazy.
can you help me understand what are we trying to achieve here.
I am happy to chat f2f if you need more clarification on the usage of this
API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
