nsivabalan commented on code in PR #13005:
URL: https://github.com/apache/hudi/pull/13005#discussion_r2106338188
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/client/BaseHoodieWriteClient.java:
##########
@@ -410,6 +410,19 @@ public void bootstrap(Option<Map<String, String>>
extraMetadata) {
*/
public abstract O upsertPreppedRecords(I preppedRecords, final String
instantTime);
+ /**
+ * Upserts the given prepared records into the Hoodie table, at the supplied
instantTime.
+ * <p>
+ * This implementation requires that the input records are already tagged,
and de-duped if needed.
+ *
+ * @param preppedRecords Prepared HoodieRecords to upsert
+ * @param instantTime Instant time of the commit
+ * @return Collection of WriteStatus to inspect errors and counts
+ */
+ public O upsertPreppedRecords(I preppedRecords, final String instantTime,
Option<List<Pair<String, String>>> partitionFileIdPairsHolderOpt) {
Review Comment:
hey @danny0405 : the arg that you are talking about, is computed on the
driver and there I don't see much value in making it lazy.
We are looking to make the entire write to metadata in a streaming manner to
be part of the same RDD stage boundary as data table writes.
So, this is the only way that can help achieve that.
and I don't see any value in making it lazy.
can you help me understand what are we trying to achieve here.
I am happy to chat f2f if you need more clarification on the usage of this
API
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
