rangareddy commented on issue #17322: URL: https://github.com/apache/hudi/issues/17322#issuecomment-5365823212
This issue was reviewed as part of the JIRA-migrated backlog triage (HUDI-8642). **Findings: cannot be dispositioned - the claim is unspecific and nothing is documented.** The body is a single line: "The two APIs behave differently and there is a chance of incorrect records." There is no failing case, no engine named (Spark, Flink and Java each have their own `HoodieData` implementation), and no call site, so there is nothing to confirm. Both APIs are present on the interface, in `hudi-common/src/main/java/org/apache/hudi/common/data/HoodieData.java`: ```java :118 <O> HoodieData<O> map(SerializableFunction<T, O> func); :133 <O> HoodieData<O> mapPartitions(SerializableFunction<Iterator<T>, Iterator<O>> func, boolean preservesPartitioning); ``` Neither javadoc records any behavioural caveat, so if a real difference exists it is undocumented - which is a problem in itself given the claim is about incorrect records. For what it is worth, the most likely candidate is the `preservesPartitioning` flag: a caller passing `true` where partitioning is **not** actually preserved would silently break any downstream logic that depends on partition boundaries. That is a hypothesis worth checking first, not a verified defect. To make progress this needs a concrete call site or a failing case from whoever filed it. If the concern turns out to be real, the fix probably includes a javadoc warning so the next caller does not repeat it. Keeping this open. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
