geserdugarov commented on PR #12729: URL: https://github.com/apache/hudi/pull/12729#issuecomment-2680041938
> Can someone help understand any overlaps of this effort vs RFC-84 and @geserdugarov 's PR that's nearing landing.. > > > > Can this work layer on top and add more speedup? I would say that this part should be done on top of RFC-84, because it expects that we already have Flink native RowData in writers. And if it done this way, it will lead to great optimization. RFC-84 ensures that there is no slowdown in operators before writers. Flink uses backpressure to adapt processing speed of separate operators in the pipeline. So, after RFC-84 the most slowing down part is writing. And any optimization of it, would lead to significant improvement of total optimization, because Flink would reduce backpressure on previous operators until we faced the next bottleneck. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
