Shawn-Hx commented on PR #3420: URL: https://github.com/apache/fluss/pull/3420#issuecomment-4912550159
> One more thought — watermark extraction uses a shared WatermarkExtractor and applies to all lakes, so it feels more like a tiering-layer concern than a per-lake writer one. Since TieringSplitReader already builds the extractor and sees every record, could we compute the watermark there and carry it in TableBucketWriteResult (next to maxTimestamp), instead of each LakeWriter doing it via LakeWriteResult? That would keep the logic in one place and avoid pushing the extends LakeWriteResult bound across the lake API. @luoyuxia what do you think about moving watermark computation up to the tiering layer? Calculating watermark in LakeWriter for each lake format involves considering that some lake formats do not support reporting watermark information. In such cases, the overhead of calculating watermark during tiering process can be omitted. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
