xinyuiscool opened a new pull request, #1705: URL: https://github.com/apache/samza/pull/1705
Currently in the Samza event-time watermark aggregation logic, it will compute the watermark as the min of watermarks from all upstream tasks. However, in the lagging cases, the upstream task might not generate watermark for a long period. In this case, the new watermarks will not be generated and the downstream aggregation will be stuck. To address this issue, this patch adds the logic to exclude the tasks that have been "idle" in generating watermark for a configured time, so that the aggregated watermarks will still be generated. Note this mechanism will unblock downstream, but also at the risk of moving event-time clock faster and the events from lagging tasks will become late arrivals. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
