Following is the code for the policy. It has a fair bit of code to handle
idle partition scenarios, for our current discussion let's assume none of
the partitions of a given job are idle but all are high throughput
partitions. For such scenarios very simply put, getTimestampForRecord() is
storing
Can you explain more about your custom watermark policy?
On Sun, Aug 24, 2025 at 11:43 AM gaurav mishra
wrote:
> Hi All,
> We have a dataflow job which is consuming from 100s of kafka topics. This
> job has time based window aggregations. From time to time we see the jobs
> dropping data due to
