This is not expected, reach out to Google Cloud support with some recent running/killed job ids or e-mail dataflow-feedb...@google.com.
On Mon, Aug 21, 2017 at 2:19 PM, Steve Anderson <st...@maestro.io> wrote: > Has anyone used allowed late data with a session window? Every time I've > tried to use it in production it seems like the time stamp gets stuck after > 3-5 hours. The pipeline keeps ingesting events but the watermark freezes. > > Context: user sends a beacon at intervals, when the beacon stops coming > for 5 minutes the window closes and their session gets marked as ended in > our BQ database (using the dataflow runner btw). However, some of our users > have bad internet connections and may send late beacons. So if I set allow > late data to 24 hours, it seems like the watermark stops progressing after > a few hours. > > Thanks in advanced, > Steve > > Sent from my iPhone
