Yes, those are the ones I was referring to.
On Tue, Apr 9, 2019 at 4:08 PM Pablo Estrada wrote:
> Yup! : ) - I think
>
> On Tue, Apr 9, 2019 at 3:52 PM Brian Hulette wrote:
>
>> Are these the blog posts?
>>
>> https://beam.apache.org/blog/2017/02/13/stateful-processing.html
>>
Yup! : ) - I think
On Tue, Apr 9, 2019 at 3:52 PM Brian Hulette wrote:
> Are these the blog posts?
>
> https://beam.apache.org/blog/2017/02/13/stateful-processing.html
> https://beam.apache.org/blog/2017/08/28/timely-processing.html
>
> On Tue, Apr 9, 2019 at 3:41 PM Pablo Estrada wrote:
>
>>
Are these the blog posts?
https://beam.apache.org/blog/2017/02/13/stateful-processing.html
https://beam.apache.org/blog/2017/08/28/timely-processing.html
On Tue, Apr 9, 2019 at 3:41 PM Pablo Estrada wrote:
> sunds good. Thanks guys <3
>
> On Tue, Apr 9, 2019 at 3:19 PM Lukasz Cwik wrote:
sunds good. Thanks guys <3
On Tue, Apr 9, 2019 at 3:19 PM Lukasz Cwik wrote:
> UnboundedSources and SplittableDoFns report watermarks which the runner
> uses to compute how much the watermark could advance if it processed some
> outstanding work. But it is always upto the runner to choose
UnboundedSources and SplittableDoFns report watermarks which the runner
uses to compute how much the watermark could advance if it processed some
outstanding work. But it is always upto the runner to choose when the
watermark advances. The runner could process each work item in watermark
priority
In state & timers and new DoFn in the past It was an explicit decision to
not allow direct observation of the watermark, but only to set a timer in
event time. Is there a design doc I can read to catch up?
Kenn
On Tue, Apr 9, 2019 at 1:44 PM Lukasz Cwik wrote:
> WatermarkReporterParam is about
hi Luke,
thanks for the prompt reply: )
That makes sense. I think I'll go back to my cave to read a bunch about
streaming. : )
I was looking for this to try to write a sequence generator for Python in
streaming, and I was trying to debug what was going on. I was trying to
allow the DoFn to
WatermarkReporterParam is about reporting the watermark. The main usecase
is for SplittableDoFns to be able to report the data watermark.
The watermark is per input and output of a DoFn. Also each bundle being
processed has its local watermarks while the runner computes the global
watermark. The
