Logically a Flatten is just a way to create a multi-input transform
downstream of the flatten (you can imagine a model in which Flatten was not
explicit, we just allowed multiple main inputs). This means that yes, the
watermark is the minimum of all inputs.

I don't see how a late tuple can become early. Can you explain?


On Mon, Mar 12, 2018 at 2:07 PM Shen Li <cs.she...@gmail.com> wrote:

> Hi Reuven,
>
> What about watermark? Should Flatten emit the min watermark of all input
> data streams? If that is the case, one late tuple can become early after
> Flatten, right? Will that cause any problem?
>
> Shen
>
> On Mon, Mar 12, 2018 at 4:09 PM, Reuven Lax <re...@google.com> wrote:
>
>> No, I don't think it makes sense for the Flatten operator to cache
>> element.
>>
>>
>> On Mon, Mar 12, 2018 at 11:55 AM Shen Li <cs.she...@gmail.com> wrote:
>>
>>> If multiple inputs of Flatten proceed at different speeds, should the
>>> Flatten operator cache tuples before emitting output watermarks? This can
>>> prevent a late tuple from becoming early. But if the watermark gap (i.e.,
>>> cache size) becomes too large among inputs, can the application tell
>>> Beam/runner to emit output watermark anyway and consider slow input tuples
>>> as late?
>>>
>>> Thanks,
>>> Shen
>>>
>>
>

Reply via email to