It makes sense.
Agreed.
Regards
JB
On 01/25/2017 08:34 PM, Kenneth Knowles wrote:
There's actually not a JIRA filed beyond BEAM-25 for what Eugene is
referring to. Context: Prior to windowing and streaming it was safe to
buffer elements in @ProcessElement and then actually perform output in
Le 25/01/2017 à 20:34, Kenneth Knowles a écrit :
There's actually not a JIRA filed beyond BEAM-25 for what Eugene is
referring to. Context: Prior to windowing and streaming it was safe to
buffer elements in @ProcessElement and then actually perform output in
@FinishBundle. This pattern is only
One more thing.
I think ideally, bundles should not leak into the model at all - e.g.
ideally, startBundle/finishBundle methods in DoFn should not exist. They
interact poorly with windowing.
The proper way to address what is commonly done in these methods is either
Setup/Teardown methods, or a
On Wed, Jan 25, 2017 at 8:23 PM Thomas Groh
wrote:
> I have a couple of points in addition to what Robert said
>
> Runners are permitted to determine bundle sizes as appropriate to their
> implementation, so long as bundles are atomically committed. The contents
> of a
I have a couple of points in addition to what Robert said
Runners are permitted to determine bundle sizes as appropriate to their
implementation, so long as bundles are atomically committed. The contents
of a PCollection are independent of the bundling of that PCollection.
Runners can process