Hi Urs,
I think Flink does not encourage to use "object reuse" feature, because in
the documentation, it warn the user it may course bug when the user-code
function of an operation is not aware of this behavior[1].
The "object reuse" is runtime behavior and it's configuration item belongs
`ExecutionConfig` (this class both for batch and streaming), so it takes
efforts for both batch and streaming[1].
"Object reuse" feature is not perfect, the main use case is for batch[2]
and FLIP-21 tries to promote this feature gracefully, but the current state
is "*Under discussion*".
[1]:
https://ci.apache.org/projects/flink/flink-docs-release-1.5/dev/execution_configuration.html#execution-configuration
[2]:
https://ci.apache.org/projects/flink/flink-docs-release-1.5/dev/batch/#operating-on-data-objects-in-functions
Thanks, vino.
2018-07-17 20:50 GMT+08:00 Urs Schoenenberger <
urs.schoenenber...@tngtech.com>:
> Hi all,
>
> we came across some interesting behaviour today.
> We enabled object reuse on a streaming job that looks like this:
>
> stream = env.addSource(source)
> stream.map(mapFnA).addSink(sinkA)
> stream.map(mapFnB).addSink(sinkB)
>
> Operator chaining is enabled, so the optimizer fuses all operations into
> a single slot.
> The same object reference gets passed to both mapFnA and mapFnB. This
> makes sense when I think about the internal implementation, but it still
> came as a bit of a surprise since the object reuse docs (for batch -
> there are no official ones for streaming, right?) don't really deal with
> splitting the DataSet/DataStream. I guess my case is *technically*
> covered by the documented warning that it is unsafe to reuse an object
> that has already been collected, only in this case this reuse is
> "hidden" behind the stream definition DSL.
>
> Is this the expected behaviour? Is object reuse for DataStreams
> encouraged at all or is it more of a "hidden beta" feature until FLIP-21
> is officially finished?
>
> Best,
> Urs
>
> --
> Urs Schönenberger - urs.schoenenber...@tngtech.com
>
> TNG Technology Consulting GmbH, Beta-Straße 13a, 85774 Unterföhring
> Geschäftsführer: Henrik Klagges, Dr. Robert Dahlke, Gerhard Müller
> Sitz: Unterföhring * Amtsgericht München * HRB 135082
>