Ok thanks for the suggestion but I think I'll wait for another Flink
version before migrating Datasets to Datastream I think...
In my experience it is very helpful to have open/close on all operators.
Best,
Flavio
On Tue, Jul 28, 2020 at 8:51 AM Aljoscha Krettek
wrote:
> I think that should wo
I think that should work with an aggregate() instead of reduce().
Best,
Aljoscha
On 24.07.20 17:02, Flavio Pompermaier wrote:
In my reduce function I want to compute some aggregation on the sub-results
of a map-partition (that I tried to migrate from DataSet to DataStream
without success).
The
In my reduce function I want to compute some aggregation on the sub-results
of a map-partition (that I tried to migrate from DataSet to DataStream
without success).
The original code was something like:
input.mapPartition(new RowToStringSketches(sketchMapSize)) //
.reduce(new SketchesStri
What are you trying to do in the ReduceFunction? Without knowing the
code, maybe an aggregate(AggregateFunction) is the solution.
Best,
Aljoscha
On 20.07.20 18:03, Flavio Pompermaier wrote:
Thanks Aljosha for the reply. So what can I do in my reduce function that
contains transient variables (
Thanks Aljosha for the reply. So what can I do in my reduce function that
contains transient variables (i.e. not serializable)?
On Mon, Jul 20, 2020 at 4:38 PM Aljoscha Krettek
wrote:
> Hi Flavio,
>
> the reason is that under the covers the ReduceFunction will be used as
> the ReduceFunction of
Hi Flavio,
the reason is that under the covers the ReduceFunction will be used as
the ReduceFunction of a ReducingState. And those cannot be rich
functions because we cannot provide all the required context "inside"
the state backend.
You can see how the ReduceFunction is used to create a
R
