I think running the program multiple times is a reasonable way to start
working on this.
I would try and see whether this can be re-written to a non-nested
iterations case. Nestes iterations algorithms may have much more overhead
to start with.
Stephan
On Tue, Apr 14, 2015 at 3:53 PM, Benoît
Hello,
I'm implementing an algorithm which requires nested iterations, and, from
what I understood, this feature was not yet available in Flink [1], and my
experiments with 2 nested bulk iterations seem to confirm that. However I
came across a Flink unit test [2] using nested iterations,
If your inner iterations happens to work only on the data of a single
partition, then you can also implement this iteration as part of a
mapPartition operator. The only problem there would be that you have to
keep all the partition's data on the heap, if you need access to it.
Cheers,
Till
On
Hi Benoît!
You are right, the nested iterations are currently not supported.
The test you found actually checks that the Optimizer gives a good error
message when encountering nested iterations.
Can you write your program as one iterations (the inner) and start the
program multiple times to
Thanks for you quick answers!
The algorithm is the following: I've got a spatial set of data and I want
to find dense regions. The space is beforehand discretized into cells of
a fixed size. Then, for each dense cell (1st iteration), starting with the
most dense, the algorithm tries to
