Hi Alexey, As pointed out by Luis the current implementation uses foreachPartition (see here <https://github.com/apache/ignite/blob/1.6.0/modules/spark/src/main/scala/org/apache/ignite/spark/IgniteContext.scala#L55>), but it's suboptimal.
Cheers, Paolo On Fri, Jul 8, 2016 at 5:03 PM, Alexey Goncharuk <[email protected] > wrote: > I remember asking this question on Spark user list and parallelize() was > the suggested option to run a closure on all Spark workers. Paolo, I like > the idea with foreachPartition() - maybe we can crete a fake RDD with > partition number equal to the number of Spark workers and then map each > partition to the corresponding worker. >
