This is exactly my case also, it worked, thanks Sean.
On 26 March 2015 at 23:35, Sean Owen wrote:
> You can do this much more simply, I think, with Scala's parallel
> collections (try .par). There's nothing wrong with doing this, no.
>
> Here, something is getting caught in your closure, maybe
>
Thanks Sean,
It works with Scala's parallel collections.
On Thu, Mar 26, 2015 at 11:35 PM, Sean Owen wrote:
> You can do this much more simply, I think, with Scala's parallel
> collections (try .par). There's nothing wrong with doing this, no.
>
> Here, something is getting caught in your closu
You can do this much more simply, I think, with Scala's parallel
collections (try .par). There's nothing wrong with doing this, no.
Here, something is getting caught in your closure, maybe
unintentionally, that's not serializable. It's not directly related to
the parallelism.
On Thu, Mar 26, 2015
Hi.
I'm trying to trigger DataFrame's save method in parallel from my driver.
For that purposes I use ExecutorService and Futures, here's my code:
val futures = [1,2,3].map( t => pool.submit( new Runnable {
override def run(): Unit = {
val commons = events.filter(_._1 == t).map(_._2.common)
