Re: Serializability: for vs. while loops

2015-01-25 Thread Tobias Pfeiffer
Aaron, On Thu, Jan 15, 2015 at 5:05 PM, Aaron Davidson wrote: > Scala for-loops are implemented as closures using anonymous inner classes > which are instantiated once and invoked many times. This means, though, > that the code inside the loop is actually sitting inside a class, which > confuses

Re: Serializability: for vs. while loops

2015-01-15 Thread Tobias Pfeiffer
Aaron, thanks for your mail! On Thu, Jan 15, 2015 at 5:05 PM, Aaron Davidson wrote: > Scala for-loops are implemented as closures using anonymous inner classes > [...] > While loops, on the other hand, involve none of this trickery, and > everyone is happy. > Ah, I was suspecting something lik

Re: Serializability: for vs. while loops

2015-01-15 Thread Aaron Davidson
Scala for-loops are implemented as closures using anonymous inner classes which are instantiated once and invoked many times. This means, though, that the code inside the loop is actually sitting inside a class, which confuses Spark's Closure Cleaner, whose job is to remove unused references from c