Right now I am not using any class variables (references to this). All my
variables are created within the scope of the method I am running.
I did more debugging and found this strange behavior.
variables here
for loop
mapPartitions call
use variables here
end mapPartitions
endfor
This will result in a serializable bug, but this won't
variables here
for loop
create new references to variables here
mapPartitions call
use new reference variables here
end mapPartitions
endfor
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Variables-outside-of-mapPartitions-scope-tp5517p5528.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
