[
https://issues.apache.org/jira/browse/SPARK-571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guoqiang Li updated SPARK-571:
------------------------------
Description:
By mistake, I wrote some code like this:
{code}
object Foo {
def main() {
val sc = new SparkContext(...)
sc.parallelize(0 to 10,10).map({ ... return 1 ... }).collect
}
}
{code}
This compiles fine and actually runs using the local scheduler. However, using
the mesos scheduler throws a NotSerializableException in the CollectTask . I
agree the result of the program above should be undefined or it should be an
error. Would it be possible to have more explicit messages?
was:
By mistake, I wrote some code like this:
<pre>
object Foo {
def main() {
val sc = new SparkContext(...)
sc.parallelize(0 to 10,10).map({ ... return 1 ... }).collect
}
}
</pre>
This compiles fine and actually runs using the local scheduler. However, using
the mesos scheduler throws a NotSerializableException in the CollectTask . I
agree the result of the program above should be undefined or it should be an
error. Would it be possible to have more explicit messages?
> Forbid return statements when cleaning closures
> -----------------------------------------------
>
> Key: SPARK-571
> URL: https://issues.apache.org/jira/browse/SPARK-571
> Project: Spark
> Issue Type: Improvement
> Reporter: tjhunter
> Assignee: William Benton
>
> By mistake, I wrote some code like this:
> {code}
> object Foo {
> def main() {
> val sc = new SparkContext(...)
> sc.parallelize(0 to 10,10).map({ ... return 1 ... }).collect
> }
> }
> {code}
> This compiles fine and actually runs using the local scheduler. However,
> using the mesos scheduler throws a NotSerializableException in the
> CollectTask . I agree the result of the program above should be undefined or
> it should be an error. Would it be possible to have more explicit messages?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
