[
https://issues.apache.org/jira/browse/SPARK-3200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15184468#comment-15184468
]
Prashant Sharma edited comment on SPARK-3200 at 3/8/16 6:06 AM:
----------------------------------------------------------------
Hi [~chrismattmann], It never worked. I have clarified above. But since no one
apart from me ever ran into this and complexity of the fix was non trivial, it
was "won't fix".
Actually now that we use the scala repl "as is" without much modifications. So
if it needs to be fixed, there is a considerably large amount of change than it
was required back then. Along with the change the maintenance overhead will
also be large.
However, if the fix is in high demand. One can go ahead and fix in the scala
repl too. It is also possible to work around it. Did you ran into this issue ?
[EDIT] The patch proposed in the Jira can still be merged for scala 2.10 port
of scala repl that lives in Spark. But then it should be done, if this fix is
highly critical.
was (Author: prashant_):
Hi [~chrismattmann], It never worked. I have clarified above. But since no one
apart from me ever ran into this and complexity of the fix was non trivial, it
was "won't fix".
Actually now that we use the scala repl "as is" without much modifications. So
if it needs to be fixed, there is a considerably large amount of change than it
was required back then. Along with the change the maintenance overhead will
also be large.
However, if the fix is in high demand. One can go ahead and fix in the scala
repl too. It is also possible to work around it. Did you ran into this issue ?
> Class defined with reference to external variables crashes in REPL.
> -------------------------------------------------------------------
>
> Key: SPARK-3200
> URL: https://issues.apache.org/jira/browse/SPARK-3200
> Project: Spark
> Issue Type: Bug
> Components: Spark Shell
> Affects Versions: 1.1.0
> Reporter: Prashant Sharma
> Assignee: Prashant Sharma
>
> Reproducer:
> {noformat}
> val a = sc.textFile("README.md").count
> case class A(i: Int) { val j = a}
> sc.parallelize(1 to 10).map(A(_)).collect()
> {noformat}
> This will happen only in distributed mode, when one refers something that
> refers sc and not otherwise.
> There are many ways to work around this, like directly assign a constant
> value instead of referring the variable.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
