[ https://issues.apache.org/jira/browse/SPARK-27137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Osira Ben updated SPARK-27137: ------------------------------ Description: If I execute this piece of code {code:java} val foo = "foo" def f(arg: Any): Unit = { Option(42).foreach(_ => java.util.Objects.requireNonNull(foo, "foo")) } sc.parallelize(Seq(1, 2), 2).foreach(f) {code} {{in spark2-shell via :paste it throws}} {code:java} scala> :paste // Entering paste mode (ctrl-D to finish) val foo = "foo" def f(arg: Any): Unit = { Option(42).foreach(_ => java.util.Objects.requireNonNull(foo, "foo")) } sc.parallelize(Seq(1, 2), 2).foreach(f) // Exiting paste mode, now interpreting. 19/03/11 15:02:06 WARN scheduler.TaskSetManager: Lost task 0.0 in stage 1.0 (TID 2, hadoop.company.com, executor 1): java.lang.NullPointerException: foo at java.util.Objects.requireNonNull(Objects.java:228) {code} However if I execute it pasting without :paste or via spark2-shell -i it doesn't. was: If I execute this piece of code {code:java} val foo = "foo" def f(arg: Any): Unit = { Option(42).foreach(_ => java.util.Objects.requireNonNull(foo, "foo")) } sc.parallelize(Seq(1, 2), 2).foreach(f) {code} {{in spark2-shell via :paste it throws}}{{}} {code:java} scala> :paste // Entering paste mode (ctrl-D to finish) val foo = "foo" def f(arg: Any): Unit = { Option(42).foreach(_ => java.util.Objects.requireNonNull(foo, "foo")) } sc.parallelize(Seq(1, 2), 2).foreach(f) // Exiting paste mode, now interpreting. 19/03/11 15:02:06 WARN scheduler.TaskSetManager: Lost task 0.0 in stage 1.0 (TID 2, hadoop.company.com, executor 1): java.lang.NullPointerException: foo at java.util.Objects.requireNonNull(Objects.java:228) {code} However if I execute it pasting without :paste or via spark2-shell -i it doesn't. > Spark captured variable is null if the code is pasted via :paste > ---------------------------------------------------------------- > > Key: SPARK-27137 > URL: https://issues.apache.org/jira/browse/SPARK-27137 > Project: Spark > Issue Type: Bug > Components: Spark Core > Affects Versions: 2.4.0 > Reporter: Osira Ben > Priority: Major > > If I execute this piece of code > {code:java} > val foo = "foo" > def f(arg: Any): Unit = { > Option(42).foreach(_ => java.util.Objects.requireNonNull(foo, "foo")) > } > sc.parallelize(Seq(1, 2), 2).foreach(f) > {code} > {{in spark2-shell via :paste it throws}} > {code:java} > scala> :paste > // Entering paste mode (ctrl-D to finish) > val foo = "foo" > def f(arg: Any): Unit = { > Option(42).foreach(_ => java.util.Objects.requireNonNull(foo, "foo")) > } > sc.parallelize(Seq(1, 2), 2).foreach(f) > // Exiting paste mode, now interpreting. > 19/03/11 15:02:06 WARN scheduler.TaskSetManager: Lost task 0.0 in stage 1.0 > (TID 2, hadoop.company.com, executor 1): java.lang.NullPointerException: foo > at java.util.Objects.requireNonNull(Objects.java:228) > {code} > However if I execute it pasting without :paste or via spark2-shell -i it > doesn't. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org