hvanhovell opened a new pull request, #42807:
URL: https://github.com/apache/spark/pull/42807
### What changes were proposed in this pull request?
Ammonite places all user code inside Helper classes which are nested inside
the class it creates for each command. This PR adds a custom code class wrapper
for the Ammonite REPL. It makes sure the Helper classes generated by ammonite
are always registered as an outer scope immediately. This way we can
instantiate classes defined inside the Helper class, even when we execute Spark
code as part of the Helper's constructor.
### Why are the changes needed?
When you currently define a class and execute a Spark command using that
class inside the same cell/line this will fail with an NullPointerException.
The reason for that is that we cannot resolve the outer scope needed to
instantiate the class. This PR fixes that issue. The following code will now
execute successfully (include the curly braces):
```scala
{
case class Thing(val value: String)
val r = (0 to 10).map( value => Thing(value.toString) )
spark.createDataFrame(r)
}
```
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
I added more tests to the `ReplE2ESuite`.
### Was this patch authored or co-authored using generative AI tooling?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
