This is an automated email from the ASF dual-hosted git repository.
uros-b pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new df23d1599157 [SPARK-57548][SQL] Avoid eager
scala.reflect.runtime.universe initialization in ScalaReflection
df23d1599157 is described below
commit df23d1599157562b9c6ab7ed195733da047e7fc6
Author: Emil Ejbyfeldt <[email protected]>
AuthorDate: Sun Aug 23 13:36:33 2026 +0000
[SPARK-57548][SQL] Avoid eager scala.reflect.runtime.universe
initialization in ScalaReflection
### What changes were proposed in this pull request?
Make initializaiton of universe in ScalaReflection lazy. This is so that
one can more easily use Spark3 from a Scala3 codebase.
An alternative approach could be to move the method
`encodeFieldNameToIdentifier` and `findConstructor` so that the universe would
only be intilizalized if needed.
These to methods uses scala reflection even if using Scala3 since they are
used from NewInstance.
### Why are the changes needed?
Because of this issue scala/scala3#25896 the change top make it possible to
use Spark from Scala 3.8.0 or later.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Existing tests ensure no regressions. That it makes the code more
compatible with Scala 3.8 was done by running a local build against the tests
in https://github.com/unum-io/tyda (a Scala 3 library that can use Spark as a
backend).
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #57303 from eejbyfeldt/SPARK-57548.
Authored-by: Emil Ejbyfeldt <[email protected]>
Signed-off-by: Uros Bojanic <[email protected]>
(cherry picked from commit 7df3ac405cf7dded626fcac4a7d45d59b93a8ebd)
Signed-off-by: Uros Bojanic <[email protected]>
---
.../main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git
a/sql/api/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
b/sql/api/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
index 6f5c4be42bbd..60a5a597d337 100644
--- a/sql/api/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
+++ b/sql/api/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala
@@ -40,7 +40,7 @@ private[catalyst] object ScalaSubtypeLock
* A default version of ScalaReflection that uses the runtime universe.
*/
object ScalaReflection extends ScalaReflection {
- val universe: scala.reflect.runtime.universe.type =
scala.reflect.runtime.universe
+ lazy val universe: scala.reflect.runtime.universe.type =
scala.reflect.runtime.universe
// Since we are creating a runtime mirror using the class loader of current
thread,
// we need to use def at here. So, every time we call mirror, it is using the
// class loader of the current thread.
@@ -218,7 +218,7 @@ object ScalaReflection extends ScalaReflection {
}
def encodeFieldNameToIdentifier(fieldName: String): String = {
- TermName(fieldName).encodedName.toString
+ scala.reflect.NameTransformer.encode(fieldName)
}
/**
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
