sarutak commented on code in PR #52956:
URL: https://github.com/apache/spark/pull/52956#discussion_r2717105892
##########
common/utils/src/main/scala/org/apache/spark/util/ClosureCleaner.scala:
##########
@@ -61,6 +61,9 @@ private[spark] object ClosureCleaner extends Logging {
cls.getName.contains("$anonfun$")
}
+ private val useCloneBasedClosureCleaner =
+ sys.env.get("SPARK_CLONE_BASED_CLOSURE_CLEANER").isDefined ||
Runtime.version().feature() >= 22
Review Comment:
@pan3793
I noticed there is no easy way to override an environment variable in the
code.
So, I'll test both a system property and environment variable like as
follows .
```
System.getProperty("spark.cloneBasedClosureCleaner.enabled") == "true" ||
System.getenv("SPARK_CLONE_BASED_CLOSURE_CLEANER") == "1" ||
javaVersion >= 22
```
Environment variable is convenient when we add a CI workflow for this new
approach.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
