dongjoon-hyun commented on a change in pull request #24540:
[SPARK-27253][SQL][FOLLOW-UP] Add a legacy flag to restore old session init
behavior.
URL: https://github.com/apache/spark/pull/24540#discussion_r281397523
##########
File path: core/src/main/scala/org/apache/spark/internal/config/package.scala
##########
@@ -1309,4 +1309,10 @@ package object config {
.bytesConf(ByteUnit.BYTE)
.createOptional
+ private[spark] val LEGACY_SESSION_INIT_WITH_DEFAULTS =
+ ConfigBuilder("spark.legacy.sessionInitWithConfigDefaults")
+ .doc("Flag to revert to legacy behavior where a cloned SparkSession
receives SparkConf " +
+ "defaults, dropping any overrides in its parent SparkSession.")
+ .booleanConf
+ .createWithDefault(false)
Review comment:
And, can we move this to `StaticSQLConf.scala` inside `sql/catalyst` module
instead of this `core` module?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
