vladimirg-db commented on code in PR #59320:
URL: https://github.com/apache/spark/pull/59320#discussion_r4227804063
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala:
##########
@@ -319,18 +319,15 @@ object AnalysisContext {
}
object Analyzer {
- // Configs with bindingPolicy SESSION or NOT_APPLICABLE are retained when
resolving views and
- // SQL UDFs, so that their values propagate from the active session rather
than falling back to
- // Spark defaults. Entries from lazily-loaded modules are recognized once
registered.
+ // Overlays the session-following configs of `existingConf` onto `newConf`,
a conf built to
+ // resolve a view or SQL function body, so their values come from the active
session rather
+ // than Spark defaults. See `SQLConf.filterRetainedResolutionConfigs` for
which configs are
+ // retained; they are memoized on `existingConf`.
def retainResolutionConfigsForAnalysis(
newConf: SQLConf,
existingConf: SQLConf,
createSparkVersion: String = ""): Unit = {
- existingConf.getAllConfs.foreach { case (key, value) =>
- if (key.startsWith("spark.sql.catalog.") ||
SQLConf.isSessionBindingPolicy(key)) {
- newConf.settings.put(key, value)
- }
- }
+ newConf.settings.putAll(existingConf.retainedResolutionConfigs)
Review Comment:
This is a very core change, can we make sure that we have a flag to flip
back to the previous behavior cleanly?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
