cloud-fan commented on PR #56437: URL: https://github.com/apache/spark/pull/56437#issuecomment-4677207275
@pan3793 Good question. First, SESSION and NOT_APPLICABLE behave identically at runtime -- the Analyzer retains both kinds when resolving views/UDFs, so the active session's value applies either way. The difference is declared intent: NOT_APPLICABLE says the config is never consulted during view/UDF/procedure resolution at all (physical planning, runtime, transport, etc.); SESSION says it is consulted and the caller's session value should win. PERSISTED is the only policy that actually changes behavior: the value captured at creation (or the Spark default, if none was saved) is used instead of the caller's. The mental model: use PERSISTED when the view/UDF/procedure should use the create-time value of the config -- think ANSI mode or session timezone, where a persisted object should keep computing the same result as it did at creation regardless of who calls it. Changing query behavior alone is not enough: a bug-fix flag also changes query behavior, but we don't want views to freeze the buggy behavior, so it gets SESSION. Agreed that by this rule `spark.sql.window.segmentTree.enabled` would more precisely be NOT_APPLICABLE -- since it's runtime-identical to SESSION there's no behavior difference, just labeling. I've also added this decision rule to the `ConfigBindingPolicy` scaladoc in this PR so it's documented where authors pick the policy. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
