[ https://issues.apache.org/jira/browse/SPARK-27253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16799574#comment-16799574 ]
Chakravarthi commented on SPARK-27253: -------------------------------------- Thanks for reporting ,will be working on this . > SparkSession clone discards SQLConf overrides in favor of SparkConf defaults > ---------------------------------------------------------------------------- > > Key: SPARK-27253 > URL: https://issues.apache.org/jira/browse/SPARK-27253 > Project: Spark > Issue Type: Bug > Components: Structured Streaming > Affects Versions: 2.4.0 > Reporter: Jose Torres > Priority: Major > > SparkSession.cloneSession() is normally supposed to create a child session > which inherits all the SQLConf values of its parent session. But when a SQL > conf is given a global default through the SparkConf, this does not happen; > the child session will receive the SparkConf default rather than its parent's > SQLConf override. > > This is particularly impactful in structured streaming, as the microbatches > run in a cloned child session. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org