[
https://issues.apache.org/jira/browse/SPARK-44458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kapil Singh updated SPARK-44458:
--------------------------------
Attachment: (was: image-2023-07-17-16-57-09-669.png)
> ThreadLocal not being copied to child thread when child thread is reused from
> pool
> -----------------------------------------------------------------------------------
>
> Key: SPARK-44458
> URL: https://issues.apache.org/jira/browse/SPARK-44458
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 3.4.1
> Reporter: Kapil Singh
> Priority: Major
>
> Currently setting the Spark configuration using the statement
> `spark.conf.set("spark.sql.caseSensitive", "true")` and subsequently
> executing a Spark action in a separate thread. An intermittent observation
> reveals that occasionally the configuration set in the main thread fails to
> propagate to the child thread
> Steps to repro: [Spark SQL configs can't get propagated properly to a new
> thread in Spark 3.1 issue ·
> GitHub|https://gist.github.com/t-rufang/25341b3678e5d7c74e3a209457fce0e9]
>
> This is limitation due to ThreadLocal not being copied to child thread when
> child thread is reused from pool and only done instead in thread creation or
> when idle thread has expired KeepAlive in Executor
> Refer:
> [https://users.scala-lang.org/t/future-executioncontext-and-threadlocal/7675/2]
>
> [https://www.stevenskelton.ca/threadlocal-variables-scala-futures/]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
