[
https://issues.apache.org/jira/browse/SPARK-44458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kapil Singh updated SPARK-44458:
--------------------------------
Attachment: image-2023-07-17-16-56-59-388.png
> ThreadLocal not being copied to child thread when child thread is reused from
> pool
> -----------------------------------------------------------------------------------
>
> Key: SPARK-44458
> URL: https://issues.apache.org/jira/browse/SPARK-44458
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 3.4.1
> Reporter: Kapil Singh
> Priority: Major
> Attachments: image-2023-07-17-16-56-59-388.png,
> image-2023-07-17-16-57-09-669.png
>
>
> Currently setting the Spark configuration using the statement
> `spark.conf.set("spark.sql.caseSensitive", "true")` and subsequently
> executing a Spark action in a separate thread. An intermittent observation
> reveals that occasionally the configuration set in the main thread fails to
> propagate to the child thread.
> !image-2023-07-17-16-54-26-372.png!
> !image-2023-07-17-16-54-36-860.png!
>
> This is limitation due to ThreadLocal not being copied to child thread when
> child thread is reused from pool and only done instead in thread creation or
> when idle thread has expired KeepAlive in Executor
> Refer:
> [https://users.scala-lang.org/t/future-executioncontext-and-threadlocal/7675/2]
>
> [https://www.stevenskelton.ca/threadlocal-variables-scala-futures/]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
