Kapil Singh created SPARK-44458:
-----------------------------------
Summary: ThreadLocal not being copied to child thread when child
thread is reused from pool
Key: SPARK-44458
URL: https://issues.apache.org/jira/browse/SPARK-44458
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 3.4.1
Reporter: Kapil Singh
Currently setting the Spark configuration using the statement
`spark.conf.set("spark.sql.caseSensitive", "true")` and subsequently executing
a Spark action in a separate thread. An intermittent observation reveals that
occasionally the configuration set in the main thread fails to propagate to the
child thread.
!image-2023-07-17-16-54-26-372.png!
!image-2023-07-17-16-54-36-860.png!
This is limitation due to ThreadLocal not being copied to child thread when
child thread is reused from pool and only done instead in thread creation or
when idle thread has expired KeepAlive in Executor
Refer:
[https://users.scala-lang.org/t/future-executioncontext-and-threadlocal/7675/2]
[https://www.stevenskelton.ca/threadlocal-variables-scala-futures/]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
