Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/2624#issuecomment-57718816
It looks like SparkEnv has two "getter" methods: `get`, which returns
either the ThreadLocal value or the last SparkEnv set _by any thread_, and
`getThreadLocal`, which just reads the current ThreadLocal.
It turns out that there's no code which calls `getThreadLocal`. We _do_
call `SparkEnv.set()` in a few places, but we seem to always set it using
either a new SparkEnv (e.g. when starting a SparkContext or Executor) or using
the value obtained from `SparkEnv.get()`. Therefore, it seems like SparkEnv
effectively acts as a global object. Why not just remove the ThreadLocals and
have `get` just return some `@volatile` field in SparkEnv that holds the
current instance (a sort of Singleton pattern)?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
