Github user andrewor14 commented on a diff in the pull request:
https://github.com/apache/spark/pull/8710#discussion_r39296051
--- Diff: core/src/main/scala/org/apache/spark/SparkContext.scala ---
@@ -348,10 +348,27 @@ class SparkContext(config: SparkConf) extends Logging
with ExecutorAllocationCli
// Thread Local variable that can be used by users to pass information
down the stack
private val localProperties = new InheritableThreadLocal[Properties] {
- override protected def childValue(parent: Properties): Properties =
new Properties(parent)
--- End diff --
I agree, I was actually going to do it in a separate patch. Incidentally
@tdas @JoshRosen and I just talked about this last night and we all agreed to
make it do a clone instead so the semantics are simpler.
However, my one concern is that doing so will change semantics for non-SQL
users in 1.5.1, so my proposal is the following: I will make the changes in
this patch and merge this patch ONLY into master. Then I'll create a new patch
for branch 1.5 that will have the current changes (the ones where we don't
clone except for SQL). I think that's the safest way forward.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
