GitHub user andrewor14 opened a pull request:
https://github.com/apache/spark/pull/8721
[SPARK-10548] [SQL] Fix concurrent SQL executions / branch-1.5
*Note: this is for branch-1.5 only*
This is the same as #8710, but without the fix for SPARK-10563, which I
consider risky to backport into branch-1.5. Note that this patch does change
the inheritance behavior but only in the SQL case, whereas SPARK-10563 changes
the behavior for all uses of Spark.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/andrewor14/spark concurrent-sql-executions-1.5
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/8721.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #8721
----
commit 2171e3db16dde963c733f03c7383d6282d2a0b60
Author: Andrew Or <[email protected]>
Date: 2015-09-10T23:37:55Z
Exclude certain local properties from being inherited
such as, cough cough, the SQL execution ID. This was a problem
because scala's parallel collections spawns threads as children
of the existing threads, causing the execution ID to be inherited
when it shouldn't be.
commit e8c7970883c7c37c28c4e552bef6ee2fd62029bd
Author: Andrew Or <[email protected]>
Date: 2015-09-11T00:45:30Z
Fix remove from Properties + add tests
Because java.util.Properties' remove method takes in an Any
instead of a String, there were some issues with matching the
key's hashCode, so removing was not successful in unit tests.
Instead, this commit fixes it by manually filtering out the keys
and adding them to the child thread's properties.
Conflicts:
core/src/main/scala/org/apache/spark/SparkContext.scala
commit 5b022011282640369e6bf969c31b21dde45c4045
Author: Andrew Or <[email protected]>
Date: 2015-09-11T01:19:28Z
Fix style
Conflicts:
core/src/main/scala/org/apache/spark/SparkContext.scala
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
