GitHub user ericl opened a pull request:
https://github.com/apache/spark/pull/12248
[SPARK-14475] Propagate user-defined context from driver to executors
## What changes were proposed in this pull request?
This adds a new API call `TaskContext.getLocalProperty` for getting
properties set in the driver from executors. These local properties are
automatically propagated from the driver to executors. For streaming, the
context for streaming tasks will be the initial driver context when ssc.start()
is called.
## How was this patch tested?
Unit tests.
cc @JoshRosen
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ericl/spark sc-2813
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/12248.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #12248
----
commit a8b07f1163a92072c1bc71cf63f804df1f430060
Author: Eric Liang <[email protected]>
Date: 2016-04-07T22:30:39Z
Thu Apr 7 15:30:39 PDT 2016
commit 9104f0cd77c30dd0acf8e2a1c53e9b57e9241c02
Author: Eric Liang <[email protected]>
Date: 2016-04-07T23:33:56Z
tests
commit b2fc541368e30ffd455c068b709d68251d09abd5
Author: Eric Liang <[email protected]>
Date: 2016-04-07T23:38:01Z
Thu Apr 7 16:38:01 PDT 2016
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
