[
https://issues.apache.org/jira/browse/BEAM-313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15307738#comment-15307738
]
ASF GitHub Bot commented on BEAM-313:
-------------------------------------
GitHub user amarouni opened a pull request:
https://github.com/apache/incubator-beam/pull/401
[BEAM-313] Enable the use of an existing spark context with the
SparkPipelineRunner
The general use case is that the SparkPipelineRunner creates its own Spark
context and uses it for the pipeline execution.
Another alternative is to provide the SparkPipelineRunner with an existing
spark context. This can be interesting for a lot of use cases where the Spark
context is managed outside of beam (context reuse, advanced context management,
spark job server, ...).
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/amarouni/incubator-beam mycbeam313
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-beam/pull/401.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #401
----
commit fec04dee06cad9bb6d8914c6ba027a5574c73e41
Author: Abbass MAROUNI <[email protected]>
Date: 2016-05-31T12:45:32Z
[BEAM-313] Enable the use of an existing spark context with the
SparkPipelineRunner
----
> Enable the use of an existing spark context with the SparkPipelineRunner
> ------------------------------------------------------------------------
>
> Key: BEAM-313
> URL: https://issues.apache.org/jira/browse/BEAM-313
> Project: Beam
> Issue Type: New Feature
> Reporter: Abbass Marouni
> Assignee: Jean-Baptiste Onofré
>
> The general use case is that the SparkPipelineRunner creates its own Spark
> context and uses it for the pipeline execution.
> Another alternative is to provide the SparkPipelineRunner with an existing
> spark context. This can be interesting for a lot of use cases where the Spark
> context is managed outside of beam (context reuse, advanced context
> management, spark job server, ...).
> code sample :
> https://github.com/amarouni/incubator-beam/commit/fe0bb517bf0ccde07ef5a61f3e44df695b75f076
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
