Kyle Weaver created BEAM-7600:
---------------------------------
Summary: Spark portable runner: reuse SDK harness
Key: BEAM-7600
URL: https://issues.apache.org/jira/browse/BEAM-7600
Project: Beam
Issue Type: Improvement
Components: runner-spark
Reporter: Kyle Weaver
Assignee: Kyle Weaver
Right now, we're creating a new SDK harness every time an executable stage is
run [1], which is expensive. We should be able to re-use code from the Flink
runner to re-use the SDK harness [2].
[1]
[https://github.com/apache/beam/blob/c9fb261bc7666788402840bb6ce1b0ce2fd445d1/runners/spark/src/main/java/org/apache/beam/runners/spark/translation/SparkExecutableStageFunction.java#L135]
[2]
[https://github.com/apache/beam/blob/c9fb261bc7666788402840bb6ce1b0ce2fd445d1/runners/flink/src/main/java/org/apache/beam/runners/flink/translation/functions/FlinkDefaultExecutableStageContext.java]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
