[
https://issues.apache.org/jira/browse/BEAM-2026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15977210#comment-15977210
]
Luke Cwik commented on BEAM-2026:
---------------------------------
The python equivalent of the java direct runner is currently not in a separate
package and is just part of the sdks/python package.
https://github.com/apache/beam/tree/master/sdks/python/apache_beam/runners
The Java portion of project was broken up to help with dependency issues,
runner selection, etc...
I'm assuming that at some point in time the python packages will get broken up
as well as more runners are added and the dependency system becomes convoluted
by embedding too many different runners within the same package.
> High performance direct runner
> ------------------------------
>
> Key: BEAM-2026
> URL: https://issues.apache.org/jira/browse/BEAM-2026
> Project: Beam
> Issue Type: New Feature
> Components: runner-direct
> Reporter: Mitar
> Assignee: Mitar
>
> In documentation (https://beam.apache.org/documentation/runners/direct/) it
> is written that direct runner does not try to run efficiently, but it serves
> mostly for development and debugging.
> I would suggest that there should be also an efficient direct runner. If Beam
> tries to be an unified programming model, for some smaller tasks I would love
> to implement them in Beam, just to keep the code in the same model, but it
> would be OK to run it as a normal smaller program (maybe inside one Docker
> container), without any distribution across multiple machines. In the future,
> if usage grows, I could then replace underlying runner with something
> distributed.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
