[
https://issues.apache.org/jira/browse/BEAM-5336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17542605#comment-17542605
]
Brian Hulette edited comment on BEAM-5336 at 5/26/22 5:38 PM:
--------------------------------------------------------------
The [Runner Guide|https://beam.apache.org/contribute/runner-guide/] would be
the best place to start for reference material, in particular the information
about building an
[SDK-independent|https://beam.apache.org/contribute/runner-guide/#writing-an-sdk-independent-runner]
(what we usually call "portable") runner. As far as getting this off the
ground, you might start by experimenting with the Python SDK's
[FnRunner|https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner.py]
which is a portable runner implemented in Python. You might try forking that
and modifying it to send work to Dask rather than running it locally. I'd also
highly recommend asking for advice on this on the [email protected] list,
lots of knowledgeable folks monitor that list and can give you helpful
high-level advice, and address any specific questions that come up. Note you
need to be [subscribed|https://beam.apache.org/community/contact-us/] to be
able to write to dev@.
was (Author: bhulette):
The [Runner Guide|https://beam.apache.org/contribute/runner-guide/] would be
the best place to start for reference material, in particular the information
about building an
[SDK-independent|https://beam.apache.org/contribute/runner-guide/#writing-an-sdk-independent-runner]
(what we usually call "portable") runner. As far as getting this off the
ground, you might start by experimenting with the Python SDK's
[FnRunner|https://github.com/apache/beam/blob/master/sdks/python/apache_beam/runners/portability/fn_api_runner/fn_runner.py]
which is a portable runner implemented in Python. You might try forking that
and modifying it to send work to Dask rather than running it locally. I'd also
highly recommend asking fort advice on this on the [email protected] list,
lots of knowledgeable folks monitor that list and can give you helpful
high-level advice, and address any specific questions that come up. Note you
need to be [subscribed|https://beam.apache.org/community/contact-us/] to be
able to write to dev@.
> Support for a Dask runner
> -------------------------
>
> Key: BEAM-5336
> URL: https://issues.apache.org/jira/browse/BEAM-5336
> Project: Beam
> Issue Type: Wish
> Components: runner-ideas
> Environment: Python
> Reporter: Georvic Tur
> Priority: P4
>
> Adding support for a Dask runner is currently under consideration?
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
