[
https://issues.apache.org/jira/browse/AIRFLOW-4843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16906258#comment-16906258
]
ASF subversion and git services commented on AIRFLOW-4843:
----------------------------------------------------------
Commit 3e2a02751cf890b780bc26b40c7cee7f1f4e0bd9 in airflow's branch
refs/heads/master from Akshesh Doshi
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=3e2a027 ]
[AIRFLOW-4843] Allow orchestration via Docker Swarm (SwarmOperator) (#5489)
* [AIRFLOW-4843] Allow orchestration via Docker Swarm (SwarmOperator)
Add support for running Docker containers via Docker Swarm
which allows the task to run on any machine (node) which
is a part of your Swarm cluster
More details: https://issues.apache.org/jira/browse/AIRFLOW-4843
Built with <3 at Agoda!
> Allow orchestration of tasks with Docker Swarm aka `SwarmOperator`
> ------------------------------------------------------------------
>
> Key: AIRFLOW-4843
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4843
> Project: Apache Airflow
> Issue Type: New Feature
> Components: operators
> Affects Versions: 1.10.3
> Reporter: Akshesh Doshi
> Assignee: Akshesh Doshi
> Priority: Major
> Labels: Docker, docker, orchestration, swarm
>
> Currently, Airflow supports spawning Docker containers for running tasks via
> the {color:#707070}_DockerOperator_{color} but these containers are run on
> the same node as the scheduler.
> It would be helpful for our use-case to be able to spawn these tasks wherever
> resources are available in our Docker Swarm cluster.
>
> This can be achieved by creating a Docker swarm service, waiting for its run
> and removing it after it has completed execution.
> This approach has been suggested/discussed at various places (and implemented
> in Golang for Swarm-cronjob):
> [https://blog.alexellis.io/containers-on-swarm/]
> [https://forums.docker.com/t/running-one-off-commands-in-swarm-containers/42436/3]
> [https://gist.github.com/alexellis/e11321b8fbfc595c208ea3e74bf5e54b]
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
