Yishan created AIRFLOW-5633:
-------------------------------
Summary: Apache Airflow Integration with HashiCorp Nomad
Key: AIRFLOW-5633
URL: https://issues.apache.org/jira/browse/AIRFLOW-5633
Project: Apache Airflow
Issue Type: New Feature
Components: executors, operators
Affects Versions: 1.10.5
Reporter: Yishan
Nomad is an orchestrator, similar to Kubernetes and used in production by
companies like PagerDuty, Target, Citadel, Trivago, Pandora, Roblox, etc.
Built by HashiCorp (company behind other tools such as Terraform, Vault, and
Consul).
We've gotten many requests from our customers and open source community for an
integration with Airflow. There are a variety of community PRs from supportive
members who are trying to build their own (linked notable ones below) - our
approach would be to build one officially supported integration ourselves for
all our users and customers.
[https://github.com/apache/airflow/pull/2708]
[https://github.com/hashicorp/nomad/issues/419]
We'd love to understand how much interest and support we can gauge from the
Airflow committee around an integration like this. Ideally, we'd aim to avoid
a situation where our integration is not accepted + Nomad users using Airflow
are subject to an incomplete/less-than-ideal experience on a lagging
HashiCorp-maintained-fork.
This has unfortunately happened with our Spark integration, so we'd love to
gauge the Airflow committee early-on in the process before we potentially build
the integration.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
