Vanessa created AIRFLOW-4030:
--------------------------------
Summary: Add Singularity Container Operator
Key: AIRFLOW-4030
URL: https://issues.apache.org/jira/browse/AIRFLOW-4030
Project: Apache Airflow
Issue Type: Improvement
Components: operators
Reporter: Vanessa
Assignee: Vanessa
Singularity containers are comparable to Docker in the level of operation -
they provide an encapsulated environment with an OS, libraries, and custom
software for the user to run. The key difference is that Docker is not
optimized for scientific compute because it could never be installed on a
shared research cluster. Singularity, on the other hand, does not have these
issues and is installed across HCP centers internationally.
This issue is to add Singularity containers as an operator to Apache Airflow,
so that we can start to explore using airflow in an HPC environment. I work
with Encode DCC at Stanford, and am hopeful to explore Airflow as an
alternative to the workflow manager(s) we are using. I am one of the [original
Singularity developers see
|https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0177459]
that manages the Singularity Python client (spython), Singularity Hub and
Singularity Registry Server, and have started working on this issue here:
[https://github.com/apache/airflow/pull/4846.] Looking forward to working with
you!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
