Brylie Christopher Oxley created AIRFLOW-3379:
-------------------------------------------------
Summary: Support for AWS triggers (S3, SNS, SQS, CloudWatch, etc.)
Key: AIRFLOW-3379
URL: https://issues.apache.org/jira/browse/AIRFLOW-3379
Project: Apache Airflow
Issue Type: Improvement
Components: aws
Reporter: Brylie Christopher Oxley
We would like to build a 'reactive', or event-driven, data pipeline. From what
I can gather, the primary Airflow DAG trigger is a timer (cron). However, it
would be useful to trigger dags on external events, namely AWS events like S3
file, SQS, SNS, and/or CloudWatch.
I note there is an experimental API, which could be triggered from an AWS
Lambda, but would add boilerplate and brittleness to the data pipeline.
What are our options for triggering Airflow DAGS from external AWS events?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
