Jacob Ferriero created AIRFLOW-4397:
---------------------------------------
Summary: Add GCSUploadSessionCompleteSensor
Key: AIRFLOW-4397
URL: https://issues.apache.org/jira/browse/AIRFLOW-4397
Project: Apache Airflow
Issue Type: New Feature
Components: contrib
Reporter: Jacob Ferriero
Assignee: Jacob Ferriero
I'd like to contribute a Sensor for Google Cloud Storage that can poke a bucket
until there has been sufficient time without a new file drop. Often times there
are cases where a third party vendor drops data to a bucket but don't send a
success flag when they are done. This sensor would allow you to poke every n
minutes to check if more files have been added since the last poke, and if
there had been `inactivity_period` minutes without a new file drop, return
`True`. This could allow SLA misses if data did not arrive by an expected time,
and have a configurable deadline past which the sensor would fail. Optionally
the user could specify a minimum number of files for the sensor to succeed.
This would be my first time contributing to an OSS project, so please let me
know if this is not the appropriate place to start.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
