jack created AIRFLOW-3146:
-----------------------------
Summary: TimeDeltaSensor - consider also end time stamp
Key: AIRFLOW-3146
URL: https://issues.apache.org/jira/browse/AIRFLOW-3146
Project: Apache Airflow
Issue Type: Wish
Reporter: jack
Currently the TimeDeltaSensor takes these arguments:
{color:#4070a0} :param delta: time length to wait after execution_date
before succeeding{color}{color:#404040}
{color}{color:#4070a0} :type delta: datetime.timedelta{color}{color:#404040}
{color}{color:#4070a0} """{color}{color:#404040}
{color}{color:#555555}@apply_defaults{color}{color:#404040}
{color}{color:#007020}def{color}
{color:#06287e}__init__{color}{color:#404040}({color}{color:#007020}self{color}{color:#404040},{color}
{color:#404040}delta{color}{color:#404040},{color}
{color:#666666}*{color}{color:#404040}args{color}{color:#404040},{color}
{color:#666666}**{color}{color:#404040}kwargs{color}{color:#404040}):{color}{color:#404040}
{color}{color:#007020}super{color}{color:#404040}({color}{color:#404040}TimeDeltaSensor{color}{color:#404040},{color}
{color:#007020}self{color}{color:#404040}){color}{color:#666666}.{color}{color:#06287e}__init__{color}{color:#404040}({color}{color:#666666}*{color}{color:#404040}args{color}{color:#404040},{color}
{color:#666666}**{color}{color:#404040}kwargs{color}{color:#404040}){color}{color:#404040}
{color}{color:#007020}self{color}{color:#666666}.{color}{color:#404040}delta{color}
{color:#666666}={color} {color:#404040}delta{color}
*The problem:*
The operator assumes that the user would like to wait execution_date + delta.
This isn't always the case, The user might want to wait delta after the task
has finished running.
Lets take this example:
{color:#4070a0} 2016-01-01 can only start running on 2016-01-02. The
timedelta here{color}{color:#404040}
{color}{color:#4070a0} represents the time after the execution period has
closed.{color}
Now, say the task ran for +4 hours.+ So if we set
{code:java}
delta = 5 minutes {code}
it actually doesn't wait at all. The sensor is useless in that senario. What
the user may be intended is :
execution_date + duration to complete task + delta.
Or in simple words: Timestamp of execution end + delta.
*My suggestion:*
Add another Boolean parameter which will choose if the delta is from
execution_date or execution_date + duration of task
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
