[
https://issues.apache.org/jira/browse/AIRFLOW-3146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jack updated AIRFLOW-3146:
--------------------------
Description:
Currently the TimeDeltaSensor takes these arguments:
{color:#4070a0} :param delta: time length to wait after execution_date
before succeeding{color}
{color:#4070a0} :type delta: datetime.timedelta{color}{color:#4070a0}
"""{color}{color:#404040}
{color}{color:#555555}@apply_defaults{color}{color:#404040}
{color}{color:#007020}def{color}
{color:#06287e}__init__{color}{color:#404040}({color}{color:#007020}self{color}{color:#404040},{color}
{color:#404040}delta{color}{color:#404040},{color}
{color:#666666}*{color}{color:#404040}args{color}{color:#404040},{color}
{color:#666666}**{color}{color:#404040}kwargs{color}{color:#404040}):{color}{color:#404040}
{color}{color:#007020}super{color}{color:#404040}({color}{color:#404040}TimeDeltaSensor{color}{color:#404040},{color}
{color:#007020}self{color}{color:#404040}){color}{color:#666666}.{color}{color:#06287e}__init__{color}{color:#404040}({color}{color:#666666}*{color}{color:#404040}args{color}{color:#404040},{color}
{color:#666666}**{color}{color:#404040}kwargs{color}{color:#404040}){color}{color:#404040}
{color}{color:#007020}self{color}{color:#666666}.{color}{color:#404040}delta{color}
{color:#666666}={color} {color:#404040}delta{color}
*The problem:*
The operator assumes that the user would like to wait execution_date + delta.
This isn't always the case, The user might want to wait delta after the task
has finished running.
Lets take this example:
{color:#4070a0} 2016-01-01 can only start running on 2016-01-02. The
timedelta here{color}{color:#4070a0} represents the time after the execution
period has closed.{color}
Now, say the task ran for +4 hours.+ So if we set
{code:java}
delta = 5 minutes {code}
it actually doesn't wait at all. The sensor is useless in that senario. What
the user may be intended is :
execution_date + duration to complete task + delta.
Or in simple words: Timestamp of execution end + delta.
*My suggestion:*
Add another Boolean parameter which will choose if the delta is from
execution_date or execution_date + duration of task
was:
Currently the TimeDeltaSensor takes these arguments:
{color:#4070a0} :param delta: time length to wait after execution_date
before succeeding{color}{color:#404040}
{color}{color:#4070a0} :type delta: datetime.timedelta{color}{color:#404040}
{color}{color:#4070a0} """{color}{color:#404040}
{color}{color:#555555}@apply_defaults{color}{color:#404040}
{color}{color:#007020}def{color}
{color:#06287e}__init__{color}{color:#404040}({color}{color:#007020}self{color}{color:#404040},{color}
{color:#404040}delta{color}{color:#404040},{color}
{color:#666666}*{color}{color:#404040}args{color}{color:#404040},{color}
{color:#666666}**{color}{color:#404040}kwargs{color}{color:#404040}):{color}{color:#404040}
{color}{color:#007020}super{color}{color:#404040}({color}{color:#404040}TimeDeltaSensor{color}{color:#404040},{color}
{color:#007020}self{color}{color:#404040}){color}{color:#666666}.{color}{color:#06287e}__init__{color}{color:#404040}({color}{color:#666666}*{color}{color:#404040}args{color}{color:#404040},{color}
{color:#666666}**{color}{color:#404040}kwargs{color}{color:#404040}){color}{color:#404040}
{color}{color:#007020}self{color}{color:#666666}.{color}{color:#404040}delta{color}
{color:#666666}={color} {color:#404040}delta{color}
*The problem:*
The operator assumes that the user would like to wait execution_date + delta.
This isn't always the case, The user might want to wait delta after the task
has finished running.
Lets take this example:
{color:#4070a0} 2016-01-01 can only start running on 2016-01-02. The
timedelta here{color}{color:#404040}
{color}{color:#4070a0} represents the time after the execution period has
closed.{color}
Now, say the task ran for +4 hours.+ So if we set
{code:java}
delta = 5 minutes {code}
it actually doesn't wait at all. The sensor is useless in that senario. What
the user may be intended is :
execution_date + duration to complete task + delta.
Or in simple words: Timestamp of execution end + delta.
*My suggestion:*
Add another Boolean parameter which will choose if the delta is from
execution_date or execution_date + duration of task
> TimeDeltaSensor - add end time stamp
> -------------------------------------
>
> Key: AIRFLOW-3146
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3146
> Project: Apache Airflow
> Issue Type: Wish
> Reporter: jack
> Priority: Minor
>
> Currently the TimeDeltaSensor takes these arguments:
> {color:#4070a0} :param delta: time length to wait after execution_date
> before succeeding{color}
> {color:#4070a0} :type delta: datetime.timedelta{color}{color:#4070a0}
> """{color}{color:#404040}
>
> {color}{color:#555555}@apply_defaults{color}{color:#404040}
> {color}{color:#007020}def{color}
> {color:#06287e}__init__{color}{color:#404040}({color}{color:#007020}self{color}{color:#404040},{color}
> {color:#404040}delta{color}{color:#404040},{color}
> {color:#666666}*{color}{color:#404040}args{color}{color:#404040},{color}
> {color:#666666}**{color}{color:#404040}kwargs{color}{color:#404040}):{color}{color:#404040}
>
> {color}{color:#007020}super{color}{color:#404040}({color}{color:#404040}TimeDeltaSensor{color}{color:#404040},{color}
>
> {color:#007020}self{color}{color:#404040}){color}{color:#666666}.{color}{color:#06287e}__init__{color}{color:#404040}({color}{color:#666666}*{color}{color:#404040}args{color}{color:#404040},{color}
>
> {color:#666666}**{color}{color:#404040}kwargs{color}{color:#404040}){color}{color:#404040}
>
> {color}{color:#007020}self{color}{color:#666666}.{color}{color:#404040}delta{color}
> {color:#666666}={color} {color:#404040}delta{color}
>
> *The problem:*
> The operator assumes that the user would like to wait execution_date + delta.
> This isn't always the case, The user might want to wait delta after the task
> has finished running.
>
> Lets take this example:
> {color:#4070a0} 2016-01-01 can only start running on 2016-01-02. The
> timedelta here{color}{color:#4070a0} represents the time after the
> execution period has closed.{color}
> Now, say the task ran for +4 hours.+ So if we set
> {code:java}
> delta = 5 minutes {code}
> it actually doesn't wait at all. The sensor is useless in that senario. What
> the user may be intended is :
>
> execution_date + duration to complete task + delta.
>
> Or in simple words: Timestamp of execution end + delta.
>
> *My suggestion:*
> Add another Boolean parameter which will choose if the delta is from
> execution_date or execution_date + duration of task
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
