paramesh opened a new issue #10082:
URL: https://github.com/apache/airflow/issues/10082
Issue:
I am facing issues when trying to use Reschedule with the http_sensor.
The following works fine without issues.
```
task_http_sensor_check_1 = HttpSensor(
task_id='http_sensor_check_1',
http_conn_id='stub_default',
endpoint='/api/v1/products',
request_params={},
response_check=lambda response: "new185" in response.text,
poke_interval=5,
dag=dag)
```
If I just add reschedule to this, I see this just run into a continuous
retry loop.
```
task_http_sensor_check_1 = HttpSensor(
task_id='http_sensor_check_1',
http_conn_id='stub_default',
endpoint='/api/v1/products',
request_params={},
response_check=lambda response: "new185" in response.text,
poke_interval=5,
**mode='reschedule',**
dag=dag)
```
The above one just runs into indefinite loop.
My Dag is quite simple.
```
default_args = {
'owner': 'Paramesh',
'depends_on_past': True,
'start_date': datetime(2020, 7, 5),
'email': '[email protected]',
'email_on_failure': False,
'email_on_retry': False,
'retries': 10,
'retry_delay': timedelta(seconds=30),
}
# Catch-up is set to false as we don't want any backfill.
dag = DAG('simplelineardagnovar185', schedule_interval='*/7 * * * *',
default_args=default_args, catchup=False)
POST_PRODUCTS = SimpleHttpOperator(
task_id='POST_PRODUCTS',
method='POST',
http_conn_id='stub_default',
endpoint='/api/v1/products',
data=json.dumps(
{"description": "new185", "name": "product3", "price": "500.0", "sleep":
"300000"}),
headers={"Content-Type": "application/json"},
xcom_push=True,
dag=dag)
task_http_sensor_check_1 = HttpSensor(
task_id='http_sensor_check_1',
http_conn_id='stub_default',
endpoint='/api/v1/products',
request_params={},
response_check=lambda response: "new185" in response.text,
mode='reschedule',
poke_interval=5,
dag=dag)
DELETE_PRODUCTS = SimpleHttpOperator(
task_id='DELETE_PRODUCTS',
method='DELETE',
http_conn_id='stub_default',
endpoint='/api/v1/products/desc/new185',
headers={"Content-Type": "application/json"},
dag=dag)
POST_PRODUCTS >> task_http_sensor_check_1 >> DELETE_PRODUCTS
```
The POST_PRODUCTS will post an async request and that will takes ~5 min to
get committed to DB and the http_sensor_check just do the get to identify
whether the POST is successful.
Airflow version used: 1.10.11
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
