ashb commented on a change in pull request #4890: [AIRFLOW-4048] HttpSensor
provide-context to response_check
URL: https://github.com/apache/airflow/pull/4890#discussion_r292970603
##########
File path: airflow/sensors/http_sensor.py
##########
@@ -81,8 +89,10 @@ def poke(self, context):
headers=self.headers,
extra_options=self.extra_options)
if self.response_check:
- # run content check on response
- return self.response_check(response)
+ if self.provide_context:
+ return self.response_check(response, context)
Review comment:
This behaviour doesn't match the description - (keyword-args) for the
description to be true this line should be
```
return self.response_check(response, **context)
```
This diff here is probably the right thing as it resembles what the
provide_context arg does for the PythonOperator.
An example in the docs would help clarify this:
```rst
Executes a HTTP GET statement and returns False on failure caused by
404 Not Found or `response_check` returning False.
HTTP Error codes other than 404 (like 403) or Connection Refused Error
would fail the sensor itself directly (no more poking).
The response check can access the template context by passing
``provide_context=True`` to the operator::
def response_check(response, **context):
# Can look at context['ti'] etc.
return True
HttpSensor(task_id='my_http_sensor', ..., provide_context=True,
response_check=response_check)
:param http_conn_id: The connection to run the sensor against
:type http_conn_id: str
:param method: The HTTP request method to use
:type method: str
:param endpoint: The relative part of the full url
:type endpoint: str
:param request_params: The parameters to be added to the GET url
:type request_params: a dictionary of string key/value pairs
:param headers: The HTTP headers to be added to the GET request
:type headers: a dictionary of string key/value pairs
:param provide_context: if set to true, Airflow will pass a set of
keyword arguments that can be used in your function. This set of
kwargs correspond exactly to what you can use in your jinja
templates.
:type provide_context: bool
:param response_check: A check against the 'requests' response object.
Returns True for 'pass' and False otherwise.
:type response_check: A lambda or defined function.
:param extra_options: Extra options for the 'requests' library, see the
'requests' documentation (options to modify timeout, ssl, etc.)
:type extra_options: A dictionary of options, where key is string and
value
depends on the option that's being modified.
```
Probably want to format the example a bit better (line wrap etc.)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
