jroachgolf84 opened a new pull request, #51893:
URL: https://github.com/apache/airflow/pull/51893
Previously, running `HttpHook.run()` altered the dictionary that was passed
via `extra_options`. This was illustrated and reproduced in #51686 using the
following logic within a Task.
```
...
extra_options = {"verify_ssl": None, "proxies": None}
original_extra_options = deepcopy(extra_options)
# jsonplaceholder Connection has
# * Host = https://jsonplaceholder.typicode.com
# * Extra = {"foo": "bar"}
# Note: some Extra value is necessary, otherwise the extra_options passed to
run are not used, see https://github.com/apache/airflow/issues/51685
http_hook = HttpHook(method='GET', http_conn_id='jsonplaceholder')
for _ in range(2):
http_hook.run(
endpoint=f"/posts/1",
extra_options=extra_options,
)
assert extra_options == original_extra_options, \
f"extra_options: {extra_options}, original_extra_options:
{original_extra_options}"
```
To fix this, two major changes were made. First, the dictionary that was
passed into hook via the `run` or `get_conn` method was "deep copied" before
being transformed. Since the previous logic relied on that dictionary being
transformed in different scopes, a `merged_extra` was added as an
instance-level attribute to the `HttpHook` class. The appropriate changes were
also made to its async counterpart.
These changes were tested using the logic that was provided by @brki, as
well as via the unit tests for this hook.
closes: #51686
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
