Taragolis commented on PR #39799: URL: https://github.com/apache/airflow/pull/39799#issuecomment-2129045571
I guess the main prom that there is no standardisation for [`no_proxy`](https://about.gitlab.com/blog/2021/01/27/we-need-to-talk-no-proxy/) and different libraries / os could work differently Most of the libraries use stdlib implementation from [Lib/urllib/request.py](https://github.com/python/cpython/tree/3.12/Lib/urllib/request.py) Request use own implementation which supports CIDR blocks, but only for ipv4: - https://github.com/psf/requests/issues/6313 ```python import os os.environ["no_proxy"] = "localhost,127.0.0.1,10.0.0.0/8,172.16.1.*,1:2:3::/64" os.environ["NO_PROXY"] = "localhost,127.0.0.1,192.168.1.0/16" # stdlib implementation from urllib.request import proxy_bypass_environment # requests implementation from requests.sessions import should_bypass_proxies hosts = ("localhost", "127.0.0.1", "127.0.0.2", "10.0.0.0", "192.168.1.32", "172.16.1.2", "[1:2:3::4]") for host in hosts: print(f" {host} ".center(80, '=')) print(f"stdlib Proxy Bypass Environment: {proxy_bypass_environment(host, None)}") print(f"requests Proxy Bypass Environment: {should_bypass_proxies(f"http://{host}";, None)}") ``` ```console ================================== localhost =================================== stdlib Proxy Bypass Environment: True requests Proxy Bypass Environment: True ================================== 127.0.0.1 =================================== stdlib Proxy Bypass Environment: True requests Proxy Bypass Environment: True ================================== 127.0.0.2 =================================== stdlib Proxy Bypass Environment: False requests Proxy Bypass Environment: False =================================== 10.0.0.0 =================================== stdlib Proxy Bypass Environment: False requests Proxy Bypass Environment: True ================================= 192.168.1.32 ================================= stdlib Proxy Bypass Environment: False requests Proxy Bypass Environment: False ================================== 172.16.1.2 ================================== stdlib Proxy Bypass Environment: False requests Proxy Bypass Environment: False ================================== [1:2:3::4] ================================== stdlib Proxy Bypass Environment: False requests Proxy Bypass Environment: False ``` I don't know what is supposed to be happen in case if we change library in the future, e.g. from `requests` to `urllib3`, or use `aiohttp` in case if we turn internal code to async implementation (`asyncio` / `anyio` / `trio`) So probably it is fine for replace `httpx` by `requests`, and I'm pretty sure that in past this was called by requests before we temporary remove usage of requests due to licence issue: https://github.com/apache/airflow/pull/15781. However there is **should not be a commitment that this behaviour remaining the same** in the future, because this part of the code usually can't be changed without monkey patching third-party library / stdlib -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
