Piyush00715 commented on issue #74469:
URL: https://github.com/apache/airflow/issues/74469#issuecomment-6141451121
Update on my comment above: I checked further and the quoting direction is
wrong — please disregard it. `httpx.URL.path` returns the decoded path and
`WSGITransport` passes it verbatim as WSGI `PATH_INFO`, so pre-quoting client
URLs converges to the same value and cannot fix this crash (real HTTP
deployments were never broken; httpx percent-encodes on the wire).
The actual fault is in the in-process harness used by `DAG.test()`:
`InProcessExecutionAPI.transport`
(`airflow-core/src/airflow/api_fastapi/execution_api/in_process.py`) chains
`httpx.WSGITransport → a2wsgi ASGIMiddleware → FastAPI`. httpx supplies decoded
Unicode as `PATH_INFO`, violating PEP 3333, and a2wsgi's
`encode("latin1").decode("utf8")` then raises the exact `UnicodeDecodeError:
byte 0xf8` from the traceback.
I have a local fix ready: a small PEP 3333 normalization wrapper around the
WSGI app in `transport` (UTF-8 → latin1 re-encode; identity for ASCII), plus
regression tests in `tests/unit/api_fastapi/execution_api/test_in_process.py`.
Verified the mechanism against real httpx 0.28.1 (raw transport reproduces the
crash, wrapped serves 200 with the exact path). I could not run the repo suite
here (no Breeze/Docker Linux env on this machine) — CI would need to confirm.
Does a transport-level fix in `in_process.py` sound like the right layer, or
would you rather handle it elsewhere? Happy to open a PR once confirmed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
