kaxil opened a new pull request, #74016:
URL: https://github.com/apache/airflow/pull/74016
The `check-conn-id-templated` hook fails intermittently in Static checks
with `FileNotFoundError` on a `__pycache__` directory under `providers/`, for
example:
```
FileNotFoundError: [Errno 2] No such file or directory:
'.../providers/cncf/kubernetes/src/airflow/providers/cncf/__pycache__'
```
The hook walks the whole providers tree with `Path.rglob("*.py")`. On Python
3.10, which the hook runs on, `rglob` lists a directory's subdirectories first
and reads each one later, and it only tolerates `PermissionError` on that read.
Other processes create and remove `__pycache__` directories under `providers/`
while static checks run, so a directory that disappears in between crashes the
hook. Python 3.12's `rglob` already copes with this, which is why it does not
reproduce locally on a newer interpreter.
The walk now uses `os.walk`, which skips a directory that has gone, and
prunes `__pycache__` and `tests` directories instead of descending into them
and filtering paths afterwards. The new test removes a sibling directory after
the walk has listed it; it raises the same `FileNotFoundError` against the old
code on 3.10.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
