potiuk commented on issue #53330: URL: https://github.com/apache/airflow/issues/53330#issuecomment-3070507186
The `./scripts/ci/pre_commit/mypy_folder.py providers/apache/pinot` should be run outside of breeze image. All the pre-commits are supposed to be run outside of the image, really. Most of them will also run inside, but this is not guaranteed. Specifically what `run_mypy.sh` script does - it actuallly **starts** the breeze image - using different docker compose project, so while the script is run in the host environment, it will start breeze image and run the mypy command inside it. There is a good reason why we are only mounting selected files and directories to inside the image - namely performance on MacOS. Part of our software chain is installing UI dependencies and compliling assets and it downloads (among others) node_modules and places them inside the sources, thera are also a number of other build artifacts that are generated in airflow sources - and mounting it all to the image is causing quite a lof of problems, for example when the folders are scanned - because MacOS filesystem used to mount files to (essentially running in a Linux VM) breeze container causes a lot of delays and slowdowns of a number of operations, also they are bad for *reproducibility* - because some of the generated folders and fiels (excluded by .gitignore and they are also ignored by .dockerignore - so not part of the CI image) might change the result of tests and starting airflow for example. That's why we only mount selected sources by default -> and Dockerfile.ci is not among those - https://github.com/apache/airflow/blob/main/scripts/ci/docker-compose/local.yml -> this is a default set of files and folders that are mounted. --mount-sources all uses https://github.com/apache/airflow/blob/main/scripts/ci/docker-compose/local-all-sources.yml which essentially mounts all "root" folder to inside the image. Generally spaeaking, if you want to easily see what should be executed "in container" vs. "in host" - all the scripts that are supposed to be executed in container are in the https://github.com/apache/airflow/tree/main/scripts/in_container folder. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
