mik-laj opened a new pull request #19737: URL: https://github.com/apache/airflow/pull/19737
The current tests for the Docker image are written in bash, which unfortunately makes them limited and difficult to maintain. They don't always find all the problems they could find if they were written in Python. For example, when we added MSSQL support to the Docker image, we don't update the list of preinstalled packages: https://github.com/apache/airflow/blob/main/scripts/ci/libraries/_initialization.sh#L419-L442 When a test is written in Python, it is easier to write a valid test because Python supports comparing complex structures like `set == set`. Thanks to `Pytest-xdist`, the tests can be executed in many threads, which significantly speeds them up. Instead of 3 m 30 sec, it now takes 1 minute on my MacBook Pro. And we don't need any specialist GNU parallels for that. In the future, I would like to rewrite [the tests for image building examples](https://github.com/apache/airflow/blob/1b89e682a1a04adf2cf5ebb37a453e245a6e7191/scripts/ci/images/ci_test_examples_of_prod_image_building.sh#L1) in a similar way, so that they can be run just as easily. In addition, I also hope to contribute [tests for docker-compose files](https://github.com/mik-laj/airflow-docker-compose-examples/blob/main/tests/test_compose_files.py), which are currently being developed in an independent repository, and despite this, they have allowed me to detect various problems many times. But for now, I would like to limit the scope of the change --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more information. In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/main/UPDATING.md). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
