Hello everyone, I would like to announce that I just merged the last (3/3) part of AIP-10 - big change in the way we run CI tests for Airflow.
Nothing changes in the tests, but the way we run them in Travis CI and locally is quite different. You should be able to reproduce Travis CI tests super-easily on your local Linux/MacOS development machine now following the documentation <https://github.com/apache/airflow/blob/master/CONTRIBUTING.md#integration-test-development-environment> . The change gets rid of tox, Dockerises everything including pylint/mypy/flake checks, and optimises it so that it can be conveniently run both in CI and locally. Including faster iterations for re-running tests several times. Travis CI builds are now better and show python version + they print summary of failed tests at the end of the log. We also have a way to run static code check super-easily locally see . I truly hope it will make it easier for casual contributors to contribute to Airflow but also a lot of experienced Airflowers might find it really helpful. Please rebase to master and make a good use of it. The CONTRIBUTING.md documentation <https://github.com/apache/airflow/blob/master/CONTRIBUTING.md#integration-test-development-environment> has changed a lot (and hopefully vastly improved).I expect some (hopefully minor) teething problems - in case you have any troubles please follow the troubleshooting <https://github.com/apache/airflow/blob/master/CONTRIBUTING.md#troubleshooting> section of the documentation - if you cannot solve it following it, reach out to me on Slack or here in the discussion. I will try to be available as much as I can over the next few days to help in case of problems. There are few follow-up PRs/Proposal that are in the making for which AIP-10 was prerequisite (Breeze - simplified development environment, Pre-commit hooks) - also we will try to migrate out of Travis CI because of the recent problems with it (AIP-10 was instrumental to enable fast migration out of Travis) . IMPORTANT! Together with Ash we are going to run webinar/screencast next week (*Thursday 25th of July, 6.30pm CEST, 5.30pm London, 12:30 pm NYC, 9.30am California*). It will present what's already merged in AIP -10 and some of the improvements coming (Breeze/pre-commit). There will be separate announcement tomorrow about it so stay tuned. Big thanks to everyone who helped with this (especially to Ash for thorough reviews and great comments, but also to Fokko, Kamil, Gerardo for their support and comments - especially those critical ones that helped to make it better. Also Daniel, Stefan for testing during last two weeks. It's great community we have here :). BTW. I think this was the longest PR I ever worked on. I wrote the first version of the proposal <https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-10+Multi-layered+and+multi-stage+official+Airflow+CI+image> mid-January and I worked on it since then. Fun fact - I hold now the 2nd place for PR with the biggest number of comments (278) in Airflow right after "[AIRFLOW-3585] - Add edges to database" (284). J. -- Jarek Potiuk Polidea <https://www.polidea.com/> | Principal Software Engineer M: +48 660 796 129 <+48660796129> [image: Polidea] <https://www.polidea.com/>
