potiuk edited a comment on pull request #11402: URL: https://github.com/apache/airflow/pull/11402#issuecomment-706665870
Hey Everyone. I think with this change we have a chance to finally reach stability of the tests. I split the tests into multiple jobs, and I think I have a very nice split. I believe with the split I introduced, we will hit resource limitations far less frequently (even if we run many more jobs). Each of the test jobs does not run longer than ~7 minutes (including pulling the image built once in the separate workflow). Additionally I've added all the test types to breeze, so that it will be really easy to reproduce every test type locally. Once (alsob built once) you have the image locally built with breeze it takes literally up to 4-5 minutes (or less depending on your machine type) to re-run the complete failed set and reproduce the failure. Additionally, in case of test job failure I print some useful instructions on how to reproduce such failed run locally. In case you have not noticed, it is now very easy to reproduce the failed build from CI using the RUN_ID from GitHub Actions (you can pull the very image that was used to run the tests). Now this information is printed out on failure of tests and it will be immediately visible by the author and committer and reproducing the failed tests will be a ..... BREEZE. ``` ******************************************************************************************************* * * ERROR! Some tests failed, unfortunately. Those might be transient errors, * but usually you have to fix something. * See the above log for details. * ******************************************************************************************************* * You can easily reproduce the failed tests on your dev machine/ * * When you have the source branch checked out locally: * * Run all tests: * * ./breeze --backend postgres --python 3.6 --db-reset --test-type Core tests * * Enter docker shell: * * ./breeze --backend postgres --python 3.6 --db-reset --test-type Core shell * * When you do not have sources: * * Run all tests: * * ./breeze --gihub-image-id NNNNNNNN --backend postgres --python 3.6 --db-reset --test-type Core tests * * Enter docker shell: * * ./breeze --gihub-image-id NNNNNNNN --backend postgres --python 3.6 --db-reset --test-type Core shell * * * NOTE! Once you are in the docker shell, you can run failed test with: * * pytest [TEST_NAME] * * You can copy the test name from the output above * *************************************************************************************************************** ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
