potiuk commented on pull request #14531: URL: https://github.com/apache/airflow/pull/14531#issuecomment-794709877
Before I merged this one I added one more change. This one should allow to dynamically choose (using gnu parallel --semaphore) how many tests we run in parallel depending on the number of CPUs (overrideable with `MAX_PARALLEL_TEST_JOBS` . This should allow to get ~20 minutes even with smaller machines I believe. Also it is very likely (testing it in https://github.com/potiuk/airflow/actions/runs/637728789 that even with default 2 CPUs for public GitHub Runners we should be able to achieve ~2x speedup. That would be awesome. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
