potiuk commented on PR #27971: URL: https://github.com/apache/airflow/pull/27971#issuecomment-1329820819
I am not sure if this is helpful, For non-hosted runners it should be no difference. We already have 900 runners by the ASF and we should have no more queueing here (at least we stopped seeing those) - so this is pretty ok to run all test types in parallel. And it will introduce significant "sequential" delays in case of non-postgres failures I am afraid. Rather than running tests in parallel, it will run them sequentially so "feedback time" will be biggeer. When optimising CI builds there are two aspects: * feedback time * cost (build hours usage) Feedback time is very important - the time between push and "result" should be as short as possible. Putting sequential dependency between tests increases feedback time, but saves cost (assuming job queue is infinite - and with 900 jobs in the ASF as of 3 weeks, for now - until ASF usage grows significantly it almost is - we used to have 300) . Cost does not matter - it is essentially free. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
