o-nikolas commented on PR #32612: URL: https://github.com/apache/airflow/pull/32612#issuecomment-1636141927
> The tricky part is that our tests are executed in various "environments" - i.e. they can often be run in parallel with other tests in order to speed up the execution - and that might get very tricky to figure out if the slowness is just environmental or a real regression. > > But that discussion actually gave me an interesting idea. Maybe we coudl extract all "timing" tests to separate test type and execute it always "alone" only with sqlite - less CPU utilization but more stability of the environment. > > We could add a separate job for those kind of tests and exclude them from regular tests easily - I think we should do it this way. Yupp, agreed, I think the limit now should be good enough for most cases. For example the regression I mentioned in description more than doubled the CLI run time from (up to about 8 seconds). So it will surely catch large regressions like that which is still helpful. But creating a new test suite as a follow-up could be a great optimization and would allow us to lower the threshold a bit. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
