potiuk commented on PR #30315: URL: https://github.com/apache/airflow/pull/30315#issuecomment-1486540171
> Greeting, Jarek. Just want to clarify why we want to change this behaviour for pytest tests? Personally for me this is a natural to have `test_` prefix or `_test` suffix for test modules, and also this is default [pytest naming convention](https://docs.pytest.org/en/7.2.x/explanation/goodpractices.html#conventions-for-python-test-discovery) > > If some tests not discoverable by the reason that python modules doesn't have `test_` prefix might be better just rename this modules? There is plenty other rules which might turn test into undiscoverable. > > But it is possible that some other reason exists and I'm just out of the context. Yes. As explained in the PR - we already hit this problem because we missed the fact that `test_` was missing from test modules during review. > This happened already: see https://github.com/apache/airflow/pull/30311 and https://github.com/apache/airflow/pull/30306 Those tests were not running when you run them in CI - simply because pytest collection rules misses them entirely because of lack of `test_*`. But we have no idea that it happens. They are just silently ignored (by design). The problem is that if you run the tests locally during development, and run them by `pytest <test_file>.py` they will run. So you are completely not aware of that - and as result our CI tests do NOT test those tests for regression. This means (and again - this already happened with databricks and smtp tests - so it will inevitably happen again unless we protect against this. We could of course add some pre-commits (and @hussein-awala attempted to do it in https://github.com/apache/airflow/pull/30306#issuecomment-1484110845 , however that was very simplistic and not fool-proof. I think the only way to avoid any such mishaps in the future is to add all files in "tests" dir to be pytest-collected. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
