potiuk commented on PR #27823: URL: https://github.com/apache/airflow/pull/27823#issuecomment-1336234191
I am afraid you are one of the few unlucky ones who will have to deal with side-effects. It is very likely that those tests that fail, rely on side effects from other tests. It's also possible that your tests are clearing/removing those side effects - that's why those tests start to fail. Did you run all tessts from providers? `breeze testing tests --test-type providers[amazon]` Those should run full "amazon providers" suite in the same sequence they are run in CI - run them on `main` and see they don't fail. If you run those tests individually and they fail - it means they are relying on side effects from other tests. This is because most of our unit tests rely on dags/dagruns/connections etc. created in the unit test DB and the DB is re-used across all the tests. The only way to solve it for now is to investigate and fix those tests that indeed rely on side effects. That might involve writing a fixture or setUp method that restores the expected state of the DB records that the test expects - many tests have those, but some of them not and rely on the DB being populated from previous tests - and this is the problem. Yes. It's not your fault. But also yes - it, unfortunately, falls on your shoulders essentially if your newly added tests or the modified ones interfere with it. Ideally by finding and resolving the side-effects in the other tests so that the situation gets improved for the future. It happens rarely enough that it is not a "common" problem, but unfortunately, you fell a victim of it likely. And yes it is NOT how it should be. But - unfortunately, we are not living in perfect world and it is, what it is. And hopefully one day we will be able to solve it better and get rid even of the possibility of the side effects to happen, but that would likely require a lot of investment into complete refactoring on how 100s if not 1000s of tests are implemented, so it's totally not feasible to solve it now once and for all, I am afraid. We can complain about it (I do) but this is - unfortuntely - quite an effort to fix. And if anyone has an idea how to solve it quickly - that would be fantastic We have tried in the past to just clean the db for every unit tests but with many thousands of them this slows down the whole suite to a crawl. Hopefully some day we will figure how to fix it better. But I am afraid quoting King Theoden - "this is not that day". BTW. This is actually one af a good ideas on how to make an overall improvement in our tests suite to fix this problem permanently. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
