potiuk commented on PR #28248: URL: https://github.com/apache/airflow/pull/28248#issuecomment-1344392785
Feel free to try :). Though I think it could be a different behaviour (the test is not mine - just read the commit message it came with: https://github.com/apache/airflow/commit/f75dd7ae6e755dad328ba6f3fd462ade194dab25 ) I belive the reason was that it could be triggered by a quick succession of those celery calls one-by-one and if we wrap it with flaky, it might simply never trigger. @yuqian90 - maybe you still remember this commit and the case and can shed some light on it In this case the whole idea of this test is not the best - I understand it was a way to make more certaintly that we do not trigger the case, but in essence this test is "flaky by design" when something happens so we might as well ignore it in the future (which we did by adding it to quarantined tests) - without actually knowing if the problem is still there or not. I believe though - that the problem is generally solved by the change, and maybe we do not even need to keep the test and can simply remove it (because regression is a) not likely b) we might not even notice the regression because of the flakey nature of the test). So it might be more distraction than help. @Taragolis : Yes, Using flaky might be a good idea in general. The problem with quarantined tests though is not only them being flaky - but also them having side-effects on other tests. Most of the tests in the "quarantined" group that we have had this nasty property, that they not only failed from time to time, but their sheer presence in the "main" group of tests caused the other tests to be also affected (that's why we invented the "quarantine" idea) to keep them isolated from the main pytest run. (and yest it was at the beginnig of COVID so the name is inspired by real-life events :D). So "flaky" library is a bit different concept. Bu yes - I would love to get rid of the quarantined tests (while leaving the possibility of quarantining some tests if we find them breaking things again). I guess we could attempt (one-by-one ideally) to fix those tests and possibly move the quarantined tests to be "flaky" ones and see if that works. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
