| Dalba added a comment. |
Yes if the site is down all will fail but that is less weird than having some builds fail and some not due to some hiccup.
May be weird, but I think it's more convenient. When a test like this fails because of a server error, I have to go through at least 13 build logs to see what has happened. This is too much work, I think many won't bother to check them all and may miss other errors.
Now let's assume there is a hiccup, i.e. a test has been skipped and we have not noticed an error in a recent commit because the build has been successful.
It is very likely that the same test in one of the other 13 build jobs will fail, or at least will fail in the very next few builds.
Finding why test has not failed during the previous build is easy because those will be marked as skipped with a summary of why the have been skipped.
I think it will still be easy to track the cause.
Would it be possible to start with a ping and only run the test if that gets a positive reply?
I'm a little sceptical on that. It could help, maybe, but I'm not convinced that it's worth it.
Alternatively we could reconsider running tests on live sites outside of our control.
I'm inclined towards this option, but it seems to require much more effort. We could also store successful responses in an optional submodule and use that to run tests.
Cc: Magul, Lokal_Profil, gerritbot, Aklapper, pywikibot-bugs-list, Dalba, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, Tbscho, MayS, Lewizho99, Mdupont, JJMC89, Maathavan, Avicennasis, Sn1per, jayvdb, Masti, Alchimista, Rxy
_______________________________________________ pywikibot-bugs mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/pywikibot-bugs
