ItamarWMDE added a comment.
After starting the week with consistent E2E test failures, it was discovered that the reson for these test failures was the request rate limit in chromatic. The frequent consecutive and parallel calls to the chromatic cause it to respond with a 503 error page, which breaks the tests. I.E. and edge seemed to be one of the slower browsers and so it seemed as though the tests were failing consistently in these browsers alone. To mitigate this on the short run, a Few PRs were created to improve our browser tests. - #293 <https://github.com/wmde/wikit/pull/293> Attempts to perform all tests on the same page, to avoid frequent "refreshes" - #294 <https://github.com/wmde/wikit/pull/294> Adds a 500ms delay between each test to ensure there is a pause between requests (to allow chromatic to "recover") - #297 <https://github.com/wmde/wikit/pull/297> Converts our browser tests into BDD style tests to enable setup and teardown - #301 <https://github.com/wmde/wikit/pull/301> Uses the `iframe.html` chromatic url to load the tests with the components only (this speeds up the tests to make up for the penalty introduced in #294) However, all of these changes serve as stopgap solutions only. A long term solution to ensure our testing infrastructure is scalable would be to use a different deployment platform for our browser tests. In which case, the solution should come within the scope of T261650 <https://phabricator.wikimedia.org/T261650>. TASK DETAIL https://phabricator.wikimedia.org/T268107 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: ItamarWMDE Cc: Aklapper, ItamarWMDE, Akuckartz, Nandana, Lahi, Gq86, GoranSMilovanovic, QZanden, LawExplorer, _jensen, rosalieper, Scott_WUaS, Volker_E, Wikidata-bugs, aude, Lydia_Pintscher, Mbch331
_______________________________________________ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
