https://bugzilla.wikimedia.org/show_bug.cgi?id=72098
Bug ID: 72098
Summary: QA: browser tests don't detect or report HTTP response
errors
Product: Wikimedia
Version: unspecified
Hardware: All
OS: All
Status: NEW
Severity: enhancement
Priority: Unprioritized
Component: Quality Assurance
Assignee: [email protected]
Reporter: [email protected]
CC: [email protected], [email protected],
[email protected]
Web browser: ---
Mobile Platform: ---
Browser tests explicitly load pages and click buttons that load pages.
Neither seems to detect outright page load failures. E.g.
https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/263/testReport/%28root%29/Actions%20menu%20Permalink/Topic_Actions_menu_Permalink/
failed with status 500 response and a printed exception, but the test instead
reported
Sauce Labs job URL:
http://saucelabs.com/jobs/b15aad5529da47ea979bb0196f17c320
timed out after 5 seconds, waiting for {:css=>".flow-topic h2", :index=>0,
:tag_name=>"h2"} to become present (Watir::Wait::TimeoutError)
Like "page has no ResourceLoader errors" (implemented) and "browser tests
should assert there are no pink error boxes on the page" (bug 61304), most
tests should have an easy way to indicate that an error in a page load should
fail outright with something like "unexpected HTTP response (500)".
However, note the comment in http://stackoverflow.com/a/5629462/451712 :
"There's a bit of a philosophical debate on whether watir or watir-webdriver
should provide HTTP return code information." so this may be hard to fix.
Since beta and production show errors slightly differently, and some HTTP
response errors have friendlier HTML content like "Wikipedia does not have an
article with this exact name", I'm not sure how to easily and generically
detect from page content that there's been a breakdown, even though it's
obvious to a human being :)
--
You are receiving this mail because:
You are the assignee for the bug.
You are on the CC list for the bug.
_______________________________________________
Wikibugs-l mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
