[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
Niedzielski added a comment. This appears to still be a failure from time to time: https://integration.wikimedia.org/ci/view/Reading-Web/job/selenium-MinervaNeue/98/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/testReport/junit/(root)/Page%20actions%20menu%20when%20anonymous/Receive_notification_message___Edit_Icon/.TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: NiedzielskiCc: Niedzielski, PokestarFan, pmiazga, Jdforrester-WMF, dduvall, Tgr, greg, Jdlrobson, Cmcmahon, hoo, Lydia_Pintscher, Jonas, Aleksey_WMDE, gerritbot, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Lewizho99, Maathavan, Liudvikas, Izno, Luke081515, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
zeljkofilipin added a comment. @pmiazga apologies for the delay, I have to finish some selenium+node tasks before I can get back to this.TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: pmiazga, Jdforrester-WMF, dduvall, Tgr, greg, Jdlrobson, Cmcmahon, hoo, Lydia_Pintscher, Jonas, Aleksey_WMDE, gerritbot, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Lewizho99, Maathavan, Liudvikas, Izno, Luke081515, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
pmiazga added a comment. @zeljkofilipin any updates? Looks like some MobileFrontend extension tests still fail from time to time: https://integration.wikimedia.org/ci/view/Reading-Web/job/selenium-MobileFrontend/451/TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: pmiazgaCc: pmiazga, Jdforrester-WMF, dduvall, Tgr, greg, Jdlrobson, Cmcmahon, hoo, Lydia_Pintscher, Jonas, Aleksey_WMDE, gerritbot, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Lewizho99, Maathavan, Liudvikas, Izno, Luke081515, Wikidata-bugs, aude, Mbch331___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
gerritbot added a comment. Change 342636 abandoned by Zfilipin: WIP: Increase in failures caused by Saucelabs https://gerrit.wikimedia.org/r/342636TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: gerritbotCc: hoo, Lydia_Pintscher, Jonas, Aleksey_WMDE, gerritbot, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Lewizho99, Maathavan, Liudvikas, Izno, Luke081515, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
gerritbot added a comment. Change 338368 abandoned by Zfilipin: WIP Increase in failures caused by Saucelabs https://gerrit.wikimedia.org/r/338368TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: gerritbotCc: hoo, Lydia_Pintscher, Jonas, Aleksey_WMDE, gerritbot, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Lewizho99, Maathavan, Liudvikas, Izno, Luke081515, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
gerritbot added a comment. Change 342636 had a related patch set uploaded (by Zfilipin): [mediawiki/extensions/Wikibase] WIP: Increase in failures caused by Saucelabs https://gerrit.wikimedia.org/r/342636TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipin, gerritbotCc: hoo, Lydia_Pintscher, Jonas, Aleksey_WMDE, gerritbot, hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, Lewizho99, Maathavan, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
Tobi_WMDE_SW added a comment. @zeljkofilipin Ok! Thanks for your effort and for the update! If there's anything we can do from our side, please let me know.TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipin, Tobi_WMDE_SWCc: hoo, Lydia_Pintscher, Jonas, Aleksey_WMDE, gerritbot, hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, Lewizho99, Maathavan, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
zeljkofilipin added a comment. @Tobi_WMDE_SW I am working on this. Looks like the only way to ensure stability is to rerun failed tests. I am investigating 2-3 ways on how to do that. No luck so far, but getting close.TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: hoo, Lydia_Pintscher, Jonas, Aleksey_WMDE, gerritbot, hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, Lewizho99, Maathavan, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
Tobi_WMDE_SW added a comment. After several days of stability, the last 2 builds had several strange failures again: One time: unknown error: $.cookie is not a function Several times: Net::ReadTimeout (Net::ReadTimeout) Several: Network is unreachable - connect(2) for "test.wikidata.org" port 443 (Faraday::ConnectionFailed) Some: Sauce could not start your job. For more information on what happened, please visit https://saucelabs.com/jobs/67b6ddb2476a4fcf852c418a8de063a4 (Selenium::WebDriver::Error::UnknownError) https://integration.wikimedia.org/ci/job/selenium-Wikibase/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=test,PLATFORM=Linux,label=BrowserTests/ https://integration.wikimedia.org/ci/job/selenium-Wikibase/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipin, Tobi_WMDE_SWCc: gerritbot, hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, Lewizho99, Maathavan, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
gerritbot added a comment. Change 341523 had a related patch set uploaded (by zfilipin): [mediawiki/selenium] WIP Problem: Can not use --retry option to retry failed tests as part of the same run https://gerrit.wikimedia.org/r/341523TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipin, gerritbotCc: gerritbot, hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, Lewizho99, Maathavan, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
gerritbot added a comment. Change 338785 abandoned by Zfilipin: WIP Increase in failures caused by Saucelabs https://gerrit.wikimedia.org/r/338785TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipin, gerritbotCc: gerritbot, hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, Lewizho99, Maathavan, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
zeljkofilipin added a comment. Albert Sison (Sauce Labs Help Center) Feb 17, 3:22 PM PST Hello, Thanks for the update and please let us know if re-running the failed tests work for you. In case you're wondering, I have attached screenshots of "Internal Server Error" and "The Sauce VMs failed to start the browser or device." error rate for jobs rans by all users for the past 30 days. Regards, Albert Internal Server Error F5740677: vmfail.png The Sauce VMs failed to start the browser or device F5740679: ISE.pngTASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: gerritbot, hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, Th3d3v1ls, Ramalepe, Liugev6, Lewizho99, Maathavan, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
gerritbot added a comment. Change 338967 had a related patch set uploaded (by Zfilipin): WIP Increase in failures caused by Saucelabs https://gerrit.wikimedia.org/r/338967TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipin, gerritbotCc: gerritbot, hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, Th3d3v1ls, Ramalepe, Liugev6, Lewizho99, Maathavan, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
zeljkofilipin added a comment. Testing: https://integration.wikimedia.org/ci/job/selenium-Math-338368/TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: gerritbot, hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, Th3d3v1ls, Ramalepe, Liugev6, Lewizho99, Maathavan, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
zeljkofilipin added a comment. http://web.archive.org/web/20160713013212/http://blog.crowdint.com/2011/08/22/auto-retry-failed-cucumber-tests.htmlTASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: gerritbot, hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, Th3d3v1ls, Ramalepe, Liugev6, Lewizho99, Maathavan, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
gerritbot added a comment. Change 338368 had a related patch set uploaded (by Zfilipin): WIP Increase in failures caused by Saucelabs https://gerrit.wikimedia.org/r/338368TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipin, gerritbotCc: gerritbot, hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, Th3d3v1ls, Ramalepe, Liugev6, Lewizho99, Maathavan, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
gerritbot added a comment. Change 336632 abandoned by Zfilipin: WIP Run selenium-Wikibase Jenkins job on Linux, Mac and Windows https://gerrit.wikimedia.org/r/336632TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipin, gerritbotCc: gerritbot, hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, Th3d3v1ls, Ramalepe, Liugev6, Lewizho99, Maathavan, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
zeljkofilipin added a comment. Sauce labs support ticket (not public): https://support.saucelabs.com/hc/en-us/requests/35513 wikimedia-jenkins a few seconds ago Running tests on Windows and Mac did not prove to be more stable. More information is available at our public bug tracker: https://phabricator.wikimedia.org/T152963 We will rerun failed tests and see if that helps. TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: gerritbot, hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, Th3d3v1ls, Ramalepe, Liugev6, Lewizho99, Maathavan, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
hashar added a comment.
From http://stackoverflow.com/questions/12787032/handling-exceptions-on-cucumber-scenarios?rq=1
Around('@handle_alert_boxes') do |scenario, block| do
begin
block.call
rescue Selenium::WebDriver::Error::UnhandledAlertError
puts "It's OK!"
end
end
So maybe just rescue the few timeout exceptions and block.call again? :}TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipin, hasharCc: gerritbot, hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, Th3d3v1ls, Ramalepe, Liugev6, Lewizho99, Maathavan, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
zeljkofilipin added a comment. Unfortunately, looks like running sequentially is supported only for executing strategy classic, but we use yaml :(TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: gerritbot, hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, Th3d3v1ls, Ramalepe, Liugev6, Lewizho99, Maathavan, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
zeljkofilipin added a comment. In T152963#3028442, @Tobi_WMDE_SW wrote: @zeljkofilipin is it possible to run the job for beta at a different time than the job for test? We could run the jobs sequentially, not in parallel: https://docs.openstack.org/infra/jenkins-job-builder/project_matrix.html?highlight=sequential sequential (bool): run builds sequentially (default false)TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: gerritbot, hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, Th3d3v1ls, Ramalepe, Liugev6, Lewizho99, Maathavan, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
zeljkofilipin added a comment. The trouble with recent data is that we had a lot of trouble with unrelated MediawikiApi::ApiError problems. Now that that is resolved, I have created a new job selenium-Wikibase-336632-4 that will run every 4 hours (because it needs over 3 hours for a run). I will let it run for a day or two.TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: gerritbot, hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, Th3d3v1ls, Ramalepe, Liugev6, Lewizho99, Maathavan, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
zeljkofilipin added a comment. I did a bit of investigation, thinking that builds targeting test wiki fail more for some reason, but that does not seem to be the case. Builds failing with MediawikiApi::ApiError are a couple of problems unrelated to this one, and should be ignored. buildbetatest failureserror(s)failureserror(s) 27001Selenium::WebDriver::Error::UnknownError 2690 2680 2670 2660 2650 26434MediawikiApi::ApiError 2631Net::ReadTimeout76MediawikiApi::ApiError 262076MediawikiApi::ApiError 261076MediawikiApi::ApiError 260076MediawikiApi::ApiError 259076MediawikiApi::ApiError 2581Net::ReadTimeout76MediawikiApi::ApiError 2571Selenium::WebDriver::Error::UnknownError76MediawikiApi::ApiError 2562Net::ReadTimeout Selenium::WebDriver::Error::UnknownErrorAttributeError 2553Selenium::WebDriver::Error::UnknownError76MediawikiApi::ApiError 254076MediawikiApi::ApiError 253076MediawikiApi::ApiError 25274MediawikiApi::ApiError76MediawikiApi::ApiError 251076MediawikiApi::ApiError 2502Selenium::WebDriver::Error::UnknownError0 24900 24800 2476Selenium::WebDriver::Error::UnknownError4Selenium::WebDriver::Error::UnknownError 2464Selenium::WebDriver::Error::UnknownError1Selenium::WebDriver::Error::UnknownError 24500 2441Net::ReadTimeout2Selenium::WebDriver::Error::UnknownError Watir::Wait::TimeoutError 24300 24200 2413Selenium::WebDriver::Error::UnknownError76MediawikiApi::ApiError 240076MediawikiApi::ApiError TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: gerritbot, hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, Th3d3v1ls, Ramalepe, Liugev6, Lewizho99, Maathavan, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
Tobi_WMDE_SW added a comment. @zeljkofilipin is it possible to run the job for beta at a different time than the job for test? At the moment they both run at 04:40 UTC but I would like one of those to start 2 hours later or earlier. Seems like the beta-job ran pretty stable during the last days while the test-job was disabled, so probably it's an issue to run them both in parallel.TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipin, Tobi_WMDE_SWCc: gerritbot, hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, Th3d3v1ls, Ramalepe, Liugev6, Lewizho99, Maathavan, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
zeljkofilipin added a comment. Number of failures in the last 10 runs LinuxWindowsMac TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: gerritbot, hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, Th3d3v1ls, Ramalepe, Liugev6, Lewizho99, Maathavan, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
zeljkofilipin added a comment. Linux selenium-WikibaseWindows selenium-Wikibase-336632-1Mac selenium-Wikibase-336632-2 green in last 5 test runs553 green in last 10 test runs563 TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: gerritbot, hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, Th3d3v1ls, Ramalepe, Liugev6, Lewizho99, Maathavan, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
zeljkofilipin added a comment. The results: Linux: selenium-Wikibase, last 5 runs are green Windows: selenium-Wikibase-336632-1, last 5 runs are green Mac: selenium-Wikibase-336632-2 3 out of 5 last runs are greenTASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: gerritbot, hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, Th3d3v1ls, Ramalepe, Liugev6, Lewizho99, Maathavan, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
zeljkofilipin added a comment. Test jobs are created and running: Windows: https://integration.wikimedia.org/ci/job/selenium-Wikibase-336632-1/ Mac: https://integration.wikimedia.org/ci/job/selenium-Wikibase-336632-2/ TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: gerritbot, hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, Th3d3v1ls, Ramalepe, Liugev6, Lewizho99, Maathavan, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
gerritbot added a comment. Change 336632 had a related patch set uploaded (by Zfilipin): WIP Run selenium-Wikibase Jenkins job using Windows 10 https://gerrit.wikimedia.org/r/336632TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipin, gerritbotCc: gerritbot, hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
zeljkofilipin added a comment. Date: Wed, 8 Feb 2017 15:55:55 +0100 Hi Albert, This Jenkins job has 211 tests and runs once a day. Each test starts a Sauce Labs job. I have investigated jobs 253-258, so 6 days, or 1266 tests. "The Sauce VMs failed to start the browser or device" error has appeared 5 times. 5/1266 = 0.0039 or 0.4%. I will run the same job on Windows and Mac and see if there are any improvements. Regards, Željko TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
zeljkofilipin added a comment. Albert Sison (Sauce Labs Help Center) Feb 3, 1:20 PM PST Hello, Thanks for writing in to Sauce Labs support. Regarding this specific ”Internal Server Error", this means the VM to run the test started and then become unresponsive. The VM appears to have hit a resource limit and crashed. It’s hard to tell exactly what limit was hit. Later our services tried to restart the VM, but by that point the test was no longer registered as an active job (i.e. it had “gone stale”), and the job failed. This is a type of failure we’ve seen before: where a VM hits a resource limit and crashes. It’s a known problem where we don’t know the exact cause, not enough to prevent it from happening again. In the engineer’s words, it is a “rare but expected scenario”. More information about this error can be found here: https://wiki.saucelabs.com/display/DOCS/Common+Error+Messages#CommonErrorMessages-InternalServerError As for the "The connection with your VM was lost and your job can't complete." error, if you only get this message rarely and randomly, it is probably a fluke on our end caused by an infrastructure blip as mentioned on our error page: https://wiki.saucelabs.com/display/DOCS/Common+Error+Messages#CommonErrorMessages-TheConnectionwithYourVirtualMachinewasLostandYourJobCan'tComplete For "The Sauce VMs failed to start the browser or device." error, what percentage of your tests are affected by this specific error? I noticed all the Sauce job URLs provided were testing against Linux/Chrome 48. For diagnostic purposes, can you try testing against a different OS such as Windows 10 or OSX and let us know if you notice any improvements? Thanks for your patience and understanding. Regards, Albert TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
zeljkofilipin added a comment. I have reported the problem to Sauce Labs support.TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
zeljkofilipin added a comment. Sauce Labs error is: Internal Server Error The Sauce VMs failed to start the browser or device. For more info, please check https://wiki.saucelabs.com/display/DOCS/Common+Error+Messages TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
zeljkofilipin added a comment. Looks like the tests are failing randomly :( BuildTestErrorSauce Labs 258Edit sitelinks.Remove multiple sitelinksNet::ReadTimeouthttp://saucelabs.com/jobs/a31d28114680489485963f5d27bea5ae 257Using url properties in statements.Check UI for invalid values (outline example : missing.http.org )Selenium::WebDriver::Error::UnknownErrorhttp://saucelabs.com/jobs/a0692d83dcd848fc890217f3785f61ce 256Setting snaktypes of statements.Change the snaktype and save (outline example : somevalue )Net::ReadTimeouthttp://saucelabs.com/jobs/2e7df09a1b4f40e9b61e3e815e03b7d1 http://saucelabs.com/jobs/2455da0a2be84a3bb91069fea3881ded 256Using monolingual properties in statements.Adding a statement of type monolingual (outline example : English )Selenium::WebDriver::Error::UnknownErrorhttps://saucelabs.com/jobs/65530e0b4f034a4291cc4b814f484b86 255Edit aliases.Type new aliasSelenium::WebDriver::Error::UnknownErrorhttps://saucelabs.com/jobs/5781c8559fbf401f8f454396bc021a21 255Edit label.Modify the labelSelenium::WebDriver::Error::UnknownErrorhttps://saucelabs.com/jobs/aa642a7c67f841e28421f60342c63f29 255Property smoke test.Click statement add buttonSelenium::WebDriver::Error::UnknownErrorhttps://saucelabs.com/jobs/914b045996e84b42b6d4b0212960abaa 254- 253- TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
zeljkofilipin added a comment. @hashar Good idea about checking the limit, but we never reach it. F5464191: sauce.png Do you have a Wikimedia Sauce Labs account? I do not see it in the list.TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
hashar added a comment. Can it be that sometime we exhaust the number of concurrent sessions? Though we run the tests serially, so unless we have more than X jobs running together we should not reach that limit. My account on sauce labs does not let me access the interface (need email verification and I never receive it). But maybe the build logs have more details?TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipin, hasharCc: hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
zeljkofilipin added a comment. Sauce Labs status says everything is fine on their end, there were no incidents in the last week. I took a quick look, the rest of selenium jobs are very rarely failing with Selenium::WebDriver::Error::UnknownError. selenium-Wikibase job is the only one that runs for hours. Most of the jobs run in a few minutes, two jobs run for about 30 minutes (selenium-MobileFrontend, selenium-MultimediaViewer). Both of the longer jobs also occasionally fail with Selenium::WebDriver::Error::UnknownError. Rarely, something goes wrong when connecting to Sauce Labs, and when a job creates many connections, one (or more) of them fail. I am not sure what to do, but to contact Sauce Labs support and ask if they can investigate on their side.TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
[Wikidata-bugs] [Maniphest] [Commented On] T152963: Increase in failures caused by Saucelabs
zeljkofilipin added a comment. Sauce Labs status: https://status.saucelabs.com/ https://twitter.com/sauceopsTASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, greg___ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
