Niedzielski added a comment.
This appears to still be a failure from time to time:
zeljkofilipin added a comment.
@pmiazga apologies for the delay, I have to finish some selenium+node tasks before I can get back to this.TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc:
pmiazga added a comment.
@zeljkofilipin any updates? Looks like some MobileFrontend extension tests still fail from time to time: https://integration.wikimedia.org/ci/view/Reading-Web/job/selenium-MobileFrontend/451/TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL
gerritbot added a comment.
Change 342636 abandoned by Zfilipin:
WIP: Increase in failures caused by Saucelabs
https://gerrit.wikimedia.org/r/342636TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To:
gerritbot added a comment.
Change 338368 abandoned by Zfilipin:
WIP Increase in failures caused by Saucelabs
https://gerrit.wikimedia.org/r/338368TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: gerritbotCc:
gerritbot added a comment.
Change 342636 had a related patch set uploaded (by Zfilipin):
[mediawiki/extensions/Wikibase] WIP: Increase in failures caused by Saucelabs
https://gerrit.wikimedia.org/r/342636TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL
Tobi_WMDE_SW added a comment.
@zeljkofilipin Ok! Thanks for your effort and for the update! If there's anything we can do from our side, please let me know.TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To:
zeljkofilipin added a comment.
@Tobi_WMDE_SW I am working on this. Looks like the only way to ensure stability is to rerun failed tests. I am investigating 2-3 ways on how to do that. No luck so far, but getting close.TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL
Tobi_WMDE_SW added a comment.
After several days of stability, the last 2 builds had several strange failures again:
One time:
unknown error: $.cookie is not a function
Several times:
Net::ReadTimeout (Net::ReadTimeout)
Several:
Network is unreachable - connect(2) for "test.wikidata.org"
gerritbot added a comment.
Change 341523 had a related patch set uploaded (by zfilipin):
[mediawiki/selenium] WIP Problem: Can not use --retry option to retry failed tests as part of the same run
https://gerrit.wikimedia.org/r/341523TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL
gerritbot added a comment.
Change 338785 abandoned by Zfilipin:
WIP Increase in failures caused by Saucelabs
https://gerrit.wikimedia.org/r/338785TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To:
zeljkofilipin added a comment.
Albert Sison (Sauce Labs Help Center)
Feb 17, 3:22 PM PST
Hello,
Thanks for the update and please let us know if re-running the failed tests work for you. In case you're wondering, I have attached screenshots of "Internal Server Error" and "The Sauce VMs failed to
gerritbot added a comment.
Change 338967 had a related patch set uploaded (by Zfilipin):
WIP Increase in failures caused by Saucelabs
https://gerrit.wikimedia.org/r/338967TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL
zeljkofilipin added a comment.
Testing: https://integration.wikimedia.org/ci/job/selenium-Math-338368/TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: gerritbot, hashar, zeljkofilipin,
zeljkofilipin added a comment.
http://web.archive.org/web/20160713013212/http://blog.crowdint.com/2011/08/22/auto-retry-failed-cucumber-tests.htmlTASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To:
gerritbot added a comment.
Change 338368 had a related patch set uploaded (by Zfilipin):
WIP Increase in failures caused by Saucelabs
https://gerrit.wikimedia.org/r/338368TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL
gerritbot added a comment.
Change 336632 abandoned by Zfilipin:
WIP Run selenium-Wikibase Jenkins job on Linux, Mac and Windows
https://gerrit.wikimedia.org/r/336632TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL
zeljkofilipin added a comment.
Sauce labs support ticket (not public): https://support.saucelabs.com/hc/en-us/requests/35513
wikimedia-jenkins
a few seconds ago
Running tests on Windows and Mac did not prove to be more stable. More information is available at our public bug tracker:
hashar added a comment.
From http://stackoverflow.com/questions/12787032/handling-exceptions-on-cucumber-scenarios?rq=1
Around('@handle_alert_boxes') do |scenario, block| do
begin
block.call
rescue Selenium::WebDriver::Error::UnhandledAlertError
puts "It's OK!"
end
end
So maybe
zeljkofilipin added a comment.
Unfortunately, looks like running sequentially is supported only for executing strategy classic, but we use yaml :(TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To:
zeljkofilipin added a comment.
In T152963#3028442, @Tobi_WMDE_SW wrote:
@zeljkofilipin is it possible to run the job for beta at a different time than the job for test?
We could run the jobs sequentially, not in parallel:
zeljkofilipin added a comment.
The trouble with recent data is that we had a lot of trouble with unrelated MediawikiApi::ApiError problems. Now that that is resolved, I have created a new job selenium-Wikibase-336632-4 that will run every 4 hours (because it needs over 3 hours for a run). I will
zeljkofilipin added a comment.
I did a bit of investigation, thinking that builds targeting test wiki fail more for some reason, but that does not seem to be the case. Builds failing with MediawikiApi::ApiError are a couple of problems unrelated to this one, and should be ignored.
buildbetatest
Tobi_WMDE_SW added a comment.
@zeljkofilipin is it possible to run the job for beta at a different time than the job for test?
At the moment they both run at 04:40 UTC but I would like one of those to start 2 hours later or earlier. Seems like the beta-job ran pretty stable during the last days
zeljkofilipin added a comment.
Number of failures in the last 10 runs
LinuxWindowsMac
TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: gerritbot, hashar, zeljkofilipin, thiemowmde,
zeljkofilipin added a comment.
Linux selenium-WikibaseWindows selenium-Wikibase-336632-1Mac selenium-Wikibase-336632-2
green in last 5 test runs553
green in last 10 test runs563
TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL
zeljkofilipin added a comment.
The results:
Linux: selenium-Wikibase, last 5 runs are green
Windows: selenium-Wikibase-336632-1, last 5 runs are green
Mac: selenium-Wikibase-336632-2 3 out of 5 last runs are greenTASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL
zeljkofilipin added a comment.
Test jobs are created and running:
Windows: https://integration.wikimedia.org/ci/job/selenium-Wikibase-336632-1/
Mac: https://integration.wikimedia.org/ci/job/selenium-Wikibase-336632-2/
TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL
gerritbot added a comment.
Change 336632 had a related patch set uploaded (by Zfilipin):
WIP Run selenium-Wikibase Jenkins job using Windows 10
https://gerrit.wikimedia.org/r/336632TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL
zeljkofilipin added a comment.
Date: Wed, 8 Feb 2017 15:55:55 +0100
Hi Albert,
This Jenkins job has 211 tests and runs once a day. Each test starts a
Sauce Labs job. I have investigated jobs 253-258, so 6 days, or 1266 tests.
"The Sauce VMs failed to start the browser or device" error has
zeljkofilipin added a comment.
Albert Sison (Sauce Labs Help Center)
Feb 3, 1:20 PM PST
Hello,
Thanks for writing in to Sauce Labs support. Regarding this specific ”Internal Server Error", this means the VM to run the test started and then become unresponsive. The VM appears to have hit a
zeljkofilipin added a comment.
I have reported the problem to Sauce Labs support.TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: hashar, zeljkofilipin, thiemowmde, Aklapper, Tobi_WMDE_SW,
zeljkofilipin added a comment.
Sauce Labs error is:
Internal Server Error
The Sauce VMs failed to start the browser or device. For more info, please check https://wiki.saucelabs.com/display/DOCS/Common+Error+Messages
TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL
zeljkofilipin added a comment.
Looks like the tests are failing randomly :(
BuildTestErrorSauce Labs
258Edit sitelinks.Remove multiple sitelinksNet::ReadTimeouthttp://saucelabs.com/jobs/a31d28114680489485963f5d27bea5ae
257Using url properties in statements.Check UI for invalid values (outline
zeljkofilipin added a comment.
@hashar Good idea about checking the limit, but we never reach it.
F5464191: sauce.png
Do you have a Wikimedia Sauce Labs account? I do not see it in the list.TASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL
hashar added a comment.
Can it be that sometime we exhaust the number of concurrent sessions? Though we run the tests serially, so unless we have more than X jobs running together we should not reach that limit.
My account on sauce labs does not let me access the interface (need email
zeljkofilipin added a comment.
Sauce Labs status says everything is fine on their end, there were no incidents in the last week. I took a quick look, the rest of selenium jobs are very rarely failing with Selenium::WebDriver::Error::UnknownError.
selenium-Wikibase job is the only one that runs
zeljkofilipin added a comment.
Sauce Labs status:
https://status.saucelabs.com/
https://twitter.com/sauceopsTASK DETAILhttps://phabricator.wikimedia.org/T152963EMAIL PREFERENCEShttps://phabricator.wikimedia.org/settings/panel/emailpreferences/To: zeljkofilipinCc: hashar, zeljkofilipin,
38 matches
Mail list logo
