Title: [252411] trunk/Tools
Revision
252411
Author
[email protected]
Date
2019-11-13 08:21:56 -0800 (Wed, 13 Nov 2019)

Log Message

Make webkitperl EWS robust against flakiness in webkitperl tests
https://bugs.webkit.org/show_bug.cgi?id=204155

Reviewed by Jonathan Bedard.

* BuildSlaveSupport/ews-build/steps.py:
(RunWebKitPerlTests): Set haltOnFailure and flunkOnFailure to false.
(RunWebKitPerlTests.evaluateCommand): Add re-run step in case of failure.
(ReRunWebKitPerlTests):
(ReRunWebKitPerlTests.evaluateCommand): Overrided this to avoid running this step again.
* BuildSlaveSupport/ews-build/steps_unittest.py:
(TestReRunJavaScriptCoreTests): Added unit-tests.

Modified Paths

Diff

Modified: trunk/Tools/BuildSlaveSupport/ews-build/steps.py (252410 => 252411)


--- trunk/Tools/BuildSlaveSupport/ews-build/steps.py	2019-11-13 16:16:42 UTC (rev 252410)
+++ trunk/Tools/BuildSlaveSupport/ews-build/steps.py	2019-11-13 16:21:56 UTC (rev 252411)
@@ -618,7 +618,8 @@
     name = 'webkitperl-tests'
     description = ['webkitperl-tests running']
     descriptionDone = ['webkitperl-tests']
-    flunkOnFailure = True
+    flunkOnFailure = False
+    haltOnFailure = False
     command = ['perl', 'Tools/Scripts/test-webkitperl']
 
     def __init__(self, **kwargs):
@@ -631,7 +632,22 @@
             return {u'step': unicode(message)}
         return {u'step': u'Failed webkitperl tests'}
 
+    def evaluateCommand(self, cmd):
+        rc = shell.ShellCommand.evaluateCommand(self, cmd)
+        if rc == FAILURE:
+            self.build.addStepsAfterCurrentStep([ReRunWebKitPerlTests()])
+        return rc
 
+
+class ReRunWebKitPerlTests(RunWebKitPerlTests):
+    name = 're-run-webkitperl-tests'
+    flunkOnFailure = True
+    haltOnFailure = True
+
+    def evaluateCommand(self, cmd):
+        return shell.ShellCommand.evaluateCommand(self, cmd)
+
+
 class RunBuildWebKitOrgUnitTests(shell.ShellCommand):
     name = 'build-webkit-org-unit-tests'
     description = ['build-webkit-unit-tests running']

Modified: trunk/Tools/BuildSlaveSupport/ews-build/steps_unittest.py (252410 => 252411)


--- trunk/Tools/BuildSlaveSupport/ews-build/steps_unittest.py	2019-11-13 16:16:42 UTC (rev 252410)
+++ trunk/Tools/BuildSlaveSupport/ews-build/steps_unittest.py	2019-11-13 16:21:56 UTC (rev 252411)
@@ -38,7 +38,7 @@
                    CheckOutSource, CheckOutSpecificRevision, CheckPatchRelevance, CheckStyle, CleanBuild, CleanUpGitIndexLock, CleanWorkingDirectory,
                    CompileJSC, CompileJSCToT, CompileWebKit, CompileWebKitToT, ConfigureBuild,
                    DownloadBuiltProduct, DownloadBuiltProductFromMaster, ExtractBuiltProduct, ExtractTestResults, InstallGtkDependencies, InstallWpeDependencies, KillOldProcesses,
-                   PrintConfiguration, ReRunAPITests, ReRunJavaScriptCoreTests, ReRunWebKitTests, RunAPITests, RunAPITestsWithoutPatch,
+                   PrintConfiguration, ReRunAPITests, ReRunJavaScriptCoreTests, ReRunWebKitPerlTests, ReRunWebKitTests, RunAPITests, RunAPITestsWithoutPatch,
                    RunBindingsTests, RunBuildWebKitOrgUnitTests, RunEWSBuildbotCheckConfig, RunEWSUnitTests, RunJavaScriptCoreTests, RunJSCTestsWithoutPatch, RunWebKit1Tests,
                    RunWebKitPerlTests, RunWebKitPyTests, RunWebKitTests, RunWebKitTestsWithoutPatch, TestWithFailureCount, Trigger, TransferToS3, UnApplyPatchIfRequired,
                    UpdateWorkingDirectory, UploadBuiltProduct, UploadTestResults, ValidatePatch)
@@ -380,8 +380,11 @@
     def tearDown(self):
         return self.tearDownBuildStep()
 
+    def configureStep(self):
+        self.setupStep(RunWebKitPerlTests())
+
     def test_success(self):
-        self.setupStep(RunWebKitPerlTests())
+        self.configureStep()
         self.expectRemoteCommands(
             ExpectShell(workdir='wkdir',
                         logEnviron=False,
@@ -394,7 +397,7 @@
         return self.runStep()
 
     def test_failure(self):
-        self.setupStep(RunWebKitPerlTests())
+        self.configureStep()
         self.expectRemoteCommands(
             ExpectShell(workdir='wkdir',
                         logEnviron=False,
@@ -411,6 +414,11 @@
         return self.runStep()
 
 
+class TestReRunJavaScriptCoreTests(TestRunWebKitPerlTests):
+    def configureStep(self):
+        self.setupStep(ReRunWebKitPerlTests())
+
+
 class TestWebKitPyTests(BuildStepMixinAdditions, unittest.TestCase):
     def setUp(self):
         self.longMessage = True

Modified: trunk/Tools/ChangeLog (252410 => 252411)


--- trunk/Tools/ChangeLog	2019-11-13 16:16:42 UTC (rev 252410)
+++ trunk/Tools/ChangeLog	2019-11-13 16:21:56 UTC (rev 252411)
@@ -1,5 +1,20 @@
 2019-11-13  Aakash Jain  <[email protected]>
 
+        Make webkitperl EWS robust against flakiness in webkitperl tests
+        https://bugs.webkit.org/show_bug.cgi?id=204155
+
+        Reviewed by Jonathan Bedard.
+
+        * BuildSlaveSupport/ews-build/steps.py:
+        (RunWebKitPerlTests): Set haltOnFailure and flunkOnFailure to false.
+        (RunWebKitPerlTests.evaluateCommand): Add re-run step in case of failure.
+        (ReRunWebKitPerlTests):
+        (ReRunWebKitPerlTests.evaluateCommand): Overrided this to avoid running this step again.
+        * BuildSlaveSupport/ews-build/steps_unittest.py:
+        (TestReRunJavaScriptCoreTests): Added unit-tests.
+
+2019-11-13  Aakash Jain  <[email protected]>
+
         Flaky API Test TestWebKitAPI.ServiceWorkers.ThrottleCrash
         https://bugs.webkit.org/show_bug.cgi?id=203734
         <rdar://problem/56814638>
_______________________________________________
webkit-changes mailing list
[email protected]
https://lists.webkit.org/mailman/listinfo/webkit-changes

Reply via email to