Log Message
Buildbot: Force crash log submission after each test run https://bugs.webkit.org/show_bug.cgi?id=210202 <rdar://problem/60508929>
Reviewed by Aakash Jain and Alexey Proskuryakov. * BuildSlaveSupport/build.webkit.org-config/factories.py: (TestFactory.__init__): Add steps triggering crash uploads and waiting for crash collection to quiesce. * BuildSlaveSupport/build.webkit.org-config/steps.py: (TriggerCrashCollection): Add step to trigger crash collection. (WaitForCrashCollection): Add step to wait for for crash collection process to quiesce. * BuildSlaveSupport/build.webkit.org-config/steps.py: (BuildStepsConstructorTest): Update Apple testing steps to include crash log collection. * BuildSlaveSupport/ews-build/factories.py: (TestFactory): (TestFactory.__init__): Add willTriggerCrashLogSubmission variable. (iOSTestsFactory): (macOSWK1Factory): (macOSWK2Factory): * BuildSlaveSupport/ews-build/factories_unittest.py: (TestLayoutTestsFactory.test_macos_wk1_release_factory): (TestLayoutTestsFactory.test_macos_wk1_debug_factory): (TestLayoutTestsFactory.test_macos_wk2_factory): (TestLayoutTestsFactory.test_ios_wk2_factory): * BuildSlaveSupport/ews-build/steps.py: (TriggerCrashLogSubmission): (WaitForCrashCollection): * BuildSlaveSupport/trigger-crash-collection: Added. * BuildSlaveSupport/wait-for-crash-collection: Added. (pid_for_name): Given a process name, return the PID for that process. (cpu_percentage): Return float representing the current CPU usage of the provided PID. (main):
Modified Paths
- trunk/Tools/BuildSlaveSupport/build.webkit.org-config/factories.py
- trunk/Tools/BuildSlaveSupport/build.webkit.org-config/steps.py
- trunk/Tools/BuildSlaveSupport/build.webkit.org-config/steps_unittest.py
- trunk/Tools/BuildSlaveSupport/ews-build/factories.py
- trunk/Tools/BuildSlaveSupport/ews-build/factories_unittest.py
- trunk/Tools/BuildSlaveSupport/ews-build/steps.py
- trunk/Tools/ChangeLog
Added Paths
Diff
Modified: trunk/Tools/BuildSlaveSupport/build.webkit.org-config/factories.py (259810 => 259811)
--- trunk/Tools/BuildSlaveSupport/build.webkit.org-config/factories.py 2020-04-09 18:22:00 UTC (rev 259810)
+++ trunk/Tools/BuildSlaveSupport/build.webkit.org-config/factories.py 2020-04-09 19:00:37 UTC (rev 259811)
@@ -84,6 +84,9 @@
if platform == 'wincairo':
self.addStep(InstallWinCairoDependencies())
+ if platform.startswith('mac') or platform.startswith('ios-simulator'):
+ self.addStep(WaitForCrashCollection())
+
if self.JSCTestClass:
self.addStep(self.JSCTestClass())
if self.LayoutTestClass:
@@ -101,6 +104,10 @@
self.addStep(RunBuiltinsTests())
if not platform.startswith('win'):
self.addStep(RunDashboardTests())
+
+ if platform.startswith('mac') or platform.startswith('ios-simulator'):
+ self.addStep(TriggerCrashLogSubmission())
+
if self.LayoutTestClass:
self.addStep(ArchiveTestResults())
self.addStep(UploadTestResults())
Modified: trunk/Tools/BuildSlaveSupport/build.webkit.org-config/steps.py (259810 => 259811)
--- trunk/Tools/BuildSlaveSupport/build.webkit.org-config/steps.py 2020-04-09 18:22:00 UTC (rev 259810)
+++ trunk/Tools/BuildSlaveSupport/build.webkit.org-config/steps.py 2020-04-09 19:00:37 UTC (rev 259811)
@@ -1,4 +1,4 @@
-# Copyright (C) 2017 Apple Inc. All rights reserved.
+# Copyright (C) 2017-2020 Apple Inc. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
@@ -140,6 +140,20 @@
command = ["python", "./Tools/BuildSlaveSupport/kill-old-processes", "buildbot"]
+class TriggerCrashLogSubmission(shell.Compile):
+ name = "trigger-crash-log-submission"
+ description = ["triggering crash log submission"]
+ descriptionDone = ["triggered crash log submission"]
+ command = ["python", "./Tools/BuildSlaveSupport/trigger-crash-log-submission"]
+
+
+class WaitForCrashCollection(shell.Compile):
+ name = "wait-for-crash-collection"
+ description = ["waiting for crash collection to quiesce"]
+ descriptionDone = ["crash collection has quiesced"]
+ command = ["python", "./Tools/BuildSlaveSupport/wait-for-crash-collection", "--timeout", str(5 * 60)]
+
+
class CleanBuildIfScheduled(shell.Compile):
name = "delete WebKitBuild directory"
description = ["deleting WebKitBuild directory"]
Modified: trunk/Tools/BuildSlaveSupport/build.webkit.org-config/steps_unittest.py (259810 => 259811)
--- trunk/Tools/BuildSlaveSupport/build.webkit.org-config/steps_unittest.py 2020-04-09 18:22:00 UTC (rev 259810)
+++ trunk/Tools/BuildSlaveSupport/build.webkit.org-config/steps_unittest.py 2020-04-09 19:00:37 UTC (rev 259811)
@@ -528,35 +528,35 @@
'Apple-Catalina-Debug-Build': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'compile-webkit', 'archive-built-product', 'upload', 'archive-built-product', 'upload', 'transfer-to-s3', 'trigger'],
'Apple-Catalina-Debug-JSC-Tests': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'jscore-test'],
'Apple-Catalina-Debug-Test262-Tests': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'test262-test'],
- 'Apple-Catalina-Debug-WK1-Tests': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'layout-test', 'run-api-tests', 'lldb-webkit-test', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'archive-test-results', 'upload', 'MasterShellCommand'],
- 'Apple-Catalina-Debug-WK2-Tests': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'layout-test', 'run-api-tests', 'lldb-webkit-test', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'archive-test-results', 'upload', 'MasterShellCommand'],
- 'Apple-Catalina-Debug-WK2-WebGL-Tests': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'layout-test', 'run-api-tests', 'lldb-webkit-test', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'archive-test-results', 'upload', 'MasterShellCommand'],
- 'Apple-Catalina-Debug-WK2-GPUProcess-Tests': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'layout-test', 'run-api-tests', 'lldb-webkit-test', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'archive-test-results', 'upload', 'MasterShellCommand'],
+ 'Apple-Catalina-Debug-WK1-Tests': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'wait-for-crash-collection', 'layout-test', 'run-api-tests', 'lldb-webkit-test', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'trigger-crash-log-submission', 'archive-test-results', 'upload', 'MasterShellCommand'],
+ 'Apple-Catalina-Debug-WK2-Tests': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'wait-for-crash-collection', 'layout-test', 'run-api-tests', 'lldb-webkit-test', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'trigger-crash-log-submission', 'archive-test-results', 'upload', 'MasterShellCommand'],
+ 'Apple-Catalina-Debug-WK2-WebGL-Tests': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'wait-for-crash-collection', 'layout-test', 'run-api-tests', 'lldb-webkit-test', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'trigger-crash-log-submission', 'archive-test-results', 'upload', 'MasterShellCommand'],
+ 'Apple-Catalina-Debug-WK2-GPUProcess-Tests': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'wait-for-crash-collection', 'layout-test', 'run-api-tests', 'lldb-webkit-test', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'trigger-crash-log-submission', 'archive-test-results', 'upload', 'MasterShellCommand'],
'Apple-Catalina-LLINT-CLoop-BuildAndTest': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'compile-webkit', 'webkit-jsc-cloop-test'],
'Apple-Catalina-Release-Build': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'compile-webkit', 'archive-built-product', 'upload', 'archive-built-product', 'upload', 'transfer-to-s3', 'trigger'],
'Apple-Catalina-Release-JSC-Tests': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'jscore-test'],
'Apple-Catalina-Release-Test262-Tests': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'test262-test'],
- 'Apple-Catalina-Release-WK1-Tests': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'layout-test', 'run-api-tests', 'lldb-webkit-test', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'archive-test-results', 'upload', 'MasterShellCommand'],
- 'Apple-Catalina-Release-WK2-Tests': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'layout-test', 'run-api-tests', 'lldb-webkit-test', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'archive-test-results', 'upload', 'MasterShellCommand'],
+ 'Apple-Catalina-Release-WK1-Tests': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'wait-for-crash-collection', 'layout-test', 'run-api-tests', 'lldb-webkit-test', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'trigger-crash-log-submission', 'archive-test-results', 'upload', 'MasterShellCommand'],
+ 'Apple-Catalina-Release-WK2-Tests': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'wait-for-crash-collection', 'layout-test', 'run-api-tests', 'lldb-webkit-test', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'trigger-crash-log-submission', 'archive-test-results', 'upload', 'MasterShellCommand'],
'Apple-Catalina-Release-WK2-Perf': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'perf-test'],
'Apple Mojave Debug (Build)': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'compile-webkit', 'archive-built-product', 'upload', 'archive-built-product', 'upload', 'transfer-to-s3', 'trigger'],
- 'Apple Mojave Debug WK1 (Tests)': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'layout-test', 'run-api-tests', 'lldb-webkit-test', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'archive-test-results', 'upload', 'MasterShellCommand'],
- 'Apple Mojave Debug WK2 (Tests)': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'layout-test', 'run-api-tests', 'lldb-webkit-test', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'archive-test-results', 'upload', 'MasterShellCommand'],
+ 'Apple Mojave Debug WK1 (Tests)': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'wait-for-crash-collection', 'layout-test', 'run-api-tests', 'lldb-webkit-test', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'trigger-crash-log-submission', 'archive-test-results', 'upload', 'MasterShellCommand'],
+ 'Apple Mojave Debug WK2 (Tests)': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'wait-for-crash-collection', 'layout-test', 'run-api-tests', 'lldb-webkit-test', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'trigger-crash-log-submission', 'archive-test-results', 'upload', 'MasterShellCommand'],
'Apple Mojave Release (Build)': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'compile-webkit', 'archive-built-product', 'upload', 'archive-built-product', 'upload', 'transfer-to-s3', 'trigger'],
- 'Apple Mojave Release WK1 (Tests)': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'layout-test', 'run-api-tests', 'lldb-webkit-test', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'archive-test-results', 'upload', 'MasterShellCommand'],
- 'Apple Mojave Release WK2 (Tests)': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'layout-test', 'run-api-tests', 'lldb-webkit-test', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'archive-test-results', 'upload', 'MasterShellCommand'],
+ 'Apple Mojave Release WK1 (Tests)': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'wait-for-crash-collection', 'layout-test', 'run-api-tests', 'lldb-webkit-test', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'trigger-crash-log-submission', 'archive-test-results', 'upload', 'MasterShellCommand'],
+ 'Apple Mojave Release WK2 (Tests)': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'wait-for-crash-collection', 'layout-test', 'run-api-tests', 'lldb-webkit-test', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'trigger-crash-log-submission', 'archive-test-results', 'upload', 'MasterShellCommand'],
'Apple Mojave Release (Build)': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'compile-webkit', 'archive-built-product', 'upload', 'archive-built-product', 'upload', 'transfer-to-s3', 'trigger'],
'Apple iOS 13 Release (Build)': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'compile-webkit'],
'Apple iOS 13 Simulator Release (Build)': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'compile-webkit', 'archive-built-product', 'upload', 'archive-built-product', 'upload', 'transfer-to-s3', 'trigger'],
- 'Apple iOS 13 Simulator Release WK2 (Tests)': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'layout-test', 'run-api-tests', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'archive-test-results', 'upload', 'MasterShellCommand'],
+ 'Apple iOS 13 Simulator Release WK2 (Tests)': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'wait-for-crash-collection', 'layout-test', 'run-api-tests', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'trigger-crash-log-submission', 'archive-test-results', 'upload', 'MasterShellCommand'],
'Apple iOS 13 Simulator Debug (Build)': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'compile-webkit', 'archive-built-product', 'upload', 'archive-built-product', 'upload', 'transfer-to-s3', 'trigger'],
- 'Apple iOS 13 Simulator Debug WK2 (Tests)': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'layout-test', 'run-api-tests', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'archive-test-results', 'upload', 'MasterShellCommand'],
+ 'Apple iOS 13 Simulator Debug WK2 (Tests)': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'wait-for-crash-collection', 'layout-test', 'run-api-tests', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'trigger-crash-log-submission', 'archive-test-results', 'upload', 'MasterShellCommand'],
- 'Apple iPadOS 13 Simulator Release WK2 (Tests)': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'layout-test', 'run-api-tests', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'archive-test-results', 'upload', 'MasterShellCommand'],
- 'Apple iPadOS 13 Simulator Debug WK2 (Tests)': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'layout-test', 'run-api-tests', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'archive-test-results', 'upload', 'MasterShellCommand'],
+ 'Apple iPadOS 13 Simulator Release WK2 (Tests)': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'wait-for-crash-collection', 'layout-test', 'run-api-tests', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'trigger-crash-log-submission', 'archive-test-results', 'upload', 'MasterShellCommand'],
+ 'Apple iPadOS 13 Simulator Debug WK2 (Tests)': ['configure build', 'svn', 'kill old processes', 'delete WebKitBuild directory', 'delete stale build files', 'download-built-product', 'extract-built-product', 'wait-for-crash-collection', 'layout-test', 'run-api-tests', 'webkitpy-test', 'webkitperl-test', 'bindings-generation-tests', 'builtins-generator-tests', 'dashboard-tests', 'trigger-crash-log-submission', 'archive-test-results', 'upload', 'MasterShellCommand'],
'JSCOnly Linux ARMv7 Thumb2 Release': ['configure build', 'svn', 'delete WebKitBuild directory', 'delete stale build files', 'compile-webkit', 'jscore-test'],
'JSCOnly Linux ARMv7 Thumb2 SoftFP Release': ['configure build', 'svn', 'delete WebKitBuild directory', 'delete stale build files', 'compile-webkit', 'jscore-test'],
Modified: trunk/Tools/BuildSlaveSupport/ews-build/factories.py (259810 => 259811)
--- trunk/Tools/BuildSlaveSupport/ews-build/factories.py 2020-04-09 18:22:00 UTC (rev 259810)
+++ trunk/Tools/BuildSlaveSupport/ews-build/factories.py 2020-04-09 19:00:37 UTC (rev 259811)
@@ -29,9 +29,9 @@
DownloadBuiltProduct, ExtractBuiltProduct, FindModifiedChangeLogs, InstallGtkDependencies,
InstallWpeDependencies, KillOldProcesses, PrintConfiguration, PushCommitToWebKitRepo,
RunAPITests, RunBindingsTests, RunBuildWebKitOrgUnitTests, RunEWSBuildbotCheckConfig, RunEWSUnitTests,
- RunResultsdbpyTests, RunJavaScriptCoreTests, RunWebKit1Tests, RunWebKitPerlTests,
- RunWebKitPyPython2Tests, RunWebKitPyPython3Tests, RunWebKitTests, SetBuildSummary, UpdateWorkingDirectory,
- ValidatePatch, ValidateChangeLogAndReviewer, ValidateCommiterAndReviewer)
+ RunResultsdbpyTests, RunJavaScriptCoreTests, RunWebKit1Tests, RunWebKitPerlTests, RunWebKitPyPython2Tests,
+ RunWebKitPyPython3Tests, RunWebKitTests, SetBuildSummary, TriggerCrashLogSubmission, UpdateWorkingDirectory,
+ ValidatePatch, ValidateChangeLogAndReviewer, ValidateCommiterAndReviewer, WaitForCrashCollection)
class Factory(factory.BuildFactory):
@@ -106,6 +106,7 @@
class TestFactory(Factory):
LayoutTestClass = None
APITestClass = None
+ willTriggerCrashLogSubmission = False
def getProduct(self):
self.addStep(DownloadBuiltProduct())
@@ -116,12 +117,17 @@
if platform == 'gtk':
self.addStep(InstallGtkDependencies())
self.getProduct()
+ if self.willTriggerCrashLogSubmission:
+ self.addStep(WaitForCrashCollection())
self.addStep(KillOldProcesses())
if self.LayoutTestClass:
self.addStep(self.LayoutTestClass())
- self.addStep(SetBuildSummary())
if self.APITestClass:
self.addStep(self.APITestClass())
+ if self.willTriggerCrashLogSubmission:
+ self.addStep(TriggerCrashLogSubmission())
+ if self.LayoutTestClass:
+ self.addStep(SetBuildSummary())
class JSCTestsFactory(Factory):
@@ -143,6 +149,7 @@
class iOSTestsFactory(TestFactory):
LayoutTestClass = RunWebKitTests
+ willTriggerCrashLogSubmission = True
class macOSBuildFactory(BuildFactory):
@@ -151,6 +158,7 @@
class macOSWK1Factory(TestFactory):
LayoutTestClass = RunWebKit1Tests
+ willTriggerCrashLogSubmission = True
def __init__(self, platform, configuration=None, architectures=None, additionalArguments=None, checkRelevance=False, **kwargs):
super(macOSWK1Factory, self).__init__(platform=platform, configuration=configuration, architectures=architectures, additionalArguments=additionalArguments, checkRelevance=True, **kwargs)
@@ -158,6 +166,7 @@
class macOSWK2Factory(TestFactory):
LayoutTestClass = RunWebKitTests
+ willTriggerCrashLogSubmission = True
class WindowsFactory(Factory):
Modified: trunk/Tools/BuildSlaveSupport/ews-build/factories_unittest.py (259810 => 259811)
--- trunk/Tools/BuildSlaveSupport/ews-build/factories_unittest.py 2020-04-09 18:22:00 UTC (rev 259810)
+++ trunk/Tools/BuildSlaveSupport/ews-build/factories_unittest.py 2020-04-09 19:00:37 UTC (rev 259811)
@@ -216,8 +216,10 @@
_BuildStepFactory(steps.ApplyPatch),
_BuildStepFactory(steps.DownloadBuiltProduct),
_BuildStepFactory(steps.ExtractBuiltProduct),
+ _BuildStepFactory(steps.WaitForCrashCollection),
_BuildStepFactory(steps.KillOldProcesses),
_BuildStepFactory(steps.RunWebKit1Tests),
+ _BuildStepFactory(steps.TriggerCrashLogSubmission),
_BuildStepFactory(steps.SetBuildSummary),
])
@@ -233,8 +235,10 @@
_BuildStepFactory(steps.ApplyPatch),
_BuildStepFactory(steps.DownloadBuiltProduct),
_BuildStepFactory(steps.ExtractBuiltProduct),
+ _BuildStepFactory(steps.WaitForCrashCollection),
_BuildStepFactory(steps.KillOldProcesses),
_BuildStepFactory(steps.RunWebKit1Tests),
+ _BuildStepFactory(steps.TriggerCrashLogSubmission),
_BuildStepFactory(steps.SetBuildSummary),
])
@@ -249,8 +253,10 @@
_BuildStepFactory(steps.ApplyPatch),
_BuildStepFactory(steps.DownloadBuiltProduct),
_BuildStepFactory(steps.ExtractBuiltProduct),
+ _BuildStepFactory(steps.WaitForCrashCollection),
_BuildStepFactory(steps.KillOldProcesses),
_BuildStepFactory(steps.RunWebKitTests),
+ _BuildStepFactory(steps.TriggerCrashLogSubmission),
_BuildStepFactory(steps.SetBuildSummary),
])
@@ -265,8 +271,10 @@
_BuildStepFactory(steps.ApplyPatch),
_BuildStepFactory(steps.DownloadBuiltProduct),
_BuildStepFactory(steps.ExtractBuiltProduct),
+ _BuildStepFactory(steps.WaitForCrashCollection),
_BuildStepFactory(steps.KillOldProcesses),
_BuildStepFactory(steps.RunWebKitTests),
+ _BuildStepFactory(steps.TriggerCrashLogSubmission),
_BuildStepFactory(steps.SetBuildSummary),
])
Modified: trunk/Tools/BuildSlaveSupport/ews-build/steps.py (259810 => 259811)
--- trunk/Tools/BuildSlaveSupport/ews-build/steps.py 2020-04-09 18:22:00 UTC (rev 259810)
+++ trunk/Tools/BuildSlaveSupport/ews-build/steps.py 2020-04-09 19:00:37 UTC (rev 259811)
@@ -1633,6 +1633,36 @@
return shell.Compile.getResultSummary(self)
+class TriggerCrashLogSubmission(shell.Compile):
+ name = 'trigger-crash-log-submission'
+ description = ['triggering crash log submission']
+ descriptionDone = ['Triggered crash log submission']
+ command = ['python', 'Tools/BuildSlaveSupport/trigger-crash-log-submission']
+
+ def __init__(self, **kwargs):
+ super(TriggerCrashLogSubmission, self).__init__(timeout=60, logEnviron=False, **kwargs)
+
+ def getResultSummary(self):
+ if self.results in [FAILURE, EXCEPTION]:
+ return {u'step': u'Failed to trigger crash log submission'}
+ return shell.Compile.getResultSummary(self)
+
+
+class WaitForCrashCollection(shell.Compile):
+ name = 'wait for crash collection'
+ description = ['waiting-for-crash-collection-to-quiesce']
+ descriptionDone = ['Crash collection has quiesced']
+ command = ['python', 'Tools/BuildSlaveSupport/wait-for-crash-collection', '--timeout', str(5 * 60)]
+
+ def __init__(self, **kwargs):
+ super(WaitForCrashCollection, self).__init__(timeout=6 * 60, logEnviron=False, **kwargs)
+
+ def getResultSummary(self):
+ if self.results in [FAILURE, EXCEPTION]:
+ return {u'step': u'Crash log collection process still running'}
+ return shell.Compile.getResultSummary(self)
+
+
class RunWebKitTests(shell.Test):
name = 'layout-tests'
description = ['layout-tests running']
Added: trunk/Tools/BuildSlaveSupport/trigger-crash-log-submission (0 => 259811)
--- trunk/Tools/BuildSlaveSupport/trigger-crash-log-submission (rev 0)
+++ trunk/Tools/BuildSlaveSupport/trigger-crash-log-submission 2020-04-09 19:00:37 UTC (rev 259811)
@@ -0,0 +1,53 @@
+#!/usr/bin/env python
+# Copyright (C) 2020 Apple Inc. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# 1. Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY APPLE AND ITS CONTRIBUTORS "AS IS" AND ANY
+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+# DISCLAIMED. IN NO EVENT SHALL APPLE OR ITS CONTRIBUTORS BE LIABLE FOR ANY
+# DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
+# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+import subprocess
+import sys
+import time
+
+WAIT_SECONDS = 3
+
+
+def main():
+ if sys.platform != 'darwin':
+ print("No crash log submission available for platform '{}'".format(sys.platform))
+ return 0
+
+ # Workaround for <rdar://problem/60507877>.
+ if subprocess.call(['/usr/bin/killall', '-9', 'diagnostics_agent']):
+ print('Failed to kill diagnostics_agent')
+ return 1
+ print('Killed diagnostics_agent')
+
+ time.sleep(WAIT_SECONDS)
+ if subprocess.call(['/usr/bin/notifyutil', '-p', 'com.apple.crashreporter.debug.submit.now']):
+ print('Failed to trigger crash log submission')
+ return 1
+ print('Triggered crash log submission')
+
+ return 0
+
+
+if __name__ == '__main__':
+ sys.exit(main())
Property changes on: trunk/Tools/BuildSlaveSupport/trigger-crash-log-submission
___________________________________________________________________
Added: svn:executable
Added: trunk/Tools/BuildSlaveSupport/wait-for-crash-collection (0 => 259811)
--- trunk/Tools/BuildSlaveSupport/wait-for-crash-collection (rev 0)
+++ trunk/Tools/BuildSlaveSupport/wait-for-crash-collection 2020-04-09 19:00:37 UTC (rev 259811)
@@ -0,0 +1,89 @@
+#!/usr/bin/env python
+# Copyright (C) 2020 Apple Inc. All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# 1. Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY APPLE AND ITS CONTRIBUTORS "AS IS" AND ANY
+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+# DISCLAIMED. IN NO EVENT SHALL APPLE OR ITS CONTRIBUTORS BE LIABLE FOR ANY
+# DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
+# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
+# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+import argparse
+import subprocess
+import sys
+import time
+
+
+SUBMIT_DIAG_INFO = '/System/Library/CoreServices/SubmitDiagInfo'
+WAIT_SECONDS = 3
+
+
+def pid_for_name(process_name):
+ process = subprocess.Popen(['/bin/ps', '-eo', 'pid,comm'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+
+ stdout, _ = process.communicate()
+ for line in stdout.splitlines():
+ try:
+ pid, candidate = line.strip().split(b' ', 1)
+ if candidate == process_name:
+ return int(pid)
+ except ValueError:
+ pass
+ return None
+
+
+def cpu_percentage(pid):
+ top_output = subprocess.check_output(['/usr/bin/top', '-pid', str(pid), '-stats', 'cpu', '-l', '2'])
+ try:
+ return float(top_output.splitlines()[-1])
+ except ValueError:
+ return 0
+
+
+def main():
+ parser = argparse.ArgumentParser(description='Wait for crash log reporting to quiesce')
+ parser.add_argument(
+ '-t', '--timeout',
+ help='Seconds to wait for process to quiesce.',
+ type=int, default=0,
+ )
+ args = parser.parse_args()
+ deadline = (time.time() + args.timeout) if args.timeout else None
+
+ if sys.platform != 'darwin':
+ print("No crash reporting available for platform '{}'".format(sys.platform))
+ return 0
+
+ pid = pid_for_name(SUBMIT_DIAG_INFO)
+ if not pid:
+ print('Failed to find {}'.format(SUBMIT_DIAG_INFO))
+ return 0
+ print('Found {} running with PID {}'.format(SUBMIT_DIAG_INFO, pid))
+
+ print('Waiting for process {} to quiesce'.format(pid))
+ while not deadline or deadline > time.time():
+ if cpu_percentage(pid) <= 5:
+ print('Process {} has quiesced'.format(pid))
+ return 0
+ time.sleep(WAIT_SECONDS)
+
+ print('Timed out waiting for {} to quiesce, continue'.format(pid))
+ return 0
+
+
+if __name__ == '__main__':
+ sys.exit(main())
Property changes on: trunk/Tools/BuildSlaveSupport/wait-for-crash-collection
___________________________________________________________________
Added: svn:executable
Modified: trunk/Tools/ChangeLog (259810 => 259811)
--- trunk/Tools/ChangeLog 2020-04-09 18:22:00 UTC (rev 259810)
+++ trunk/Tools/ChangeLog 2020-04-09 19:00:37 UTC (rev 259811)
@@ -1,3 +1,39 @@
+2020-04-09 Jonathan Bedard <jbed...@apple.com>
+
+ Buildbot: Force crash log submission after each test run
+ https://bugs.webkit.org/show_bug.cgi?id=210202
+ <rdar://problem/60508929>
+
+ Reviewed by Aakash Jain and Alexey Proskuryakov.
+
+ * BuildSlaveSupport/build.webkit.org-config/factories.py:
+ (TestFactory.__init__): Add steps triggering crash uploads and waiting for crash
+ collection to quiesce.
+ * BuildSlaveSupport/build.webkit.org-config/steps.py:
+ (TriggerCrashCollection): Add step to trigger crash collection.
+ (WaitForCrashCollection): Add step to wait for for crash collection process to quiesce.
+ * BuildSlaveSupport/build.webkit.org-config/steps.py:
+ (BuildStepsConstructorTest): Update Apple testing steps to include crash log collection.
+ * BuildSlaveSupport/ews-build/factories.py:
+ (TestFactory):
+ (TestFactory.__init__): Add willTriggerCrashLogSubmission variable.
+ (iOSTestsFactory):
+ (macOSWK1Factory):
+ (macOSWK2Factory):
+ * BuildSlaveSupport/ews-build/factories_unittest.py:
+ (TestLayoutTestsFactory.test_macos_wk1_release_factory):
+ (TestLayoutTestsFactory.test_macos_wk1_debug_factory):
+ (TestLayoutTestsFactory.test_macos_wk2_factory):
+ (TestLayoutTestsFactory.test_ios_wk2_factory):
+ * BuildSlaveSupport/ews-build/steps.py:
+ (TriggerCrashLogSubmission):
+ (WaitForCrashCollection):
+ * BuildSlaveSupport/trigger-crash-collection: Added.
+ * BuildSlaveSupport/wait-for-crash-collection: Added.
+ (pid_for_name): Given a process name, return the PID for that process.
+ (cpu_percentage): Return float representing the current CPU usage of the provided PID.
+ (main):
+
2020-04-09 Aakash Jain <aakash_j...@apple.com>
[ews] rename CompileJSCToT to CompileJSCWithoutPatch
_______________________________________________ webkit-changes mailing list webkit-changes@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-changes
