Title: [281297] trunk/Tools
- Revision
- 281297
- Author
- [email protected]
- Date
- 2021-08-20 00:50:20 -0700 (Fri, 20 Aug 2021)
Log Message
Remove all WTR output before Content-Type:text/plain for WebRTC tests in case of known excessive logging
https://bugs.webkit.org/show_bug.cgi?id=229188
Reviewed by Jonathan Bedard.
Some logging beneath WebKit is making WebRTC tests flaky.
While we work on the cause of this bug, we can still beef-up our test output clean up steps to handle this logging.
Instead of stripping lines based on a known pattern, which does not always work as per current results,
we can simply remove all logging before the 'Content-Type:text/plain\n' string that WebKitTestRunner is adding before
printing the text generated by the test itself.
We update webkitpy scripts accordingly and add this new clean-up for:
- tests that have webrtc in their name
- tests whose output contain some known logging strings like 'LRP' or 'Negotiation String'.
This should hopefully be more robust while not hiding real regressions.
* Scripts/webkitpy/layout_tests/controllers/single_test_runner.py:
(SingleTestRunner):
* Scripts/webkitpy/port/base.py:
(Port.logging_detectors_to_strip_text_start):
* Scripts/webkitpy/port/driver.py:
(DriverOutput.strip_text_start_if_needed):
* Scripts/webkitpy/port/mac.py:
(MacPort.logging_patterns_to_strip):
(MacPort.logging_detectors_to_strip_text_start):
Modified Paths
Diff
Modified: trunk/Tools/ChangeLog (281296 => 281297)
--- trunk/Tools/ChangeLog 2021-08-20 06:20:46 UTC (rev 281296)
+++ trunk/Tools/ChangeLog 2021-08-20 07:50:20 UTC (rev 281297)
@@ -1,3 +1,30 @@
+2021-08-20 Youenn Fablet <[email protected]>
+
+ Remove all WTR output before Content-Type:text/plain for WebRTC tests in case of known excessive logging
+ https://bugs.webkit.org/show_bug.cgi?id=229188
+
+ Reviewed by Jonathan Bedard.
+
+ Some logging beneath WebKit is making WebRTC tests flaky.
+ While we work on the cause of this bug, we can still beef-up our test output clean up steps to handle this logging.
+ Instead of stripping lines based on a known pattern, which does not always work as per current results,
+ we can simply remove all logging before the 'Content-Type:text/plain\n' string that WebKitTestRunner is adding before
+ printing the text generated by the test itself.
+ We update webkitpy scripts accordingly and add this new clean-up for:
+ - tests that have webrtc in their name
+ - tests whose output contain some known logging strings like 'LRP' or 'Negotiation String'.
+ This should hopefully be more robust while not hiding real regressions.
+
+ * Scripts/webkitpy/layout_tests/controllers/single_test_runner.py:
+ (SingleTestRunner):
+ * Scripts/webkitpy/port/base.py:
+ (Port.logging_detectors_to_strip_text_start):
+ * Scripts/webkitpy/port/driver.py:
+ (DriverOutput.strip_text_start_if_needed):
+ * Scripts/webkitpy/port/mac.py:
+ (MacPort.logging_patterns_to_strip):
+ (MacPort.logging_detectors_to_strip_text_start):
+
2021-08-19 Carlos Alberto Lopez Perez <[email protected]>
[build.webkit.org] Port old unit test with the expected build steps to the new buildbot
Modified: trunk/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py (281296 => 281297)
--- trunk/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py 2021-08-20 06:20:46 UTC (rev 281296)
+++ trunk/Tools/Scripts/webkitpy/layout_tests/controllers/single_test_runner.py 2021-08-20 07:50:20 UTC (rev 281297)
@@ -132,6 +132,8 @@
expected_driver_output.strip_patterns(patterns)
driver_output.strip_patterns(patterns)
+ driver_output.strip_text_start_if_needed(self._port.logging_detectors_to_strip_text_start(self._driver_input().test_name))
+
driver_output.strip_stderror_patterns(self._port.stderr_patterns_to_strip())
test_result = self._compare_output(expected_driver_output, driver_output)
Modified: trunk/Tools/Scripts/webkitpy/port/base.py (281296 => 281297)
--- trunk/Tools/Scripts/webkitpy/port/base.py 2021-08-20 06:20:46 UTC (rev 281296)
+++ trunk/Tools/Scripts/webkitpy/port/base.py 2021-08-20 07:50:20 UTC (rev 281297)
@@ -1420,6 +1420,9 @@
def stderr_patterns_to_strip(self):
return []
+ def logging_detectors_to_strip_text_start(self, test_name):
+ return []
+
def test_expectations_file_position(self):
# By default baseline search path schema is i.e. port-wk2 -> wk2 -> port -> generic, so port expectations file is at second to last position.
return 1
Modified: trunk/Tools/Scripts/webkitpy/port/driver.py (281296 => 281297)
--- trunk/Tools/Scripts/webkitpy/port/driver.py 2021-08-20 06:20:46 UTC (rev 281296)
+++ trunk/Tools/Scripts/webkitpy/port/driver.py 2021-08-20 07:50:20 UTC (rev 281297)
@@ -114,6 +114,20 @@
for pattern in patterns:
self.text = re.sub(pattern[0], pattern[1], self.text)
+ def strip_text_start_if_needed(self, detectors):
+ if not self.text or not len(detectors):
+ return
+
+ result = self.text.split('Content-Type: text/plain\n')
+ if len(result) != 2:
+ return
+
+ for detector in detectors:
+ if detector in result[0]:
+ self.text = result[1]
+ self.error += '\nRemoved logging from stdout:\n' + result[0]
+ return
+
def strip_stderror_patterns(self, patterns):
if not self.error:
return
Modified: trunk/Tools/Scripts/webkitpy/port/mac.py (281296 => 281297)
--- trunk/Tools/Scripts/webkitpy/port/mac.py 2021-08-20 06:20:46 UTC (rev 281296)
+++ trunk/Tools/Scripts/webkitpy/port/mac.py 2021-08-20 07:50:20 UTC (rev 281297)
@@ -290,15 +290,17 @@
# FIXME: Find where this is coming from and file a bug to have it removed (then remove this line).
logging_patterns.append((re.compile('VP9 Info:.*\n'), ''))
- # FIXME: Replace this with a defaults write after bots update to a build that has rdar://problem/74031400 fixed.
- logging_patterns.append((r'Negotiation String:.*\n', ''))
- logging_patterns.append((r'LRP.*\n', ''))
- logging_patterns.append((r'.* \(seconds\) ', ''))
- logging_patterns.append((r'.* 0 0 \[\]\n', ''))
- logging_patterns.append((r'\[\]\n', ''))
-
return logging_patterns
+ def logging_detectors_to_strip_text_start(self, test_name):
+ logging_detectors = []
+
+ if 'webrtc' in test_name:
+ logging_detectors.append('Negotiation String:')
+ logging_detectors.append('LRP')
+
+ return logging_detectors
+
def stderr_patterns_to_strip(self):
worthless_patterns = []
worthless_patterns.append((re.compile('.*(Fig|fig|itemasync|vt|mv_|PullParamSetSPS|ccrp_|client).* signalled err=.*\n'), ''))
_______________________________________________
webkit-changes mailing list
[email protected]
https://lists.webkit.org/mailman/listinfo/webkit-changes
