Hi Everyone,
I want to update everyone with the further improvements I have made to new EWS.
As always, please feel free to provide any feedback (either by filing bugs or
contacting me directly).
New Features:
- Moved following queues to from old to new EWS:
• iOS 12 Tests EWS
• macOS WK1 and WK2 Release Tests EWS
• macOS WK1 Debug Tests EWS
• GTK and WPE EWS
• WinCairo EWS
• Style (and watchlist) EWS
- Added a new 'services' EWS to run Buildbot unit tests
https://webkit.org/b/199994
- Separate status bubbles for builders and layout-testers
Infrastructure Improvements:
- EWS is now able to automatically recover (and retry the build) in case of
certain git and network errors https://webkit.org/b/199722
- Some EWS bots now reboot periodically to ensure that start from clean state
- Added KillOldProcesses step before running API or Layout tests to increase
robustness https://webkit.org/b/199592
- 'view layout test results' option is now displayed next to layout-test build
step https://webkit.org/b/200048
- EWS now displays pre-existing Layout test failure names in the build summary
https://webkit.org/b/199941
- Machine uptime is now reported in PrintConfiguration step
https://webkit.org/b/200812
- Added EWS Buildbot support to Internal scripts to easily find/reconfigure bots
- Flakiness in API tests has been further reduced (thanks to multiple fixes by
WebKit developers)
- Added more unit tests
Bug fixes:
- EWS status-bubbles are sometimes multi-row with scroll-bar
https://webkit.org/b/199939
- EWS: Cannot see build status page when patch is waiting for tester
https://webkit.org/b/200333
- EWS: Do not append additional '(failure)' string at the end of custom failure
message in EWS Buildbot https://webkit.org/b/201140
- [ews] Status bubble should display only important messages in pop-over
https://webkit.org/b/201308
Only remaining queues on old EWS are windows, jsc and commit-queue, which I
will be working on next.
Thanks
Aakash
> On Jul 11, 2019, at 10:18 am, Aakash Jain <aakash_j...@apple.com> wrote:
>
> Hi Everyone,
>
> I want to update everyone with the further improvements I have made to new
> EWS. As always, please feel encouraged to provide any feedback (either by
> filing bugs or contacting me directly).
>
> New Features:
> - Launched 'Security EWS'
> - Added iOS-12 Builder queue on new EWS (moved from old to new EWS)
> - Added WPE and GTK queues on new EWS (moved from old to new EWS)
> - New EWS can now process large patches (larger than 640kb)
> https://webkit.org/b/198851
>
>
> Infrastructure Improvements:
> - EWS now automatically retries builds in case of certain infrastructure
> issues and ToT compile failures https://bugs.webkit.org/show_bug.cgi?id=199025
> - Improved stability by adding KillOldProcesses step before API and Layout
> tests https://webkit.org/b/199592
> - Shared bots across queues to improve efficiency https://webkit.org/b/198370
> - Added check for duplicate workers in config.json https://webkit.org/b/199240
> - Patch link now opens the pretty-patch url https://webkit.org/b/199031
> - Triggered builds should use same revision as parent build (fixes a corner
> case) https://webkit.org/b/198289
> - Display pre-existing API test failures in build summary to help
> bot-watchers https://webkit.org/b/199525
> - Send email notifications for failures (for maintenance/bot-watching)
> https://webkit.org/b/198919
> - Remove unused buildbot tabs for better readability
> https://webkit.org/b/198108
> - Allow skipping uploading built product for few builders
> https://webkit.org/b/199422
> - EWS should provide option to download layout test results zip file
> https://webkit.org/b/199121
> - Retry Layout test in case of failures https://webkit.org/b/199194
> - Upload test results after running layout-tests https://webkit.org/b/199120
> - Improved error message on performing certain actions (like Rebuild) which
> requires authentication
> - Added more unit-tests
>
>
> Bug fixes:
> - Results are clobbered in UploadTestResults and ExtractTestResults steps in
> case of multiple layout test runs https://webkit.org/b/199178
> - New EWS: api-ios does not differentiate between patch compile failure and
> ToT compile failure https://webkit.org/b/197850
> - Buildbot worker not pinged https://webkit.org/b/199438
> - Make PrintConfiguration platform aware https://webkit.org/b/196657
> - Status bubble should not turn orange when any build step is skipped
> https://webkit.org/b/199079
> - Do not print worker environment variables in each build step
> https://webkit.org/b/197319
> - Do not run unix commands for windows in PrintConfiguration
> https://webkit.org/b/199605
>
> Thanks
> Aakash
>
>> On May 22, 2019, at 7:36 PM, Aakash Jain <aakash_j...@apple.com> wrote:
>>
>> Hi Everyone,
>>
>> I just wanted to update everyone with the recent improvements I have made to
>> new EWS. As always, please feel encouraged to provide any feedback (either
>> by filing bugs or contacting me directly).
>>
>> New Features:
>> • EWS status-bubble now display position in queue while patch is
>> waiting to be processed
>> • Added webkitpy and bindings-tests EWS (moved from old to new EWS)
>> • Status bubbles for webkitpy and bindings-tests EWS now display the
>> exact test failures in hover-over message (https://webkit.org/b/197395,
>> https://webkit.org/b/197423)
>> • Added support for 'new EWS' in webkit-patch tool
>> • Added 'EWS Build Archives' (similar to 'WebKit Build Archives'
>> https://webkit.org/blog/7978/introducing-webkit-build-archives/). For every
>> patch uploaded to Bugzilla, EWS builders build the patch for various
>> platforms (currently macOS and iOS) and upload the archives to S3. These
>> archives are available to download by anyone (for 14 days). The S3 URL is in
>> corresponding build (e.g.: notice 'uploaded archive' link in
>> https://ews-build.webkit.org/#/builders/7/builds/2477). So, if for any
>> reason, you want to get a built archive for your patch, you can simply
>> upload the patch to Bugzilla. (Note that if there is interest in this, we
>> can enhance it further)
>>
>> Infrastructure Improvements:
>> • Flakiness in API tests has been reduced (thanks to many WebKit
>> developers)
>> • Infrastructure improvements to prevent build failure due to "worker
>> not pinged" (e.g.: https://ews-build.webkit.org/#/builders/9/builds/332)
>> • New EWS polls bugzilla more frequently https://webkit.org/b/197138
>> • Configured DEBUG mode appropriately for Production and Development
>> env https://webkit.org/b/197700
>> • Ensured that Buildbot worker logs are not lost on restarting worker
>> • Do not run clean build by default on EWS builders (to improve
>> efficiency) https://webkit.org/b/196897
>> • build.webkit.org and ews-build.webkit.org starting sharing code
>> (although very little as of now, however the plan is to share more code)
>> • Added migrations file to repository https://webkit.org/b/197729
>> • Added EWS bots information to Internal scripts to easily monitor bots
>> • Added more unit-tests
>>
>> Bug fixes:
>> • Clicking 'submit to new ews' doesn't reload status-bubble
>> https://webkit.org/b/196675
>> • Clicking on white bubble navigates to page with only bubbles
>> https://webkit.org/b/197520
>> • Submit to EWS buttons are not aligned properly with status-bubbles
>> https://webkit.org/b/197139
>> • Status bubble should turn orange when any build step fails
>> https://webkit.org/b/197812
>> • Handle bug titles with unicode characters https://webkit.org/b/196802
>> • Scripts using Buildbot API have CORS error https://webkit.org/b/196709
>> • PrintConfiguration should display Xcode version instead of SDKVersion
>> https://webkit.org/b/196780
>> • Trigger queues only after uploading the archive
>> https://webkit.org/b/197180
>> • Do not upload archive when Compile Fails https://webkit.org/b/196674
>> • Exception while loading status-bubble when no build step has started
>> https://webkit.org/b/196676
>> • Use singular verb in failure description in case of single api test
>> failure https://webkit.org/b/197013
>> • EWS should clearly indicate flaky test failures
>> https://webkit.org/b/196947
>> • Use explicit imports instead of wildcard imports
>> https://webkit.org/b/197194
>> • New EWS: patches on recently added queues listed as #1 for older bugs
>> https://webkit.org/b/197496
>> • Improved summary text for various build steps
>>
>> Interesting info: Since last month, 'EWS for API tests' prevented API test
>> breakage on 50+ patches
>> (https://ews-build.webkit.org/api/v2/builders/3/builds?state_string__contains=new%20API%20Test&property=bug_id&order=-number).
>>
>> Thanks
>> Aakash
>>
>>> On Apr 4, 2019, at 10:00 PM, Aakash Jain <aakash_j...@apple.com> wrote:
>>>
>>> Introducing brand new EWS
>>>
>>> with EWS for API Tests (for macOS and iOS)
>>>
>>> and EWS for WebKitPerl Tests!
>>>
>>>
>>> Starting today, when you upload a patch to bugs.webkit.org, you will see
>>> few more bubbles (for API tests and webkitperl tests). You might also see
>>> additional button 'Submit to new EWS' (if the patch doesn't have r? flag).
>>>
>>> The new EWS comes with many new features and there are lot more I want to
>>> add. But, I don't want you guys to wait more to start getting the benefits.
>>> That's why I am rolling it out in phases, starting with EWS for API Tests
>>> and WebKitPerl Tests. These are tests which are not currently covered by
>>> the existing EWS. Next step would be to move queues from existing EWS to
>>> new EWS one by one, with the eventual goal of moving over everything to new
>>> EWS.
>>>
>>>
>>> Why new EWS?
>>> The existing EWS has certain architectural limitations. One of the
>>> prominent limitation is that there is no concept of building and testing
>>> the patch on different queues. If we have three queues: WK1 tests, WK2
>>> tests and API tests, all three queues would need to compile WebKit
>>> independently. So WebKit would be compiled thrice instead of once. This is
>>> inefficient and thereby require more hardware.
>>>
>>> The new EWS has separate builder and tester queues. Builder queues build
>>> once and upload the archive. Multiple tester queues download that same
>>> archive and run tests on that. That way WebKit is compiled only once, and
>>> re-used on multiple tester queues. This improves system efficiency and
>>> allows us to add new test queues with substantially less hardware.
>>>
>>> The new EWS uses Buildbot at the back-end, which is a production-level CI
>>> system. It is easier to maintain and automatically provide various features
>>> like historical build logs, real-time log streaming, easier bot management,
>>> ability to retry a build etc. Plus, it’s a system most of you are already
>>> familiar with (build.webkit.org).
>>>
>>>
>>> How can you contribute:
>>> If you are interested in contributing, the source code is located at:
>>> ews-build (Buildbot): Tools/BuildSlaveSupport/ews-build
>>> ews-app (web-app): Tools/BuildSlaveSupport/ews-app
>>>
>>> Detailed instructions are at:
>>> https://trac.webkit.org/wiki/EarlyWarningSystem#ContributingtoEarlyWarningSystem
>>>
>>>
>>> Upcoming features:
>>> - status-bubble should display position in queue
>>> https://bugs.webkit.org/show_bug.cgi?id=196607
>>> - EWS should comment on Bugzilla bugs about failures
>>> https://bugs.webkit.org/show_bug.cgi?id=196598
>>> - EWS should have a way to retry a patch
>>> https://bugs.webkit.org/show_bug.cgi?id=196599
>>> - Security EWS https://bugs.webkit.org/show_bug.cgi?id=196605
>>>
>>>
>>> If you notice any issue, please feel free to file bugs (and assign to me).
>>>
>>> Thanks & Regards
>>> Aakash Jain
>>
>
_______________________________________________
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev
