Here’s what Engineering Productivity has accomplished in Q3. In Q4, we’re
switching to OKR’s for tracking goals and progress; these should be
== Build System ==
Overall goal: Reduce build times for local developers and automation;
* TaskCluster tasks now running on faster AWS instances. Execution time
decreases of several minutes across the board.
* Initial support for building with Tup build system.
* Libffi’s configure script, formerly a long pole when running configure,
is no longer run, and its functionality has been incorporated into our
Python configure infrastructure. Additionally, libffi no longer builds with
its own build system and is now incorporated into moz.build.
* Significant progress has been made to convert our autoconf based
configure script to a more performant and maintainable Python based system.
The quantity of .m4 and .sh code invoked during this part of the build has
been reduced by ⅓.
== MozReview ==
Overall goal: Increase adoption of MozReview for mozilla-central code
* New, custom theme
* “Finish Review” dialog reworked for improved usability
* Automatic LDAP association
* Display diff statistics in MozReview and BMO
* Prototyped review-request UI changes to improve clarity and
* New Vagrant-based development environment to encourage contributions
== Debugging UX ==
Overall goal: Make it easier to debug and resolve automated test failures
Sub-goal: Created simple workflow to run test suites from an interactive
* Deployed automation changes that provide developers with the ability to
run tests, clone mozilla-central, and obtain shell access on actual linux64
test workers with just a few clicks from Treeherder.
Sub-goal: Reduce median end-to-end times on Try to 60 minutes
* Download zip files into memory before unzipping to save writing the zip
file to disk first
* Using ‘--artifact’ flag in try syntax replaces all scheduled build jobs
with an opt artifact build on linux, linux64, mac64, win32, win64. |./mach
try| also recognizes --artifact flag and prevents the user from scheduling
compiled-code tests with it. Compiled-code test jobs report an ‘exception’
result when run against an artifact build on try.
* Made progress on rewriting sccache in Rust, after which we can deploy a
two-tier sccache for Try builds, that will allow Try builds to consume
sccache objects from mozilla-central
* Build jobs in automation are now using version control clone+checkout
best practices; this results in faster VCS operations, less overhead during
* WPT tests now running from source checkout
* Work in progress to show high level “end-to-end times” across all
platforms and tests, on Try. This is building on previous work, which is a
prototype limited to build times:
== Other Projects ==
* Memory leak stuff previously discussed was implemented this quarter
* BMO implemented CSP for some pages, yielding an A- on Mozilla Observatory
* BMO switched to vendored dependencies which will result in a much faster
* Legacy bzapi saw a performance improve 226s to 184ms or 122826% faster (
* Work continues on search speed improvements; when landed, keyword search
for intermittent-failure goes from 2 to 0.08s
* Migrated Treeherder from SCL3 to Heroku.
* Use ReactJS to make the exclusion editor (Sheriff panel / Admin) usable
* Support submitting Github resultsets via Pulse
Continuous Integration and Test Harnesses
* Stood up flake8 lint job with mozlint, and converted eslint job to use
* Finished migrating majority of jobs from windows hardware to windows VM
in AWS, only a few remain running on HW with no plans to move to windows vm
VM to limitations of the VM’s
* Implemented Edge support in wptrunner
* Did initial exploratory work on cross-browser comparisons using
web-platform-tests and wptrunner
* Our Outreachy intern, Anjana Vakil, landed log parsing for
marionette-harness tests on Treeherder by integrating a pytest-mozlog
plugin into mozlog. Marionette-harness tests are thus ready to be Tier-1 on
* Documented some best practices for code backouts; see
* Refactored the Janitor service into a multi-server cluster (multiple
Docker hosts with Node.js routing agent, TLS / OAuth2 token-based
* Completed preliminary service review, started hardening (formalized +
implemented firewall rules, started implementing Mozilla’s security
guidelines, scheduled rapid risk assessment)
W3C WebDriver and Marionette
* Released geckodriver 0.10 with multiple fixes including support for
configuring Firefox through capabilities
* Extensive work on geckodriver 0.11 with improved configuration support,
win32 support, better logging
* Added webdriver spec tests to treeherder (on linux64).
* Wrote specification for “actions” support in WebDriver, and Marionette
implementation is in review
* Deployed Marionette tests on Fennec in TaskCluster; they are Tier-3 on
Treeherder as we make progress with greening up the tests.
* Fall session of UCOSP has started! There are three students, plus a
student mentor, continuing work from last year; this time adding C++ code
coverage to our tests so it is easy to trigger with a mach command.
Meeting notes are here:
* UCOSP is a program that brings together students from across Canada to
work together on open source projects.
dev-platform mailing list