Have you ever been looking at a test file in Searchfox and wondered how
the test could possibly work? Then you do some more investigation and
it turns out that the test, in fact, does not work and is disabled?
Perhaps you even melodramatically threw your hands up in the air in
frustration?
Well, prepare to throw your hands up in excitement, because
https://bugzilla.mozilla.org/show_bug.cgi?id=1653986 has landed,
bringing with it the gift of info boxes at the top of every test page!
Now you can know:
- How long your tests take to run (on average)!
- How many times your tests were run in the preceding 7 days[1]!
- How many times your tests were skipped in the preceding 7 days!
- What skip-patterns govern the skips from your test's
mochitest.ini-style files!
- What WPT disabled patterns govern the skips from your WPT test's meta
files!
- How many WPT subtests defy expectations!
- The wpt.fyi URL of the test you're looking at... by clicking on the
"Web Platform Tests Dashboard" link in the "Navigation" panel on the
right side of the searchfox page!
This information is brought to you by:
- The "source-test-file-metadata-test-info-all" taskcluster job defined
at
https://searchfox.org/mozilla-central/source/taskcluster/ci/source-test/file-metadata.yml
that provides statistics on runs, skipped runs, and (non-WPT) skip
conditions.
- The "source-test-wpt-metadata-summary" taskcluster job defined at
https://searchfox.org/mozilla-central/source/taskcluster/ci/source-test/wpt-metadata.yml
that derives its data from `mach wpt-metadata-summary`. (Thanks :jgraham!)
- The people who make taskcluster and the taskcluster jobs and the
testing infrastructure and the test data pipeline go! Having been around
for the tail end of the days of the tinderbox waterfall, it is so
incredibly fantastic that adding this data to searchfox is just a matter
of expressing a few task dependencies, adding some calls to curl using a
normalized URL structure that is exposed by taskcluster/treeherder's
introspection capabilities, some JSON plumbing, and some HTML formatting!
Do you think these info boxes are good, but not yet worth putting your
hands up for? But you'd like to put your hand up to volunteer to help
improve the info boxes so that you can put both your hands up in
excitement? Then put your hands up in excitement at the prospect of
putting your hand up to volunteer because that is a thing you can do!
Because:
- Anyone can contribute to Searchfox!
- Bugs and enhancements are tracked at bugzilla.mozilla.org under
Webtools::Searchfox. You can see existing open bugs at
https://bugzilla.mozilla.org/buglist.cgi?product=Webtools&component=Searchfox&bug_status=__open__
and file new bugs at
https://bugzilla.mozilla.org/enter_bug.cgi?product=Webtools&component=Searchfox
- The source can be found at https://github.com/mozsearch/mozsearch and
the production configurations at
https://github.com/mozsearch/mozsearch-mozilla.
- You can spin up a Searchfox VM easily[2] with Vagrant and then build
the test repo quickly with a single `make build-test-repo` that indexes
the repo and starts the web server in as little as 3.4 seconds!
- There's a cool "Searchfox" chatroom on https://chat.mozilla.org/ where
you can discuss potential enhancements to searchfox, how you use
searchfox, how you'd like to use searchfox, and where the point of
diminishing returns was probably located on the hands up shtick.
Andrew
1: Searchfox indexing jobs run once a day for branches that are expected
to change and only on demand for historical branches. Infrequently, the
indexing jobs fail, which could put Searchfox a day or two behind the
current state of things. More info at
https://github.com/mozsearch/mozsearch-mozilla#how-searchfoxorg-stays-up-to-date
2: Fingers crossed, but we tried really hard to sand down all the rough
edges. Computers are still involved though so YMMV, but the members of
the "Searchfox" chatroom are happy to help you out if you run into problems!
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
