On 3/2/14, 1:46 PM, Axel Hecht wrote:
Hi,
I've watched you guys thinking for an hour ;-)
Some comments from me.
Yes to moving build flows that generate assets into the tree.
Yes to having a way for developers to reproduce what automation does.
Yes to having jobs being executed more on demand than on push, and
having that have idempotent results.
Sceptical on the vision that we'll see the end of inbounds. The
interactions between test results and rebase don't seem to be trivial
enough to me to hope for non-backout always-open trees via auto land.
We kind of got sidetracked with that part of the discussion. Let's
discuss that elsewhere :)
I'm having an 'oh noes' for single command called by automation. My main
point here is the usefulness of logs generated. When you put all
sequential and parallel tasks into one single wrapper process, you end
up with one big log file on ftp, like today. And if anything happens,
one needs to read that log and reverse engineer which characters in this
log are stdout/stderr, and to which task they belong. I know I can't
tell good from bad in our logs.
Not necessarily.
We have the potential to slurp output from certain commands to specific
log files and to upload those as separate assets as part of the build.
This is possible today. It's annoying with make. It's less annoying when
"not make" (read: Python) is doing high-level DAG and process execution
because we should get better control than "interleave every process's
stdout/stderr to a unified stdout."
As I stated in the conference call, one of the first things I'll do
after automation is calling into our in-tree "build" command is to hook
up the existing output parsing (that mach uses) to extract compiler
warnings and have a machine-readable file (likely JSON) uploaded as a
build artifact.
A few comments on mozharness. One of the earliest tasks it offered,
IIRC, was multi-locale android builds. Sadly, it happens that it's not
helping those developers that want to create and test multi-locale
builds. It's monolithic deliverable isn't what developers need at the
point when they test multi-locale builds, nor does it blend in to the
developer's setup. Folks like rnewman were glad once I explained how to
avoid using mozharness for their builds. To me that's a sign of an
inadequate level of abstraction.
I /think/ the primary goal of mozharness has always been to move
automation logic out of buildbot land. In that regard, I view it as a
success: the existence of mozharness is helping to port our automation
off buildbot to a modern and more efficient platform.
I do agree that there is room for developers' needs to be better met. I
think that by moving things into the tree - where tree (not automation)
developers can access and change them more easily - will facilitate
improving developers' needs.
And, as it's been mentioned all over the call, l10n repacks:
Testing: repacks are hard to test, and they should be. They're designed
to be infallible, so that, no matter what happens in a localization,
they're producing runnable builds. A test is challenged to tell between
a broken localization and a broken build system. We shouldn't
overestimate the amount of errors in the build that end up in a build
bustage, and which of those are actually test failures. And which are
not generating build failures, but are bustages. One example would be
broken locale merge dirs, for example. Anything can be in those, and the
builds build and run fine. They're just not showing the right strings.
More generally, repacks are basically unowned at this point. There's a
bit of ownership in build, in releng, and me, as to how they're done.
There's absolutely nothing as far as reporting goes. The agreement
between John and me was "if there's anything odd, file a bug on releng
to dig in".
Repacks are special. They are a perfect example of a procedure that has
organically evolved to be defined both inside the tree and automation.
They are needlessly complex and fragile because of this dual-homed
nature. I hope to one day consolidate the logic inside the tree as part
of this proxy script. The reason this hasn't been done yet is quite
frankly nobody is willing to put in the effort. While each break is
painful and I would very much like for repacks to be bullet-proof and
tested as part of every checkin, we've more or less made the calculation
that investment in other parts of the build and automation is more
important.
FWIW, as of January 2013, we have basic unit tests for packaging and
repacks in the tree. In theory, this should help address the problems
you've noted. A problem is that without the repack logic in the tree, we
have no guarantee that the code paths in our tests match up to what
automation does. Again, moving logic into the tree should help us
establish order and raise our test coverage.
_______________________________________________
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform
