(CCing auto-to...@mozilla.com)
jmaher and jhammel will be able to comment more on the talos specifics,
but few thoughts off the top of my head:
It seems like we're conflating multiple issues here:
1) [talos] is a separate repo from mc
2) [it's a hassle to] test the test to be sure it’s
- Original Message -
From: Ed Morley emor...@mozilla.com
To: Jim Mathies jmath...@mozilla.com, auto-to...@mozilla.com
Cc: dev-platform@lists.mozilla.org
Sent: Monday, March 4, 2013 8:42:39 AM
Subject: Re: proposal: replace talos with inline tests
(CCing auto-to...@mozilla.com)
jmaher and jhammel
Good points, comments below.
Ed Morley emor...@mozilla.com wrote in message
news:mailman.1992.1362404580.24452.dev-platf...@lists.mozilla.org...
(CCing auto-to...@mozilla.com)
jmaher and jhammel will be able to comment more on the talos specifics,
but few thoughts off the top of my head:
On 3/4/13 8:15 AM, Jim Mathies wrote:
So to work around this I’ve been putting together some basic perf tests I can
use to measure performance using the mochitest framework.
How are you dealing with the fact that mochitest runs on heterogeneous
hardware (including VMs and the like last I
On 3/4/13 5:15 AM, Jim Mathies wrote:
For metrofx we’ve been working on getting omtc and apzc running in the browser.
One of the things we need to be able to do is run performance tests that tell
us whether or not the work we’re doing is having a positive effect on perf. We
currently don’t
Boris Zbarsky bzbar...@mit.edu wrote in message
news:o7ydnyp6n66okqnmnz2dnuvz_uwdn...@mozilla.org...
On 3/4/13 8:15 AM, Jim Mathies wrote:
So to work around this I’ve been putting together some basic perf tests I
can use to measure performance using the mochitest framework.
How are you
1) something checked into mc anyone can easily author or run (for tracking
down regressions) without having to checkout a separate repo, or setup and
run a custom perf test framework.
I don't oppose the gist of what you're suggesting here, but please
keep in mind that small perf changes are
On 3/4/13 9:36 AM, Gregory Szorc wrote:
If all your tests are declared the same way, then presumably the test
running code would be similar and capturing performance data would
require a single implementation affecting all test suites instead of N
1-off solutions.
We've talked about this
On Monday, March 4, 2013 5:15:56 AM UTC-8, Jim Mathies wrote:
For metrofx we’ve been working on getting omtc and apzc running in the
browser. One of the things we need to be able to do is run performance tests
that tell us whether or not the work we’re doing is having a positive effect
on
Writing a lot of performance tests creates the problem that those tests
will take a long time to run. The nature of performance tests is that each
test must run for a relatively long time to get meaningful results.
Therefore I doubt writing lots of different performance tests can scale.
(Maybe we
I'll point out and really this is about all I have to say on this thread
that while perf testing (that is, recording results) may bewell, not
easy, but not too awful that rigorous analysis of what the data means
and if there is a regression is often hard since it is often the case,
as
On Monday, March 4, 2013 5:42:39 AM UTC-8, Ed Morley wrote:
(CCing auto-to...@mozilla.com)
jmaher and jhammel will be able to comment more on the talos specifics,
but few thoughts off the top of my head:
It seems like we're conflating multiple issues here:
1) [talos] is a separate repo
On Monday, March 4, 2013 5:17:29 PM UTC-8, Gregory Szorc wrote:
On 3/4/13 5:09 PM, Dave Mandelin wrote:
We already don't back back out changes for regressing a benchmark like
we back them out for regressing tests. I think this is at least
partially because a general sentiment that not
13 matches
Mail list logo
