Re: Shutting off leak tests?
On Monday, July 15, 2013 3:09:05 PM UTC-7, Kyle Huey wrote: FWIW now that we have AWSY and we don't really care about shutdown leaks specifically I don't think these tests are very useful to memshrink anymore. AWSY is not a replacement for shutdown-leak testing. It's limited to code exercised by TP5. Small leaks are masked by normal variation in memory use. Fuzzing with partial* leak checks frequently catches regressions missed by AWSY. MemShrink does care about these bugs. I imagine the same would be true for full** leak checks if Firefox passed enough of its own tests to be fuzzable. https://bugzil.la/ALL+reporter:jruderman+kw:mlk * trace-refcnt ** trace-malloc, LSan, or valgrind (which has modes for distinguishing still-reachable memory from totally leaked memory) ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Shutting off leak tests?
On 7/17/2013 2:05 AM, Jesse Ruderman wrote: AWSY is not a replacement for shutdown-leak testing. It's limited to code exercised by TP5. Small leaks are masked by normal variation in memory use. Note, though, that we still run almost all of our test suites on debug builds with leak checking enabled, so we catch shutdown leaks in all the Mochitest variants, reftest and crashtest. The actual leak test is much simpler, it just loads a few web pages and quits. The only valuable thing we're losing from shutting this off is tracemalloc coverage, which we don't have elsewhere. I don't have any evidence to show that anyone has actually looked at the tracemalloc data or done anything useful with it in recent history, though. Leaks discovered by tracemalloc don't make the test fail, so we're not measuring much there. -Ted ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Shutting off leak tests?
Steven, anyone else that can take a look at this mac bug? Not that I'm aware of. And are we sure this *is* a Mac bug? I've never seen anything like it in any other context, so I think this is most likely to be a tests bug. So the best person to look at it is probably someone who knows a lot about our tests infrastructure and our tests. The Unknown event type 0xff90 error is displayed from trace-malloc code: https://hg.mozilla.org/mozilla-central/annotate/e5d74eebd0e2/tools/trace-malloc/tmreader.c#l290 So is the no callsite for 'F' (70)! error: https://hg.mozilla.org/mozilla-central/annotate/e5d74eebd0e2/tools/trace-malloc/tmreader.c#l710 https://hg.mozilla.org/mozilla-central/annotate/e5d74eebd0e2/tools/trace-malloc/tmreader.c#l765 On 7/15/13 6:08 PM, Doug Turner wrote: Makes me sad that the knee jerk reaction is to turn leak testing off before anyone actually does any engineering. Steven, anyone else that can take a look at this mac bug? Steven Michaud mailto:smich...@pobox.com July 15, 2013 2:15 PM I'd say go ahead and shut them off. I'm not going to have time to investigate this for the foreseeable future. I'm already dealing with one very difficult (and possibly intractable) tests bug (https://bugzilla.mozilla.org/show_bug.cgi?id=884471), and that's more than enough at one time :-( Doug Turner mailto:doug.tur...@gmail.com July 15, 2013 2:07 PM Has a developer investigated? Steven, do you know anything about this? doug ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Shutting off leak tests?
It's not exactly a knee jerk reaction. We discussed this quite a while ago as I mentioned earlier and there were more arguments in favor of turning these off. It's just that the work was never done. And in the mean time we have got much more useful test suites, such as AWSY which actually help us catch memory usage regressions. Cheers, -- Ehsan http://ehsanakhgari.org/ On Mon, Jul 15, 2013 at 7:08 PM, Doug Turner doug.tur...@gmail.com wrote: Makes me sad that the knee jerk reaction is to turn leak testing off before anyone actually does any engineering. Steven, anyone else that can take a look at this mac bug? Steven Michaud mailto:smich...@pobox.com July 15, 2013 2:15 PM I'd say go ahead and shut them off. I'm not going to have time to investigate this for the foreseeable future. I'm already dealing with one very difficult (and possibly intractable) tests bug (https://bugzilla.mozilla.org/**show_bug.cgi?id=884471https://bugzilla.mozilla.org/show_bug.cgi?id=884471), and that's more than enough at one time :-( Doug Turner mailto:doug.tur...@gmail.com July 15, 2013 2:07 PM Has a developer investigated? Steven, do you know anything about this? doug __**_ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/**listinfo/dev-platformhttps://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Shutting off leak tests?
Hi! Leak tests on OSX have been failing intermittently for nearly a year now[1]. As yet, we don't have any ideas why they're failing, and nobody is working on fixing them. Would anybody be very sad if we shut them off? Are these tests providing useful information any more? If they are still important to run, can we get some help fixing them? Cheers, Chris [1] https://bugzilla.mozilla.org/show_bug.cgi?id=774844 signature.asc Description: Digital signature ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Shutting off leak tests?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 13-07-15 1:45 PM, Chris AtLee wrote: Would anybody be very sad if we shut them off? I would be happy if you did, for the reasons you state. Please shut them off. -r -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.13 (Darwin) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQEcBAEBAgAGBQJR5GANAAoJEEcAD3uxRB3vaF8H/0gq4xVs/rOGumSonllWZ50X xh8HxJkvakk3mytq0Umcgxe/eVHnFQXesfXOcsg0PuvUNuWOFeo5o0iW0EBqoSAU FXr1CCfJfYB6E2C1holesMd9y6yWc4swaB5u4k/MRwUFUIgnTNjxMbY3/OwzUWyv Uozo6De42A3FFcpwo993w2eHr3jid0C6Mr45xr3D4G8Tbb0RNonD9cDJZoMXX97P wiPftPBz6a/Ql1+49lEN19kX3PlqyKW9e+6z/rwjcglnyjg3nGkVn3bQhOT902yi nylx0URcAO4T9VXEUQZ8AmrQdtVfir7re1eV6yxTEiUoEsVMiDYL0gbtyuhArG0= =bl35 -END PGP SIGNATURE- ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Shutting off leak tests?
Has a developer investigated? Steven, do you know anything about this? doug Chris AtLee wrote: Hi! Leak tests on OSX have been failing intermittently for nearly a year now[1]. As yet, we don't have any ideas why they're failing, and nobody is working on fixing them. Would anybody be very sad if we shut them off? Are these tests providing useful information any more? If they are still important to run, can we get some help fixing them? Cheers, Chris [1] https://bugzilla.mozilla.org/show_bug.cgi?id=774844 ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Shutting off leak tests?
I'd say go ahead and shut them off. I'm not going to have time to investigate this for the foreseeable future. I'm already dealing with one very difficult (and possibly intractable) tests bug (https://bugzilla.mozilla.org/show_bug.cgi?id=884471), and that's more than enough at one time :-( On 7/15/13 4:07 PM, Doug Turner wrote: Has a developer investigated? Steven, do you know anything about this? doug Chris AtLee wrote: Hi! Leak tests on OSX have been failing intermittently for nearly a year now[1]. As yet, we don't have any ideas why they're failing, and nobody is working on fixing them. Would anybody be very sad if we shut them off? Are these tests providing useful information any more? If they are still important to run, can we get some help fixing them? Cheers, Chris [1] https://bugzilla.mozilla.org/show_bug.cgi?id=774844 ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Shutting off leak tests?
I think we can only make this decision once we know the worst case scenario these tests are currently preventing, so that we can mitigate or plan for it. -Alex On Jul 15, 2013, at 1:45 PM, Chris AtLee cat...@mozilla.com wrote: Hi! Leak tests on OSX have been failing intermittently for nearly a year now[1]. As yet, we don't have any ideas why they're failing, and nobody is working on fixing them. Would anybody be very sad if we shut them off? Are these tests providing useful information any more? If they are still important to run, can we get some help fixing them? Cheers, Chris [1] https://bugzilla.mozilla.org/show_bug.cgi?id=774844 ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Shutting off leak tests?
On Mon, Jul 15, 2013 at 3:05 PM, Alex Keybl ake...@mozilla.com wrote: I think we can only make this decision once we know the worst case scenario these tests are currently preventing, so that we can mitigate or plan for it. -Alex On Jul 15, 2013, at 1:45 PM, Chris AtLee cat...@mozilla.com wrote: Hi! Leak tests on OSX have been failing intermittently for nearly a year now[1]. As yet, we don't have any ideas why they're failing, and nobody is working on fixing them. Would anybody be very sad if we shut them off? Are these tests providing useful information any more? If they are still important to run, can we get some help fixing them? Cheers, Chris [1] https://bugzilla.mozilla.org/show_bug.cgi?id=774844 ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform FWIW now that we have AWSY and we don't really care about shutdown leaks specifically I don't think these tests are very useful to memshrink anymore. - Kyle ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Shutting off leak tests?
I brought this up ~2 years ago https://groups.google.com/forum/#!topic/mozilla.dev.platform/0lkjbtBK8eQ, and we concluded that discussion saying that we should turn these tests off, so bug 617441 was filed and then nothing happened. I don't think anything has changed since we had that discussion. On 2013-07-15 4:45 PM, Chris AtLee wrote: Hi! Leak tests on OSX have been failing intermittently for nearly a year now[1]. As yet, we don't have any ideas why they're failing, and nobody is working on fixing them. Would anybody be very sad if we shut them off? Are these tests providing useful information any more? If they are still important to run, can we get some help fixing them? Cheers, Chris [1] https://bugzilla.mozilla.org/show_bug.cgi?id=774844 ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Shutting off leak tests?
Makes me sad that the knee jerk reaction is to turn leak testing off before anyone actually does any engineering. Steven, anyone else that can take a look at this mac bug? Steven Michaud mailto:smich...@pobox.com July 15, 2013 2:15 PM I'd say go ahead and shut them off. I'm not going to have time to investigate this for the foreseeable future. I'm already dealing with one very difficult (and possibly intractable) tests bug (https://bugzilla.mozilla.org/show_bug.cgi?id=884471), and that's more than enough at one time :-( Doug Turner mailto:doug.tur...@gmail.com July 15, 2013 2:07 PM Has a developer investigated? Steven, do you know anything about this? doug ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
