We poison the GC heap with 0xda on nightly and aurora [1]. Lots of our processMarkStackTop are at this address [2]. It looks like we only poison Ion code in debug builds [3]. That should be easy to fix by removing the #ifdef DEBUG.
I don't think we've found GC heap poisoning to be all that helpful for debugging in crashstats. The stacks are almost always really generic. All you know is that some code didn't root something properly, but it could be anywhere in Gecko. Poisoning has been very helpful when we have an STR, since it improves reproducibility. However, we rarely get an STR for these crashes. One thing I've noticed in the past is that certain add-ons seem to increase the frequency of GC crashes (not necessarily though any misbehavior on their part). The correlations that show up in crashstats aren't very helpful here since so many different crashes are lumped together. I wonder if we could do a more sophisticated analysis to try to find the problematic add-ons. Then we could try to do some manual (or automatic) testing with those add-ons and hopefully get some full crashdumps, which would be more useful than what we have now. -Bill [1] http://mxr.mozilla.org/mozilla-central/source/js/src/jsgc.cpp#477 [2] https://crash-stats.mozilla.com/report/index/50bb404b-5b0b-458a-b3b6-5a2ba2140226 [3] http://mxr.mozilla.org/mozilla-central/source/js/src/jit/Ion.cpp#681 ----- Original Message ----- > From: "Benjamin Smedberg" <[email protected]> > To: "JS Internals list" <[email protected]> > Sent: Thursday, February 27, 2014 6:23:07 AM > Subject: [JS-internals] Memory poisoning and other ways to make stability > issues more reproducible > > [tl;dr: please poison GC and JIT memory when it is freed] > > I mentioned this at the engineering meeting last Tuesday but I don't > think I hit the right audience so I thought I'd repost it here. > > We currently have some important crashes that are currently not > actionable based on crash stats. If you take a look at the current 28 > beta topcrashes, the #3 topcrash (js::GCMarker::processMarkStackTop) is > a GC crash probably due to a corrupt GC heap/dangling pointers. > > The #5 crash (EnterBaseline) is a generic signature for any crash that > happens in the JIT. > > There are a few others such as > JSScript::markChildren/ScanRope/FinalizeArenas. > > A lot of our current efforts have revolved around using fuzzers to > detect these kinds of situations. But we really need to try and use our > release population more effectively to discover why these issues are > happening and fix them. > > We recently landed memory poisoning on free() in the Firefox codebase > (bug 860254). This provides some defense in depth for individual users, > but its primary purpose is to make use-after-free crashes happen more > quickly and with stack traces that are more useful. For the most part > this change does not appear to have affected performance (except for > tsvg tests on macOS which have strange symptoms). > > I'd like the JS team to consider doing the same kind of memory poisoning > in release builds for the various JS heaps which aren't the jemalloc heap: > > * The GC > * The various JIT code buffers > * Any other relevant caches or specialized allocators where normal > free() poisoning doesn't work > > Thoughts? I don't know enough about this to file specific bugs, so I'm > looking to you on the JS team to identify what things are candidates and > filing the bugs. Especially if there are mentors for the bugs, I have > some great volunteers who have been fixing OOM issues recently who might > love to help take some of these bugs and fix them. > > --BDS > > _______________________________________________ > dev-tech-js-engine-internals mailing list > [email protected] > https://lists.mozilla.org/listinfo/dev-tech-js-engine-internals > _______________________________________________ dev-tech-js-engine-internals mailing list [email protected] https://lists.mozilla.org/listinfo/dev-tech-js-engine-internals
