Hi Alexander, we are seeing such crashes too. They are notoriously hard to debug. Generally, what happens is that the garbage collector stumbles over a broken object. This could have all sorts of reasons, notably including (1) V8 bugs pretty much *anywhere* in the system leading to stale pointer fields in objects (e.g. missing write barriers, missing slot set clearing when trimming objects, accidental writes to the wrong address), (2) memory corruption caused by bad hardware, malware or misbehaving antimalware products, cosmic rays, and who-knows-what (I'm not just saying that; bit flips are real). Given that the symptom (=the crash) is typically far away from the root cause (whether it's bug or bit flip), it is next to impossible to determine the relative frequency of these reasons.
Indirect investigation strategies include: - if the number of such crashes spikes, chances are it's a new bug. However, the garbage collector code has been undergoing changes recently, so in many cases, the crash signatures have changed, but the overall number of memory corruption crashes has not. - if a certain website exhibits more crashes than others, then chances are it's a real bug, that might be reproducible with enough stress testing (maybe an automated script/extension that reloads it over and over, or navigates randomly: https://chromium.googlesource.com/v8/v8/+/master/tools/link_clicker.extension/ ) - if a few machines are sending a significant part of the crash reports, then chances are it's just bad hardware - if a machine that sent a lot of crash reports that are likely to be memory corruption *also* sends some reports with other signatures, then chances are that those other signatures are also caused by memory corruption. Direct investigation strategies include: - if you can repro locally in a debugger (possibly after hours of trying... I've been there!), you'll at least have the entire heap to inspect. Sometimes that's enough to figure out what's going on. Sometimes it's not, as in: you might be able to pinpoint an object with a stale field, but have no way to find out why/how/when that field got corrupted. - if you stare at the contents of crash reports in great detail, maybe you can spot a pattern. If you're lucky, that can give a clue. Example: if you notice that it's always the same field in the same type of object that's corrupted, then you can search through the source for all places modifying that field, and audit them for correctness. For SlotSet crashers this is probably not applicable though :-( Sorry I don't have anything better to go on. FWIW, I don't think we have observed M66 to be crashier than other milestones. Good luck with your investigation! Let me know if you find anything. Jakob On Fri, Aug 10, 2018 at 5:14 AM <atimo...@yandex-team.ru> wrote: > Hi, > > We develop Chromium-based product and frequently receive renderer crash > dumps from our users with segfault in > function v8::internal::SlotSet::Iterate(). > We had this crash in previous versions but it become very frequent in > version based on Chromium 66. > > Crash occurs here: > https://cs.chromium.org/chromium/src/v8/src/heap/slot-set.h?g=0&l=206 > Crashed thread stack: https://pastebin.com/raw/ZDNCfsiX > Main thread stack: https://pastebin.com/raw/G6N40V7w > > Also i have done some disassembly and have extracted V8 heap page > fragments from some of our crash dumps: https://pastebin.com/raw/TdxQEwLB > EBX points to slot with broken pointer (enclosed by parenthesis in memory > dumps) and crashes were caused by access to memory pointed by this pointer. > > Unfortunately, we can't reproduce this crash locally. > > Can anyone take a brief look at this heap fragments? Maybe we can extract > some additional information that can help to understand what's going wrong? > Or maybe there is already known crash with this signature? > > Thanks, > > Alexander Timokhin, > Yandex LLC. > > -- > -- > v8-dev mailing list > v8-dev@googlegroups.com > http://groups.google.com/group/v8-dev > --- > You received this message because you are subscribed to the Google Groups > "v8-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to v8-dev+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- -- v8-dev mailing list v8-dev@googlegroups.com http://groups.google.com/group/v8-dev --- You received this message because you are subscribed to the Google Groups "v8-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to v8-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
