Apologies for sending this email to some people twice, but the information is pertinent to y'alls.
/will ---------- Forwarded message --------- From: William Kahn-Greene <wil...@mozilla.com> Date: Tue, Feb 2, 2021 at 5:30 PM Subject: this sprint in Crash Stats (2021.1) To: William Kahn-Greene <wil...@mozilla.com> *This sprint in Crash Stats. Volume 2021.1* *New features and changes in Crash Stats* *Crash Stats crash report view pages show Breadcrumbs information* In 2020q3, Roger and I worked out a new Breadcrumbs crash annotation for crash reports generated by the android-components crash reporter. It's a JSON-encoded field with a structure just like the one that the sentry-sdk sends. Incoming crash reports that have this annotation will show the data on the crash report view page to people who have protected data access. [image: Screenshot_2021-02-02 [ java lang OutOfMemoryError at java io ByteArrayOutputStream toByteArray(ByteArrayOutputStream java)[...].png] I implemented it based on what we get in the structure and what's in the Sentry interface. Breadcrumbs information is not searchable with Supersearch. Currently, it's only shown in the crash report view in the Details tab. Does this help your work? Are there ways to improve this? If so, let me know! This work was done in: https://bugzilla.mozilla.org/show_bug.cgi?id=1671276 *Crash Stats crash report view pages show Java exceptions* For the longest of long times, crash reports from a Java process included a JavaStackTrace annotation which was a big unstructured string of problematic parseability and I couldn't do much with it. In 2020q4, Roger and I worked out a new JavaException crash annotation which was a JSON-encoded structured string containing the exception information. Not only does it have the final exception, but it also has cascading exceptions if there are any! With a structured form of the exception, we can suddenly do a lot of interesting things. As a first step, I added display of the Java exception information to the crash report view page in the Display tab. It's in the same place that you would see the crashing thread stack if this were a C++/Rust crash. Just like the JavaStackTrace, the JavaException has some data in it that can have PII in it. Because of that, the Socorro processor generates two versions of the data: one that's sanitized (no java exception message values) and one that's raw. If you have protected data access, you can see the raw one. The interface is pretty wide and exceeds the screenshot. Sorry about that. [image: Screenshot_2021-02-02 [ java lang OutOfMemoryError at java io ByteArrayOutputStream toByteArray(ByteArrayOutputStream java)[...](2).png] My next step is to use the structured exception information to improve Java crash report signatures. I'm doing that work in https://bugzilla.mozilla.org/show_bug.cgi?id=1541120 and hoping to land that in 2021q1. More on that later in this email. Does this help your work? Are there ways to improve this? If so, let me know! This work was done in: https://bugzilla.mozilla.org/show_bug.cgi?id=1675560 *Changes to crash report view* One of the things I don't like about the crash report view is that it's impossible to intuit where the data you're looking at is from. Further, some of the tabs were unclear about what bits of data were protected data and what bits weren't. I've been improving that over time. The most recent step involved the following changes: 1. The "Metadata" tab was renamed to "Crash Annotations". This tab holds the crash annotation data from the raw crash before processing as well as a few fields that the collector adds when accepting a crash report from the client. Most of the fields are defined in the CrashAnnotations.yaml <https://hg.mozilla.org/mozilla-central/file/tip/toolkit/crashreporter/CrashAnnotations.yaml> file in mozilla-central. The ones that aren't, yet, should get added. I have that on my list of things to get to. 2. The "Crash Annotations" tab is now split into public and protected data sections. I hope this makes it a little clearer which is which. 3. I removed some unneeded fields that the collector adds at ingestion. Does this help your work? Are there ways to improve this? If so, let me know! *What's in the queue* In addition to the day-to-day stuff, I'm working on the following big projects in 2021q1. *Remove the Email field* Last year, I looked into who's using the Email field and for what, whether the data was any good, and in what circumstances do we even get Email data. That work was done in this bug: https://bugzilla.mozilla.org/show_bug.cgi?id=1626277 The consensus is that since not all of the crash reporter clients let a user enter in an email address, it doesn't seem like we use the data, and it's pretty toxic data to have, we should remove it. The first step of that is to delete the field from the crash report at ingestion. I'll be doing that work in: https://bugzilla.mozilla.org/show_bug.cgi?id=1688905 The second step is to remove it from the webapp. I'll be doing that work in: https://bugzilla.mozilla.org/show_bug.cgi?id=1688907 Once that's done, I'll write up some bugs to remove it from the crash reporter clients and wherever else it is in products. Does this affect you? If so, let me know! *Redo signature generation for Java crashes* Currently, signature generation for Java crashes is pretty basic and it's not flexible in the ways we need it. Now we can fix that. I need some Java crash expertise to bounce ideas off of and to help me verify "goodness" of signatures. If you're interested in helping in any capacity or if you have opinions on how it should work or what you need out of it, please let me know. I'm hoping to do this work in 2021q1. The tracker bug is: https://bugzilla.mozilla.org/show_bug.cgi?id=1541120 *Closing* Thank you to Roger Yang who implemented Breadcrumbs and JavaException reporting and Gabriele Svelto who advised on new annotations and how things should work! Thank you to everyone who submits signature generation changes--I really appreciate your efforts! If you want to forward this email to other people, please do! There's nothing in this email that's protected data or confidential or anything like that. My apologies for any shoddiness in this email. I've never done an email like this before and I have no idea what I'm doing. If there are things you wish were covered or things you wish were better, please let me know. /will _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
