Re: sentry evaluation

[Sharaf Zaman]

Hi!

Nate Graham  writes:

> To be honest, I haven’t found Sentry to be that useful in its current
> implementation. The primary issue is that it represents a second source of 
> truth
> for where crash reports live. As a result, developers who already struggle to
> notice Bugzilla-based crash reports have to look in a second place, further
> diving their scarce attention. I haven’t seen evidence of people regularly
> interacting with it or looking at its crashes beyond the excitement of the
> initial rollout, so now it’s largely just a new graveyard of issues being
> ignored due to insufficient developer time.
>

I think looking at Sentry as just another tool to get reports is a wrong way to
look at it (: It should be more like a tool to look at reports which *most*
people are facing, in this case crashes. Sentry isn’t currently implemented in
Krita, but on Android we have similar reporting, thanks to google play and it
has helped me a lot to detect crashes in our beta (most people never report
them!). Similarly it has helped me ignore the obscure error that 0.001% of the
user space gets on their out of ordinary device (and you have no idea a crash is
obscure from bugzilla). In that sense sentry is something very similar and could
be a great metric to direct your attention at issues rather than a distraction 
:)

> I think Sentry could make sense to keep if it was implemented for us more as a
> kind of automatic symbolication service that can take a bad backtrace, add 
> debug
> symbols, retrace it, and then pass *that* off to DrKonqi so that the resulting
> Bugzilla ticket is guaranteed to have a *good* backtrace in it. That’s a real
> problem we currently have that could benefit from being solved in a targeted
> way.

I would absolutely love if sentry could also symbolicate from binary factory
artifacts, but I know it can be difficult!

>
> If we can’t or don’t want to do that, then Sentry might make sense if it were
> possible for us to bypass the “multiple sources of crash truth” issue by
> completely disabling Bugzilla for crash reports and migrating old Bugzilla
> crashes into sentry. But Then we’d run into the new issue that Sentry doesn’t
> permit discussion with the person experiencing the crash to ask for more 
> details
> if needed. This isn’t always needed, but it sometimes is. Sentry also isn’t
> integrated into our issue priority tracking system in Bugzilla, but that’s a
> more minor issue.
>
> Nate
>
>
>
> On 6/1/23 04:35, Harald Sitter wrote:
>> Hey,
>> We’ve had almost a year, albeit in a super limited capacity for git
>> builds, with sentry () and we should
>> probably render a final verdict on the evaluation.
>> Did you like it?
>> What could be improved?
>> Should we move ahead with a more permanent setup?
>> The overall game plan would be to have drkonqi ask the user to opt
>> into automatic crash submission when they open drkonqi so we get close
>> to all crashes (the ones caught by kcrash) automatically filed into
>> sentry from then on out. To increase exposure of this feature I’d also
>> like to glue it into the feedback KCM but I’m not yet sure if it
>> should be a separate setting or linked to the regular feedback
>> categories, the former sounds more accessible. The current bugzilla
>> based workflow would still be available for when the user actually
>> wants to write a description.
>> (previous discussion: )
>> HS

Re: sentry evaluation

[Nate Graham]

To be honest, I haven't found Sentry to be that useful in its current 
implementation. The primary issue is that it represents a second source 
of truth for where crash reports live. As a result, developers who 
already struggle to notice Bugzilla-based crash reports have to look in 
a second place, further diving their scarce attention. I haven't seen 
evidence of people regularly interacting with it or looking at its 
crashes beyond the excitement of the initial rollout, so now it's 
largely just a new graveyard of issues being ignored due to insufficient 
developer time.


I think Sentry could make sense to keep if it was implemented for us 
more as a kind of automatic symbolication service that can take a bad 
backtrace, add debug symbols, retrace it, and then pass *that* off to 
DrKonqi so that the resulting Bugzilla ticket is guaranteed to have a 
*good* backtrace in it. That's a real problem we currently have that 
could benefit from being solved in a targeted way.


If we can't or don't want to do that, then Sentry might make sense if it 
were possible for us to bypass the "multiple sources of crash truth" 
issue by completely disabling Bugzilla for crash reports and migrating 
old Bugzilla crashes into sentry. But Then we'd run into the new issue 
that Sentry doesn't permit discussion with the person experiencing the 
crash to ask for more details if needed. This isn't always needed, but 
it sometimes is. Sentry also isn't integrated into our issue priority 
tracking system in Bugzilla, but that's a more minor issue.


Nate



On 6/1/23 04:35, Harald Sitter wrote:

Hey,

We've had almost a year, albeit in a super limited capacity for git
builds, with sentry (https://errors-eval.kde.org) and we should
probably render a final verdict on the evaluation.

Did you like it?
What could be improved?
Should we move ahead with a more permanent setup?

The overall game plan would be to have drkonqi ask the user to opt
into automatic crash submission when they open drkonqi so we get close
to all crashes (the ones caught by kcrash) automatically filed into
sentry from then on out. To increase exposure of this feature I'd also
like to glue it into the feedback KCM but I'm not yet sure if it
should be a separate setting or linked to the regular feedback
categories, the former sounds more accessible. The current bugzilla
based workflow would still be available for when the user actually
wants to write a description.

(previous discussion: https://markmail.org/thread/6y5paczdposz3aoj)

HS

sentry evaluation

[Harald Sitter]

Hey,

We've had almost a year, albeit in a super limited capacity for git
builds, with sentry (https://errors-eval.kde.org) and we should
probably render a final verdict on the evaluation.

Did you like it?
What could be improved?
Should we move ahead with a more permanent setup?

The overall game plan would be to have drkonqi ask the user to opt
into automatic crash submission when they open drkonqi so we get close
to all crashes (the ones caught by kcrash) automatically filed into
sentry from then on out. To increase exposure of this feature I'd also
like to glue it into the feedback KCM but I'm not yet sure if it
should be a separate setting or linked to the regular feedback
categories, the former sounds more accessible. The current bugzilla
based workflow would still be available for when the user actually
wants to write a description.

(previous discussion: https://markmail.org/thread/6y5paczdposz3aoj)

HS