Re: Shutdown hangs are very common
Can we fix the UX? Presumably we will never have zero shutdown hangs and there may be different/better ways to prompt the user about it. On Mon, Jul 6, 2015 at 10:48 PM, Vladan D vdje...@mozilla.com wrote: KaiRo pointed out another reason to reduce shutdown hang rates on IRC: it's lousy UX. The crash-reporter dialog pops up a minute after the user closed Firefox On Monday, July 6, 2015 at 4:37:41 PM UTC-4, RyanVM wrote: On 7/6/2015 4:34 PM, Vladan D wrote: Background: Firefox shutdown hangs are turned into shutdown crashes by a watchdog thread [1] that forces a crash if shutdown hasn't completed within 1 minute. Thanks to the watchdog and the Windows profile unlocker [2], shutdown hangs aren't as frustrating as they used to be. However, shutdown hangs might still be causing data loss and they are indicative of potentially-serious bugs in the code. According to this graph of Firefox crash rate history, shutdown hangs (crashes) make up about one third of all browser crashes [3]: https://crash-analysis.mozilla.com/rkaiser/crash-report-tools/longtermgraph/?fxrel I've been told shutdown hangs often don't get enough attention. Should fixing shutdown hangs be higher priority? And if so, should we allow features with shutdown hangs to be released? Notes: 1. Force Firefox crash if shutdown hangs https://bugzilla.mozilla.org/show_bug.cgi?id=1038342 2. win32 implementation of nsIProfileUnlocker https://bugzilla.mozilla.org/show_bug.cgi?id=286355 3. The graph above shows that the overall crash rate jumped up by roughly a third when the watchdog code shipped in Firefox 36. Hover over the 36 box on the blue line Windows mochitest-bc shutdown hangs have been on of the top oranges in our automation for months now. See bug 1121145. Would be great if we could get more eyes on the problem. -Ryan ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Shutdown hangs are very common
On Monday, July 6, 2015 at 3:52:39 PM UTC-5, Kyle Huey wrote: On Mon, Jul 6, 2015 at 1:37 PM, Ryan VanderMeulen rya...@gmail.com wrote: On 7/6/2015 4:34 PM, Vladan D wrote: Background: Firefox shutdown hangs are turned into shutdown crashes by a watchdog thread [1] that forces a crash if shutdown hasn't completed within 1 minute. Thanks to the watchdog and the Windows profile unlocker [2], shutdown hangs aren't as frustrating as they used to be. However, shutdown hangs might still be causing data loss and they are indicative of potentially-serious bugs in the code. According to this graph of Firefox crash rate history, shutdown hangs (crashes) make up about one third of all browser crashes [3]: https://crash-analysis.mozilla.com/rkaiser/crash-report-tools/longtermgraph/?fxrel I've been told shutdown hangs often don't get enough attention. Should fixing shutdown hangs be higher priority? And if so, should we allow features with shutdown hangs to be released? Notes: 1. Force Firefox crash if shutdown hangs https://bugzilla.mozilla.org/show_bug.cgi?id=1038342 2. win32 implementation of nsIProfileUnlocker https://bugzilla.mozilla.org/show_bug.cgi?id=286355 3. The graph above shows that the overall crash rate jumped up by roughly a third when the watchdog code shipped in Firefox 36. Hover over the 36 box on the blue line Windows mochitest-bc shutdown hangs have been on of the top oranges in our automation for months now. See bug 1121145. Would be great if we could get more eyes on the problem. -Ryan ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform The last five logs in that bug are all hanging in QuotaClient code. I'll take a look. - Kyle Bug 1160459 was filed on the QuotaClient problem, it's a parent of the test failure bug. There's some discussion in there about the problem which you were involved in fyi. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Shutdown hangs are very common
Should fixing shutdown hangs be higher priority? Starting and shutting down properly (and quickly), IMO, fall into the core quality / no compromise / we have to get this right bucket. So making it higher priority sounds aligned with the Three Pillars. And if so, should we allow features with shutdown hangs to be released? Probably depends on the feature and how frequent / severe the shutdown hangs are with it. Not sure we can make a blanket judgement right here and now about it, unless we want to go hard-line on this. On 06/07/2015 4:34 PM, Vladan D wrote: Should fixing shutdown hangs be higher priority? And if so, should we allow features with shutdown hangs to be released? ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Shutdown hangs are very common
On Mon, Jul 6, 2015 at 1:37 PM, Ryan VanderMeulen rya...@gmail.com wrote: On 7/6/2015 4:34 PM, Vladan D wrote: Background: Firefox shutdown hangs are turned into shutdown crashes by a watchdog thread [1] that forces a crash if shutdown hasn't completed within 1 minute. Thanks to the watchdog and the Windows profile unlocker [2], shutdown hangs aren't as frustrating as they used to be. However, shutdown hangs might still be causing data loss and they are indicative of potentially-serious bugs in the code. According to this graph of Firefox crash rate history, shutdown hangs (crashes) make up about one third of all browser crashes [3]: https://crash-analysis.mozilla.com/rkaiser/crash-report-tools/longtermgraph/?fxrel I've been told shutdown hangs often don't get enough attention. Should fixing shutdown hangs be higher priority? And if so, should we allow features with shutdown hangs to be released? Notes: 1. Force Firefox crash if shutdown hangs https://bugzilla.mozilla.org/show_bug.cgi?id=1038342 2. win32 implementation of nsIProfileUnlocker https://bugzilla.mozilla.org/show_bug.cgi?id=286355 3. The graph above shows that the overall crash rate jumped up by roughly a third when the watchdog code shipped in Firefox 36. Hover over the 36 box on the blue line Windows mochitest-bc shutdown hangs have been on of the top oranges in our automation for months now. See bug 1121145. Would be great if we could get more eyes on the problem. -Ryan ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform The last five logs in that bug are all hanging in QuotaClient code. I'll take a look. - Kyle ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Shutdown hangs are very common
On 06/07/15 22:48, Vladan D wrote: KaiRo pointed out another reason to reduce shutdown hang rates on IRC: it's lousy UX. The crash-reporter dialog pops up a minute after the user closed Firefox That's bug 1137941, but how would that reduce shutdown hangs? -- David Rajchenbach-Teller, PhD Performance Team, Mozilla ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Shutdown hangs are very common
Background: Firefox shutdown hangs are turned into shutdown crashes by a watchdog thread [1] that forces a crash if shutdown hasn't completed within 1 minute. Thanks to the watchdog and the Windows profile unlocker [2], shutdown hangs aren't as frustrating as they used to be. However, shutdown hangs might still be causing data loss and they are indicative of potentially-serious bugs in the code. According to this graph of Firefox crash rate history, shutdown hangs (crashes) make up about one third of all browser crashes [3]: https://crash-analysis.mozilla.com/rkaiser/crash-report-tools/longtermgraph/?fxrel I've been told shutdown hangs often don't get enough attention. Should fixing shutdown hangs be higher priority? And if so, should we allow features with shutdown hangs to be released? Notes: 1. Force Firefox crash if shutdown hangs https://bugzilla.mozilla.org/show_bug.cgi?id=1038342 2. win32 implementation of nsIProfileUnlocker https://bugzilla.mozilla.org/show_bug.cgi?id=286355 3. The graph above shows that the overall crash rate jumped up by roughly a third when the watchdog code shipped in Firefox 36. Hover over the 36 box on the blue line ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Shutdown hangs are very common
On 7/6/2015 4:34 PM, Vladan D wrote: Background: Firefox shutdown hangs are turned into shutdown crashes by a watchdog thread [1] that forces a crash if shutdown hasn't completed within 1 minute. Thanks to the watchdog and the Windows profile unlocker [2], shutdown hangs aren't as frustrating as they used to be. However, shutdown hangs might still be causing data loss and they are indicative of potentially-serious bugs in the code. According to this graph of Firefox crash rate history, shutdown hangs (crashes) make up about one third of all browser crashes [3]: https://crash-analysis.mozilla.com/rkaiser/crash-report-tools/longtermgraph/?fxrel I've been told shutdown hangs often don't get enough attention. Should fixing shutdown hangs be higher priority? And if so, should we allow features with shutdown hangs to be released? Notes: 1. Force Firefox crash if shutdown hangs https://bugzilla.mozilla.org/show_bug.cgi?id=1038342 2. win32 implementation of nsIProfileUnlocker https://bugzilla.mozilla.org/show_bug.cgi?id=286355 3. The graph above shows that the overall crash rate jumped up by roughly a third when the watchdog code shipped in Firefox 36. Hover over the 36 box on the blue line Windows mochitest-bc shutdown hangs have been on of the top oranges in our automation for months now. See bug 1121145. Would be great if we could get more eyes on the problem. -Ryan ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Shutdown hangs are very common
KaiRo pointed out another reason to reduce shutdown hang rates on IRC: it's lousy UX. The crash-reporter dialog pops up a minute after the user closed Firefox On Monday, July 6, 2015 at 4:37:41 PM UTC-4, RyanVM wrote: On 7/6/2015 4:34 PM, Vladan D wrote: Background: Firefox shutdown hangs are turned into shutdown crashes by a watchdog thread [1] that forces a crash if shutdown hasn't completed within 1 minute. Thanks to the watchdog and the Windows profile unlocker [2], shutdown hangs aren't as frustrating as they used to be. However, shutdown hangs might still be causing data loss and they are indicative of potentially-serious bugs in the code. According to this graph of Firefox crash rate history, shutdown hangs (crashes) make up about one third of all browser crashes [3]: https://crash-analysis.mozilla.com/rkaiser/crash-report-tools/longtermgraph/?fxrel I've been told shutdown hangs often don't get enough attention. Should fixing shutdown hangs be higher priority? And if so, should we allow features with shutdown hangs to be released? Notes: 1. Force Firefox crash if shutdown hangs https://bugzilla.mozilla.org/show_bug.cgi?id=1038342 2. win32 implementation of nsIProfileUnlocker https://bugzilla.mozilla.org/show_bug.cgi?id=286355 3. The graph above shows that the overall crash rate jumped up by roughly a third when the watchdog code shipped in Firefox 36. Hover over the 36 box on the blue line Windows mochitest-bc shutdown hangs have been on of the top oranges in our automation for months now. See bug 1121145. Would be great if we could get more eyes on the problem. -Ryan ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Shutdown hangs are very common
On Monday, July 6, 2015 at 5:00:36 PM UTC-4, David Rajchenbach-Teller wrote: On 06/07/15 22:48, Vladan D wrote: KaiRo pointed out another reason to reduce shutdown hang rates on IRC: it's lousy UX. The crash-reporter dialog pops up a minute after the user closed Firefox That's bug 1137941, but how would that reduce shutdown hangs? I think you misunderstood, I wasn't talking about the reducing the watchdog timeout :) I was saying that shutdown hangs result in bad Crash Reporter UX, so that is one more reason to *fix* shutdown hangs, i.e. reduce the shutdown hang rate by fixing the causes of hangs ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Shutdown hangs are very common
Should fixing shutdown hangs be higher priority? And if so, should we allow features with shutdown hangs to be released? I admit to skimming over shutdown-hang signatures when looking at topcrash lists. In my experience they're often longstanding issues that are difficult to diagnose and rarely end in a successful fix. Given infinite resources, we should absolutely work on these hangs. In practice though, we always have more bugs than time, so I tend to focus my efforts on other topcrashes that we have a better chance of fixing. A possible exception is for new, never-before-seen shutdown hangs. Those are likely to be more actionable since we often have a clear regression range. I think it would be fair to consider those a blocker for the associated feature. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Shutdown hangs are very common
Vladan D. writes: Should fixing shutdown hangs be higher priority? _exit() after profile-before-change notification would be the many-holes-with-one-plug bug to prioritize. https://wiki.mozilla.org/XPCOM_Shutdown https://bugzilla.mozilla.org/show_bug.cgi?id=662444 ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Shutdown hangs are very common
On Mon, Jul 6, 2015 at 3:05 PM, Karl Tomlinson mozn...@karlt.net wrote: Vladan D. writes: Should fixing shutdown hangs be higher priority? _exit() after profile-before-change notification would be the many-holes-with-one-plug bug to prioritize. https://wiki.mozilla.org/XPCOM_Shutdown https://bugzilla.mozilla.org/show_bug.cgi?id=662444 ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform I think most of our hangs are before that. The one RyanVM pointed out certainly is. - Kyle ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Shutdown hangs are very common
Ah, right. On 06/07/15 23:05, Vladan D wrote: I think you misunderstood, I wasn't talking about the reducing the watchdog timeout :) I was saying that shutdown hangs result in bad Crash Reporter UX, so that is one more reason to *fix* shutdown hangs, i.e. reduce the shutdown hang rate by fixing the causes of hangs ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform -- David Rajchenbach-Teller, PhD Performance Team, Mozilla ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
