Re: Gecko shutdown (London session)

[Nicholas Alexander]

Hi Aaron, others,

On Thu, Jun 30, 2016 at 8:41 AM, Aaron Klotz  wrote:

> Did the now-defunct exit(0) project ever come up in this discussion?
>
> See bugs 662444 and 826143. This was a perf team project back in the
> Snappy days where the objective was to write important data to disk ASAP,
> then exit(0) without doing a bunch of cleanup. The argument for this was
> that, yes, during development we want to make sure that we are properly
> cleaning up after ourselves, but there is no reason for end users with opt
> builds to be waiting around while Firefox spends a bunch of time destroying
> things that are going to be wiped anyway by process termination.
>

I'd like to (re-)surface that exit(0) is never going to be feasible on
Android [1].  The Android process hosts many long-lived services with
lifecycles distinct from the lifecycle of the Gecko rendering engine.  In
some way, Fennec needs to be able to "shutdown" Gecko without Gecko killing
its process.  (For the record: separating Gecko into its own process is
technically feasible but I expect it to be a great deal of work that I
definitely think we should not do.  JNI across processes is probably
possible but definitely not pleasant!)

Nick

[1] If necessary, I can dig out Bugzilla links to discussions that have
taken place about this.
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Gecko shutdown (London session)

[smaug]

On 06/30/2016 11:49 AM, Nicolas Silva wrote:

Hi dev-platform,

We had a session about shutdown problems during the London workweek. I
did a writeup of what was discussed and as it grew into a large-ish
piece, I put it in a wiki page (instead of the email I intended to send
initially) [1].
There's been a lot of work on reducing shutdown crashes/hangs
(especially with e10s) this year, but we still have a lot of work to do
[2].

The page contains some discussions about the kinds of issues we ran into
in the gfx team and of how we went about addressing some of them. If you
are working on shutdown problems, you may (or may not) find useful
information in there. If you have some experience with dealing with
shutdown bugs, I encourage you to do a brain dump on that wiki page.

There's also a two-phase shutdown proposal briefly described in there
[3], that is worth discussing because it would involve some cooperation
between different modules.


"Phase 1: All modules destroy all of their dynamic objects, but stay in a usable state. No more DOM objects or documents or gpu textures floating 
around at the end of this phase. If the cycle collector is still retaining resources at this point, it's a bug. "


Not sure if this matters here, but cycle collector doesn't really retain objects. Sure, we could explicitly clear so called jsholders, so that C++->JS 
edges are manually cut. But in general, CC doesn't know about a cycle collectable objects unless the object has been addrefed/released after the 
previous cycle collection.

(those operations make the object suspected, so the object is put to cycle 
collector's purple buffer).

Note, we do have shutdown leaks occasionally dealing with cycle collectable objects. It may be because of some JS using random services in a bad way 
or someone forgetting to traverse/unlink some cycle collabtable member variable, or non-cycle collectable object keeping a strong reference to a cycle 
collectable object etc.
Shutdown leaks are always considered bugs, so nothing new there, but they do happen and other code needs to be aware of it and not crash in a bad way 
if that happens.







Cheers,

Nical

[1] https://wiki.mozilla.org/Gecko:Shutdown_issues
[2]
https://crash-stats.mozilla.com/search/?product=Firefox_progress=shutdown&_facets=signature&_columns=date&_columns=signature&_columns=version&_columns=build_id&_columns=platform&_columns=shutdown_progress#facet-signature
[3]
https://wiki.mozilla.org/Gecko:Shutdown_issues#Two-phase_shutdown_proposal



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Gecko shutdown (London session)

[Andrew McCreight]

On Thu, Jun 30, 2016 at 8:41 AM, Aaron Klotz  wrote:

> Did the now-defunct exit(0) project ever come up in this discussion?
>
> See bugs 662444 and 826143. This was a perf team project back in the
> Snappy days where the objective was to write important data to disk ASAP,
> then exit(0) without doing a bunch of cleanup. The argument for this was
> that, yes, during development we want to make sure that we are properly
> cleaning up after ourselves, but there is no reason for end users with opt
> builds to be waiting around while Firefox spends a bunch of time destroying
> things that are going to be wiped anyway by process termination.
>

We already do _exit(0) in content processes in non-debug e10s,
in ContentChild::ActorDestroy(). A lot of the mess that Nicolas has had to
deal with in shutdown is in debug builds (see bug 1215265 and friends), so
exit(0) won't help with that.



>
> Aaron
>
>
> On 6/30/2016 2:49 AM, Nicolas Silva wrote:
>
>> Hi dev-platform,
>>
>> We had a session about shutdown problems during the London workweek. I
>> did a writeup of what was discussed and as it grew into a large-ish
>> piece, I put it in a wiki page (instead of the email I intended to send
>> initially) [1].
>> There's been a lot of work on reducing shutdown crashes/hangs
>> (especially with e10s) this year, but we still have a lot of work to do
>> [2].
>>
>> The page contains some discussions about the kinds of issues we ran into
>> in the gfx team and of how we went about addressing some of them. If you
>> are working on shutdown problems, you may (or may not) find useful
>> information in there. If you have some experience with dealing with
>> shutdown bugs, I encourage you to do a brain dump on that wiki page.
>>
>> There's also a two-phase shutdown proposal briefly described in there
>> [3], that is worth discussing because it would involve some cooperation
>> between different modules.
>>
>> Cheers,
>>
>> Nical
>>
>> [1] https://wiki.mozilla.org/Gecko:Shutdown_issues
>> [2]
>>
>> https://crash-stats.mozilla.com/search/?product=Firefox_progress=shutdown&_facets=signature&_columns=date&_columns=signature&_columns=version&_columns=build_id&_columns=platform&_columns=shutdown_progress#facet-signature
>> [3]
>> https://wiki.mozilla.org/Gecko:Shutdown_issues#Two-phase_shutdown_proposal
>>
>> ___
>> dev-platform mailing list
>> dev-platform@lists.mozilla.org
>> https://lists.mozilla.org/listinfo/dev-platform
>>
>
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Gecko shutdown (London session)

[Aaron Klotz]

Cool, thanks for the refresh on those details. Clearly this still 
involves a bunch of work, but so would any other shutdown improvement 
project.


My concern is that if we are going to reexamine shutdown, then I think 
that exit(0) needs to fit into this somehow. If we're going to spend the 
resources to refactor a bunch of stuff in the shutdown sequence, we're 
not maximizing the benefit to the user without it.


On 6/30/2016 9:44 AM, David Rajchenbach-Teller wrote:

There were plenty of blockers for _exit(0), including the fact that
pretty much none of the async code in Firefox/Gecko was shutdown-safe.

That's one of the reasons we had to come up with
nsIAsyncShutdown/AsyncShutdown.jsm. One of the not-entirely-stated goals
was that once everything registered with AsyncShutdown had finished
shutting down, we could call _exit(0).

Cheers,
  David

On 30/06/16 17:41, Aaron Klotz wrote:

Did the now-defunct exit(0) project ever come up in this discussion?

See bugs 662444 and 826143. This was a perf team project back in the
Snappy days where the objective was to write important data to disk
ASAP, then exit(0) without doing a bunch of cleanup. The argument for
this was that, yes, during development we want to make sure that we are
properly cleaning up after ourselves, but there is no reason for end
users with opt builds to be waiting around while Firefox spends a bunch
of time destroying things that are going to be wiped anyway by process
termination.

Aaron


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Gecko shutdown (London session)

[David Rajchenbach-Teller]

There were plenty of blockers for _exit(0), including the fact that
pretty much none of the async code in Firefox/Gecko was shutdown-safe.

That's one of the reasons we had to come up with
nsIAsyncShutdown/AsyncShutdown.jsm. One of the not-entirely-stated goals
was that once everything registered with AsyncShutdown had finished
shutting down, we could call _exit(0).

Cheers,
 David

On 30/06/16 17:41, Aaron Klotz wrote:
> Did the now-defunct exit(0) project ever come up in this discussion?
> 
> See bugs 662444 and 826143. This was a perf team project back in the
> Snappy days where the objective was to write important data to disk
> ASAP, then exit(0) without doing a bunch of cleanup. The argument for
> this was that, yes, during development we want to make sure that we are
> properly cleaning up after ourselves, but there is no reason for end
> users with opt builds to be waiting around while Firefox spends a bunch
> of time destroying things that are going to be wiped anyway by process
> termination.
> 
> Aaron
> 
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Gecko shutdown (London session)

[Aaron Klotz]

Did the now-defunct exit(0) project ever come up in this discussion?

See bugs 662444 and 826143. This was a perf team project back in the 
Snappy days where the objective was to write important data to disk 
ASAP, then exit(0) without doing a bunch of cleanup. The argument for 
this was that, yes, during development we want to make sure that we are 
properly cleaning up after ourselves, but there is no reason for end 
users with opt builds to be waiting around while Firefox spends a bunch 
of time destroying things that are going to be wiped anyway by process 
termination.


Aaron

On 6/30/2016 2:49 AM, Nicolas Silva wrote:

Hi dev-platform,

We had a session about shutdown problems during the London workweek. I
did a writeup of what was discussed and as it grew into a large-ish
piece, I put it in a wiki page (instead of the email I intended to send
initially) [1].
There's been a lot of work on reducing shutdown crashes/hangs
(especially with e10s) this year, but we still have a lot of work to do
[2].

The page contains some discussions about the kinds of issues we ran into
in the gfx team and of how we went about addressing some of them. If you
are working on shutdown problems, you may (or may not) find useful
information in there. If you have some experience with dealing with
shutdown bugs, I encourage you to do a brain dump on that wiki page.

There's also a two-phase shutdown proposal briefly described in there
[3], that is worth discussing because it would involve some cooperation
between different modules.

Cheers,

Nical

[1] https://wiki.mozilla.org/Gecko:Shutdown_issues
[2]
https://crash-stats.mozilla.com/search/?product=Firefox_progress=shutdown&_facets=signature&_columns=date&_columns=signature&_columns=version&_columns=build_id&_columns=platform&_columns=shutdown_progress#facet-signature
[3]
https://wiki.mozilla.org/Gecko:Shutdown_issues#Two-phase_shutdown_proposal

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform


___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Re: Gecko shutdown (London session)

[Nicolas Silva]

On Thu, Jun 30, 2016, at 04:35 PM, smaug wrote:
> Not sure if this matters here, but cycle collector doesn't really retain
> objects. Sure, we could explicitly clear so called jsholders, so that
> C++->JS 
> edges are manually cut. But in general, CC doesn't know about a cycle
> collectable objects unless the object has been addrefed/released after
> the 
> previous cycle collection.
> (those operations make the object suspected, so the object is put to
> cycle collector's purple buffer).

Yeah, sorry, the wording was inaccurate. Something phrased like this
would make more sense:

All modules should deallocate all or as much as they can of their
short-lived objects during phase 1 so that at the end of phase 1 a cycle
collection is run and all cycles are already broken. If a cycle still
exist at this point, it should be considered a bug.

The problems we currently are not the CC's "fault", it comes from
garbage collected and reference-counted objects being destroyed at
various times during ShutdownXPCOM, which means we have to shut the CC
down (and run the last cycle-collection which will free some more
dynamic objects) at the very end when there isn't much left of gecko
that is in a valid state.

I'll clarify in the document.

Cheers,

Nical
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

Gecko shutdown (London session)

[Nicolas Silva]

Hi dev-platform,

We had a session about shutdown problems during the London workweek. I
did a writeup of what was discussed and as it grew into a large-ish
piece, I put it in a wiki page (instead of the email I intended to send
initially) [1].
There's been a lot of work on reducing shutdown crashes/hangs
(especially with e10s) this year, but we still have a lot of work to do
[2]. 

The page contains some discussions about the kinds of issues we ran into
in the gfx team and of how we went about addressing some of them. If you
are working on shutdown problems, you may (or may not) find useful
information in there. If you have some experience with dealing with
shutdown bugs, I encourage you to do a brain dump on that wiki page.

There's also a two-phase shutdown proposal briefly described in there
[3], that is worth discussing because it would involve some cooperation
between different modules.

Cheers,

Nical

[1] https://wiki.mozilla.org/Gecko:Shutdown_issues
[2]
https://crash-stats.mozilla.com/search/?product=Firefox_progress=shutdown&_facets=signature&_columns=date&_columns=signature&_columns=version&_columns=build_id&_columns=platform&_columns=shutdown_progress#facet-signature
[3]
https://wiki.mozilla.org/Gecko:Shutdown_issues#Two-phase_shutdown_proposal

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform