Re: Finding windows and docshells leaked until shutdown

2019-10-04 Thread Greg Tatum
I started a section on dealing with leaks on the MDN mochitest page. Feel
free to contribute to it with any tips that may be useful to others. I got
pointed to this thread when I was asking for some help on leaks yesterday.
I couldn't find any details when searching online for anything around the
"leakcheck" mechanism, so I figured docs would be good to add.

https://developer.mozilla.org/en-US/docs/Mozilla/Projects/Mochitest#Diagnosing_and_fixing_leakcheck_failures

On Thu, Oct 3, 2019 at 6:10 AM smaug  wrote:

> On 10/3/19 1:18 AM, Andrew McCreight wrote:
> > On Wed, Oct 2, 2019 at 2:35 PM Geoff Lankow 
> wrote:
> >
> >> Hi all, I need some advice.
> >>
> >> I'm trying to enable Mochitest on debug builds of Thunderbird. It works
> >> fine, except for the leak check, which finds a number of leaked windows
> >> and docshells. Obviously we don't want to leak these things, but I can't
> >> find what's holding onto them.
> >>
> >> I've ruled out a number of likely candidates and I'm quickly running out
> >> of ideas. Can you offer any tips or tricks that might help?
> >>
> >> Here's an example log:
> >>
> >>
> https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=267576044=try-comm-central=8051
> >
> >
> > What this leak checker means is that a docshell or window was created
> while
> > the test was running, but was not destroyed until after the test finished
> > (and we ran a bunch of GCs and CCs). However, these docshells and windows
> > don't show up in the leakcheck output, which means that they were cleaned
> > up before we shut down.
>
> Ah, good, runtime leak :)
> Runtime leaks are in general way easier to debug than leaks which are
> there until shutdown.
> Add a conditional break point to the Release method of the relevant object
> and see all the cases Release is called. One (or more) of the Release
> calls ends
> up revealing where the leak is coming from.
>
> To get access to the right C++ object, CC log is one way to get the
> address.
>
>
>
> >
> > A typical cause for these is that there's some top level data structure,
> > often in JS but it could be in C++, too, that keeps track of every
> docshell
> > or window that is created, and does so by just adding a reference to it.
> > When we start shutdown, the data structure gets torn down, and we don't
> > leak things forever.
> >
> > One small tip is that if you look at the line after the "leaked until
> > shutdown" message and you'll see something like this:
> > [task 2019-09-20T03:42:16.942Z] 03:42:16 INFO - TEST-INFO |
> > comm/calendar/test/browser/browser_calendarList.js | windows(s) leaked:
> > [pid = 1155] [serial = 38], [pid = 1155] [serial = 36], [pid = 1155]
> > [serial = 39], [pid = 1155] [serial = 37]
> >
> > The entries like "[pid = 1155] [serial = 38]" are the descriptors we use
> > for windows (and docshells have a similar thing). If you search for
> "[pid =
> > 1155] [serial = 38]" in the log, you can see where exactly each window
> was
> > created and destroyed. Sometimes that can give a hint about what is going
> > wrong.
> >
> > Like Emilio said, the best approach to fixing these leaks is going to be
> > getting GC/CC logs from the point where it complains about the window
> being
> > alive, and then you can figure out from there what is keeping the window
> > alive, which may or may not point at what the leak is.
> >
> > In practice, doing all of this leak fixing is rather a big project by
> > itself, so I'd recommend that you disable this leak checking in
> Thunderbird
> > somehow, and then just try to get the rest of the debug tests up and
> > running, and then maybe come back to it later. This leak checking is done
> > in this file (by doing postprocessing of the browser log):
> > testing/mochitest/leaks.py
> >
> >
> >
> >>
> >> GL
> >> ___
> >> dev-platform mailing list
> >> dev-platform@lists.mozilla.org
> >> https://lists.mozilla.org/listinfo/dev-platform
> >>
>
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform


Re: Finding windows and docshells leaked until shutdown

2019-10-03 Thread smaug

On 10/3/19 1:18 AM, Andrew McCreight wrote:

On Wed, Oct 2, 2019 at 2:35 PM Geoff Lankow  wrote:


Hi all, I need some advice.

I'm trying to enable Mochitest on debug builds of Thunderbird. It works
fine, except for the leak check, which finds a number of leaked windows
and docshells. Obviously we don't want to leak these things, but I can't
find what's holding onto them.

I've ruled out a number of likely candidates and I'm quickly running out
of ideas. Can you offer any tips or tricks that might help?

Here's an example log:

https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=267576044=try-comm-central=8051



What this leak checker means is that a docshell or window was created while
the test was running, but was not destroyed until after the test finished
(and we ran a bunch of GCs and CCs). However, these docshells and windows
don't show up in the leakcheck output, which means that they were cleaned
up before we shut down.


Ah, good, runtime leak :)
Runtime leaks are in general way easier to debug than leaks which are there 
until shutdown.
Add a conditional break point to the Release method of the relevant object
and see all the cases Release is called. One (or more) of the Release calls ends
up revealing where the leak is coming from.

To get access to the right C++ object, CC log is one way to get the address.





A typical cause for these is that there's some top level data structure,
often in JS but it could be in C++, too, that keeps track of every docshell
or window that is created, and does so by just adding a reference to it.
When we start shutdown, the data structure gets torn down, and we don't
leak things forever.

One small tip is that if you look at the line after the "leaked until
shutdown" message and you'll see something like this:
[task 2019-09-20T03:42:16.942Z] 03:42:16 INFO - TEST-INFO |
comm/calendar/test/browser/browser_calendarList.js | windows(s) leaked:
[pid = 1155] [serial = 38], [pid = 1155] [serial = 36], [pid = 1155]
[serial = 39], [pid = 1155] [serial = 37]

The entries like "[pid = 1155] [serial = 38]" are the descriptors we use
for windows (and docshells have a similar thing). If you search for "[pid =
1155] [serial = 38]" in the log, you can see where exactly each window was
created and destroyed. Sometimes that can give a hint about what is going
wrong.

Like Emilio said, the best approach to fixing these leaks is going to be
getting GC/CC logs from the point where it complains about the window being
alive, and then you can figure out from there what is keeping the window
alive, which may or may not point at what the leak is.

In practice, doing all of this leak fixing is rather a big project by
itself, so I'd recommend that you disable this leak checking in Thunderbird
somehow, and then just try to get the rest of the debug tests up and
running, and then maybe come back to it later. This leak checking is done
in this file (by doing postprocessing of the browser log):
testing/mochitest/leaks.py





GL
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform



___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform


Re: Finding windows and docshells leaked until shutdown

2019-10-02 Thread Andrew McCreight
On Wed, Oct 2, 2019 at 2:35 PM Geoff Lankow  wrote:

> Hi all, I need some advice.
>
> I'm trying to enable Mochitest on debug builds of Thunderbird. It works
> fine, except for the leak check, which finds a number of leaked windows
> and docshells. Obviously we don't want to leak these things, but I can't
> find what's holding onto them.
>
> I've ruled out a number of likely candidates and I'm quickly running out
> of ideas. Can you offer any tips or tricks that might help?
>
> Here's an example log:
>
> https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=267576044=try-comm-central=8051


What this leak checker means is that a docshell or window was created while
the test was running, but was not destroyed until after the test finished
(and we ran a bunch of GCs and CCs). However, these docshells and windows
don't show up in the leakcheck output, which means that they were cleaned
up before we shut down.

A typical cause for these is that there's some top level data structure,
often in JS but it could be in C++, too, that keeps track of every docshell
or window that is created, and does so by just adding a reference to it.
When we start shutdown, the data structure gets torn down, and we don't
leak things forever.

One small tip is that if you look at the line after the "leaked until
shutdown" message and you'll see something like this:
[task 2019-09-20T03:42:16.942Z] 03:42:16 INFO - TEST-INFO |
comm/calendar/test/browser/browser_calendarList.js | windows(s) leaked:
[pid = 1155] [serial = 38], [pid = 1155] [serial = 36], [pid = 1155]
[serial = 39], [pid = 1155] [serial = 37]

The entries like "[pid = 1155] [serial = 38]" are the descriptors we use
for windows (and docshells have a similar thing). If you search for "[pid =
1155] [serial = 38]" in the log, you can see where exactly each window was
created and destroyed. Sometimes that can give a hint about what is going
wrong.

Like Emilio said, the best approach to fixing these leaks is going to be
getting GC/CC logs from the point where it complains about the window being
alive, and then you can figure out from there what is keeping the window
alive, which may or may not point at what the leak is.

In practice, doing all of this leak fixing is rather a big project by
itself, so I'd recommend that you disable this leak checking in Thunderbird
somehow, and then just try to get the rest of the debug tests up and
running, and then maybe come back to it later. This leak checking is done
in this file (by doing postprocessing of the browser log):
testing/mochitest/leaks.py



>
> GL
> ___
> dev-platform mailing list
> dev-platform@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-platform
>
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform


Re: Finding windows and docshells leaked until shutdown

2019-10-02 Thread Emilio Cobos Álvarez
Not an expert in leak-hunting, but generally this happens every time you 
leave something like an observer or such registered and you don't 
de-register it where the page goes away. It'd keep all the window alive. 
Probably can also happen with some native event listeners or what not...


A recent example if it may help: This patch got backed out due to 
leaking the layout debugger window:


  https://hg.mozilla.org/integration/autoland/rev/3f8c2c77cd63

This is the fixed version:

  https://hg.mozilla.org/integration/autoland/rev/ddfb38a12cdf

And this the interdiff:

  https://phabricator.services.mozilla.com/D47715?vs=170954=171025

That one was pretty obvious to fix, but yeah, tracking pre-existing ones 
is tricky... I think the best way to do that is using:


  https://github.com/amccreight/heapgraph

On the logs generated using:


https://developer.mozilla.org/en-US/docs/Mozilla/Performance/GC_and_CC_logs

(There are docs there about how to generate those from automation)

This is useful to find leaks of individual objects too, where you may 
want to find where the object is allocated rather than what keeps it alive:


  https://developer.mozilla.org/en-US/docs/Mozilla/Performance/BloatView

But I guess that's not all that useful for chasing window leaks.

I'm not an expert on these tools myself, hopefully Andrew or one of the 
people with more experience with them can help more...


Hope it helps,

 -- Emilio

On 10/2/19 11:29 PM, Geoff Lankow wrote:

Hi all, I need some advice.

I'm trying to enable Mochitest on debug builds of Thunderbird. It works 
fine, except for the leak check, which finds a number of leaked windows 
and docshells. Obviously we don't want to leak these things, but I can't 
find what's holding onto them.


I've ruled out a number of likely candidates and I'm quickly running out 
of ideas. Can you offer any tips or tricks that might help?


Here's an example log: 
https://treeherder.mozilla.org/logviewer.html#/jobs?job_id=267576044=try-comm-central=8051 



GL
___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform

___
dev-platform mailing list
dev-platform@lists.mozilla.org
https://lists.mozilla.org/listinfo/dev-platform