Re: Improvements to the Sphinx in-tree documentation generation
On 12/16/2013 03:09 PM, Gregory Szorc wrote: Perhaps Reflect.parse() could grow a new option to expose comment nodes or could attach comment metadata to specific node types? It's really not possible to do the latter. Comments don't appertain to specific nodes at all. They're just random non-token things interspersed in the token stream. Note that the Esprima stuff mentioned in the other email doesn't attempt to put comments in the parse tree -- comment text and locations are stored in a side-along array. The user has to re-associate the comments with particular nodes after the fact. Fundamentally, comments are a token-level feature, not a parse tree-level feature. We sort of can do whatever we want with the parser API. Except for Esprima compatibility for one. And also except that our parser doesn't track comments at all, so we'd have to spend extra time and memory to record locations and such, beyond what we do now. The side-along Esprima approach is probably the only way this can work sensibly, given just how many places comments can be put; I don't think any API we might have should attempt associations itself. Jeff ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: We should write memory reporters for new features as they're being developed
Generally, I like the idea. Is it possible to write memory reporters for JS-implemented code? Also, is it possible to write memory reporters for Chrome Worker code? Cheers, David On 12/17/13 4:57 AM, Nicholas Nethercote wrote: So I want to propose something: if you're working on a change that will introduce significant new causes of memory consumption, you should write a memory reporter for it at the same time, rather than (maybe) doing it later, or letting someone else do it. And in this context, significant may be smaller than you expect. For example, we have numerous reporters for things that are typically only 100s of KBs. On B2G, 100KB per process is significant. -- David Rajchenbach-Teller, PhD Performance Team, Mozilla ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: We should write memory reporters for new features as they're being developed
On Tue, Dec 17, 2013 at 2:55 PM, David Rajchenbach-Teller dtel...@mozilla.com wrote: Is it possible to write memory reporters for JS-implemented code? It's possible. Some points about this... - Memory used by the JS engine already has good coverage in the explicit tree. - It's hard to measure the sizes of things in JS code. And there are lots of JS things (e.g. shapes) that aren't even visible from JS code. It might be interesting to measure counts of some things, though. - https://bugzilla.mozilla.org/show_bug.cgi?id=932156 is an idea to mark particular JS objects as special, and thus get identified specially in about:memory. IMO this is the most promising direction for improving reporting of JS code. - The DownloadThemAll! add-on implements some JS-side reporters. I'm not sure exactly what is measured, though. But in general, memory reporters are more relevant for C++ code. Also, is it possible to write memory reporters for Chrome Worker code? That's also JS code, right? JS runtimes for workers get measured the same way that the main JS runtime does. So I don't think Chrome worker code needs any treatment different to other kinds of JS code. Nick ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: We should write memory reporters for new features as they're being developed
First of all, sorry for neglecting that bug! And +1 on this suggestion. How does one write a memory reporter that captures data stored on more than just the main thread? In the case of Web Audio for example, some of our buffers live in another thread, and as far as I can see the memory reporter API is mostly synchronous. Is locking the right way to do this? Thanks! Ehsan On 12/16/2013, 10:57 PM, Nicholas Nethercote wrote: Hi, For over a month I've been working with a user to identify the cause of a bad memory leak (https://bugzilla.mozilla.org/show_bug.cgi?id=936784). Just today we got DMD working sufficiently well on Windows that the user was able to run it, and it pointed the finger at webaudio. Which is great progress! But if we had memory reporters for web audio, all this would have been so much easier. And (queue sad-face) we actually have a six-month old bug open for that: https://bugzilla.mozilla.org/show_bug.cgi?id=884368 So I want to propose something: if you're working on a change that will introduce significant new causes of memory consumption, you should write a memory reporter for it at the same time, rather than (maybe) doing it later, or letting someone else do it. And in this context, significant may be smaller than you expect. For example, we have numerous reporters for things that are typically only 100s of KBs. On B2G, 100KB per process is significant. Understanding the data structures is usually the hard part of writing a memory reporter. The actual reporter registration side isn't hard, and there are plenty of examples to refer to. So the author of the new code is typically the best person to write a reporter for it. And I'm happy to help (and review). Furthermore, memory reporters are best verified by doing a DMD run, and DMD now runs on all tier-1 platforms and is well documented (https://wiki.mozilla.org/Performance/MemShrink/DMD). So that shouldn't be an obstacle. This couldn't be a hard-and-fast rule, but I would like for it to be something that developers and reviewers keep in mind -- Does this code need a memory reporter? And have you verified it with DMD? Thoughts? Nick p.s.: The web audio bug prompted me to make this suggestion, and is a good example of the potential benefits. But I don't mean to criticize those who implemented web audio; apologies if it comes across that way. In that spirit, let's keep discussion on the general proposal as much as possible, rather than web audio. Thanks! ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: We should write memory reporters for new features as they're being developed
On Tue, Dec 17, 2013 at 6:54 PM, Ehsan Akhgari ehsan.akhg...@gmail.com wrote: How does one write a memory reporter that captures data stored on more than just the main thread? In the case of Web Audio for example, some of our buffers live in another thread, and as far as I can see the memory reporter API is mostly synchronous. Is locking the right way to do this? Good question. It depends on the details. Each memory reporter's CollectReports() method is called only from the main thread, and the CollectReports() call is synchronous. There isn't a good way to do asynchronous reporting. Web workers use a hack: code running on the main thread completely pauses (via locks) each worker's actions and then the main thread code measures the worker's data structures. So you may need to do something similar. One thing that's possibly in your favour is that memory reporting isn't performance-critical, since it only happens in response to user requests. Nick ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
