Re: Proposal: Remove Linux PGO Testing
On Wed, Oct 10, 2012 at 05:57:53PM -0400, Justin Lebar wrote: By turning off Linux PGO testing, you really mean stop making and distributing Linux PGO builds, right? The main reason I'd want Linux PGO is for mobile. On desktop Linux, most users (I expect) don't run our builds, so it's not a big deal if they're some percent slower. Many people have made claims about that at several different occasions. Can we once and for all come up with actual data on that? That being said, PGO on Linux is between 5 and 20% improvement on our various talos tests. That's with the version of gcc we currently use, which is 4.5. I'd expect 4.7 to do a better job even, especially if we added lto to the equation (and since we are now building on x86-64 machines, we wouldn't have to worry about memory usage ; link time could be a problem, though). Also note that disabling PGO currently also means disabling the optimizations we do on omni.ja (central directory optimizations and reordering). This is somehow covered by bug 773171. Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal: Remove Linux PGO Testing
On 10/10/2012 11:57 PM, Justin Lebar wrote: The main reason I'd want Linux PGO is for mobile. On desktop Linux, most users (I expect) don't run our builds, so it's not a big deal if they're some percent slower. (Unless distros commonly do PGO builds of Firefox?) But we're not doing mobile Linux PGO builds (that I know of), and I don't expect success with desktop PGO is much related to success with mobile PGO. You may be right for release builds but that doesn't hold true for Nightly/Aurora/Beta users. I don't think it's a good idea to make those builds ~20% slower when of course we want and need more testers. Don't forget that testers on Linux do not only test Linux-only features but also features we have on every platform. Nobody likes running debug builds because they're slower so why would that be different for non-PGO builds? Also, I'm not sure how this affects Telemetry results. In terms of perf measurements we'd probably need to completely ignore everything from non-release builds as the results might differ heavily for some use cases. - Tim ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal: Remove Linux PGO Testing
On 10/11/12 3:05 AM, Tim Taubert wrote: Also, I'm not sure how this affects Telemetry results. In terms of perf measurements we'd probably need to completely ignore everything from non-release builds as the results might differ heavily for some use cases. I'm not following. The suggestion, as far as I can tell, is to drop Linux PGO completely. We woudln't have it in nightly, Aurora, Beta, or releases. Compiling with PGO on Linux would be an unsupported configuration that we'd probably advise distros against, because it wouldn't be particularly well-tested. So the real question is whether PGO on Linux is worth it in general to us, again as far as I can tell. -Boris ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal: Remove Linux PGO Testing
On 10/11/2012 09:32 AM, Boris Zbarsky wrote: The suggestion, as far as I can tell, is to drop Linux PGO completely. We woudln't have it in nightly, Aurora, Beta, or releases. Compiling with PGO on Linux would be an unsupported configuration that we'd probably advise distros against, because it wouldn't be particularly well-tested. Yes, if we don't distribute PGO builds at all we'd see a one-time bump and Telemetry is then a non-issue. I was misguided by the thread's title. If we turn it off for testing we can't ship it. - Tim ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal: Remove Linux PGO Testing
Right, exactly. I am arguing that testing PGO, which is a buggy optimization pass, incurs too much developer cost to justify a 5-20% talos improvement on select benchmarks. On Linux, which is a very small percentage of our market share, and where distributions make their own builds anyway. Whether we'd tell distributions that PGO was unsupported: it actually seems difficult to say that it *is* supported, even now. PGO bugs will likely be highly dependent on the environment and compiler version, which are won't be exactly the same anywhere as they are on Windows. -David On Thursday, October 11, 2012 12:32:10 AM UTC-7, Boris Zbarsky wrote: On 10/11/12 3:05 AM, Tim Taubert wrote: Also, I'm not sure how this affects Telemetry results. In terms of perf measurements we'd probably need to completely ignore everything from non-release builds as the results might differ heavily for some use cases. I'm not following. The suggestion, as far as I can tell, is to drop Linux PGO completely. We woudln't have it in nightly, Aurora, Beta, or releases. Compiling with PGO on Linux would be an unsupported configuration that we'd probably advise distros against, because it wouldn't be particularly well-tested. So the real question is whether PGO on Linux is worth it in general to us, again as far as I can tell. -Boris ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal: Remove Linux PGO Testing
Keep in mind that debug builds are probably at least an order of magnitude slower (or a large factor), whereas PGO is a very small factor. (After all, we do not PGO on Mac and it doesn't seem to be a problem.) -David On Thursday, October 11, 2012 12:05:35 AM UTC-7, Tim Taubert wrote: On 10/10/2012 11:57 PM, Justin Lebar wrote: The main reason I'd want Linux PGO is for mobile. On desktop Linux, most users (I expect) don't run our builds, so it's not a big deal if they're some percent slower. (Unless distros commonly do PGO builds of Firefox?) But we're not doing mobile Linux PGO builds (that I know of), and I don't expect success with desktop PGO is much related to success with mobile PGO. You may be right for release builds but that doesn't hold true for Nightly/Aurora/Beta users. I don't think it's a good idea to make those builds ~20% slower when of course we want and need more testers. Don't forget that testers on Linux do not only test Linux-only features but also features we have on every platform. Nobody likes running debug builds because they're slower so why would that be different for non-PGO builds? Also, I'm not sure how this affects Telemetry results. In terms of perf measurements we'd probably need to completely ignore everything from non-release builds as the results might differ heavily for some use cases. - Tim ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
How to be notified when a node gets detached/reparented?
Context: in the firefox devtools, we need to track some nodes and update different views based on what's happening to this node (show its parents, show its child, show its attributes, …). The new Mutation observers are very helpful. But there's one thing I am not really sure how to handle correctly . When a node gets detached (parent.removeChild(node)) or reparented, I need to be notified. My current idea is to listen to childList mutations from the parent, then, on this mutation, check if the node is still part of the children of the parent, if not, check if it has a parent, if so, the node has been *relocated*, then I need re-listen to a childList mutation from this new parent, if no parent, the node has been *detached*. I was wondering if there was any better way to do that. Thanks, -- Paul ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: How to be notified when a node gets detached/reparented?
Marcio Galli wrote: Hi Paul, so this means we do not have anymore DOMnodeRemoved from the mutation events? There's no DOMNodeRemoved type: https://developer.mozilla.org/en-US/docs/DOM/MutationObserver#MutationObserverInit But there's a removedNodes from the mutation record. Maybe this array include the target if we listen to the subtree. I'll try. I find your use case sort of important specially now that I believe pages will suffer more changes based in template operations in the client. So detecting context is key for client apps to know where they were and what to do next. A more specific case is a 4x4 grid abcd that loses quadrant a. It is pretty important to signal the new state, let's say null,bcd to consumers — for example a grid rearrangement may take place. m On Thu, Oct 11, 2012 at 8:40 AM, Paul Rouget p...@mozilla.com wrote: Context: in the firefox devtools, we need to track some nodes and update different views based on what's happening to this node (show its parents, show its child, show its attributes, …). The new Mutation observers are very helpful. But there's one thing I am not really sure how to handle correctly . When a node gets detached (parent.removeChild(node)) or reparented, I need to be notified. My current idea is to listen to childList mutations from the parent, then, on this mutation, check if the node is still part of the children of the parent, if not, check if it has a parent, if so, the node has been *relocated*, then I need re-listen to a childList mutation from this new parent, if no parent, the node has been *detached*. I was wondering if there was any better way to do that. Thanks, -- Paul ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform -- www.telasocial.com -- Paul ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: How to be notified when a node gets detached/reparented?
Paul Rouget wrote: Marcio Galli wrote: Hi Paul, so this means we do not have anymore DOMnodeRemoved from the mutation events? There's no DOMNodeRemoved type: https://developer.mozilla.org/en-US/docs/DOM/MutationObserver#MutationObserverInit But there's a removedNodes from the mutation record. Maybe this array include the target if we listen to the subtree. I'll try. Nope. I find your use case sort of important specially now that I believe pages will suffer more changes based in template operations in the client. So detecting context is key for client apps to know where they were and what to do next. A more specific case is a 4x4 grid abcd that loses quadrant a. It is pretty important to signal the new state, let's say null,bcd to consumers — for example a grid rearrangement may take place. m On Thu, Oct 11, 2012 at 8:40 AM, Paul Rouget p...@mozilla.com wrote: Context: in the firefox devtools, we need to track some nodes and update different views based on what's happening to this node (show its parents, show its child, show its attributes, …). The new Mutation observers are very helpful. But there's one thing I am not really sure how to handle correctly . When a node gets detached (parent.removeChild(node)) or reparented, I need to be notified. My current idea is to listen to childList mutations from the parent, then, on this mutation, check if the node is still part of the children of the parent, if not, check if it has a parent, if so, the node has been *relocated*, then I need re-listen to a childList mutation from this new parent, if no parent, the node has been *detached*. I was wondering if there was any better way to do that. Thanks, -- Paul ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform -- www.telasocial.com -- Paul ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform -- Paul ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: How to be notified when a node gets detached/reparented?
Paul Rouget wrote: Context: in the firefox devtools, we need to track some nodes and update different views based on what's happening to this node (show its parents, show its child, show its attributes, …). The new Mutation observers are very helpful. But there's one thing I am not really sure how to handle correctly . When a node gets detached (parent.removeChild(node)) or reparented, I need to be notified. My current idea is to listen to childList mutations from the parent, then, on this mutation, check if the node is still part of the children of the parent, if not, check if it has a parent, if so, the node has been *relocated*, then I need re-listen to a childList mutation from this new parent, if no parent, the node has been *detached*. I was wondering if there was any better way to do that. Another way to do it is to listen to subtree mutations from the documentElement, and then check if the targeted node is part of the removeNodes list. Would that deteriorate the performance? -- Paul ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Xulrunner UniversalXPConnect confusion
On Thu, Oct 11, 2012 at 11:53 AM, matthew.pain...@import.io wrote: Hi Scott, Could you expand on your hack? I'm in a similar situation here :) In the end we opted to get the filepath via a Java dialog but here's the gist: In user.js (same folder as the prefs.js file) I added the following:lines: user_pref(signed.applets.codebase_principal_support, true); user_pref(capability.principal.codebase.p0.granted, UniversalXPConnect UniversalFileRead); user_pref(capability.principal.codebase.p0.id, URL-TO-YOUR-APP-HERE); user_pref(capability.principal.codebase.p0.subjectName, ); Now when you read the value property of a file input it should return the full path to the file selected by the user. Unfortunately this is taken from some notes that I haven't looked at in a few months - if it doesn't work, let me know and I'll try to whip up a quick sample. Best, -- Scott Elcomb @psema4 on Twitter / Identi.ca / Github more Atomic OS: Self Contained Microsystems http://code.google.com/p/atomos/ Member of the Pirate Party of Canada http://www.pirateparty.ca/ ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Proposed W3C Charter: Pointer Events Working Group
W3C is proposing a charter for a new Pointer Events Working Group. For more details, see: http://lists.w3.org/Archives/Public/public-new-work/2012Sep/0017.html http://www.w3.org/2012/pointerevents/charter/charter-proposed.html Mozilla has the opportunity to send comments or objections through Thursday, October 25. Please reply to this thread if you think there's something we should say. -David -- 턞 L. David Baron http://dbaron.org/ 턂 턢 Mozilla http://www.mozilla.org/ 턂 ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal: Remove Linux PGO Testing
On 10/11/2012 02:33 AM, Mike Hommey wrote: On Wed, Oct 10, 2012 at 05:57:53PM -0400, Justin Lebar wrote: By turning off Linux PGO testing, you really mean stop making and distributing Linux PGO builds, right? The main reason I'd want Linux PGO is for mobile. On desktop Linux, most users (I expect) don't run our builds, so it's not a big deal if they're some percent slower. Many people have made claims about that at several different occasions. Can we once and for all come up with actual data on that? That being said, PGO on Linux is between 5 and 20% improvement on our various talos tests. That's with the version of gcc we currently use, which is 4.5. I'd expect 4.7 to do a better job even, especially if we added lto to the equation (and since we are now building on x86-64 machines, we wouldn't have to worry about memory usage ; link time could be a problem, though). Also note that disabling PGO currently also means disabling the optimizations we do on omni.ja (central directory optimizations and reordering). This is somehow covered by bug 773171. I wouldn't be surprised if most of the pgo benefit is because of bad inline decisions by gcc. If we can narrow the gap by adding MOZ_ALWAYS_INLINE, then maybe we can drop pgo. I did a talos run during the last clang update to compare it with the previous version on OS X, but I now added linux runs to compare gcc 4.5 and clang on linux. The results are interesting (see attached file). dromaeo_dom shows a 31% improvement on 64 bits for example. This also suggests another option: using clang on linux too. This would have the added benefit of using the same compiler for OS X and Linux, which would remove most of the argument of developers spending time on linux only issues. I can do a comparison of gcc+pgo and clang if others are interested. Mike Cheers, Rafael a11yr_paint Fedora 12 - Constantine | (527.681818182, 2.59651983057) - (413.15, 1.41403960494) 1.2772x better Fedora 12 x64 - Constantine | (451.454545455, 1.24422992408) - (349.45, 0.819984718342) 1.2919x better dromaeo_css Fedora 12 - Constantine | (2152.09181818, 12.0144869607) - (2639.839, 8.95312637755) 1.2266x better Fedora 12 x64 - Constantine | (2485.29272727, 13.7659449067) - (2921.445, 17.8096790459) 1.1755x better dromaeo_dom Fedora 12 - Constantine | (142.806181818, 0.744898237637) - (187.9765, 0.650195498534) 1.3163x better Fedora 12 x64 - Constantine | (181.576818182, 0.806482009967) - (232.0514, 1.34925457235) 1.2780x better kraken Fedora 12 - Constantine | (3939.40909091, 71.9270328223) - (3767.41, 71.3917310166) 1.0457x better Fedora 12 x64 - Constantine | (3579.29090909, 15.596652341) - (3446.56, 9.33275577468) 1.0385x better num_ctors CentOS (x86_64) release 5 (Final) | (232.0,None) - (174.0, None) 1.x better CentOS release 5 (Final) | (232.0,None) - (174.0, None) 1.x better sunspider Fedora 12 - Constantine | (364.172727273, 1.5583412188) - (345.08, 1.26271089553) 1.0553x better Fedora 12 x64 - Constantine | (334.363636364, 2.07058055282) - (321.97, 2.64172734312) 1.0385x better tdhtmlr_nochrome_paint Fedora 12 - Constantine | (417.558727273, 0.728348424302) - (392.9735, 0.424360042529) 1.0626x better Fedora 12 x64 - Constantine | (390.3705, 0.633414181048) - (359.1264, 6.1131889195) 1.0870x better tdhtmlr_paint Fedora 12 - Constantine | (433.884909091, 4.67646883558) - (402.9236, 4.62643754814) 1.0768x better Fedora 12 x64 - Constantine | (398.0735, 3.51583823606) - (368.2883, 1.94210482286) 1.0809x better tp5n_main_rss_paint Fedora 12 - Constantine | (114134400.0, 271577.523722) - (115129700.0, 280881.107982) 1.0087x worse Fedora 12 x64 - Constantine | (149395900.0, 563431.271779) - (148169900.0, 430915.422528) 1.0083x better tp5n_paint Fedora 12 - Constantine | (377.5355, 2.28065749844) - (316.3663, 2.72038848381) 1.1933x better Fedora 12 x64 - Constantine | (325.3768, 3.31696113133) - (278.6602, 3.33314780988) 1.1676x better tp5n_xres_paint Fedora 12 - Constantine | (7249195.0,114562.779547) - (7640897.0,218688.473551) 1.0540x worse tpaint Fedora 12 - Constantine | (214.727272727, 2.47305127989) - (187.4, 7.74235973322) 1.1458x better Fedora 12 x64 - Constantine | (203.545454545, 3.0539930444) - (173.2, 4.81738641223) 1.1752x better tresize Fedora 12 - Constantine | (13.0813363636, 0.276579872514) - (11.8753, 0.318387813206) 1.1016x better Fedora 12 x64 - Constantine | (12.0, 0.0)- (11.0, 0.0)1.0909x better ts_paint Fedora 12 - Constantine |
Re: Proposal: Remove Linux PGO Testing
I filed bug 800471 for considering using Clang on Linux. -Gary This also suggests another option: using clang on linux too. This would have the added benefit of using the same compiler for OS X and Linux, which would remove most of the argument of developers spending time on linux only issues. I can do a comparison of gcc+pgo and clang if others are interested. Cheers, Rafael ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
W3C Proposed Recommendation: WOFF File Format 1.0
W3C recently published the following Proposed Recommendation (the final stage in the W3C process before Recommendation) WOFF File Format 1.0 http://www.w3.org/TR/2012/PR-WOFF-20121011/ Mozilla's Jonathan Kew is one of the authors of this specification. If there are comments you think Mozilla should send as part of the review, or if you think Mozilla should voice support or opposition to the specification, please say so in this thread. Given Mozilla's involvement, I'd expect to express unqualified support for the specification's advancement. (I'd also note that there have been many previous opportunities to make comments, so it's somewhat bad form to bring up fundamental issues for the first time at this stage.) The deadline for Mozilla to send comments is November 8. -David -- 턞 L. David Baron http://dbaron.org/ 턂 턢 Mozilla http://www.mozilla.org/ 턂 ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Imported code
In Bug 794510, ehsan said in response to me: Isaac makes a good point; we should clearly mark imported code, both for our own purposes and for scripts. Biesi and I were commiserating about the lack of a standard for this (third_party/blah such as netwerk/third_party/sctp instead of netwerk/sctp/src might be one way to do it, or put them ALL in a top-level third-party directory, or add a file in the root of a third-party directory (IMPORTED, with info on where/how/etc), ...) Well, marking imported code is definitely helpful, but really we should have a clear process on how to modify the imported code. It is entirely unreasonable to render ourselves unable to modify our imported code (just look at the current situation with NSPR which causes developers to go through huge pain in order to work around things which would be very simply dealt with if only we had the option of fixing NSPR...). We currently do a very good job in some of the projects (see angle for example: http://mxr.mozilla.org/mozilla-central/source/gfx/angle/) and in some other cases we do an extremely poor job (case in point: nsprpub/!). Really, whoever maintains a given imported code should come up with a clear process on how to take patches to that code and either try to upstream it if it makes sense or maintain a local patch to assist updates to the imported code in the future. Right. I've tried to create automatic import scripts for all the libraries I've imported as part of webrtc; some more complex (webrtc/trunk, where we expect to be upstreaming a lot of stuff and editing out a lot from the import) and others that basically import updates from the source repo and require reapplying local updates (which I maintained as independent patches when we need them); this is mostly for libraries that we don't expect/want to modify, and if there are changes needed we ask for updates upstream (netwerk/sctp/src for example, and netwerk/srtp/src; see netwerk/srtp/update_srtp.sh). Standardizing how patches are applied on top of imports would be good, and also how to handle patches slated for upstreaming (that will hopefully be backed out when upstream updates). A tarball of patches in the directory is one way I believe is used (though very un-source-control-management style). Most do updates entirely by hand, which makes updates extrodinarily painful and infrequent (or never). We could also keep a list of bugs to apply patches from (somewhat better), but still a very manual process. I built a more complex process for media/webrtc/trunk, which I've had to only partly use on alder and not m-c, because of the size of the resultant imported changeset history. I plan to revise this process to resolve that issue, but the fundamental way it works (in media/webrtc/webrtc_update.sh) is to: 1) import a complete update (using hg addremove --similarity XX to catch renames) 2) merge it to another head which has pending upstream patches 3) merge *that* to another head which has deletions not intended for upstream to winnow it down to what we want in our tree. In webrtc, this involves removing large number of third-party modules, remove 40MB video YUV test files, etc. 4) take that, and merge it to a mozilla repo (alder) which merges any local (non-upstream) mods to our tree with the imported update. I've used this to produce what I want on alder, and have released stuff to m-c by copying the code over and running hg addremove, instead of using hg pull from alder to m-c, because the imported patch history would enormously expand the repo size. For smaller projects this would not be an issue. This may also be overkill for smaller projects, however. This itself may be overly complex, especially in the separation of patches-for-upstream from our normal dev tree. When I designed it, I didn't know how many upstream patches I'd be producing. -- Randell Jesup, Mozilla Corp remove .news for personal email ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: How to be notified when a node gets detached/reparented?
On Thu, Oct 11, 2012 at 6:04 AM, Paul Rouget p...@mozilla.com wrote: Paul Rouget wrote: Context: in the firefox devtools, we need to track some nodes and update different views based on what's happening to this node (show its parents, show its child, show its attributes, …). The new Mutation observers are very helpful. But there's one thing I am not really sure how to handle correctly . When a node gets detached (parent.removeChild(node)) or reparented, I need to be notified. My current idea is to listen to childList mutations from the parent, then, on this mutation, check if the node is still part of the children of the parent, if not, check if it has a parent, if so, the node has been *relocated*, then I need re-listen to a childList mutation from this new parent, if no parent, the node has been *detached*. I was wondering if there was any better way to do that. Another way to do it is to listen to subtree mutations from the documentElement, and then check if the targeted node is part of the removeNodes list. You also would have to check if any of the the node's ancestors is part of the removeNodes list. Would that deteriorate the performance? That is obviously more work that has to be done both by the platform and by the webpage, so yes, it's worse performance. How much work I couldn't say offhand. It would be worth bringing this use-case to the webapps WG where MutationObservers are defined. Especially getting notifications when a node is moved in and out of a document seems like it would be worth having explicit notifications about. / Jonas ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: How to be notified when a node gets detached/reparented?
@Paul, What is your use case BTW? when you say update views based on mutations, is the goal is to let the user know what is going on? Or you actually performing other mutations back to the DOM or logging things or creating reports? m On Thu, Oct 11, 2012 at 4:27 PM, Jonas Sicking jo...@sicking.cc wrote: On Thu, Oct 11, 2012 at 6:04 AM, Paul Rouget p...@mozilla.com wrote: Paul Rouget wrote: Context: in the firefox devtools, we need to track some nodes and update different views based on what's happening to this node (show its parents, show its child, show its attributes, …). The new Mutation observers are very helpful. But there's one thing I am not really sure how to handle correctly . When a node gets detached (parent.removeChild(node)) or reparented, I need to be notified. My current idea is to listen to childList mutations from the parent, then, on this mutation, check if the node is still part of the children of the parent, if not, check if it has a parent, if so, the node has been *relocated*, then I need re-listen to a childList mutation from this new parent, if no parent, the node has been *detached*. I was wondering if there was any better way to do that. Another way to do it is to listen to subtree mutations from the documentElement, and then check if the targeted node is part of the removeNodes list. You also would have to check if any of the the node's ancestors is part of the removeNodes list. Would that deteriorate the performance? That is obviously more work that has to be done both by the platform and by the webpage, so yes, it's worse performance. How much work I couldn't say offhand. It would be worth bringing this use-case to the webapps WG where MutationObservers are defined. Especially getting notifications when a node is moved in and out of a document seems like it would be worth having explicit notifications about. / Jonas ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform -- www.telasocial.com ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal: Remove Linux PGO Testing
On 2012-10-11 3:12 PM, Anthony Jones wrote: On 11/10/12 19:33, Mike Hommey wrote: That being said, PGO on Linux is between 5 and 20% improvement on our various talos tests. That's with the version of gcc we currently use, which is 4.5. I'd expect 4.7 to do a better job even, especially if we added lto to the equation (and since we are now building on x86-64 machines, we wouldn't have to worry about memory usage ; link time could be a problem, though). Perhaps the problems can be resolved or ameliorated by bumping the minimum version of GCC that we support for PGO. Link-time optimization is described as an experimental new feature in the GCC 4.5.0 release notes[1]. The 4.6.0 release notes[2] say that it has now stabilized to the point of being usable, and the 4.7.0 release notes[3] describe it as still further improved both in reliability and code quality. If we're trying to use the 4.5 LTO, I'm not at all surprised to hear it's causing more trouble than it's worth. PGO is not the same thing as LTO, of course, but GCC's PGO was kind of an unloved stepchild until they got serious about LTO, so that, too, is likely to be much improved in 4.7. zw [1] http://gcc.gnu.org/gcc-4.5/changes.html [2] http://gcc.gnu.org/gcc-4.6/changes.html [3] http://gcc.gnu.org/gcc-4.7/changes.html ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Imported code
On 2012-10-11 3:16 PM, Randell Jesup wrote: In Bug 794510, ehsan said in response to me: Isaac makes a good point; we should clearly mark imported code, both for our own purposes and for scripts. Biesi and I were commiserating about the lack of a standard for this (third_party/blah such as netwerk/third_party/sctp instead of netwerk/sctp/src might be one way to do it, or put them ALL in a top-level third-party directory, or add a file in the root of a third-party directory (IMPORTED, with info on where/how/etc), ...) Well, marking imported code is definitely helpful, but really we should have a clear process on how to modify the imported code. It is entirely unreasonable to render ourselves unable to modify our imported code (just look at the current situation with NSPR which causes developers to go through huge pain in order to work around things which would be very simply dealt with if only we had the option of fixing NSPR...). We currently do a very good job in some of the projects (see angle for example: http://mxr.mozilla.org/mozilla-central/source/gfx/angle/) and in some other cases we do an extremely poor job (case in point: nsprpub/!). Really, whoever maintains a given imported code should come up with a clear process on how to take patches to that code and either try to upstream it if it makes sense or maintain a local patch to assist updates to the imported code in the future. Right. I've tried to create automatic import scripts for all the libraries I've imported as part of webrtc; some more complex (webrtc/trunk, where we expect to be upstreaming a lot of stuff and editing out a lot from the import) and others that basically import updates from the source repo and require reapplying local updates (which I maintained as independent patches when we need them); this is mostly for libraries that we don't expect/want to modify, and if there are changes needed we ask for updates upstream (netwerk/sctp/src for example, and netwerk/srtp/src; see netwerk/srtp/update_srtp.sh). Standardizing how patches are applied on top of imports would be good, and also how to handle patches slated for upstreaming (that will hopefully be backed out when upstream updates). A tarball of patches in the directory is one way I believe is used (though very un-source-control-management style). Most do updates entirely by hand, which makes updates extrodinarily painful and infrequent (or never). I think owners of imported code should be responsible for updating it, and it is nice of them if they include update scripts (it makes it easier for them to do future updates, and it decreases the bus factor), but I don't know how much we can enforce that. Other projects such as WebRTC which have more complicated workflows can pick whatever works best for them. However, keeping the history of the changes over upstream code is another issue. Really, we should not need to do anything special there since the history of modifications is already recorded in the revision control systems, but some people like to store them as patches etc, probably because of CVS-nostalgia (you didn't use to have the entire history of the repo on your hard disk!). What I really don't want us to do is to prohibit people from fixing things in the imported code. That is the absolute worst situation we can face with a given piece of code, as we already have learned painfully. Cheers, Ehsan ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal: Remove Linux PGO Testing
On Thu, Oct 11, 2012 at 02:26:33PM -0400, Rafael Ávila de Espíndola wrote: On 10/11/2012 02:33 AM, Mike Hommey wrote: On Wed, Oct 10, 2012 at 05:57:53PM -0400, Justin Lebar wrote: By turning off Linux PGO testing, you really mean stop making and distributing Linux PGO builds, right? The main reason I'd want Linux PGO is for mobile. On desktop Linux, most users (I expect) don't run our builds, so it's not a big deal if they're some percent slower. Many people have made claims about that at several different occasions. Can we once and for all come up with actual data on that? That being said, PGO on Linux is between 5 and 20% improvement on our various talos tests. That's with the version of gcc we currently use, which is 4.5. I'd expect 4.7 to do a better job even, especially if we added lto to the equation (and since we are now building on x86-64 machines, we wouldn't have to worry about memory usage ; link time could be a problem, though). Also note that disabling PGO currently also means disabling the optimizations we do on omni.ja (central directory optimizations and reordering). This is somehow covered by bug 773171. I wouldn't be surprised if most of the pgo benefit is because of bad inline decisions by gcc. If we can narrow the gap by adding MOZ_ALWAYS_INLINE, then maybe we can drop pgo. A non-unsignificant part of the performance improvements PGO gives come from code reordering to improve branch prediction. Presumably, we can use NS_LIKELY/NS_UNLIKELY to improve some branches manually. Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Imported code
On Thu, Oct 11, 2012 at 4:20 PM, Ehsan Akhgari ehsan.akhg...@gmail.com wrote: What I really don't want us to do is to prohibit people from fixing things in the imported code. That is the absolute worst situation we can face with a given piece of code, as we already have learned painfully. This should absolutely be at the discretion of the maintainer of the imported code in our tree. From personal experience, allowing local changes to land in Breakpad before they are landed upstream has caused huge headaches. Our in-tree copy of Breakpad was nearly 2 years out-of-date because of a few large patches that landed in support of OOP work and were difficult to upstream. -Ted ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Imported code
On 2012-10-11 6:36 PM, Mike Hommey wrote: On Thu, Oct 11, 2012 at 06:14:51PM -0400, Ted Mielczarek wrote: On Thu, Oct 11, 2012 at 4:20 PM, Ehsan Akhgari ehsan.akhg...@gmail.com wrote: What I really don't want us to do is to prohibit people from fixing things in the imported code. That is the absolute worst situation we can face with a given piece of code, as we already have learned painfully. This should absolutely be at the discretion of the maintainer of the imported code in our tree. From personal experience, allowing local changes to land in Breakpad before they are landed upstream has caused huge headaches. Our in-tree copy of Breakpad was nearly 2 years out-of-date because of a few large patches that landed in support of OOP work and were difficult to upstream. Same experience with jemalloc, which is why the rule is to have things applied upstream first. What is the nature of this problem? Is it the difficulty to reapply the patches when pulling new code from upstream? Note that I'm mostly talking about fixing small problems in our copy, not doing major architectural changes. Ehsan ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Imported code
On Thu, Oct 11, 2012 at 06:38:57PM -0400, Ehsan Akhgari wrote: On 2012-10-11 6:36 PM, Mike Hommey wrote: On Thu, Oct 11, 2012 at 06:14:51PM -0400, Ted Mielczarek wrote: On Thu, Oct 11, 2012 at 4:20 PM, Ehsan Akhgari ehsan.akhg...@gmail.com wrote: What I really don't want us to do is to prohibit people from fixing things in the imported code. That is the absolute worst situation we can face with a given piece of code, as we already have learned painfully. This should absolutely be at the discretion of the maintainer of the imported code in our tree. From personal experience, allowing local changes to land in Breakpad before they are landed upstream has caused huge headaches. Our in-tree copy of Breakpad was nearly 2 years out-of-date because of a few large patches that landed in support of OOP work and were difficult to upstream. Same experience with jemalloc, which is why the rule is to have things applied upstream first. What is the nature of this problem? Is it the difficulty to reapply the patches when pulling new code from upstream? Note that I'm mostly talking about fixing small problems in our copy, not doing major architectural changes. Experience shows that adding patches to our copies of third party libraries lead to, at least: - long-time forks - undocumented patches (no corresponding patch file in the tree to reapply = fun to handle upgrades, or broken upgrades if unnoticed) - outdated patches (a .patch exists but doesn't correspond to what is in the tree = most likely broken upgrades) - unapplied patches after an upgrade (= broken upgrade) I have seen multiple iterations of each of the above. This is not a theoretical problem. And some even happened with things that are much less third-party than most third-party code we have in the tree: nspr and nss. Mike ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal: Remove Linux PGO Testing
On 2012-10-11 4:34 PM, Mike Hommey wrote: On Thu, Oct 11, 2012 at 02:26:33PM -0400, Rafael Ávila de Espíndola wrote: On 10/11/2012 02:33 AM, Mike Hommey wrote: On Wed, Oct 10, 2012 at 05:57:53PM -0400, Justin Lebar wrote: By turning off Linux PGO testing, you really mean stop making and distributing Linux PGO builds, right? The main reason I'd want Linux PGO is for mobile. On desktop Linux, most users (I expect) don't run our builds, so it's not a big deal if they're some percent slower. Many people have made claims about that at several different occasions. Can we once and for all come up with actual data on that? That being said, PGO on Linux is between 5 and 20% improvement on our various talos tests. That's with the version of gcc we currently use, which is 4.5. I'd expect 4.7 to do a better job even, especially if we added lto to the equation (and since we are now building on x86-64 machines, we wouldn't have to worry about memory usage ; link time could be a problem, though). Also note that disabling PGO currently also means disabling the optimizations we do on omni.ja (central directory optimizations and reordering). This is somehow covered by bug 773171. I wouldn't be surprised if most of the pgo benefit is because of bad inline decisions by gcc. If we can narrow the gap by adding MOZ_ALWAYS_INLINE, then maybe we can drop pgo. A non-unsignificant part of the performance improvements PGO gives come from code reordering to improve branch prediction. Presumably, we can use NS_LIKELY/NS_UNLIKELY to improve some branches manually. Don't both of these proposals map to tons of manual work? I'm not convinced that doing either of those would necessarily be easier than finding and fixing the PGO bug at hand. Ehsan ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: How to be notified when a node gets detached/reparented?
On 10/11/2012 02:40 PM, Paul Rouget wrote: Context: in the firefox devtools, we need to track some nodes and update different views based on what's happening to this node (show its parents, show its child, show its attributes, …). The new Mutation observers are very helpful. But there's one thing I am not really sure how to handle correctly . When a node gets detached (parent.removeChild(node)) or reparented, I need to be notified. My current idea is to listen to childList mutations from the parent, then, on this mutation, check if the node is still part of the children of the parent, if not, check if it has a parent, if so, the node has been *relocated*, then I need re-listen to a childList mutation from this new parent, if no parent, the node has been *detached*. Why do you need to re-listen anywhere? You get the node in a MutationRecord and when the callback is called you check where it is. ( node.contains can be useful and certainly faster than anything in JS. ) If the node doesn't have parent, it is detached. I was wondering if there was any better way to do that. Thanks, -- Paul ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal: Remove Linux PGO Testing
On 10/11/2012 03:49 PM, Ehsan Akhgari wrote: Don't both of these proposals map to tons of manual work? I'm not convinced that doing either of those would necessarily be easier than finding and fixing the PGO bug at hand. The problem is that fixing this one bug might take only a few days, but that underestimates the true cost of PGO. The fact is that, as long as we have it turned on, PGO will keep introducing new bugs. Each new line of code that we write gives PGO an opportunity to miscompile it. This represents an unbounded amount of work for us. This can be compared to the benefits PGO provides, which are growing very slowly, if at all (about 2% per year [1]). Additionally, I don't think we have a good idea of how many bugs are caused by GCC PGO. Windows PGO issues often turn into topcrashes. On Linux, we may not have enough users for these bugs to bubble up far enough for us to investigate them. -Bill [1] http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.29.434 ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Why we avoid making private modifications to NSPR and NSS (was Re: Imported code)
Randell quoted: Ehsan wrote: It is entirely unreasonable to render ourselves unable to modify our imported code (just look at the current situation with NSPR which causes developers to go through huge pain in order to work around things which would be very simply dealt with if only we had the option of fixing NSPR...). First, I think that a lot of Mozillian's concerns about how we can(not) change NSPR and NSS are based on old events and circumstances that do not apply now. For example, the time where NSS 3.whatever was being FIPS validated seemed to cause a lot of unfortunate misunderstandings all around. But, that was literally years ago. I encourage people to be more optimistic about things NSPR and NSS related. And, please keep in mind that there is already progress being made (if not a definitive agreement) on the Mercurial vs. CVS issue. There is a very practical reason for avoiding making private changes to NSPR and NSS. The most obvious reason is that it makes it esay to merge changes between upstream and our codebase. However, the more critical reason is that we support --use-system-nss and --use-system-nspr, and some Linux developers build their Firefox packages with those options. As long as we support --use-system-nss and --use-system-nspr, we need to make sure the upstream NSPR and NSS contain every bugfix and every feature that we require. If we were to give up on the idea of supporting --use-system-nspr and --use-system-nss, then we could gain more flexibility in how we change NSPR and/or NSS in mozilla-central. However, at least in theory, --use-system-nss and --use-system-nspr should be superior performance-wise, on Linux, because usually the system NSPR and NSS libraries are already loaded before Firefox starts. Thus, our startup time should be lower. Also, according to some conversation on dev-tech-crypto, the system NSS may eventually provide better platform integration regarding centralized certificate and smartcard handling, allowing us to share various security-related features with other applications (between Firefox and Thunderbird, or Firefox and whatever-the-Gnome-email-app-is). So, I think there are still advantages to supporting system NSS. In the event that we really do need to make private changes to NSPR and NSS, we should be able to do so. I think there's been a lot of unnecessary misunderstanding about this. If you *need* a change made to NSPR or NSS before we're ready to make a NSPR or NSS release, please make sure that the NSPR and NSS teams are aware of that, so we can help. And, whenever possible, try to avoid creating emergency situations. I have found everybody to be quite reasonable about it, especially when my request didn't involve me needing them to drop their work to do a code review for me. Usually we upstream those changes into NSS and NSPR first, because we need NSPR and NSS peers to do the review anyway. We try to avoid making temporary fixes only in mozilla-central, because, well, what happens if they don't get accepted upstream? Then we've broken --use-system-nspr and --use-system-nss. Still, I think there is more flexibility here than people realize. In general, it is harder to get changes made to NSS and NSPR than it is to get changes made to the rest of Gecko. One reason is that the reviewing standards are different/stricter in these modules than they are in some/many (but not all) Gecko modules. I actually prefer the stricter NSPR/NSS reviews and I hope that doesn't change. Another reason is that, generally mozilla-central is primarily geared towards Mozilla products (especially Firefox), whereas NSPR and NSS are shared between us, Chrome, and all Linux distributions. NSPR and NSS are part of the Linux Standard Base, which means that it is difficult to make compatibility-breaking changes to them. Sometimes when people suggest changes to NSPR and/or NSS, it isn't clear how urgent that change is. Definitely, I have sat on a review for too long because I didn't realize it was actually as urgent as other work I am doing. Because many of the NSPR and NSS peers do not work for Mozilla, and do not work on Mozilla stuff full-time, it definitely isn't as obvious to them what is a high-priority request and what isn't. And, also, because they have their own jobs and their own schedules, sometimes schedules are not aligned as well as we would like/need them to be. IMO, the solution to that is to have more MoCo employees and other Mozillians become peers on the NSPR and NSS projects, so that we can help with the code reviews in these projects. I know on the NSS part, we're definitely trying to make progress in getting more Mozilla people involved. For NSPR, my understanding is that we're generally migrating away from using NSPR in Gecko, except for networking. Lots of stuff that's in NSPR already has replacements in mfbt and/or in ISO C/C++. One reason for doing this is that we can make use of
Re: Proposal: Remove Linux PGO Testing
Zack Weinberg wrote: Link-time optimization is described as an experimental new feature in the GCC 4.5.0 release notes[1]. The 4.6.0 release notes[2] say that it has now stabilized to the point of being usable, and the 4.7.0 release notes[3] describe it as still further improved both in reliability and code quality. If we're trying to use the 4.5 LTO, I'm not at all surprised to hear it's causing more trouble than it's worth. PGO is not the same thing as LTO, of course, but GCC's PGO was kind of an unloved stepchild until they got serious about LTO, so that, too, is likely to be much improved in 4.7. I think it is important to give Linux users the fastest browser we can give them, because: 1. Linux users tend to be disproportionately influential in the markets we care the most about (web developers, techies) 2. Linux is the foundation of B2G and Firefox for Android, where we *definitely* must deliver the fastest product we can Now, if it were up to me, I'd try to reproduce this on a build built with the latest stable GCC or latest stable clang, and if that fixes the issue, I'd consider this a big motivation for upgrading to GCC 4.5 to a better compiler, which we need to do anyway for language feature support. Definitely, I don't think we should be adding hacks to our code to work around GCC problems that are already fixed in later releases of GCC. It's better to just make the build fail when the user attempts to use one of those older GCC releases. Now, if PGO doesn't result in the fastest browser, then of course we should disable PGO. Or, if there is no better compiler possible, then yes, I think it makes sense to disable PGO temporarily until there is a better compiler available. (And/or, help fix the compiler, either by contributing a patch, or by commissioning somebody else to contribute a patch.) Cheers, Brian ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Why we avoid making private modifications to NSPR and NSS (was Re: Imported code)
Ehsan wrote: It is entirely unreasonable to render ourselves unable to modify our imported code (just look at the current situation with NSPR which causes developers to go through huge pain in order to work around things which would be very simply dealt with if only we had the option of fixing NSPR...). I don't know what the current situation with NSPR is. I guess I am not reviewing patches fast enough. That's certainly my fault. As for the option of fixing NSPR: you have that option. NSPR public functions need to stay backward compatible. This means the prototypes of public functions and the definitions of public types cannot change. (There are exceptions, if done carefully.) Bugs in function behavior can certainly be fixed. Wan-Teh ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposed W3C Charter: Pointer Events Working Group
On 10/11/2012 07:55 PM, L. David Baron wrote: W3C is proposing a charter for a new Pointer Events Working Group. For more details, see: http://lists.w3.org/Archives/Public/public-new-work/2012Sep/0017.html http://www.w3.org/2012/pointerevents/charter/charter-proposed.html Mozilla has the opportunity to send comments or objections through Thursday, October 25. Please reply to this thread if you think there's something we should say. -David We should join PEWG. Nicer API than touch API. The spec needs some work but is a good approach. -Olli ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
PRtypes [was Re: Why we avoid making private modifications to NSPR and NSS]
On 10/11/2012 7:52 PM, Wan-Teh Chang wrote: NSPR public functions need to stay backward compatible. This means the prototypes of public functions and the definitions of public types cannot change. (There are exceptions, if done carefully.) Bugs in function behavior can certainly be fixed. Wan-Teh I think the discussion that's currently going around the block with respect to NSPR is the entire PR types issue, particularly the fact that PRUint64 != uint64_t on some platforms (notably BSD). I do realize that changing this in NSPR carries backwards-compatibility issues that are important to manage, but I wonder if there is a way to let users of NSPR choose whether or not to have stdint-compliant definitions of PR types instead of the backwards-compatible variants. ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
Re: Proposal: Remove Linux PGO Testing
2. Linux is the foundation of B2G and Firefox for Android, where we *definitely* must deliver the fastest product we can I totally agree, but it's not clear to me whether continuing to do PGO on desktop Linux has any effect on our ability to potentially do PGO on Android/B2G. If we were to stop doing PGO on desktop Linux tomorrow, would that make it much harder to do PGO on mobile in the future? Not to beg the question as to whether we want to do PGO on desktop Linux. We may want to for the other reasons you suggest... On Thu, Oct 11, 2012 at 8:49 PM, Brian Smith bsm...@mozilla.com wrote: Zack Weinberg wrote: Link-time optimization is described as an experimental new feature in the GCC 4.5.0 release notes[1]. The 4.6.0 release notes[2] say that it has now stabilized to the point of being usable, and the 4.7.0 release notes[3] describe it as still further improved both in reliability and code quality. If we're trying to use the 4.5 LTO, I'm not at all surprised to hear it's causing more trouble than it's worth. PGO is not the same thing as LTO, of course, but GCC's PGO was kind of an unloved stepchild until they got serious about LTO, so that, too, is likely to be much improved in 4.7. I think it is important to give Linux users the fastest browser we can give them, because: 1. Linux users tend to be disproportionately influential in the markets we care the most about (web developers, techies) 2. Linux is the foundation of B2G and Firefox for Android, where we *definitely* must deliver the fastest product we can Now, if it were up to me, I'd try to reproduce this on a build built with the latest stable GCC or latest stable clang, and if that fixes the issue, I'd consider this a big motivation for upgrading to GCC 4.5 to a better compiler, which we need to do anyway for language feature support. Definitely, I don't think we should be adding hacks to our code to work around GCC problems that are already fixed in later releases of GCC. It's better to just make the build fail when the user attempts to use one of those older GCC releases. Now, if PGO doesn't result in the fastest browser, then of course we should disable PGO. Or, if there is no better compiler possible, then yes, I think it makes sense to disable PGO temporarily until there is a better compiler available. (And/or, help fix the compiler, either by contributing a patch, or by commissioning somebody else to contribute a patch.) Cheers, Brian ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform ___ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
