Re: [Wikitech-l] ResourceLoader JavaScript validation on trunk (bug 28626)
On Wed, Jul 6, 2011 at 3:04 PM, Brion Vibber br...@pobox.com wrote: Only the parser is being used right now, in two places: - on the JavaScriptMinifier test cases to confirm that results are valid JS (should be extended to a fuzz tester, probably) - on each individual file loaded via ResourceLoaderFileModule or ResourceLoaderWikiModule, so we can throw a JavaScript exception with details of the parse error *with line numbers for the original input file* This can be disabled by turning off $wgResourceLoaderValidateJs, but I'm setting it on by default to aid testing. I'd like for folks to keep an eye out to make sure they don't get any false positive parse errors in real-world modules, and to see if there are any noticeable performance regressions. Like ResourceLoader's minification itself the validation parses are cached so shouldn't cause too much ongoing load, but it still takes some time. Per feedback from TranslateWiki (yay testers!) I've disabled validation for JS modules coming from on-disk files; large libs like jQuery can hit the PHP memory limits if you're unlucky. This kills the load.php process, neatly defeating the purpose of validating the code. ;) Validation is still on by default for JS modules pulled from wiki pages -- which are editable and therefore what we really cared about to begin with. :) May still be nice to reduce the memory footprint of the syntax tree as it's being built, as it's likely a very inefficient structure by default. Gadgets that stuff big JS libraries into their pages are probably the most likely to still be able to fail this way. -- brion ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
[Wikitech-l] ResourceLoader JavaScript validation on trunk (bug 28626)
Some of you may have found that ResourceLoader's bundled minified JavaScript loads can be a bit frustrating when syntax errors creep into your JavaScript code -- not only are the line numbers reported in your browser of limited help, but a broken file can cause *all* JS modules loaded in the same request to fail[1]. This can manifest as for instance a jquery-using Gadget breaking the initial load of jquery itself because it gets bundled together into the same request. I've taken a copy of JSMin+ (MPL 1.1/GPL 2.0/LGPL 2.1 triple-license) to our includes/libs -- it's a JS minification library that had originally gotten knocked out of the running for merging due to being a bit slow, but has the advantage of coming with an actual JavaScript parser [2]. Only the parser is being used right now, in two places: - on the JavaScriptMinifier test cases to confirm that results are valid JS (should be extended to a fuzz tester, probably) - on each individual file loaded via ResourceLoaderFileModule or ResourceLoaderWikiModule, so we can throw a JavaScript exception with details of the parse error *with line numbers for the original input file* This can be disabled by turning off $wgResourceLoaderValidateJs, but I'm setting it on by default to aid testing. I'd like for folks to keep an eye out to make sure they don't get any false positive parse errors in real-world modules, and to see if there are any noticeable performance regressions. Like ResourceLoader's minification itself the validation parses are cached so shouldn't cause too much ongoing load, but it still takes some time. [1] https://bugzilla.wikimedia.org/show_bug.cgi?id=28626 [2] http://crisp.tweakblogs.net/blog/1856/jsmin+-version-13.html -- brion ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] ResourceLoader JavaScript validation on trunk (bug 28626)
How is JSMin+ different to the plain JSMin that we had and was removed due to licensing conflicts? (See: http://lists.wikimedia.org/pipermail/wikitech-l/2011-January/051308.html https://bugzilla.wikimedia.org/show_bug.cgi?id=26791) ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] ResourceLoader JavaScript validation on trunk (bug 28626)
On Wed, Jul 6, 2011 at 3:18 PM, K. Peachey p858sn...@gmail.com wrote: How is JSMin+ different to the plain JSMin that we had and was removed due to licensing conflicts? It's a different program, written by different people, based on code from another unrelated project, under a different license. Open Source is a wonderful thing; we use it every day and in this case it would be a waste if we were the only one to benefit from the hard work of others. We dubbed our little project JSMin+ because essentially it acts as Douglas Crockford's JSMin but is far less restrictive, and released it under the same MPL/GPL/LGPL tri-license as the original Narcissus code. http://crisp.tweakblogs.net/blog/1665/a-new-javascript-minifier-jsmin+.html -- brion ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
Re: [Wikitech-l] ResourceLoader JavaScript validation on trunk (bug 28626)
On 07/06/2011 03:04 PM, Brion Vibber wrote: Some of you may have found that ResourceLoader's bundled minified JavaScript loads can be a bit frustrating when syntax errors creep into your JavaScript code -- not only are the line numbers reported in your browser of limited help, but a broken file can cause *all* JS modules loaded in the same request to fail[1]. This can manifest as for instance a jquery-using Gadget breaking the initial load of jquery itself because it gets bundled together into the same request. Long term I wonder if we should not be looking at closure compiler [1], we could gain an additional 10% or so compression with simple optimisations, and it has tools for inspecting compiled output [2] Long term we could work toward making code compatible with advanced optimisations, as a side effect we could get improved jsDoc docs and even better compression and optimisations would be possible. [1] http://code.google.com/closure/compiler/ [2] http://code.google.com/closure/compiler/docs/inspector.html ___ Wikitech-l mailing list Wikitech-l@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikitech-l
