There is too much content in this thread so I am going to break it down to smaller threads.
On Fri, Apr 4, 2014 at 5:44 PM, Anis KADRI <[email protected]> wrote: > > > > On Thu, Apr 3, 2014 at 11:47 AM, Michal Mocny <[email protected]> wrote: > >> On Thu, Apr 3, 2014 at 11:18 AM, Andrew Grieve <[email protected]> >> wrote: >> >> > On Tue, Apr 1, 2014 at 12:49 PM, Brian LeRoux <[email protected]> wrote: >> > >> > > So, this is a huge refactor, but its also a step in the right >> direction. >> > In >> > > order for this to mainline though we need everyone to buy into the >> > > approach. >> > > >> > > - Simplified code (just less of it in general too) >> > > - Not maintaining our own module loader >> > > - Enables npm dep workflows instead of inventing our own >> > > - No double load or script injection >> > > - Simplified plugin authoring (no weird clobbers logic in plugin.xml / >> > just >> > > use vanilla JS to achieve this) >> > > - Optimized builds per platform (only build what a platform needs >> instead >> > > of the kitchen sink) >> > > - Future path to killing off cordova-js repo >> > > >> > > What we need to make this happen >> > > >> > > - Backwards compat: existing core plugins must work (thus must be >> tested) >> > > - Tested on all platforms for 4.x release >> > > - Well documented for new plugin authors (and migrating) >> > > >> > > We need your feedback and concerns / once this starts rolling things >> will >> > > break and we'll need to work together >> > > >> > > (And big thanks Anis for taking this path to get us to this point) >> > > >> > > >> > > >> > > On Wed, Apr 2, 2014 at 12:27 AM, Anis KADRI <[email protected]> >> > wrote: >> > > >> > > > I've been working on an alternative build system for cordova-js over >> > the >> > > > last little while and I finally have something to share. >> > > > >> > > > TL;DR >> > > > >> > > > - cordova.js includes all the plugins (no more cordova_plugins.js >> and >> > > > double plugin loading/mapping). >> > > >> > > - cordova-js is an npm module that is used by plugman to build the >> final >> > > > javascript file with all the plugins bundled in. >> > > >> > > - All the existing plugins are backward compatible with this new >> build >> > > > system without modification. >> > > > >> > > > As I briefly mentioned at our last Google Hangout, I use browserify >> to >> > > > build the final cordova.js. For those of you who are not familiar >> with >> > > > browserify [1], it is a tool that makes it possible to run node >> modules >> > > in >> > > > the browser. >> > > > >> > > > I slightly modified the cordova-js code to remove the dependency on >> > > > pluginloader and modulemapper as well as the initialization code. >> > > > Everything else stays the same (common, platforms...etc). The >> reason I >> > am >> > > > able to keep the existing source is because I use a feature of >> > browserify >> > > > called "transforms" that allows me to modify each file before it is >> > added >> > > > to the bundle. The feature allows me to parse all the javascript >> files >> > > and >> > > > map the old-style require() calls with node-js/browserify compatible >> > > > requires(). Basically mapping all the old IDs with real file paths. >> > > >> > >> > Just want to clarify that app code can still call >> > cordova.require('module/name'), or >> cordova.require('org.my.plugin.module') >> > at runtime? >> > >> > >> > > > >> > > > I modified plugman/prepare to use cordova-js as an npm module >> > dependency >> > > > and build a cordova.js with all the plugins. >> > > >> > > clobbers/merges are supported as well. I am not sure what runs does >> but I >> > > > am guessing it's code that should run onload? If that is the case >> then >> > > > everything always runs onload with browserify (no need to >> require()). >> > > >> > <runs/> means that the module will be require()d on start-up after all >> of >> > the other modules have been mapped to their symbol paths. >> > >> > To be clear, this means that all modules will be parsed, and they they >> will >> > all be require()'ed at start-up? Why would you want to do this? >> > >> >> I don't think its true that browserify does require() all modules on >> startup. We must absolutely support some form of runs, since plugins >> depend on this for correctness. >> >> >> > >> > > >> > > > Some random thoughts for the future: >> > > > - clobbers/merges/runs should be useless. plugins should >> clobber/merge >> > if >> > > > they want/need to. Each plugin can just be written as a commonJS >> > module. >> > > >> > I don't think getting rid of <merges>,<clobbers>,<runs> is an >> improvement. >> > 1. Clobbering symbols can sometimes be hard (e.g. there is special code >> for >> > clobbering properties). It's also now more code that all modules >> exporting >> > symbols will have to write. >> > >> >> In full honesty, we could just ship a cordova plugin that defines >> clobbers/merges/etc helpers. Many javascript libraries do this. >> >> However, while in hindsight I think we should probably not have added >> first >> class support for these tags, right now it seems like an unnecessary pain >> to drop support for them. Maybe its not a big deal, lets take a look at >> the plugin registry.. we could probably just reach out to every single >> plugin author if we wanted to honestly, cordova dev community wrote at >> least half of total plugins. >> > > I don't think, dropping support for them is a big deal. Are they used > anywhere else other than plugman/prepare ? By dropping support, I don't > mean dropping support now. We can just follow our deprecation policy. > > >> >> >> > 2. It removes our ability to have modules that are mapped to symbols >> > lazy-loaded (e.g. don't load the module until the first time the symbol >> > path is referenced). >> > >> >> Well, I was thinking about this more generally: <runs/> is pretty >> heavyweight right now, you are right. But perhaps we should brainstorm >> ways to just have "startup" code split out from "require" code to have all >> startup code run fast. >> >> I did something like this for tests: by convention, any js-module named >> "test" is included in the test suite on startup. We could document that >> any module named "main" will be required() on startup, and then plugins >> can >> ship a separate "main" js-module if they want a "runs" and use our >> clobbers/merges helpers from there. >> >> Again, not sure we want this pain, but it would remove some of our tooling >> code out into userland, which I think is good. >> > > I just need one or two example of some plugin code that should not run. > Most modules are just function/variable/exports definitions. If there is > some other code (function calls etc...) in the module than it will just run > in the context of that module. > So if there is code in a module that shouldn't run for whatever reason, I > would like to see an example. > > >> >> >> > 3. Our Chrome apps on cordova use modulemapper to inject plugin symbols >> > into the background window. I think this is a reasonable thing to want >> to >> > do (separate symbol mappings from code so that you can apply them in >> > multiple contexts: non-main window, webworker, etc). >> > >> > >> > >> > >> > > > - pluginloader/modulemapper will no no longer used. >> > > >> > This could also be an issue for cordova-app-harness. It needs to be >> able to >> > run the JS only for the plugins that apply to the currently running app. >> > It's fine if excess modules get loaded, since they will never be run >> (never >> > require()'ed), but we'd want to maintain a way to only apply <runs/> and >> > friends for a subset of the plugins that cordova.js was built with. >> > >> >> Since I'm almost certain that browserify doesn't require() everything on >> startup, we can easily support this as port of implementing <runs/> in >> cordova.js. Its good to make a note of its importance. >> >> >> > >> > >> > > > - cordova-cli shouldn't be impacted by this change? >> > > >> > Might be. cordova-cli keeps a copy of the base cordova.js file at >> > platforms/foo/platform_www/cordova.js. Probably this should be used or >> > removed depending on if it's still useful. >> > >> > >> > > > - each platform could have its own (exec/platform) JS and be an npm >> > > module >> > > > (different thread currently open on the topic). We could then remove >> > the >> > > > platforms/ folder from cordova-js and only keep common/ and plugman >> > could >> > > > use it to build the final bundle. Since we talked about versioning >> > > > platforms separately, this also allows us to version the javascript >> > along >> > > > with the platform? >> > > >> > We already version the JS with platforms (we copy a snapshot of it in). >> I >> > think what this does is allows us to update parts of cordova.js >> > *separately* from platforms. Platforms will version exec() and bootstrap >> > code with their native bits, and other misc. cordova.js things can be >> > versioned separately. >> > >> > >> > >> > > > - We could actually leverage NPM dependencies for javascript >> modules. >> > > > Native code will still have to be resolved but this could be a big >> win >> > > for >> > > > js-only platforms. >> > > > - Since we decided to ditch everything for node, I think browserify >> is >> > a >> > > > step in the right direction. >> > > > >> > > > Things to know: >> > > > - I've only been testing on iOS. I suspect that all the web-based >> > > platforms >> > > > such as blackberry10/firefoxos and especially Windows might be >> broken. >> > > > - Right now, everything works just the way it used to. I just >> generate >> > a >> > > > cordova-b.js (a browserify version of cordova.js) that includes all >> the >> > > > plugins but the old cordova.js and cordova_plugins.js are still >> there. >> > > All >> > > > the cordova-js tasks are still there too, I just added a new one >> called >> > > > compile-browserify that generates a browserify bundle. >> > > > - I wasn't able to run mobile-spec because of outstanding iOS issues >> > with >> > > > XCode 5.1 :-( >> > > > >> > > > If mobile-spec tests pass and we all agree on using this new build >> > system >> > > > I'd like to: >> > > > - Keep existing plugins, cordova-js code/structure as is for the >> next 6 >> > > > months and get rid of cordova_plugins.js in favor of just >> cordova.js. >> > > > - 6 months from now: restructure cordova-js and move platform >> specific >> > > code >> > > > to the platforms (maybe?) as well as update all the plugins to >> support >> > > > browserify out of the box and rely on npm-style dependencies. >> > > > >> > > > If you want to check it all out, you can: >> > > > - Look at the browserify branch of cordova-js if you want to analyze >> > how >> > > > the javascript is built. Specifically look at: compile-browserify >> and >> > > > bundle-browserify tasks (and compare them to the original ones). >> > > > - Look at the browserify branch of plugman and maybe run it with the >> > > device >> > > > plugin (or any plugin other than contacts) on a freshly created >> project >> > > and >> > > > include the generated cordova-b.js (in place of cordova.js). >> > > > >> > > > I consider this to be a big change so I would like us to address all >> > the >> > > > concerns before/if we move forward with it. >> > > >> > >> > I suspect the overhead of browserify is a more than our current require >> > system in terms of complexity and JS size. >> > >> > I think selling this as "simplifying" is not a good way to go because >> > digging into the guts of browserify, is likely much more complicated >> than >> > our require() system (which hasn't required much maintenance at all, >> and is >> > really a glorified concatenating of files). >> > >> > I do see this as a net win though. Removing the runtime script >> injection as >> > well as being able to do tree-shaking is what makes the added complexity >> > worth it in my eyes. >> > >> >> Its also a popular project with many eyeballs, tests, familiarity, and >> interesting future features. >> >> Replacing custom internal solutions for appropriate public ones is a great >> idea. >> >> One question, though: did we explore Web Pack? ( >> https://github.com/webpack/docs/wiki/webpack-for-browserify-users) >> >> I have no personal preference (yet), I've just been hearing rumblings.. > > >> >> > >> > >> > >> > > > >> > > > Please share everything you have on your mind if you read this far >> and >> > > > thank you. >> > > > >> > > > [1] https://github.com/substack/node-browserify >> > > > >> > > >> > >> > >
