I just converted geolocation to the new test style [1] I'm happy with the process overall, and I find the jasmine 2 tests are more succinct.
There are a few things worth noting: - I kept the eval code in. At google today, it was discussed that this may not be the best approach. - Jasmine 2: You must hit at least one expect statement or the test will timeout even though done was called. - I did not remove the manual test page [2]. It would be great if we had a convention for importing these. (ie test harness looks for a page at www/test/plugin-id.html and adds a link to it if it exists) [1] https://git-wip-us.apache.org/repos/asf?p=cordova-plugin-geolocation.git;a=commit;h=075850a460d8171a04809cf6317fb4c4ef998603 [2] https://git-wip-us.apache.org/repos/asf?p=cordova-plugin-geolocation.git;a=blob;f=test/manual.html;h=6ec2eed03e18c2efaa8710094d60930bb32227ba;hb=459a01c888801e8dfa2a688d25483bb48c46d8e2 On Tue, Oct 15, 2013 at 12:50 PM, David Kemp <drk...@chromium.org> wrote: > In spite of that fact that it needs a tooling change, I like the added > <mode> tag / prepare steps. > The tooling change should be small and it means no runtime impact on apps. > > I love the approach - a very positive step to cleaning up testing. > > David Kemp > > > > On Tue, Oct 15, 2013 at 12:18 PM, Michal Mocny <mmo...@chromium.org> > wrote: > > > Braden, you're right, good catch. > > > > Discussing locally how we could support "prepare --mode=..." in the most > > generalized form, we remembered an old suggestion to just support <mode> > > tags. > > > > The benefits seem to be: > > - No need to add custom tag-prefix/attributes for the combinations of > > js-module mode=, asset mode=, etc etc > > - More visually apparent when reading a plugin.xml file, akin to > > <platforms> tag > > > > The drawbacks seem to be: > > - Not all descendant tags are easy to support for a given mode (ie, > > <dependency>) > > > > > > Summarizing the options currently discussed in this thread: > > > > - new <js-test-module> tag. Not general enough solution to support tests > > bundling <assets>, so -1 from me for this reason. > > - mode="..." attribute for a set of whitelisted tags (<js-module>, > <asset>, > > ...) > > - <mode name="..."> tag for a set of whitelisted descendant > > tags (<js-module>, <asset>, ...) > > > > The last two options are really functionally equivalent, but given that > we > > opted for <platform> tag instead of attribute, I'm also favoring a <mode> > > tag. > > > > -Michal > > > > > > On Tue, Oct 15, 2013 at 11:43 AM, Braden Shepherdson < > bra...@chromium.org > > >wrote: > > > > > It's not true that adding these tests only creates larger binaries. > They > > > will be fetched and parsed by the plugin loader code at app startup > time. > > > It goes through the list of all plugins in cordova_plugins.js and loads > > > them all. That parses them, and runs the outermost layer, which is the > > > wrapping function function(require, module, exports) { ... }. So there > is > > > still runtime memory and CPU impact. > > > > > > I support <js-test-module> tags or <js-module ... test> or whatever. > > > Similarly, prepare for tests. I realize we want to avoid tooling > support, > > > but I think tooling support is a lesser evil than production > performance > > > impact. > > > > > > Overall approach makes me happy. > > > > > > Braden > > > > > > > > > On Fri, Oct 11, 2013 at 9:43 PM, Michal Mocny <mmo...@chromium.org> > > wrote: > > > > > >> On Fri, Oct 11, 2013 at 9:08 PM, Andrew Grieve <agri...@chromium.org> > > >> wrote: > > >> > > >> > The eval of the jasmine interface deserves mention. Is the > motivation > > >> > there that tests can choose to use another testing framework? That's > > why > > >> > you don't just make jasmine functions globals? > > >> > > > >> > > >> I was hoping the plugins would need to depend on anything but CDVTest, > > and > > >> not expect any globals. I guess, though, that CDVTest still expects > the > > >> app to provide to a test framework and some other stuff, so in the end > > its > > >> no different. I was hedging on being able to update CDVTest in the > > future > > >> for whatever we need, and all 3rdparty plugins would not need > updating. > > >> eval() could be used to do all sorts of clever things if we needed.. > > >> > > >> > > >> > > > >> > One nit pick just from reading your email is that this will cause > the > > >> test > > >> > js-modules to be injected into apps that use the plugins. I think > > >> probably > > >> > we will want to update the tools to recognize a <js-test-module>. We > > >> > *could* work around it by adding the tests to new plugins that > depend > > on > > >> > the thing they are testing, but I think changing the tools would be > > >> nicer. > > >> > > > >> > > >> I also mentioned splitting tests into second plugin but thats overkill > > >> except in extreme circumstances. Note that the tests aren't actually > > >> loaded unless you require them, so its just a matter of larger > binaries > > >> which could be filtered out manually. > > >> > > >> My person preference would be to support conditional build types, > which > > >> have come up before. ie, cordova prepare debug, cordova prepare > > release, > > >> cordova prepare test -- and plugin.xml could specify a different set > of > > >> js-module for either. > > >> > > >> A specific <js-test-module> would be fine, but isnt "0 new tooling". > > >> > > >> Also, I forgot to mention, but we do need to add support for getting > the > > >> full list of plugins installed, which should be trivial to add to > > >> modulemapper (maybe there is already a way to reach in, but I don't > > think > > >> we have a documented api for it). > > >> > > >> > > >> > Another nit is that it would be nice if the CordovaTests app didn't > > >> depend > > >> > on any plugins. e.g., don't have it depend on device plugin. > > >> > > > >> > > >> CordovaTests doesn't explicitly depend on device plugin, except that > at > > >> the > > >> moment I have it printing the same info that MobileSpec does at > startup. > > >> This could be wrapped in a try{}catch in case the require fails, but > > its > > >> good info to have in the common case. > > >> > > >> > > >> > > > >> > Overall, really like the approach! > > >> > > > >> > > >> Thanks! > > >> > > >> > > >> > > > >> > > > >> > On Fri, Oct 11, 2013 at 5:17 PM, Michal Mocny <mmo...@chromium.org> > > >> wrote: > > >> > > > >> >> TLDR; I've implemented a plugin testing strategy that requires 0 > new > > >> >> tooling features, can support non-core plugins, and hopefully even > > >> support > > >> >> a variety of methods for running/reporting the test results. Super > > >> alpha > > >> >> preview, but take a look if you like the direction! > > >> >> > > >> >> > > >> >> NEW: CDVTest Plugin: https://github.com/mmocny/CDVTest > > >> >> > > >> >> NEW: CordovaTest App: https://github.com/mmocny/CordovaTests > > >> >> > > >> >> UPDATED: Converted three existing plugins to this "new style". > Code > > >> is on > > >> >> feature branches of respective repos (cdvtest branch): > > >> >> org.apache.cordova.device > > >> >> org.apache.cordova.device-motion > > >> >> org.chromium.storage > > >> >> (must clone locally, switch to branch, and plugin add from local > > path) > > >> >> > > >> >> > > >> >> Now, any plugin that wants to join in on the fun needs to provide a > > >> >> <js-module > > >> >> src="..." name="test" /> and use this template: > > >> >> > > >> >> exports.init = function() { > > >> >> > > >> >> > > >> > > eval(require('org.apache.cordova.test.test').injectJasmineInterface(this, > > >> >> 'this')); > > >> >> > > >> >> // TESTS GO HERE > > >> >> describe(..., function() { > > >> >> it(...); > > >> >> }); > > >> >> }; > > >> >> (The eval is optional but super useful; it will inject the jasmine > > >> >> interface into your scope, so you don't have to type > > jasmine.describe, > > >> >> jasmine.it, etc. Not sure of any cleaner way to do this.) > > >> >> > > >> >> > > >> >> STEPS: > > >> >> 1. create a new cordova project > > >> >> 2. import the CordovaTest app into your www > > >> >> 3. add any or all of the above plugins > > >> >> 4. give it a run, and try out the auto tests (all pass on ipad, > some > > >> still > > >> >> fail on android) > > >> >> > > >> >> Lots of work left to do, but hopefully good enough to whet your > > >> appetite. > > >> >> > > >> >> One thing to note, I've attempted to make CDVTest and plugin tests > > >> unaware > > >> >> of the specific jasmine version the app is hosting (so that it can > be > > >> >> swapped without changing all plugins), but it must support the new > > >> style > > >> >> interface for async tests (ie, accept a done callback). This is > the > > >> style > > >> >> that node-jasmine uses, mocha uses, and jasmine-2.0 is going to use > > >> (I'm > > >> >> using jasmine 2.0 rc3). This means that core plugin tests require > > some > > >> >> code transformations, but the net effect is cleaner tests and more > > >> common > > >> >> style with our node tools' tests. > > >> >> > > >> >> Manual tests are not implemented yet. > > >> >> > > >> >> -Michal > > >> >> > > >> >> On Fri, Oct 11, 2013 at 12:54 PM, Michal Mocny < > mmo...@chromium.org> > > >> >> wrote: > > >> >> > > >> >> > I'm throwing something together right now, actually. I'll post > my > > >> >> current > > >> >> > progress today so you can take a look. > > >> >> > > > >> >> > > > >> >> > On Fri, Oct 11, 2013 at 12:41 PM, Brian LeRoux <b...@brian.io> > wrote: > > >> >> > > > >> >> >> Sorry keep meaning to respond. I like Michal's first step but > > >> growing > > >> >> to a > > >> >> >> full suite of tools. Are you currently tackling this Braden? I > > feel > > >> >> like > > >> >> >> it > > >> >> >> is related to the Medic stuff and maybe we should throw one of > our > > >> >> guys on > > >> >> >> the problem fully. > > >> >> >> > > >> >> >> > > >> >> >> On Sep 27, 2013 5:10 PM, "Braden Shepherdson" < > > bra...@chromium.org> > > >> >> >> wrote: > > >> >> >> > > >> >> >> > Which one? > > >> >> >> > > > >> >> >> > > > >> >> >> > On Fri, Sep 27, 2013 at 10:09 AM, Brian LeRoux <b...@brian.io> > > >> wrote: > > >> >> >> > > > >> >> >> > > I really like your proposal as a starting point. Very simple > > but > > >> >> would > > >> >> >> > > allow for in-app testing as well as on the cmd line if we so > > >> wish. > > >> >> >> > > > > >> >> >> > > > > >> >> >> > > On Fri, Sep 27, 2013 at 3:28 PM, Michal Mocny < > > >> mmo...@chromium.org > > >> >> > > > >> >> >> > wrote: > > >> >> >> > > > > >> >> >> > > > I was looking over some old emails from this list on > plugin > > >> >> testing, > > >> >> >> > and > > >> >> >> > > an > > >> >> >> > > > idea that was proposed way back was to ship plugin tests > as > > a > > >> >> second > > >> >> >> > > > plugin. That way, you can chose to install tests, or not, > > and > > >> >> know > > >> >> >> > > > explicitly if they are being copied into your final > project. > > >> >> >> > > > > > >> >> >> > > > An alternative would be to support build targets a la > > >> >> >> "release/debug" > > >> >> >> > and > > >> >> >> > > > have target-specific plugin.xml tags (assets, js-modules, > > >> >> >> > source-file..). > > >> >> >> > > > > > >> >> >> > > > -Michal > > >> >> >> > > > > > >> >> >> > > > > > >> >> >> > > > On Fri, Sep 27, 2013 at 4:52 AM, Brian LeRoux <b...@brian.io > > > > >> >> wrote: > > >> >> >> > > > > > >> >> >> > > > > I think this is basically what we've been proposing for > a > > >> while > > >> >> >> now. > > >> >> >> > > > > > > >> >> >> > > > > > > >> >> >> > > > > On Thu, Sep 26, 2013 at 8:29 PM, Michal Mocny < > > >> >> >> mmo...@chromium.org> > > >> >> >> > > > wrote: > > >> >> >> > > > > > > >> >> >> > > > > > I would suggest perhaps a simpler approach, which > > doesn't > > >> add > > >> >> >> > > anything > > >> >> >> > > > > new > > >> >> >> > > > > > to cordova-cli/plugman: > > >> >> >> > > > > > > > >> >> >> > > > > > - Each plugin ships with a "tests" js-module, and we > > >> >> document a > > >> >> >> > > > > convention > > >> >> >> > > > > > of where they should live, and what signature it > should > > >> have > > >> >> >> (i.e., > > >> >> >> > > > > > cordova.require('plugin.name.Tests').forEach(...) ). > > >> >> >> > > > > > - Will need a common way to describe/report results > > >> (others > > >> >> >> have > > >> >> >> > > > > > mentioned TAP). > > >> >> >> > > > > > - Any app is free to run those plugin tests in any > which > > >> way, > > >> >> >> but > > >> >> >> > we > > >> >> >> > > > > ship a > > >> >> >> > > > > > mobile-spec app which is one opinionated way to do so. > > >> >> >> > > > > > - It attempts to require the test module for each > > >> installed > > >> >> >> > plugin, > > >> >> >> > > > > runs > > >> >> >> > > > > > them, and aggregates results. > > >> >> >> > > > > > - It could report results to some shared server, > allow > > >> >> >> toggling > > >> >> >> > of > > >> >> >> > > > > tests, > > >> >> >> > > > > > etc, but no plugin should know or care about those > > >> features. > > >> >> >> > > > > > > > >> >> >> > > > > > Using that as a generic base: > > >> >> >> > > > > > > > >> >> >> > > > > > - We ship a "CDVTests" (or whatever) plugin which has > a > > >> >> bunch of > > >> >> >> > > > library > > >> >> >> > > > > > code for creating tests, and plugins can use it to > > >> register > > >> >> >> their > > >> >> >> > > > tests. > > >> >> >> > > > > > - This makes it easier to register manual tests in a > > >> common > > >> >> >> format > > >> >> >> > > for > > >> >> >> > > > > core > > >> >> >> > > > > > plugins, and prevents code duplication for core auto > > >> tests. > > >> >> >> > > > > > - External plugins can chose to use our testing > library, > > >> or > > >> >> not. > > >> >> >> > > > > > > > >> >> >> > > > > > -Michal > > >> >> >> > > > > > > > >> >> >> > > > > > > > >> >> >> > > > > > On Thu, Sep 26, 2013 at 10:34 AM, Braden Shepherdson < > > >> >> >> > > > > bra...@chromium.org > > >> >> >> > > > > > >wrote: > > >> >> >> > > > > > > > >> >> >> > > > > > > Here's an off-the-top-of-my-head sketch of how we > > might > > >> do > > >> >> >> > Voltron > > >> >> >> > > > > tests: > > >> >> >> > > > > > > > > >> >> >> > > > > > > - Add a tag to plugin.xml that names each test file: > > >> >> >> > > > > > > <test type="automatic" src="spec/foo.js" > name="Foo > > >> >> >> Automated" > > >> >> >> > > /> > > >> >> >> > > > > > > <test type="manual" src="spec/bar.js" name="Foo > > >> >> Manual" /> > > >> >> >> > > > > > > - Add a new command, cordova test (maybe > > prepare-test), > > >> >> that: > > >> >> >> > > > > > > - Ignores the top-level www. > > >> >> >> > > > > > > - Instead copies in a basic testing index.html > > >> similar > > >> >> to > > >> >> >> the > > >> >> >> > > > > current > > >> >> >> > > > > > > mobile-spec's > > >> >> >> > > > > > > - That index reads a file akin to > > cordova_plugins.js > > >> >> >> > > > > > (cordova_tests.js, > > >> >> >> > > > > > > maybe?) generated by the CLI, containing the info > from > > >> the > > >> >> >> <test> > > >> >> >> > > > tags. > > >> >> >> > > > > > > - It has navigation similar to the current > > >> mobile-spec, > > >> >> >> with > > >> >> >> > > > > buttons > > >> >> >> > > > > > > for the automatic and manual sections. Auto has > "All" > > >> and > > >> >> then > > >> >> >> > each > > >> >> >> > > > > > module, > > >> >> >> > > > > > > manual just has the list of modules. > > >> >> >> > > > > > > > > >> >> >> > > > > > > Thoughts? > > >> >> >> > > > > > > > > >> >> >> > > > > > > Braden > > >> >> >> > > > > > > > > >> >> >> > > > > > > > > >> >> >> > > > > > > On Thu, Sep 26, 2013 at 6:33 AM, Carlos Santana < > > >> >> >> > > > csantan...@gmail.com > > >> >> >> > > > > > > >wrote: > > >> >> >> > > > > > > > > >> >> >> > > > > > > > I like the idea can we call mobilespec now > > >> >> cordova-voltron > > >> >> >> and > > >> >> >> > be > > >> >> >> > > > DRY > > >> >> >> > > > > > and > > >> >> >> > > > > > > > use the tests form the plugins. > > >> >> >> > > > > > > > > > >> >> >> > > > > > > > Voltron by itself creates an App that tests only > > core, > > >> >> but > > >> >> >> as > > >> >> >> > you > > >> >> >> > > > > > > > use plugman to add plugins to voltron it has more > > test > > >> >> >> cases. > > >> >> >> > > > > > > > > > >> >> >> > > > > > > > It would not be a bad idea to enhance plugin.xml > in > > >> the > > >> >> >> future > > >> >> >> > to > > >> >> >> > > > > > include > > >> >> >> > > > > > > > information about testing (i.e. Directory > containing > > >> >> tests > > >> >> >> > files, > > >> >> >> > > > > test > > >> >> >> > > > > > > > command, etc..) > > >> >> >> > > > > > > > > > >> >> >> > > > > > > > --Carlos > > >> >> >> > > > > > > > > > >> >> >> > > > > > > > On Thursday, September 26, 2013, Anis KADRI wrote: > > >> >> >> > > > > > > > > > >> >> >> > > > > > > > > What's the challenge of having us use the tests > > that > > >> >> come > > >> >> >> > with > > >> >> >> > > > the > > >> >> >> > > > > > > > > individual plugins ? > > >> >> >> > > > > > > > > > > >> >> >> > > > > > > > > On Thu, Sep 26, 2013 at 8:13 AM, David Kemp < > > >> >> >> > drk...@google.com > > >> >> >> > > > > > > > <javascript:;>> > > >> >> >> > > > > > > > > wrote: > > >> >> >> > > > > > > > > > Currently, the automated test system that we > > have > > >> >> >> running > > >> >> >> > > > > (derived > > >> >> >> > > > > > > from > > >> >> >> > > > > > > > > > Medic) uses only the mobilespec tests. It does > > not > > >> >> yet > > >> >> >> use > > >> >> >> > > > tests > > >> >> >> > > > > > > > > collected > > >> >> >> > > > > > > > > > from the plugins. Its been talked about, but > not > > >> gone > > >> >> >> > > anywhere. > > >> >> >> > > > > > > > > > > > >> >> >> > > > > > > > > > David Kemp > > >> >> >> > > > > > > > > > > > >> >> >> > > > > > > > > > > > >> >> >> > > > > > > > > > On Wed, Sep 25, 2013 at 7:58 PM, Jesse < > > >> >> >> > > > purplecabb...@gmail.com > > >> >> >> > > > > > > > <javascript:;>> > > >> >> >> > > > > > > > > wrote: > > >> >> >> > > > > > > > > > > > >> >> >> > > > > > > > > >> Yeah, I have pushed some changes to > > mobile-spec, > > >> and > > >> >> >> when > > >> >> >> > I > > >> >> >> > > > did > > >> >> >> > > > > I > > >> >> >> > > > > > > also > > >> >> >> > > > > > > > > >> copied the tests into the plugin involved. > > >> >> >> > > > > > > > > >> Until we get the magic test runner > happening, I > > >> >> think > > >> >> >> we > > >> >> >> > > just > > >> >> >> > > > > keep > > >> >> >> > > > > > > > > >> duplicating. > > >> >> >> > > > > > > > > >> > > >> >> >> > > > > > > > > >> @purplecabbage > > >> >> >> > > > > > > > > >> risingj.com > > >> >> >> > > > > > > > > >> > > >> >> >> > > > > > > > > >> > > >> >> >> > > > > > > > > >> On Wed, Sep 25, 2013 at 4:38 PM, Steven Gill > < > > >> >> >> > > > > > > stevengil...@gmail.com > > >> >> >> > > > > > > > <javascript:;> > > >> >> >> > > > > > > > > > > > >> >> >> > > > > > > > > >> wrote: > > >> >> >> > > > > > > > > >> > > >> >> >> > > > > > > > > >> > We copied over tests into plugins when we > > first > > >> >> broke > > >> >> >> > them > > >> >> >> > > > > out, > > >> >> >> > > > > > > but > > >> >> >> > > > > > > > I > > >> >> >> > > > > > > > > >> don't > > >> >> >> > > > > > > > > >> > believe they have been updated. > > >> >> >> > > > > > > > > >> > > > >> >> >> > > > > > > > > >> > I would say for now to just add the tests > to > > >> >> mobile > > >> >> >> > spec, > > >> >> >> > > > and > > >> >> >> > > > > > > > > possibly in > > >> >> >> > > > > > > > > >> > the future we go all voltron to build > mobile > > >> spec > > >> >> and > > >> >> >> > keep > > >> >> >> > > > > tests > > >> >> >> > > > > > > > with > > >> >> >> > > > > > > > > >> their > > >> >> >> > > > > > > > > >> > corresponding plugins. > > >> >> >> > > > > > > > > >> > > > >> >> >> > > > > > > > > >> > > > >> >> >> > > > > > > > > >> > On Wed, Sep 25, 2013 at 4:22 PM, Joe > Bowser < > > >> >> >> > > > > bows...@gmail.com > > >> >> >> > > > > > > > <javascript:;>> > > >> >> >> > > > > > > > > wrote: > > >> >> >> > > > > > > > > >> > > > >> >> >> > > > > > > > > >> > > Hey > > >> >> >> > > > > > > > > >> > > > > >> >> >> > > > > > > > > >> > > Right now, I'm working on a weird file > > issue > > >> >> that > > >> >> >> > > requires > > >> >> >> > > > > me > > >> >> >> > > > > > to > > >> >> >> > > > > > > > > >> > > update mobile-spec, but I'm wondering > where > > >> the > > >> >> >> tests > > >> >> >> > > > should > > >> >> >> > > > > > > live. > > >> >> >> > > > > > > > > >> > > Should it all keep living in mobile-spec, > > or > > >> is > > >> >> it > > >> >> >> > with > > >> >> >> > > > the > > >> >> >> > > > > > > > plugins. > > >> >> >> > > > > > > > > >> > > And if it's with the plugins, will there > be > > >> >> >> scripts to > > >> >> >> > > > > > assemble > > >> >> >> > > > > > > > > >> > > mobile-spec all Voltron style? > > >> >> >> > > > > > > > > >> > > > > >> >> >> > > > > > > > > >> > > This came up earlier, but I haven't found > > any > > >> >> fix > > >> >> >> that > > >> >> >> > > > > needed > > >> >> >> > > > > > a > > >> >> >> > > > > > > > > >> > > mobile-spec test. (Many that need native > > >> >> testing, > > >> >> >> > like > > >> >> >> > > > > > > recursive > > >> >> >> > > > > > > > > file > > >> >> >> > > > > > > > > >> > > copy, etc). Any thoughts? > > >> >> >> > > > > > > > > >> > > > > >> >> >> > > > > > > > > >> > > Joe > > >> >> >> > > > > > > > > >> > > > > >> >> >> > > > > > > > > >> > > > >> >> >> > > > > > > > > >> > > >> >> >> > > > > > > > > > > >> >> >> > > > > > > > > > >> >> >> > > > > > > > > > >> >> >> > > > > > > > -- > > >> >> >> > > > > > > > Carlos Santana > > >> >> >> > > > > > > > <csantan...@gmail.com> > > >> >> >> > > > > > > > > > >> >> >> > > > > > > > > >> >> >> > > > > > > > >> >> >> > > > > > > >> >> >> > > > > > >> >> >> > > > > >> >> >> > > > >> >> >> > > >> >> > > > >> >> > > > >> >> > > >> > > > >> > > > >> > > > > > > > > >
