Braden, you're right, good catch.
Discussing locally how we could support "prepare --mode=..." in the most
generalized form, we remembered an old suggestion to just support <mode>
tags.
The benefits seem to be:
- No need to add custom tag-prefix/attributes for the combinations of
js-module mode=, asset mode=, etc etc
- More visually apparent when reading a plugin.xml file, akin to
<platforms> tag
The drawbacks seem to be:
- Not all descendant tags are easy to support for a given mode (ie,
<dependency>)
Summarizing the options currently discussed in this thread:
- new <js-test-module> tag. Not general enough solution to support tests
bundling <assets>, so -1 from me for this reason.
- mode="..." attribute for a set of whitelisted tags (<js-module>, <asset>,
...)
- <mode name="..."> tag for a set of whitelisted descendant
tags (<js-module>, <asset>, ...)
The last two options are really functionally equivalent, but given that we
opted for <platform> tag instead of attribute, I'm also favoring a <mode>
tag.
-Michal
On Tue, Oct 15, 2013 at 11:43 AM, Braden Shepherdson <bra...@chromium.org>wrote:
> It's not true that adding these tests only creates larger binaries. They
> will be fetched and parsed by the plugin loader code at app startup time.
> It goes through the list of all plugins in cordova_plugins.js and loads
> them all. That parses them, and runs the outermost layer, which is the
> wrapping function function(require, module, exports) { ... }. So there is
> still runtime memory and CPU impact.
>
> I support <js-test-module> tags or <js-module ... test> or whatever.
> Similarly, prepare for tests. I realize we want to avoid tooling support,
> but I think tooling support is a lesser evil than production performance
> impact.
>
> Overall approach makes me happy.
>
> Braden
>
>
> On Fri, Oct 11, 2013 at 9:43 PM, Michal Mocny <mmo...@chromium.org> wrote:
>
>> On Fri, Oct 11, 2013 at 9:08 PM, Andrew Grieve <agri...@chromium.org>
>> wrote:
>>
>> > The eval of the jasmine interface deserves mention. Is the motivation
>> > there that tests can choose to use another testing framework? That's why
>> > you don't just make jasmine functions globals?
>> >
>>
>> I was hoping the plugins would need to depend on anything but CDVTest, and
>> not expect any globals. I guess, though, that CDVTest still expects the
>> app to provide to a test framework and some other stuff, so in the end its
>> no different. I was hedging on being able to update CDVTest in the future
>> for whatever we need, and all 3rdparty plugins would not need updating.
>> eval() could be used to do all sorts of clever things if we needed..
>>
>>
>> >
>> > One nit pick just from reading your email is that this will cause the
>> test
>> > js-modules to be injected into apps that use the plugins. I think
>> probably
>> > we will want to update the tools to recognize a <js-test-module>. We
>> > *could* work around it by adding the tests to new plugins that depend on
>> > the thing they are testing, but I think changing the tools would be
>> nicer.
>> >
>>
>> I also mentioned splitting tests into second plugin but thats overkill
>> except in extreme circumstances. Note that the tests aren't actually
>> loaded unless you require them, so its just a matter of larger binaries
>> which could be filtered out manually.
>>
>> My person preference would be to support conditional build types, which
>> have come up before. ie, cordova prepare debug, cordova prepare release,
>> cordova prepare test -- and plugin.xml could specify a different set of
>> js-module for either.
>>
>> A specific <js-test-module> would be fine, but isnt "0 new tooling".
>>
>> Also, I forgot to mention, but we do need to add support for getting the
>> full list of plugins installed, which should be trivial to add to
>> modulemapper (maybe there is already a way to reach in, but I don't think
>> we have a documented api for it).
>>
>>
>> > Another nit is that it would be nice if the CordovaTests app didn't
>> depend
>> > on any plugins. e.g., don't have it depend on device plugin.
>> >
>>
>> CordovaTests doesn't explicitly depend on device plugin, except that at
>> the
>> moment I have it printing the same info that MobileSpec does at startup.
>> This could be wrapped in a try{}catch in case the require fails, but its
>> good info to have in the common case.
>>
>>
>> >
>> > Overall, really like the approach!
>> >
>>
>> Thanks!
>>
>>
>> >
>> >
>> > On Fri, Oct 11, 2013 at 5:17 PM, Michal Mocny <mmo...@chromium.org>
>> wrote:
>> >
>> >> TLDR; I've implemented a plugin testing strategy that requires 0 new
>> >> tooling features, can support non-core plugins, and hopefully even
>> support
>> >> a variety of methods for running/reporting the test results. Super
>> alpha
>> >> preview, but take a look if you like the direction!
>> >>
>> >>
>> >> NEW: CDVTest Plugin: https://github.com/mmocny/CDVTest
>> >>
>> >> NEW: CordovaTest App: https://github.com/mmocny/CordovaTests
>> >>
>> >> UPDATED: Converted three existing plugins to this "new style". Code
>> is on
>> >> feature branches of respective repos (cdvtest branch):
>> >> org.apache.cordova.device
>> >> org.apache.cordova.device-motion
>> >> org.chromium.storage
>> >> (must clone locally, switch to branch, and plugin add from local path)
>> >>
>> >>
>> >> Now, any plugin that wants to join in on the fun needs to provide a
>> >> <js-module
>> >> src="..." name="test" /> and use this template:
>> >>
>> >> exports.init = function() {
>> >>
>> >>
>> eval(require('org.apache.cordova.test.test').injectJasmineInterface(this,
>> >> 'this'));
>> >>
>> >> // TESTS GO HERE
>> >> describe(..., function() {
>> >> it(...);
>> >> });
>> >> };
>> >> (The eval is optional but super useful; it will inject the jasmine
>> >> interface into your scope, so you don't have to type jasmine.describe,
>> >> jasmine.it, etc. Not sure of any cleaner way to do this.)
>> >>
>> >>
>> >> STEPS:
>> >> 1. create a new cordova project
>> >> 2. import the CordovaTest app into your www
>> >> 3. add any or all of the above plugins
>> >> 4. give it a run, and try out the auto tests (all pass on ipad, some
>> still
>> >> fail on android)
>> >>
>> >> Lots of work left to do, but hopefully good enough to whet your
>> appetite.
>> >>
>> >> One thing to note, I've attempted to make CDVTest and plugin tests
>> unaware
>> >> of the specific jasmine version the app is hosting (so that it can be
>> >> swapped without changing all plugins), but it must support the new
>> style
>> >> interface for async tests (ie, accept a done callback). This is the
>> style
>> >> that node-jasmine uses, mocha uses, and jasmine-2.0 is going to use
>> (I'm
>> >> using jasmine 2.0 rc3). This means that core plugin tests require some
>> >> code transformations, but the net effect is cleaner tests and more
>> common
>> >> style with our node tools' tests.
>> >>
>> >> Manual tests are not implemented yet.
>> >>
>> >> -Michal
>> >>
>> >> On Fri, Oct 11, 2013 at 12:54 PM, Michal Mocny <mmo...@chromium.org>
>> >> wrote:
>> >>
>> >> > I'm throwing something together right now, actually. I'll post my
>> >> current
>> >> > progress today so you can take a look.
>> >> >
>> >> >
>> >> > On Fri, Oct 11, 2013 at 12:41 PM, Brian LeRoux <b...@brian.io> wrote:
>> >> >
>> >> >> Sorry keep meaning to respond. I like Michal's first step but
>> growing
>> >> to a
>> >> >> full suite of tools. Are you currently tackling this Braden? I feel
>> >> like
>> >> >> it
>> >> >> is related to the Medic stuff and maybe we should throw one of our
>> >> guys on
>> >> >> the problem fully.
>> >> >>
>> >> >>
>> >> >> On Sep 27, 2013 5:10 PM, "Braden Shepherdson" <bra...@chromium.org>
>> >> >> wrote:
>> >> >>
>> >> >> > Which one?
>> >> >> >
>> >> >> >
>> >> >> > On Fri, Sep 27, 2013 at 10:09 AM, Brian LeRoux <b...@brian.io>
>> wrote:
>> >> >> >
>> >> >> > > I really like your proposal as a starting point. Very simple but
>> >> would
>> >> >> > > allow for in-app testing as well as on the cmd line if we so
>> wish.
>> >> >> > >
>> >> >> > >
>> >> >> > > On Fri, Sep 27, 2013 at 3:28 PM, Michal Mocny <
>> mmo...@chromium.org
>> >> >
>> >> >> > wrote:
>> >> >> > >
>> >> >> > > > I was looking over some old emails from this list on plugin
>> >> testing,
>> >> >> > and
>> >> >> > > an
>> >> >> > > > idea that was proposed way back was to ship plugin tests as a
>> >> second
>> >> >> > > > plugin. That way, you can chose to install tests, or not, and
>> >> know
>> >> >> > > > explicitly if they are being copied into your final project.
>> >> >> > > >
>> >> >> > > > An alternative would be to support build targets a la
>> >> >> "release/debug"
>> >> >> > and
>> >> >> > > > have target-specific plugin.xml tags (assets, js-modules,
>> >> >> > source-file..).
>> >> >> > > >
>> >> >> > > > -Michal
>> >> >> > > >
>> >> >> > > >
>> >> >> > > > On Fri, Sep 27, 2013 at 4:52 AM, Brian LeRoux <b...@brian.io>
>> >> wrote:
>> >> >> > > >
>> >> >> > > > > I think this is basically what we've been proposing for a
>> while
>> >> >> now.
>> >> >> > > > >
>> >> >> > > > >
>> >> >> > > > > On Thu, Sep 26, 2013 at 8:29 PM, Michal Mocny <
>> >> >> mmo...@chromium.org>
>> >> >> > > > wrote:
>> >> >> > > > >
>> >> >> > > > > > I would suggest perhaps a simpler approach, which doesn't
>> add
>> >> >> > > anything
>> >> >> > > > > new
>> >> >> > > > > > to cordova-cli/plugman:
>> >> >> > > > > >
>> >> >> > > > > > - Each plugin ships with a "tests" js-module, and we
>> >> document a
>> >> >> > > > > convention
>> >> >> > > > > > of where they should live, and what signature it should
>> have
>> >> >> (i.e.,
>> >> >> > > > > > cordova.require('plugin.name.Tests').forEach(...) ).
>> >> >> > > > > > - Will need a common way to describe/report results
>> (others
>> >> >> have
>> >> >> > > > > > mentioned TAP).
>> >> >> > > > > > - Any app is free to run those plugin tests in any which
>> way,
>> >> >> but
>> >> >> > we
>> >> >> > > > > ship a
>> >> >> > > > > > mobile-spec app which is one opinionated way to do so.
>> >> >> > > > > > - It attempts to require the test module for each
>> installed
>> >> >> > plugin,
>> >> >> > > > > runs
>> >> >> > > > > > them, and aggregates results.
>> >> >> > > > > > - It could report results to some shared server, allow
>> >> >> toggling
>> >> >> > of
>> >> >> > > > > tests,
>> >> >> > > > > > etc, but no plugin should know or care about those
>> features.
>> >> >> > > > > >
>> >> >> > > > > > Using that as a generic base:
>> >> >> > > > > >
>> >> >> > > > > > - We ship a "CDVTests" (or whatever) plugin which has a
>> >> bunch of
>> >> >> > > > library
>> >> >> > > > > > code for creating tests, and plugins can use it to
>> register
>> >> >> their
>> >> >> > > > tests.
>> >> >> > > > > > - This makes it easier to register manual tests in a
>> common
>> >> >> format
>> >> >> > > for
>> >> >> > > > > core
>> >> >> > > > > > plugins, and prevents code duplication for core auto
>> tests.
>> >> >> > > > > > - External plugins can chose to use our testing library,
>> or
>> >> not.
>> >> >> > > > > >
>> >> >> > > > > > -Michal
>> >> >> > > > > >
>> >> >> > > > > >
>> >> >> > > > > > On Thu, Sep 26, 2013 at 10:34 AM, Braden Shepherdson <
>> >> >> > > > > bra...@chromium.org
>> >> >> > > > > > >wrote:
>> >> >> > > > > >
>> >> >> > > > > > > Here's an off-the-top-of-my-head sketch of how we might
>> do
>> >> >> > Voltron
>> >> >> > > > > tests:
>> >> >> > > > > > >
>> >> >> > > > > > > - Add a tag to plugin.xml that names each test file:
>> >> >> > > > > > > <test type="automatic" src="spec/foo.js" name="Foo
>> >> >> Automated"
>> >> >> > > />
>> >> >> > > > > > > <test type="manual" src="spec/bar.js" name="Foo
>> >> Manual" />
>> >> >> > > > > > > - Add a new command, cordova test (maybe prepare-test),
>> >> that:
>> >> >> > > > > > > - Ignores the top-level www.
>> >> >> > > > > > > - Instead copies in a basic testing index.html
>> similar
>> >> to
>> >> >> the
>> >> >> > > > > current
>> >> >> > > > > > > mobile-spec's
>> >> >> > > > > > > - That index reads a file akin to cordova_plugins.js
>> >> >> > > > > > (cordova_tests.js,
>> >> >> > > > > > > maybe?) generated by the CLI, containing the info from
>> the
>> >> >> <test>
>> >> >> > > > tags.
>> >> >> > > > > > > - It has navigation similar to the current
>> mobile-spec,
>> >> >> with
>> >> >> > > > > buttons
>> >> >> > > > > > > for the automatic and manual sections. Auto has "All"
>> and
>> >> then
>> >> >> > each
>> >> >> > > > > > module,
>> >> >> > > > > > > manual just has the list of modules.
>> >> >> > > > > > >
>> >> >> > > > > > > Thoughts?
>> >> >> > > > > > >
>> >> >> > > > > > > Braden
>> >> >> > > > > > >
>> >> >> > > > > > >
>> >> >> > > > > > > On Thu, Sep 26, 2013 at 6:33 AM, Carlos Santana <
>> >> >> > > > csantan...@gmail.com
>> >> >> > > > > > > >wrote:
>> >> >> > > > > > >
>> >> >> > > > > > > > I like the idea can we call mobilespec now
>> >> cordova-voltron
>> >> >> and
>> >> >> > be
>> >> >> > > > DRY
>> >> >> > > > > > and
>> >> >> > > > > > > > use the tests form the plugins.
>> >> >> > > > > > > >
>> >> >> > > > > > > > Voltron by itself creates an App that tests only core,
>> >> but
>> >> >> as
>> >> >> > you
>> >> >> > > > > > > > use plugman to add plugins to voltron it has more test
>> >> >> cases.
>> >> >> > > > > > > >
>> >> >> > > > > > > > It would not be a bad idea to enhance plugin.xml in
>> the
>> >> >> future
>> >> >> > to
>> >> >> > > > > > include
>> >> >> > > > > > > > information about testing (i.e. Directory containing
>> >> tests
>> >> >> > files,
>> >> >> > > > > test
>> >> >> > > > > > > > command, etc..)
>> >> >> > > > > > > >
>> >> >> > > > > > > > --Carlos
>> >> >> > > > > > > >
>> >> >> > > > > > > > On Thursday, September 26, 2013, Anis KADRI wrote:
>> >> >> > > > > > > >
>> >> >> > > > > > > > > What's the challenge of having us use the tests that
>> >> come
>> >> >> > with
>> >> >> > > > the
>> >> >> > > > > > > > > individual plugins ?
>> >> >> > > > > > > > >
>> >> >> > > > > > > > > On Thu, Sep 26, 2013 at 8:13 AM, David Kemp <
>> >> >> > drk...@google.com
>> >> >> > > > > > > > <javascript:;>>
>> >> >> > > > > > > > > wrote:
>> >> >> > > > > > > > > > Currently, the automated test system that we have
>> >> >> running
>> >> >> > > > > (derived
>> >> >> > > > > > > from
>> >> >> > > > > > > > > > Medic) uses only the mobilespec tests. It does not
>> >> yet
>> >> >> use
>> >> >> > > > tests
>> >> >> > > > > > > > > collected
>> >> >> > > > > > > > > > from the plugins. Its been talked about, but not
>> gone
>> >> >> > > anywhere.
>> >> >> > > > > > > > > >
>> >> >> > > > > > > > > > David Kemp
>> >> >> > > > > > > > > >
>> >> >> > > > > > > > > >
>> >> >> > > > > > > > > > On Wed, Sep 25, 2013 at 7:58 PM, Jesse <
>> >> >> > > > purplecabb...@gmail.com
>> >> >> > > > > > > > <javascript:;>>
>> >> >> > > > > > > > > wrote:
>> >> >> > > > > > > > > >
>> >> >> > > > > > > > > >> Yeah, I have pushed some changes to mobile-spec,
>> and
>> >> >> when
>> >> >> > I
>> >> >> > > > did
>> >> >> > > > > I
>> >> >> > > > > > > also
>> >> >> > > > > > > > > >> copied the tests into the plugin involved.
>> >> >> > > > > > > > > >> Until we get the magic test runner happening, I
>> >> think
>> >> >> we
>> >> >> > > just
>> >> >> > > > > keep
>> >> >> > > > > > > > > >> duplicating.
>> >> >> > > > > > > > > >>
>> >> >> > > > > > > > > >> @purplecabbage
>> >> >> > > > > > > > > >> risingj.com
>> >> >> > > > > > > > > >>
>> >> >> > > > > > > > > >>
>> >> >> > > > > > > > > >> On Wed, Sep 25, 2013 at 4:38 PM, Steven Gill <
>> >> >> > > > > > > stevengil...@gmail.com
>> >> >> > > > > > > > <javascript:;>
>> >> >> > > > > > > > > >
>> >> >> > > > > > > > > >> wrote:
>> >> >> > > > > > > > > >>
>> >> >> > > > > > > > > >> > We copied over tests into plugins when we first
>> >> broke
>> >> >> > them
>> >> >> > > > > out,
>> >> >> > > > > > > but
>> >> >> > > > > > > > I
>> >> >> > > > > > > > > >> don't
>> >> >> > > > > > > > > >> > believe they have been updated.
>> >> >> > > > > > > > > >> >
>> >> >> > > > > > > > > >> > I would say for now to just add the tests to
>> >> mobile
>> >> >> > spec,
>> >> >> > > > and
>> >> >> > > > > > > > > possibly in
>> >> >> > > > > > > > > >> > the future we go all voltron to build mobile
>> spec
>> >> and
>> >> >> > keep
>> >> >> > > > > tests
>> >> >> > > > > > > > with
>> >> >> > > > > > > > > >> their
>> >> >> > > > > > > > > >> > corresponding plugins.
>> >> >> > > > > > > > > >> >
>> >> >> > > > > > > > > >> >
>> >> >> > > > > > > > > >> > On Wed, Sep 25, 2013 at 4:22 PM, Joe Bowser <
>> >> >> > > > > bows...@gmail.com
>> >> >> > > > > > > > <javascript:;>>
>> >> >> > > > > > > > > wrote:
>> >> >> > > > > > > > > >> >
>> >> >> > > > > > > > > >> > > Hey
>> >> >> > > > > > > > > >> > >
>> >> >> > > > > > > > > >> > > Right now, I'm working on a weird file issue
>> >> that
>> >> >> > > requires
>> >> >> > > > > me
>> >> >> > > > > > to
>> >> >> > > > > > > > > >> > > update mobile-spec, but I'm wondering where
>> the
>> >> >> tests
>> >> >> > > > should
>> >> >> > > > > > > live.
>> >> >> > > > > > > > > >> > > Should it all keep living in mobile-spec, or
>> is
>> >> it
>> >> >> > with
>> >> >> > > > the
>> >> >> > > > > > > > plugins.
>> >> >> > > > > > > > > >> > > And if it's with the plugins, will there be
>> >> >> scripts to
>> >> >> > > > > > assemble
>> >> >> > > > > > > > > >> > > mobile-spec all Voltron style?
>> >> >> > > > > > > > > >> > >
>> >> >> > > > > > > > > >> > > This came up earlier, but I haven't found any
>> >> fix
>> >> >> that
>> >> >> > > > > needed
>> >> >> > > > > > a
>> >> >> > > > > > > > > >> > > mobile-spec test. (Many that need native
>> >> testing,
>> >> >> > like
>> >> >> > > > > > > recursive
>> >> >> > > > > > > > > file
>> >> >> > > > > > > > > >> > > copy, etc). Any thoughts?
>> >> >> > > > > > > > > >> > >
>> >> >> > > > > > > > > >> > > Joe
>> >> >> > > > > > > > > >> > >
>> >> >> > > > > > > > > >> >
>> >> >> > > > > > > > > >>
>> >> >> > > > > > > > >
>> >> >> > > > > > > >
>> >> >> > > > > > > >
>> >> >> > > > > > > > --
>> >> >> > > > > > > > Carlos Santana
>> >> >> > > > > > > > <csantan...@gmail.com>
>> >> >> > > > > > > >
>> >> >> > > > > > >
>> >> >> > > > > >
>> >> >> > > > >
>> >> >> > > >
>> >> >> > >
>> >> >> >
>> >> >>
>> >> >
>> >> >
>> >>
>> >
>> >
>>
>
>
