On a related note, I have been looking at using cordova-coho and cordova-cli as the primary components of the build process. That way, the whole tool-chain gets tested too. The down-side is that if you break the tools, it all stops pretty fast....
On Thu, Jul 25, 2013 at 2:19 PM, Filip Maj <[email protected]> wrote: > Nice! > > Definitely want to see medic get more love and ideally get it into apache > so contribution is easier (it's existing on my github at the moment) > > Totally agree the data structure needs to be updated to account for > changed in architecture. That said, with a document store backing it, we > don't have to put too much thought into a schema. One thing we know for > sure: it will probably change in the future anyways :) > > In no particular order, things that need to be done: > > - handling plugins post-breakout > - using cordova-cli as a npm module. There is a TON of duplicated code in > there that pretty much exactly mirrors cordova-cli. This will need some > investigation as medic does multi-device deployments and synchronizes > results across multiple deploys, which cordova-cli does not provide. > Perhaps there is some overlap here and perhaps some of that functionality > can trickle back to cordova-ci. > - integrating with our (other) couchdb store for the plugin registry. > Figuring out replication. > - ability to distribute medic as another node in a larger network. > Implementing the vision of a distributed testing lab. > > On 7/25/13 11:07 AM, "Andrew Grieve" <[email protected]> wrote: > > >+dev > > > > > >On Thu, Jul 25, 2013 at 2:03 PM, David Kemp <[email protected]> wrote: > > > >> Hi all, > >> > >> In the spirit of trying to share the goods, I would like to see if we > >>can > >> co-ordinate some of the efforts, and more importantly some of the data. > >> With the changes in 3.0, I don't think the data structure used > >>previously > >> is sufficient. > >> > >> I would like to see results that: > >> * for mobilespec: include the platform, platform version, js version, > >> plugin list (with versions) > >> * include results for cordova-cli > >> * include results for cordova-coho > >> * include results for cordova-js tests > >> * include results for other tools that are testable > >> > >> The current medic structure tags tests with <platform><sha>. > >> > >> A possible solution is to add an additional dataset that ties a Test_ID > >>to > >> a test description. The test description could be a a structured object > >> indicating the list of <component><sha> that was in the test. If we > >>expand > >> the <platform> to mean <component>, then the test results get saved > >>with a > >> tag of <component><test_id>. > >> > >> So you would get a test like: > >> test_id=ANDROID_1234 > >> results="270 passed, 0 failed" > >> components=" > >> cordova-android: 980c4699b0993b1718c2f0a1761f7d3cc49a0c9b > >> cordova-cli: 1fe77b0be1997c42da31606cdbca9eb9e90a954e > >> cordova-js: d716e31b39bd39a06f1db80040e2c8a6a78a34e8 > >> cordova-mobile-spec: f5c0a54d056c6e9b76f59eac2b2908ddbfed3e65 > >> cordova-plugin-battery-status: f52824aba6aa0f470d2ee5b3d53466fd8668ac56 > >> cordova-plugin-camera: 5d747c739ddd41b731f77fb531481a709ea497cd > >> cordova-plugin-console: 3cb1165a3f95e86045e9800ba3e8b4ffb0ef5a91 > >> cordova-plugin-contacts: 5f0847d4a13a90df5b37bdcbabae6936610a5345 > >> cordova-plugin-device: 3c5832d2cca354bb1309eef738a848e2d4b87e3e > >> cordova-plugin-device-motion: 43e653378bb3bc98ffd39c1b8d0e2a0a3d412c6c > >> cordova-plugin-device-orientation: > >>a522c97336d859a57c89d11a2dde7cb9a5771b15 > >> cordova-plugin-dialogs: 96c52b4d8411f7745fec5e154ad75474a4fc4d07 > >> cordova-plugin-file: 698d7f6d2d2f2e9364451d5c0b4c763eae97c192 > >> cordova-plugin-file-transfer: e37895a56e314832d8480dfde221a1f72d915140 > >> cordova-plugin-geolocation: d8f8aa5beef0b16d656662d59b31a0bee1b69b0d > >> cordova-plugin-globalization: 8caedd2c6674e485dd46941f3f0106f7b17f1bbf > >> cordova-plugin-inappbrowser: dd7b20543f20660b4be71721e404e4ba7219aa7c > >> cordova-plugin-media: 9538b8c10d01fac785b3764985e7d836046802fb > >> cordova-plugin-media-capture: ba76f1121b34376fe11c58bb30ebe8ab8481281a > >> cordova-plugin-network-information: > >> e45286df35487a4414756d8fb93597ea454cb1ff > >> cordova-plugin-splashscreen: 33f353d2e9684e189f8b7d59c066253f2fe73f48 > >> cordova-plugin-vibration: 5c08769e83a09e900fe301f140bbe143390e8eb0 " > >> > >> Better ideas anyone? > >> > >> > >> > >> > >> On Fri, Jul 19, 2013 at 2:38 PM, David Kemp <[email protected]> wrote: > >> > >>> Hi Mike, > >>> So, we are on much the same road. I also had medic working briefly > >>> pre-3.0. > >>> When the 3.0 structure changed, I put it aside for a while since it was > >>> clearly going to take a bit of effort to change medic to run with 3.0. > >>> > >>> Today, while the dust settles a bit I have been looking at the idea of > >>> using buildbot for the primary tooling, and using coho and cordova-cli > >>>for > >>> a bunch of the build steps. > >>> > >>> My goal was to include coho and cli in the test chain so we can detect > >>> failures there too. > >>> > >>> I also looked at the issue of tracking versions and had not yet decided > >>> on any good way to describe a test set in a similar way to the simple > >>> <platform><sha>. In the case of 3.X, it seems like it should be: > >>> <platform><sha><mobile-spec><sha>[..<plugin><sha>..] > >>> > >>> David > >>> > >>> > >>> > >>> > >>> On Fri, Jul 19, 2013 at 2:01 PM, Mike Billau > >>><[email protected]>wrote: > >>> > >>>> Hi, > >>>> > >>>> Initially I was just working on understanding what medic was doing and > >>>> how. It's a pretty neat piece of code, Fil did a great job. I am new > >>>>to > >>>> node so this step has taken a bit longer than expected. > >>>> > >>>> You'll notice that the code is pretty old, so at first I was just > >>>> updating it to work with the 2.X stream - doing things like removing > >>>>the > >>>> -VERSION suffix from cordova.js file, figuring out how to write my > >>>>couchdb > >>>> URL into the whitelist, etc. I was able to get medic to work for a few > >>>> hours with three different Android devices, it was cool. > >>>> > >>>> Things are a bit different now though with the 3.X stream and having > >>>> everything split apart. The last few days I have been working on > >>>> integrating the CLI. On one hand, using the CLI seems to simplify a > >>>>lot of > >>>> the code, especially with the mobile spec create script that Andrew > >>>>checked > >>>> in yesterday (haven't integrated this yet but seems simple enough.) > >>>>On the > >>>> other hand, with everything split out I am not sure if it is going to > >>>>be > >>>> possible to pull down the code from a specific SHA and test it out - > >>>>at > >>>> least not until the medic tests themselves get added into their > >>>>specific > >>>> repos. (How would you know which version of mobile spec matches the > >>>>current > >>>> SHA? By timestamp?) I don't really think this is an issue though > >>>>because > >>>> it is a CI...if we just pull the HEAD of every repo when a new commit > >>>>comes > >>>> in and run the test, then it should be fine. (If there is a need to > >>>>test > >>>> old SHA, one could build that specific version of mobile spec, run > >>>>through > >>>> the automated tests, and hit a "Report to DB" button or something.) > >>>> > >>>> My current goals are to 1. integrate Andrew's script so I can have a > >>>> single node command instead of 15 nested .exec callbacks, 2. Figure > >>>>out why > >>>> the gitpubsub always gives me JSON errors when a new commit goes in, > >>>>3. Get > >>>> it running and reporting data to the couchdb, 4. Clean up the code / > >>>>remove > >>>> the specific SHA part 5. Check in with Ryan Willoughby who has been > >>>>working > >>>> on integrating PhoneGapBuild and see if he has done any 3.0 work 6. > >>>>Poke > >>>> the apache infra guys to see if they have set up a couch db machine > >>>>for us > >>>> > >>>> I have also been researching types of USB hubs to use - the Adobe > >>>>guys a > >>>> while back told me about their huge 42 port hub. So far I have been > >>>>testing > >>>> the 4 port powered Pluggable devices and they daisy chain well and > >>>>can do > >>>> data+power with all of our devices save Samsung tablets and some > >>>>iDevices. > >>>> I have been compiling a table internally of the device/hub > >>>>compatibility > >>>> that I'd like to share/collaborate on. > >>>> > >>>> Hopefully we can collaborate and get this thing running. A few days > >>>>ago > >>>> we talked about how it'd be cool to add a "-ci" parameter to the CLI > >>>>so > >>>> that other offices can easily get medic up and running with whatever > >>>> devices they have. I think this would be a pretty useful thing to add. > >>>> > >>>> Hope ya'ller having fun at the conference! > >>>> > >>>> Mike > >>>> > >>>> > >
