Agree to all of this. Currently, our CI system runs all plugin tests on every commit. Creating a test matrix like you mention below would be a good improvement.
Thanks, Nikhil -----Original Message----- From: Jesse [mailto:[email protected]] Sent: Monday, July 13, 2015 2:41 PM To: [email protected] Subject: Re: [CI] Medic Update This particular issue has been addressed. [1] Ideally a failed test does not break a build, I think we need to address this in the test framework. I will look into this. I see a couple problems with the current system: a. All plugins are tested together, meaning a commit to any one of our multitude of plugins can break everything. b. Dependencies are NOT tested. Again, We only test ALL the plugins together, so if pluginA requires pluginB but does not declare the dependency, we are already installing both, so we don't see an error. I think we need better granularity in our testing, so we can see where something is broken immediately. 1. A commit to any particular plugin repo should trigger a test of that plugin for each platform. 2. A commit to any particular platform should trigger the platforms tests, and queue all individual plugin tests for that platform. 3. A commit to lib/cli/plugman should trigger all the individual platform tests, and queue all individual plugin tests ... [1] https://git-wip-us.apache.org/repos/asf?p=cordova-plugin-media.git;a=commit;h=5f8738c5267be70562df9714e6f3971790b37d19 My team is hiring! @purplecabbage risingj.com On Mon, Jul 13, 2015 at 2:24 PM, Nikhil Khandelwal <[email protected]> wrote: > Looks like this commit [1] breaks mobilespec on all platforms. Test > failure log: > > > https://ci.apache.org/builders/cordova-android-osx/builds/31/steps/run > ning-tests/logs/stdio > > Logs here: > https://ci.apache.org/builders/cordova-android-osx/builds/31/steps/gat > hering-logs/logs/stdio > > W/PluginManager( 970): THREAD WARNING: exec() call to Media.release > blocked the main thread for 18ms. Plugin should use > CordovaInterface.getThreadPool(). > I/chromium( 970): [INFO:CONSOLE(115)] "Error in Success callbackId: > Media1993552855 : TypeError: Wrong type for parameter "src" of Media: > Expected String, but got Undefined.", source: > file:///android_asset/www/plugins/cordova-plugin-test-framework/www/ma > in.js > (115) > > Looks like we need to do: new Media("dummy") to prevent failure. We > really need CI on PR to validate changes like these. > > Thanks, > Nikhil > > [1] > https://github.com/apache/cordova-plugin-media/commit/072656de2a12e6e8 > deb94ce8ce3fad9eb9ce42a3 > > Thanks, > Nikhil > > > -----Original Message----- > From: Shazron [mailto:[email protected]] > Sent: Monday, July 13, 2015 1:28 PM > To: [email protected] > Subject: Re: [CI] Medic Update > > Since all the builds report the same error "didn't get test results: > timed out", Perhaps the CouchDB server is not running? > > CouchDB server: > http://cordova-vm.apache.org:5984 > > I verified that mobilespec for iOS builds and runs (with failing tests). > > > On Mon, Jul 13, 2015 at 1:01 AM, Dmitry Blotsky > <[email protected]> > wrote: > > > Hi folks, > > > > TL;DR: all builds are failing; please help investigate > > > > Last week we had some CI downtime due to the Apache buildbot master > > machine being rebuilt. Since then, we’ve had some successful runs > > and some hickups getting resolved. Current status is: > > - nightly builds now fire unconditionally at 00:01 GMT > > - changes are still not arriving from GitHub (tracked by INFRA-9979) > > - status reporting to GitHub is still in progress (tracked by > > INFRA-9811 and CB-9243) > > > > (Unfortunate) current status: > > - all builds are failing! > > > > Let’s get the build back up. Please run mobilespec on your own > > machines with the latest code of all plugins and help resolve what’s > > making the builds fail. > > > > Kindly, > > Dmitry > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] >
