Thanks for the clarification Alexander! I think I got confused between the PR and periodic builds. I see in the periodic builds, indeed the job is cloning down the relevant platform and using the latest HEAD from master. In the PR builds, the logs are sparser and don't explicitly log out versions of dependent libraries consumed, so I will assume you are right!
I have one more question: how does everyone feel about cordova-qa's GitHub comments with test results (passing example: [1], failing example: [2])? What about integrating these kinds of checks into the GitHub UI for per-commit checks (this is the green checkmark you'll see in GH UI)? I'm probably missing a reason why it is integrated in this way, so forgive me if it's a naive question :) It looks like Travis test runs are integrated using the GitHub pass/fail integrations / UI. I'm wondering why not the Jenkins tests? I believe we can set up the integration such that each discrete check (i.e. a specific plugin/platform combination) can set up as a separate check, posting a separate result. Might be less confusing / less "made in house" than a custom QA bot? Just a thought. Currently, the 'fail' comment from cordova-qa requires contributors to click through all the various links to figure out which particular platform errored out. By splitting the checks out, it would make that more apparent more immediately. I think the UI as viewed from GitHub would be cleaner: from the pull req list it would be instantly visible which PRs are passing and which are failing, and the merge instructions could be simplified to "if the green check mark shows up, merge away", instead of right now needing to wait for both the green checkmark representing Travis + cordova-qa bot comment. It's a UI nitpick but I think it might improve the contribution experience. Feel free to shoot me down or direct me to more pressing issues :D [1] https://github.com/apache/cordova-plugin-device/pull/58#issuecomment-263793786 [2] https://github.com/apache/cordova-plugin-camera/pull/131#issuecomment-263013516 On Tue, Nov 29, 2016 at 12:28 AM, <alsoro...@apache.org> wrote: > Hi again Filip, you are most welcome! > >>> - cordova-paramedic configs are pulled from cordova-medic repo. >>> (?) requires an extra pull in CI. > Yeah. This is kind of rudimentary thing, I think we can safely transfer them > to cordova-paramedic repo. This will require some Jenkins jobs changes, I can > assist with that. > >> > - paramedic setup for individual plugins install latest HEAD of >> > master of platform code (at least, cordova-android + device plugin) > What makes you think so? I double-checked, our current CI setup for per-PR > jobs is using the released versions of all the platforms. Master versions are > used only for periodic build. > >> > - there are plugins tests that run via a jenkins instance on >> > cloudapp.net, and there are travis tests too. travis is pull-req >> > triggered, cloud app runs nightly. why? > Regarding cloudapp (Jenkins) builds: > The goal of nightly tests is to verify the master versions of plugins against > the master versions of platforms and CLI. We also run per-PR jobs against the > released versions of platform and CLI, you can find them here: > http://cordova-ci.cloudapp.net:8080/view/Pull%20requests/ > > Feel free to contact me via email or Slack if you have any questions > regarding our current CI setup or need an assistance. > > Thanks, > Alexander Sorokin > > -----Original Message----- > From: Jesse [mailto:purplecabb...@gmail.com] > Sent: Monday, November 28, 2016 11:00 PM > To: dev@cordova.apache.org > Subject: Re: Hello again! > > Welcome back! > > > @purplecabbage > risingj.com > > On Mon, Nov 28, 2016 at 11:50 AM, Simon MacDonald <simon.macdon...@gmail.com >> wrote: > >> Never heard of this guy. >> Simon Mac Donald >> https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsimonm >> acdonald.com&data=02%7C01%7Cv-alsoro%40microsoft.com%7Cdcdee276682a4ff >> 9353408d417c92e87%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6361596 >> 00215033744&sdata=zPpzT0F5p5rlW71wFWe0DtB2cG8rednMPQyiqtUw%2BZg%3D&res >> erved=0 >> >> >> On Mon, Nov 28, 2016 at 2:48 PM, Filip Maj <maj....@gmail.com> wrote: >> > Hi everyone! >> > >> > Just wanted to (re)introduce myself after a 3 year or so hiatus :) >> > >> > I used to be an active member of the group between 2011 and 2013 >> > when I was on the Adobe PhoneGap team. I took a 3 year detour >> > focusing on mobile testing infrastructure at Sauce Labs, but >> > recently rejoined the Adobe PhoneGap team. I have been lurking more >> > intently on this list for the past month or so and aim to be more involved >> > these days. >> > >> > I've been poking around and getting my bearings around the testing >> > suites, infrastructure and CI in Cordova the past week or so. I >> > think I will try to contribute in that area initially. In >> > particular, I am interested in enabling functional end-to-end >> > testing for all repos in cordova that could benefit from that sort >> > of testing, and seamlessly integrating running the tests and >> > reporting their results back into the standard Cordova dev workflow >> > (I assume that is focussed around GitHub?). I see there are >> > different kinds of test coverage and CI systems at play (cloudapp, >> > travis, appveyor, plus unit and functional tests), so initially just >> > wrapping my head around all that. >> > >> > If anyone here has suggestions on areas that need work, have >> > grievances around how they are frustrated by manually needing to >> > test something, or having any other helpful tips on what needs work >> > or what could be improved, feel free to reply to this thread! >> > >> > My generic notes on this topic so far, in case that is helpful: >> > >> > cordova testing overview >> > ——————————— >> > notes / weird things: >> > - cordova-paramedic configs are pulled from cordova-medic repo. (?) >> > requires an extra pull in CI. >> > - paramedic setup for individual plugins install latest HEAD of >> > master of platform code (at least, cordova-android + device plugin) >> > - there are plugins tests that run via a jenkins instance on >> > cloudapp.net, and there are travis tests too. travis is pull-req >> > triggered, cloud app runs nightly. why? >> > >> > road to testing utopia: >> > - how do platforms get tested? integration tests with what: tooling? >> plugins? >> > - unit tests run on travis/appveyor? >> > - understand what needs to be tested for a release. work >> > backwards to automate that from there. Steve sent some helpful links my >> > way: >> > - platform: >> > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit >> > hub.com%2Fapache%2Fcordova-coho%2Fblob%2Fmaster%2Fdocs%2F&data=02%7C >> > 01%7Cv-alsoro%40microsoft.com%7Cdcdee276682a4ff9353408d417c92e87%7C7 >> > 2f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636159600215033744&sdata=q >> > oBxneF1%2FwzJTPwFxp7CsvbdEUUq8u5ALYCqXbjyuzM%3D&reserved=0 >> platforms-release-process.md#testing >> > - plugins: >> > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit >> > hub.com%2Fapache%2Fcordova-coho%2Fblob%2Fmaster%2Fdocs%2F&data=02%7C >> > 01%7Cv-alsoro%40microsoft.com%7Cdcdee276682a4ff9353408d417c92e87%7C7 >> > 2f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636159600215033744&sdata=q >> > oBxneF1%2FwzJTPwFxp7CsvbdEUUq8u5ALYCqXbjyuzM%3D&reserved=0 >> plugins-release-process.md#test >> > - tools: >> > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit >> > hub.com%2Fapache%2Fcordova-coho%2Fblob%2Fmaster%2Fdocs%2F&data=02%7C >> > 01%7Cv-alsoro%40microsoft.com%7Cdcdee276682a4ff9353408d417c92e87%7C7 >> > 2f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636159600215033744&sdata=q >> > oBxneF1%2FwzJTPwFxp7CsvbdEUUq8u5ALYCqXbjyuzM%3D&reserved=0 >> tools-release-process.md#test >> > - how do plugins get tested? >> > - make sure dependencies / artifact versions are locked down. >> > - what is the difference between "local" vs appium tests >> > >> > Looking forward to collaborating with y'all in here once more :) >> > >> > Cheers, >> > Fil >> > >> > -------------------------------------------------------------------- >> > - To unsubscribe, e-mail: dev-unsubscr...@cordova.apache.org >> > For additional commands, e-mail: dev-h...@cordova.apache.org >> > >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@cordova.apache.org >> For additional commands, e-mail: dev-h...@cordova.apache.org >> >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@cordova.apache.org > For additional commands, e-mail: dev-h...@cordova.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@cordova.apache.org For additional commands, e-mail: dev-h...@cordova.apache.org
