On Wed, Jul 2, 2014 at 1:27 PM, Bill Hunt <[email protected]> wrote: > Ok, checking in again on the testing front progress... > > To re-summarize the situation, here's the criteria what I'm trying to > accomplish for this first phase: > > * Run the existing test suite on a cloud service in multiple browsers. > * Have the failure data automatically reported back in a team-viewable (or > ideally publicly-viewable) way. > * Be able to automate this testing via CI of some sort. > * Use existing tools as much as possible. > > > On Jun 23, 2014, at 2:52 PM, Randall Leeds <[email protected]> wrote: > > On Jun 23, 2014 11:29 AM, "Bill Hunt" <[email protected]> wrote: > > > > I spent the morning working with BrowserStack, and it doesn't appear to > be quite as mature as SauceLabs. > > > > You can definitely still run tests in a cloud-based browser as all of > the other services offer, but like Browserling, the failures are not logged > and reported in the dashboard. Or, at least, it doesn't appear to do this > by default and they don't really have very deep documentation on how to do > this. It still looks like those errors can probably be reported back to > the script running them, but I haven't had time to dig in very deep on that > and again, it doesn't look as easy as what SauceLabs already has in place. > If anyone on the list has other experience they can contribute, please > chime in! > > > > Did you try it with yeti? A brief look at that example seems like that's > the thing for running mocha tests with browserstack > > > On Jun 24, 2014, at 4:41 AM, Nick Stenning <[email protected]> wrote: > > If I had to express a preference on strategy I'd be inclined to dig into > the issues with Browserling and get in touch with James Halliday to get > any problems fixed. He's 'substack' in the relevant IRC channels and on > Twitter. > > > > Again, I did not try either Browserling or BrowserStack any further once > it appeared they both do not have the ability to report detailed failures. > Unless I'm missing something huge here, I don't really see the advantage > of running tests in the cloud if there's not clear, publishable reporting > results. Having to manually copy and paste that information seems like a > big waste of time, when Sauce does this by default - we have a *lot* of > browsers to test here. > > I don't know what you mean by detailed failures. But it seems I can't even find the reference I remembered and maybe BrowserStack isn't free for OSS.
> > > On Jun 23, 2014, at 2:52 PM, Randall Leeds <[email protected]> wrote: > > I just found this: > https://github.com/mantoni/mochify.js/blob/master/README.md > > I spent a few days trying to get this to work, but it appears that it's > just a wrapper for browserify, mocaccino, and min-webdriver - but, it > doesn't allow support for coffeescript from what I can see - there's simply > no configuration options to drop in "-t coffeeify" like you'd do with > browserify alone. I've been chatting with the developer to add support for > these things, but that's going very slowly. The relevant ticket is here: > https://github.com/mantoni/mochify.js/issues/17 > > I've separately gotten the individual pieces to work ok with browserify > and mocaccino, but I still don't have the piece to interact with webdriver > to send the whole stack to saucelabs. > > Sorry if that was a dead end. > > On Jun 23, 2014, at 2:52 PM, Randall Leeds <[email protected]> wrote: > > It's rather the point of gulp to be more a set of simple tools to > construct the process pipeline rather than a build system. One doesn't > publish gulp plugins as much because there isn't as much boilerplate to > introduce custom build steps as grunt. > > At least, that's my understanding of it. > > But I'm not sure any of this should be necessary. > > On Jun 24, 2014, at 4:41 AM, Nick Stenning <[email protected]> wrote: > > But as for having to rewrite the build to use Grunt to get this working, > no thank you. I'm perfectly happy to see the build process change, but > Grunt is a bad reimplementation of half of GNU Make, so I'd rather stick > with the original. > > > > So far, the only tool that I've found that "works" 100%, end-to-end, that > fulfills the criteria above is the grunt+saucelabs tool I mentioned a few > messages back - everything else either just doesn't work, or doesn't do > enough. > > At this point, whether we use Gulp or Make or plain Javascript or even a > Bash script, we're in the same boat - there's no tool that's working and > does all of this out of the box, and much more work will need to be done to > proceed - or we wait until those issues are resolved upstream. The only > reason I brought Grunt up in the first place is that there *is* a tool that > does everything we want here, with no issues, ready to go right now. > > Anyway, we've wasted about two weeks already on just this step, trying > different options. At this point, I'm inclined to go back to the option > that I know works (Grunt + Sauce) to at least get the results of the tests, > to make some forward progress on this, rather than diving into any more > rabbitholes. > > Let's go with Sauce then. But I don't want to switch to Grunt. Writing a little runner isn't hard. Here's an example: https://github.com/pouchdb/pouchdb/blob/master/bin/test-browser.js
_______________________________________________ annotator-dev mailing list [email protected] https://lists.okfn.org/mailman/listinfo/annotator-dev Unsubscribe: https://lists.okfn.org/mailman/options/annotator-dev
