On Jul 2, 2014, at 8:40 PM, Randall Leeds <[email protected]> wrote:
>
> Again, I did not try either Browserling or BrowserStack any further once it
> appeared they both do not have the ability to report detailed failures.
> Unless I'm missing something huge here, I don't really see the advantage of
> running tests in the cloud if there's not clear, publishable reporting
> results. Having to manually copy and paste that information seems like a big
> waste of time, when Sauce does this by default - we have a *lot* of browsers
> to test here.
>
>
> I don't know what you mean by detailed failures.
> But it seems I can't even find the reference I remembered and maybe
> BrowserStack isn't free for OSS.
By "detailed failures" I mean that SauceLabs keeps track of the errors that (in
our case) Mocha reports back, so you can easily browse the results of the test
on the web. The others do not do this, they just fail silently. I asked the
Browserling guys about it months ago and they said it wasn't something they
were going to implement.
You can still get the output from the runner on your local machine and copy and
paste it somewhere, but that would require lots of extra work to automate into
something usable - you'd need a place to store the results and some mechanism
for updating them and lots of other hurdles would need to be leaped.
> So far, the only tool that I've found that "works" 100%, end-to-end, that
> fulfills the criteria above is the grunt+saucelabs tool I mentioned a few
> messages back - everything else either just doesn't work, or doesn't do
> enough.
>
> At this point, whether we use Gulp or Make or plain Javascript or even a Bash
> script, we're in the same boat - there's no tool that's working and does all
> of this out of the box, and much more work will need to be done to proceed -
> or we wait until those issues are resolved upstream. The only reason I
> brought Grunt up in the first place is that there *is* a tool that does
> everything we want here, with no issues, ready to go right now.
>
> Anyway, we've wasted about two weeks already on just this step, trying
> different options. At this point, I'm inclined to go back to the option that
> I know works (Grunt + Sauce) to at least get the results of the tests, to
> make some forward progress on this, rather than diving into any more
> rabbitholes.
>
>
> Let's go with Sauce then.
> But I don't want to switch to Grunt.
>
> Writing a little runner isn't hard. Here's an example:
> https://github.com/pouchdb/pouchdb/blob/master/bin/test-browser.js
Thanks - I'll dive into that one today.
Cheers,
-Bill
Bill Hunt
Senior Developer
OpenGov Foundation
http://opengovfoundation.org/
_______________________________________________
annotator-dev mailing list
[email protected]
https://lists.okfn.org/mailman/listinfo/annotator-dev
Unsubscribe: https://lists.okfn.org/mailman/options/annotator-dev
