Thanks! That looks very useful. I started work on a feature/testing branch. I’ll try to merge what you did into what Josh did. I’m going to try to get the existing Core and/or Basic tests working in both swf and node js output tomorrow. We’ll see how well I do… ;-)
We might need different configs for different testing environments, but I’ll see if we can keep the divergence of the environs is minimal as possible. Once I get the basics worked out, I’ll likely have an idea what others can help with. Let’s see if we can whip the Royale tests into shape. :-) Harbs > On Nov 4, 2017, at 10:30 PM, Greg Dove <[email protected]> wrote: > > Harbs, I don't know if it still works since changes to Royale, but I had > something rudimentary for cross-target unit testing quite shortly after > working on the reflection support last year, in fact that was my primary > reason for choosing to focus on reflection at the time. > > It was visual output only (i.e. just to look at the test results output in > the browser) and the goal was to get some compatibility with flexunit (so > that the same swf-based flexunit tests that were running in the build -e..g > BinaryData tests- could be run visually side by side between swf and js). > I also added a new TestVariance metadata to account for known (expected?) > variation between swf and js. This was needed to cover (for example) things > like testing Reflection classes themselves, where the results can be > different between the targets based on the 'native' base classes (e..g > EventDispatcher inheritance chain). > > Some of that code might also be useful to harvest for what you are doing, > I'm not sure... > > It's here: > https://github.com/apache/royale-asjs/tree/develop/manualtests/UnitTests > > > > On Sun, Nov 5, 2017 at 7:30 AM, Harbs <[email protected]> wrote: > >> Awesome! >> >> I assume that this means I can code the code to the Royale repo under >> Apache 2. Right? >> >> Harbs >> >>> On Nov 3, 2017, at 5:37 PM, Josh Tynjala <[email protected]> wrote: >>> >>> Please feel free to use my test runner. I'm happy to commit it somewhere >> to make it official Apache code, if necessary. >>> >>> - Josh >>> >>> On 2017-11-03 04:19, Harbs <[email protected]> wrote: >>>> One topic which keeps coming up is better test coverage for Royale. >>>> >>>> I think this is becoming a critical issue for a few reasons: >>>> 1. As we get close to version 1.0 it’s necessary to have good test >> coverage for confidence of quality and confidence that we don’t introduce >> recessive bugs. >>>> 2. It’s really hard to accept Github pull requests without examining >> the code VERY well that it does not introduce recessive bugs. CI which runs >> automated tests could give a preliminary test on pull requests to ensure >> that they don’t break anything. If the pull requests do break something, >> it allows the requester to fix the problem with confidence without taking >> others’ time. >>>> >>>> I think we need to break up testing into pieces and figure out a >> strategy to implement automated tests in a way that they are maintainable. >>>> >>>> Some points: >>>> 1. I think integration into something like Travis would be very helpful. >>>> 2. I think there’s a Jenkins plugin for building pull requests. Not >> sure how useful it is.[1] >>>> 3. Josh has created a Node.js-compatible test-runner architecture which >> could be useful for unit tests on parts of the framework which don’t rely >> on browser features. (i.e. models and the like) He mentioned the >> possibility of donating it, and I think that would be a very useful feature. >>>> 4. For UI and integration tests, there seem to be some pretty cool >> integrations using Selenium.[2][3] >>>> 5. I think the main testing effort needs to be using JS and something >> like Josh’s testing framework for non-UI pieces and some easy-to-use >> Selenium integration for visual pieces and integration tests. >>>> 6. We probably also want some API endpoints we can test off of for >> integration testing. >>>> >>>> I’m willing to invest time into this. >>>> >>>> It’s going to be a lot of work building out the tests and I think we >> need a plan for that. My thoughts: >>>> >>>> 1. Step one is to make it easy to write meaningful automated tests and >> establish a clear pattern. >>>> 2. Step two is to start writing tests starting from the >> most-used/easiest to beak pieces and work out from there. >>>> 3. Once the pattern is established, any new pieces MUST have testing >> coverage. >>>> 4. When fixing bugs, attention should be paid to adding testing for >> that component. >>>> 5. When a pull request comes in on a piece which does not have unit >> test, a test must be written before accepting the pull request. The test >> does not need to be written by the requester. Before examining the request, >> the test should be written to pass for expected behavior and fail for the >> bug that the pull request is attempting to fix (assuming the pull request >> is to fix a bug). >>>> >>>> Thoughts? >>>> Harbs >>>> >>>> P.S. I’m thinking of coming to the US in late December/early January. >> I would be interested in getting together for a hacking session with folks >> who are available. >>>> >>>> [1]https://wiki.jenkins.io/display/JENKINS/GitHub+pull+ >> request+builder+plugin <https://wiki.jenkins.io/ >> display/JENKINS/GitHub+pull+request+builder+plugin> >>>> [2]https://docs.travis-ci.com/user/gui-and-headless-browsers/ < >> https://docs.travis-ci.com/user/gui-and-headless-browsers/> >>>> [3]https://saucelabs.com/products/integrations <https://saucelabs.com/ >> products/integrations> >> >>
