Some progress on the unit tests: I got the Core tests to run on the swf side when doing a full build. In order to do that I needed to move the running of the tests until after Basic finished building so the test app could be built.
We’re going to need to do something similar with the Maven build. The Testing project is compiling and is usable (on a basic level) too. Tests should run with both a SWF and JS target. The SWF target uses FlexUnit, while the JS target uses Josh’s implementation. I also migrated the Core tests to use the new project and that is building as well. I still need to add the bits that Greg wrote and wire up some JS tests. Hopefully I’ll make some more progress tomorrow. Harbs > On Nov 5, 2017, at 8:17 AM, Alex Harui <aha...@adobe.com.INVALID> wrote: > > Those warnings do not seem to affect the success of the tests on SWF for > me. > > IMO, the big challenge is in capturing test results so it can be reported > in a CI dashboard. Having a GUI display results in Flash or in a browser > is a great first step, but I believe going from there to getting Selenium > to report individual test failures may be trickier. > > Peter tried to write up Mustella a long time ago. It might need updating. > See: > https://cwiki.apache.org/confluence/display/FLEX/Mustella+Overview > > The advantage of declarative testing languages is that you get to > constrain the kinds of things that a test can do, and abstract when the > test harness does it. It is much harder to interrupt sequential lines of > ActionScript/JavaScript. > > HTH, > -Alex > > On 11/4/17, 3:13 PM, "Harbs" <harbs.li...@gmail.com> wrote: > >> It looks like FlexUnit is currently broken. >> >> I’m getting the following output when trying to run the FlexUnit tests on >> Core and Basic: >> >> [mxmlc] >> /Users/harbs/Documents/ApacheRoyale/flex-flexunit/FlexUnit4/target/flexuni >> t-4.3.0-20140410-as3_4.12.0.swc Warning: The definition >> mx.utils.ObjectUtil depended on by org.flexunit.runner.Description in the >> SWC >> /Users/harbs/Documents/ApacheRoyale/flex-flexunit/FlexUnit4/target/flexuni >> t-4.3.0-20140410-as3_4.12.0.swc could not be found >> [mxmlc] >> [mxmlc] >> [mxmlc] >> /Users/harbs/Documents/ApacheRoyale/flex-flexunit/FlexUnit4/target/flexuni >> t-4.3.0-20140410-as3_4.12.0.swc Warning: The definition >> mx.utils.StringUtil depended on by flexunit.framework.AsyncTestHelper in >> the SWC >> /Users/harbs/Documents/ApacheRoyale/flex-flexunit/FlexUnit4/target/flexuni >> t-4.3.0-20140410-as3_4.12.0.swc could not be found >> >> Has anyone looked at this yet? >> >> Harbs >> >>> On Nov 4, 2017, at 10:45 PM, Harbs <harbs.li...@gmail.com> wrote: >>> >>> Thanks! That looks very useful. >>> >>> I started work on a feature/testing branch. I’ll try to merge what you >>> did into what Josh did. I’m going to try to get the existing Core and/or >>> Basic tests working in both swf and node js output tomorrow. We’ll see >>> how well I do… ;-) >>> >>> We might need different configs for different testing environments, but >>> I’ll see if we can keep the divergence of the environs is minimal as >>> possible. >>> >>> Once I get the basics worked out, I’ll likely have an idea what others >>> can help with. Let’s see if we can whip the Royale tests into shape. :-) >>> >>> Harbs >>> >>>> On Nov 4, 2017, at 10:30 PM, Greg Dove <greg.d...@gmail.com> wrote: >>>> >>>> Harbs, I don't know if it still works since changes to Royale, but I >>>> had >>>> something rudimentary for cross-target unit testing quite shortly after >>>> working on the reflection support last year, in fact that was my >>>> primary >>>> reason for choosing to focus on reflection at the time. >>>> >>>> It was visual output only (i.e. just to look at the test results >>>> output in >>>> the browser) and the goal was to get some compatibility with flexunit >>>> (so >>>> that the same swf-based flexunit tests that were running in the build >>>> -e..g >>>> BinaryData tests- could be run visually side by side between swf and >>>> js). >>>> I also added a new TestVariance metadata to account for known >>>> (expected?) >>>> variation between swf and js. This was needed to cover (for example) >>>> things >>>> like testing Reflection classes themselves, where the results can be >>>> different between the targets based on the 'native' base classes (e..g >>>> EventDispatcher inheritance chain). >>>> >>>> Some of that code might also be useful to harvest for what you are >>>> doing, >>>> I'm not sure... >>>> >>>> It's here: >>>> >>>> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub. >>>> com%2Fapache%2Froyale-asjs%2Ftree%2Fdevelop%2Fmanualtests%2FUnitTests&da >>>> ta=02%7C01%7C%7C67a58a332b6942a2022108d523d15d35%7Cfa7b1b5a7b34438794aed >>>> 2c178decee1%7C0%7C0%7C636454304479378270&sdata=DX9gXiLqUoFQizp6xrakUwxjl >>>> cdyHMlZO5mOzDSiKDQ%3D&reserved=0 >>>> >>>> >>>> >>>> On Sun, Nov 5, 2017 at 7:30 AM, Harbs <harbs.li...@gmail.com> wrote: >>>> >>>>> Awesome! >>>>> >>>>> I assume that this means I can code the code to the Royale repo under >>>>> Apache 2. Right? >>>>> >>>>> Harbs >>>>> >>>>>> On Nov 3, 2017, at 5:37 PM, Josh Tynjala <joshtynj...@apache.org> >>>>>> wrote: >>>>>> >>>>>> Please feel free to use my test runner. I'm happy to commit it >>>>>> somewhere >>>>> to make it official Apache code, if necessary. >>>>>> >>>>>> - Josh >>>>>> >>>>>> On 2017-11-03 04:19, Harbs <harbs.li...@gmail.com> wrote: >>>>>>> One topic which keeps coming up is better test coverage for Royale. >>>>>>> >>>>>>> I think this is becoming a critical issue for a few reasons: >>>>>>> 1. As we get close to version 1.0 it’s necessary to have good test >>>>> coverage for confidence of quality and confidence that we don’t >>>>> introduce >>>>> recessive bugs. >>>>>>> 2. It’s really hard to accept Github pull requests without >>>>>>> examining >>>>> the code VERY well that it does not introduce recessive bugs. CI >>>>> which runs >>>>> automated tests could give a preliminary test on pull requests to >>>>> ensure >>>>> that they don’t break anything. If the pull requests do break >>>>> something, >>>>> it allows the requester to fix the problem with confidence without >>>>> taking >>>>> others’ time. >>>>>>> >>>>>>> I think we need to break up testing into pieces and figure out a >>>>> strategy to implement automated tests in a way that they are >>>>> maintainable. >>>>>>> >>>>>>> Some points: >>>>>>> 1. I think integration into something like Travis would be very >>>>>>> helpful. >>>>>>> 2. I think there’s a Jenkins plugin for building pull requests. >>>>>>> Not >>>>> sure how useful it is.[1] >>>>>>> 3. Josh has created a Node.js-compatible test-runner architecture >>>>>>> which >>>>> could be useful for unit tests on parts of the framework which >>>>> don’t rely >>>>> on browser features. (i.e. models and the like) He mentioned the >>>>> possibility of donating it, and I think that would be a very useful >>>>> feature. >>>>>>> 4. For UI and integration tests, there seem to be some pretty cool >>>>> integrations using Selenium.[2][3] >>>>>>> 5. I think the main testing effort needs to be using JS and >>>>>>> something >>>>> like Josh’s testing framework for non-UI pieces and some easy-to-use >>>>> Selenium integration for visual pieces and integration tests. >>>>>>> 6. We probably also want some API endpoints we can test off of for >>>>> integration testing. >>>>>>> >>>>>>> I’m willing to invest time into this. >>>>>>> >>>>>>> It’s going to be a lot of work building out the tests and I think >>>>>>> we >>>>> need a plan for that. My thoughts: >>>>>>> >>>>>>> 1. Step one is to make it easy to write meaningful automated tests >>>>>>> and >>>>> establish a clear pattern. >>>>>>> 2. Step two is to start writing tests starting from the >>>>> most-used/easiest to beak pieces and work out from there. >>>>>>> 3. Once the pattern is established, any new pieces MUST have testing >>>>> coverage. >>>>>>> 4. When fixing bugs, attention should be paid to adding testing for >>>>> that component. >>>>>>> 5. When a pull request comes in on a piece which does not have unit >>>>> test, a test must be written before accepting the pull request. The >>>>> test >>>>> does not need to be written by the requester. Before examining the >>>>> request, >>>>> the test should be written to pass for expected behavior and fail for >>>>> the >>>>> bug that the pull request is attempting to fix (assuming the pull >>>>> request >>>>> is to fix a bug). >>>>>>> >>>>>>> Thoughts? >>>>>>> Harbs >>>>>>> >>>>>>> P.S. I’m thinking of coming to the US in late December/early >>>>>>> January. >>>>> I would be interested in getting together for a hacking session with >>>>> folks >>>>> who are available. >>>>>>> >>>>>>> >>>>>>> [1]https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fw >>>>>>> iki.jenkins.io%2Fdisplay%2FJENKINS%2FGitHub%2Bpull&data=02%7C01%7C%7C >>>>>>> 67a58a332b6942a2022108d523d15d35%7Cfa7b1b5a7b34438794aed2c178decee1%7 >>>>>>> C0%7C0%7C636454304479378270&sdata=cnyBtnVycpg3Aa7Hfel3dkAlez2m7M0uSl3 >>>>>>> f0ezbSZY%3D&reserved=0+ >>>>> request+builder+plugin >>>>> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki. >>>>> jenkins.io%2F&data=02%7C01%7C%7C67a58a332b6942a2022108d523d15d35%7Cfa7b >>>>> 1b5a7b34438794aed2c178decee1%7C0%7C0%7C636454304479378270&sdata=eqe6ec% >>>>> 2F%2FIatSxdUuHU0Lo8o3WxySBkdSvUNa6i9NQ%2FI%3D&reserved=0 >>>>> display/JENKINS/GitHub+pull+request+builder+plugin> >>>>>>> >>>>>>> [2]https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fd >>>>>>> ocs.travis-ci.com%2Fuser%2Fgui-and-headless-browsers%2F&data=02%7C01% >>>>>>> 7C%7C67a58a332b6942a2022108d523d15d35%7Cfa7b1b5a7b34438794aed2c178dec >>>>>>> ee1%7C0%7C0%7C636454304479378270&sdata=zK%2FdOBmWJUsF7SylIQtMQQpAeOhw >>>>>>> 03sjOytutB4rGCY%3D&reserved=0 < >>>>> >>>>> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.t >>>>> ravis-ci.com%2Fuser%2Fgui-and-headless-browsers%2F&data=02%7C01%7C%7C67 >>>>> a58a332b6942a2022108d523d15d35%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7 >>>>> C0%7C636454304479378270&sdata=zK%2FdOBmWJUsF7SylIQtMQQpAeOhw03sjOytutB4 >>>>> rGCY%3D&reserved=0> >>>>>>> >>>>>>> [3]https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fs >>>>>>> aucelabs.com%2Fproducts%2Fintegrations&data=02%7C01%7C%7C67a58a332b69 >>>>>>> 42a2022108d523d15d35%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636 >>>>>>> 454304479378270&sdata=kwfto9xiP0ygeBrEqGz9ucdC%2FNXSRVOQho9qUW1mqdw%3 >>>>>>> D&reserved=0 >>>>>>> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsau >>>>>>> celabs.com%2F&data=02%7C01%7C%7C67a58a332b6942a2022108d523d15d35%7Cfa >>>>>>> 7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636454304479378270&sdata=xpG >>>>>>> G85mRtcFwwG7nbRfT7oLMPm4VPeHxe4trkG1wrAw%3D&reserved=0 >>>>> products/integrations> >>>>> >>>>> >>> >> >
