BTW: Was there a good reason FlexUnit uses the spread operator for assert arguments instead of typed arguments?
> On Nov 5, 2017, at 11:36 PM, Harbs <[email protected]> wrote: > > Some progress on the unit tests: > > I got the Core tests to run on the swf side when doing a full build. In order > to do that I needed to move the running of the tests until after Basic > finished building so the test app could be built. > > We’re going to need to do something similar with the Maven build. > > The Testing project is compiling and is usable (on a basic level) too. Tests > should run with both a SWF and JS target. The SWF target uses FlexUnit, while > the JS target uses Josh’s implementation. > > I also migrated the Core tests to use the new project and that is building as > well. > > I still need to add the bits that Greg wrote and wire up some JS tests. > > Hopefully I’ll make some more progress tomorrow. > > Harbs > >> On Nov 5, 2017, at 8:17 AM, Alex Harui <[email protected]> wrote: >> >> Those warnings do not seem to affect the success of the tests on SWF for >> me. >> >> IMO, the big challenge is in capturing test results so it can be reported >> in a CI dashboard. Having a GUI display results in Flash or in a browser >> is a great first step, but I believe going from there to getting Selenium >> to report individual test failures may be trickier. >> >> Peter tried to write up Mustella a long time ago. It might need updating. >> See: >> https://cwiki.apache.org/confluence/display/FLEX/Mustella+Overview >> >> The advantage of declarative testing languages is that you get to >> constrain the kinds of things that a test can do, and abstract when the >> test harness does it. It is much harder to interrupt sequential lines of >> ActionScript/JavaScript. >> >> HTH, >> -Alex >> >> On 11/4/17, 3:13 PM, "Harbs" <[email protected]> wrote: >> >>> It looks like FlexUnit is currently broken. >>> >>> I’m getting the following output when trying to run the FlexUnit tests on >>> Core and Basic: >>> >>> [mxmlc] >>> /Users/harbs/Documents/ApacheRoyale/flex-flexunit/FlexUnit4/target/flexuni >>> t-4.3.0-20140410-as3_4.12.0.swc Warning: The definition >>> mx.utils.ObjectUtil depended on by org.flexunit.runner.Description in the >>> SWC >>> /Users/harbs/Documents/ApacheRoyale/flex-flexunit/FlexUnit4/target/flexuni >>> t-4.3.0-20140410-as3_4.12.0.swc could not be found >>> [mxmlc] >>> [mxmlc] >>> [mxmlc] >>> /Users/harbs/Documents/ApacheRoyale/flex-flexunit/FlexUnit4/target/flexuni >>> t-4.3.0-20140410-as3_4.12.0.swc Warning: The definition >>> mx.utils.StringUtil depended on by flexunit.framework.AsyncTestHelper in >>> the SWC >>> /Users/harbs/Documents/ApacheRoyale/flex-flexunit/FlexUnit4/target/flexuni >>> t-4.3.0-20140410-as3_4.12.0.swc could not be found >>> >>> Has anyone looked at this yet? >>> >>> Harbs >>> >>>> On Nov 4, 2017, at 10:45 PM, Harbs <[email protected]> wrote: >>>> >>>> Thanks! That looks very useful. >>>> >>>> I started work on a feature/testing branch. I’ll try to merge what you >>>> did into what Josh did. I’m going to try to get the existing Core and/or >>>> Basic tests working in both swf and node js output tomorrow. We’ll see >>>> how well I do… ;-) >>>> >>>> We might need different configs for different testing environments, but >>>> I’ll see if we can keep the divergence of the environs is minimal as >>>> possible. >>>> >>>> Once I get the basics worked out, I’ll likely have an idea what others >>>> can help with. Let’s see if we can whip the Royale tests into shape. :-) >>>> >>>> Harbs >>>> >>>>> On Nov 4, 2017, at 10:30 PM, Greg Dove <[email protected]> wrote: >>>>> >>>>> Harbs, I don't know if it still works since changes to Royale, but I >>>>> had >>>>> something rudimentary for cross-target unit testing quite shortly after >>>>> working on the reflection support last year, in fact that was my >>>>> primary >>>>> reason for choosing to focus on reflection at the time. >>>>> >>>>> It was visual output only (i.e. just to look at the test results >>>>> output in >>>>> the browser) and the goal was to get some compatibility with flexunit >>>>> (so >>>>> that the same swf-based flexunit tests that were running in the build >>>>> -e..g >>>>> BinaryData tests- could be run visually side by side between swf and >>>>> js). >>>>> I also added a new TestVariance metadata to account for known >>>>> (expected?) >>>>> variation between swf and js. This was needed to cover (for example) >>>>> things >>>>> like testing Reflection classes themselves, where the results can be >>>>> different between the targets based on the 'native' base classes (e..g >>>>> EventDispatcher inheritance chain). >>>>> >>>>> Some of that code might also be useful to harvest for what you are >>>>> doing, >>>>> I'm not sure... >>>>> >>>>> It's here: >>>>> >>>>> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub. >>>>> com%2Fapache%2Froyale-asjs%2Ftree%2Fdevelop%2Fmanualtests%2FUnitTests&da >>>>> ta=02%7C01%7C%7C67a58a332b6942a2022108d523d15d35%7Cfa7b1b5a7b34438794aed >>>>> 2c178decee1%7C0%7C0%7C636454304479378270&sdata=DX9gXiLqUoFQizp6xrakUwxjl >>>>> cdyHMlZO5mOzDSiKDQ%3D&reserved=0 >>>>> >>>>> >>>>> >>>>> On Sun, Nov 5, 2017 at 7:30 AM, Harbs <[email protected]> wrote: >>>>> >>>>>> Awesome! >>>>>> >>>>>> I assume that this means I can code the code to the Royale repo under >>>>>> Apache 2. Right? >>>>>> >>>>>> Harbs >>>>>> >>>>>>> On Nov 3, 2017, at 5:37 PM, Josh Tynjala <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>> Please feel free to use my test runner. I'm happy to commit it >>>>>>> somewhere >>>>>> to make it official Apache code, if necessary. >>>>>>> >>>>>>> - Josh >>>>>>> >>>>>>> On 2017-11-03 04:19, Harbs <[email protected]> wrote: >>>>>>>> One topic which keeps coming up is better test coverage for Royale. >>>>>>>> >>>>>>>> I think this is becoming a critical issue for a few reasons: >>>>>>>> 1. As we get close to version 1.0 it’s necessary to have good test >>>>>> coverage for confidence of quality and confidence that we don’t >>>>>> introduce >>>>>> recessive bugs. >>>>>>>> 2. It’s really hard to accept Github pull requests without >>>>>>>> examining >>>>>> the code VERY well that it does not introduce recessive bugs. CI >>>>>> which runs >>>>>> automated tests could give a preliminary test on pull requests to >>>>>> ensure >>>>>> that they don’t break anything. If the pull requests do break >>>>>> something, >>>>>> it allows the requester to fix the problem with confidence without >>>>>> taking >>>>>> others’ time. >>>>>>>> >>>>>>>> I think we need to break up testing into pieces and figure out a >>>>>> strategy to implement automated tests in a way that they are >>>>>> maintainable. >>>>>>>> >>>>>>>> Some points: >>>>>>>> 1. I think integration into something like Travis would be very >>>>>>>> helpful. >>>>>>>> 2. I think there’s a Jenkins plugin for building pull requests. >>>>>>>> Not >>>>>> sure how useful it is.[1] >>>>>>>> 3. Josh has created a Node.js-compatible test-runner architecture >>>>>>>> which >>>>>> could be useful for unit tests on parts of the framework which >>>>>> don’t rely >>>>>> on browser features. (i.e. models and the like) He mentioned the >>>>>> possibility of donating it, and I think that would be a very useful >>>>>> feature. >>>>>>>> 4. For UI and integration tests, there seem to be some pretty cool >>>>>> integrations using Selenium.[2][3] >>>>>>>> 5. I think the main testing effort needs to be using JS and >>>>>>>> something >>>>>> like Josh’s testing framework for non-UI pieces and some easy-to-use >>>>>> Selenium integration for visual pieces and integration tests. >>>>>>>> 6. We probably also want some API endpoints we can test off of for >>>>>> integration testing. >>>>>>>> >>>>>>>> I’m willing to invest time into this. >>>>>>>> >>>>>>>> It’s going to be a lot of work building out the tests and I think >>>>>>>> we >>>>>> need a plan for that. My thoughts: >>>>>>>> >>>>>>>> 1. Step one is to make it easy to write meaningful automated tests >>>>>>>> and >>>>>> establish a clear pattern. >>>>>>>> 2. Step two is to start writing tests starting from the >>>>>> most-used/easiest to beak pieces and work out from there. >>>>>>>> 3. Once the pattern is established, any new pieces MUST have testing >>>>>> coverage. >>>>>>>> 4. When fixing bugs, attention should be paid to adding testing for >>>>>> that component. >>>>>>>> 5. When a pull request comes in on a piece which does not have unit >>>>>> test, a test must be written before accepting the pull request. The >>>>>> test >>>>>> does not need to be written by the requester. Before examining the >>>>>> request, >>>>>> the test should be written to pass for expected behavior and fail for >>>>>> the >>>>>> bug that the pull request is attempting to fix (assuming the pull >>>>>> request >>>>>> is to fix a bug). >>>>>>>> >>>>>>>> Thoughts? >>>>>>>> Harbs >>>>>>>> >>>>>>>> P.S. I’m thinking of coming to the US in late December/early >>>>>>>> January. >>>>>> I would be interested in getting together for a hacking session with >>>>>> folks >>>>>> who are available. >>>>>>>> >>>>>>>> >>>>>>>> [1]https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fw >>>>>>>> iki.jenkins.io%2Fdisplay%2FJENKINS%2FGitHub%2Bpull&data=02%7C01%7C%7C >>>>>>>> 67a58a332b6942a2022108d523d15d35%7Cfa7b1b5a7b34438794aed2c178decee1%7 >>>>>>>> C0%7C0%7C636454304479378270&sdata=cnyBtnVycpg3Aa7Hfel3dkAlez2m7M0uSl3 >>>>>>>> f0ezbSZY%3D&reserved=0+ >>>>>> request+builder+plugin >>>>>> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki. >>>>>> jenkins.io%2F&data=02%7C01%7C%7C67a58a332b6942a2022108d523d15d35%7Cfa7b >>>>>> 1b5a7b34438794aed2c178decee1%7C0%7C0%7C636454304479378270&sdata=eqe6ec% >>>>>> 2F%2FIatSxdUuHU0Lo8o3WxySBkdSvUNa6i9NQ%2FI%3D&reserved=0 >>>>>> display/JENKINS/GitHub+pull+request+builder+plugin> >>>>>>>> >>>>>>>> [2]https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fd >>>>>>>> ocs.travis-ci.com%2Fuser%2Fgui-and-headless-browsers%2F&data=02%7C01% >>>>>>>> 7C%7C67a58a332b6942a2022108d523d15d35%7Cfa7b1b5a7b34438794aed2c178dec >>>>>>>> ee1%7C0%7C0%7C636454304479378270&sdata=zK%2FdOBmWJUsF7SylIQtMQQpAeOhw >>>>>>>> 03sjOytutB4rGCY%3D&reserved=0 < >>>>>> >>>>>> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.t >>>>>> ravis-ci.com%2Fuser%2Fgui-and-headless-browsers%2F&data=02%7C01%7C%7C67 >>>>>> a58a332b6942a2022108d523d15d35%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7 >>>>>> C0%7C636454304479378270&sdata=zK%2FdOBmWJUsF7SylIQtMQQpAeOhw03sjOytutB4 >>>>>> rGCY%3D&reserved=0> >>>>>>>> >>>>>>>> [3]https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fs >>>>>>>> aucelabs.com%2Fproducts%2Fintegrations&data=02%7C01%7C%7C67a58a332b69 >>>>>>>> 42a2022108d523d15d35%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636 >>>>>>>> 454304479378270&sdata=kwfto9xiP0ygeBrEqGz9ucdC%2FNXSRVOQho9qUW1mqdw%3 >>>>>>>> D&reserved=0 >>>>>>>> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsau >>>>>>>> celabs.com%2F&data=02%7C01%7C%7C67a58a332b6942a2022108d523d15d35%7Cfa >>>>>>>> 7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636454304479378270&sdata=xpG >>>>>>>> G85mRtcFwwG7nbRfT7oLMPm4VPeHxe4trkG1wrAw%3D&reserved=0 >>>>>> products/integrations> >>>>>> >>>>>> >>>> >>> >> >
