Hi Jeff, You are suggesting that if someone has the horsepower available (i.e., access to an IaaS cloud), the mustella tests should run, in their entirety in 10 mins or less - versus - anyone being able to do this, correct? I think I'm reading more into your 10-minutes-or-less suggestion than intended.
I do think the tests can be broken down into better units as they are all kind of lumped together, so that's a particularly good goal. --peter On 8/14/12 12:01 PM, "Jeff Conrad" <jeff...@gmail.com> wrote: >Hi, > >I'd like to help the project get to a point to where we can run the entire >test suite for the sdk in 10 minutes or less. I think that's a worthy >goal, and I'm willing to help make that a reality. > >If we get the testing time down to being that fast, that will mean we can >review a lot more contributions, and it could also mean that potential >contributors can submit patches that they know don't fail any tests before >submitting the patch. It also means that if one of those tests fail, the >person writing the code still has the context of what they did fresh in >their mind and can fix it quickly. > >For reference, I ran the entire Mustella suite last night, and it took 4 >hours, 3 minutes, and 50 seconds to run ./mini_run.sh -createImages -all >last night on a quad-core machine running Windows 7 using the Git Bash >(yay! no cygwin!). To make it work with the Git Bash, I changed the shell >variable in mustella/build.xml to just sh.exe. As a plus, shellrunner.sh >somewhat intelligently parallel-ized the compilation of all the test swfs >so it was compiling 4 swfs at a time, and then the ant script ran all the >test swfs one at a time. I know mustella is more a functional / >integration test suite, so by definition it's going to run slower than a >suite of unit tests. > >I also know that at least a few people on this list want to refactor the >sdk so it's unit-testable. I definitely support this and would like to >help in any way I can on that front. > >I want the entire suite: unit, integration, and functional to be able to >run in 10 minutes or less. I have two ideas as to how we can make this >happen, and I'm open to more. > >One idea would be to intelligently look at the files that a given patch / >changeset affects and only build and run the tests that test that >functionality. > >The other idea that comes to mind is spin up a group of virtual servers in >an IaaS cloud and split up the workload. If we can evenly split this up, >and my original number is right, it means 24+ servers. There are easy >ways >to get the files across that many servers quickly, so the file transfer >part wouldn't be that hard. > >What do you guys think? Where would you start? > >Jeff
