On 6/21/05, Mind Bridge <[EMAIL PROTECTED]> wrote: > Hi, > > I just want to mention that I do not like much the current unit testing > approach.
I'm reconciled that I'm going to write several line of test code for every line of production code. It encourages me to be organized when writing the tests (thus the number of newFoo() convienience methods). > > This is not related to the fact that I have actually spent more time to > create unit tests than to create the actual components -- I see this as > an investment in the future. > > My problem is more conceptual. If we separate the component behaviour > into a 'result' and an 'algorithm to achieve that result', I think that > the unit tests should perform an end-to-end testing and verify the > 'result', rather than emulate the 'algorithm' that achieves that result. > Please note that EasyMock does the latter. I think integration tests still have thier place, and I hope to resume work on and improved, productized approach (it wa the first thing I started in 3.1/4.0 but switched over to EasyMock). Integration testing is always going to be slower, especially because we now pay the HiveMind startup costs repeatedly. I run > 1200 unit tests, most with EasyMock, in about 30 seconds. Add in the 30-35 integration tests and we're up to five minutes. There's value in having a unit test suite you can run constantly. I run the "fast" suite, which omits the integration tests, many times for each change, and try to run the integration tests just once. > > In the specific case of web development, I believe the unit tests should > verify the output of the components, rather than the methods the > components invoke (or cause the invocation of). Note that the approach > to unit testing of the previous version of Tapestry was doing precisely > that (it is still possible, but it is not the focus). I think there is value here; I'm 90% confident in my code after the unit tests run, but would feel better with a quick, but comprehensive, integration suite ... that would be checking that the HTML and XML files are put together properly. However, wehn things go wrong in the integration tests, it can be very painful to intepret the results and backwards chain to what broke. This is much less the case in the unit tested code, especially the modern, less monolithic code, that was designed with easy of unit testing in mind. It is muchmore obvious to me that the code is changed and why because of the narrow focus. > > The reason for this thinking is that unit testing should provide an > alternative path to reaching the goal, rather than an emulation of the > existing path. Consider what happens when a modification in a component > is performed in order to make it more efficient (i.e. changing the > 'algorithm'). First, with the EasyMock approach pretty much all unit > tests containing that component will need to be modified, even though > the result is the same. Second, because of the work involved, the > developer will soon start to simply modify the unit tests to make them > work rather than think about whether what he is doing makes sense. Any > faults in the new algorithm will not be caught as a result. Proper testing is hard. But nightmare testing is a bad design smell ... one that says that the code needs refactoring. In some cases, I've taken a mixed approach. Some of the component unit tests use a real MarkupWriter that collects output in a buffer and the unit test code checks the content of that buffer at intervals. Some of that support has percolated up to BaseComponentTestCase. > > I think it is important for the unit tests to verify the algorithms > correctness by checking the results using a different method. Otherwise > a duplication of any faults is just too easy to happen. I think of unit testing as like a crossword puzzle. You can't be sure of 5 across until you've fit in solutions to 3 down and 7 down at the very least. Likewise, no single test is going to decisively prove anything, but when intersecting tests reinforce each other, you're on to something. > > Just a some of my thoughts.... Right now, I'm willing to take the trade off of speed and precision vs. ease of creating the unit tests. Even assuming I sped up the integration tests by 10x (1 second per test, and that might simply not be possible) and reduced the number of tests by 5x (say 250 tests), that would still take over four minutes to run the test suite. > > -mb > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > -- Howard M. Lewis Ship Independent J2EE / Open-Source Java Consultant Creator, Jakarta Tapestry Creator, Jakarta HiveMind Professional Tapestry training, mentoring, support and project work. http://howardlewisship.com --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
