On 4/29/2013 8:45 AM, Wayne Stambaugh wrote: > On 4/28/2013 8:15 PM, Dick Hollenbeck wrote: >> >> On Apr 28, 2013 10:54 AM, "Brian Sidebotham" <[email protected] >> <mailto:[email protected]>> wrote: >>> >>> Hi Guys, >>> >>> I'm just catching up with the list, and I saw something that caught my >> eye as it's something that's been on my mind for a while: >>> >>> -------------------------- >>> >>> Dick Hollenbeck wrote: >>> >>> - Right now, I am finding too many bugs in the software ... >>> >>> - We might do well as a team by slowing down and focusing >>> - on reliability and quality not features for awhile. Firstly, >>> - the bugs are damaging to the project. >>> >>> --------------------------- >>> >>> I agree with this, there are things I'd like to add to KiCad, but only >> on-top of something I can be confident I'm not breaking, especially by >> creating corner case issues. >>> >>> I would like us to think about regression testing using something like >> CTest (Which would make sense as we're currently using CMake anyway!). >> We could then publish dashboard regression testing results. >>> >>> I'm aware work is going into making eeschema and PCBNEW essentially >> into DLL's, so perhaps it's best to wait until that work is complete >> before starting down this road? >>> >>> In particular I'd like to see regression testing on the DRC, Gerber >> generation, and the Python exposed API. Probably in that order of >> priority too. Certainly the Python API changes are already tripping us >> up, but only when they have already been broken in committed code. >>> >>> Being able to regression test changes to optimisations and code >> tidying will help that move along anyway as you can be more confident in >> your changes having complete coverage once the number of tests increases. >>> >>> I am prepared to say that I'll undertake this work too. Obviously it >> can't start straight away as I'm currently doing work on the Windows >> scripting build system and python-a-mingw-us packaging. >>> >>> Is anyone against regression testing, or have alternatives that would >> achieve similar confidence in committed code? My vote is for regression >> testing. > > I think it's good idea as long as we think it through before we start > implementing them. I want avoid a free for all mentality and then have > to go back and clean up the mess. We should set down some preliminary > guidelines for testing along the lines of the coding policy before we > start actually writing test code. This way developers will no what is ^^know^^
Sorry, the caffeine hasn't kicked in yet. > expected. > >>> >> >> I fully support the idea. It will expand the size of the source tree >> significantly over time, and increase maintainence, but these costs are >> dwarfed by the benefits. >> >> Pyhon itself has quite a developed test harness environment with lots of >> tests. They were helpful in getting a-ming-us up to a certain >> confidence level. I did not need an understanding of the test harness >> environment to use it. >> >> The other thing I learned was that python can call arbitrary C functions >> in an arbitrary dll, even if they are not swigged. > > I've used the Python unit test framework for testing Python code. I > never thought about using it for testing C++ code but it does seem > feasible. Does anybody have any experience doing this or are there any > examples of anyone else doing this so we can get an idea of what is > involved? If the Python testing framework is not adequate, we can > always take a look at the Boost testing framework. I looked at this > several months ago and it looked like a good fit given that we already > have Boost as a dependency and it's feature parity seemed as good as any > of the other open source C/C++ testing frameworks and even many of the > commercial testing frameworks. > >> >>> We would need good support from all, new code requires new tests, and >> nobody is better suited to devising the tests than the person who >> specified the new functionality - which means all developers would >> probably end up writing test cases at some point. >>> >>> Best Regards, Brian. >>> >>> >>> _______________________________________________ >>> Mailing list: https://launchpad.net/~kicad-developers >>> Post to : [email protected] >> <mailto:[email protected]> >>> Unsubscribe : https://launchpad.net/~kicad-developers >>> More help : https://help.launchpad.net/ListHelp >>> >> >> >> >> _______________________________________________ >> Mailing list: https://launchpad.net/~kicad-developers >> Post to : [email protected] >> Unsubscribe : https://launchpad.net/~kicad-developers >> More help : https://help.launchpad.net/ListHelp >> > > > _______________________________________________ > Mailing list: https://launchpad.net/~kicad-developers > Post to : [email protected] > Unsubscribe : https://launchpad.net/~kicad-developers > More help : https://help.launchpad.net/ListHelp > _______________________________________________ Mailing list: https://launchpad.net/~kicad-developers Post to : [email protected] Unsubscribe : https://launchpad.net/~kicad-developers More help : https://help.launchpad.net/ListHelp
