On 2/26/07, Paul McNett <[EMAIL PROTECTED]> wrote: > Nate Lowrie wrote: > > On 2/26/07, Dabo <[EMAIL PROTECTED]> wrote: > >> #4: Improve unit testing coverage > >> ----------------------+----------------------------------------------------- > >> Reporter: paul | Owner: paul > >> Type: task | Status: new > >> Priority: critical | Milestone: 1.0 > >> Component: app | Version: > >> Keywords: | > >> ----------------------+----------------------------------------------------- > >> We are on the right track with unit testing, but we desperately need to > >> improve coverage. Most (all?) properties of dCursorMixin and dBizobj are > >> currently being tested in some way, but hardly any methods are. Start by > >> improving the coverage here, and then move to the ui layer. > > Thanks for replying. I had no idea that the dabo-dev list was getting > these notifications, although I was indeed trying to make that happen. > > > > Notes on unit testing...... > > > > 1) I think that unit tests are absolutely vital to the continuing > > success of Dabo and that I am surprised that the code has a high level > > of cohesion with a relatively low number of defects without it. We > > need to bring all tiers under the unit test. > > Yep. > > > > 2) I like the simplicity that the no API of py.test brings, but I > > think that in the case of Dabo this is not good. For such a large > > structure as Dabo I see the need for at least some form of API in the > > test suite. I like that Paul went with PyUnit for the test library. > > It has a nice structured API and the formatting is easy. On another > > note, I have an absolute disdain for the doctest suite and would like > > to avoid its use if at all possible. > > I don't really like doctest that much, either. > > > > 3) I do think that we need to completely separate the unit test > > structure from the Dabo directory (and move it into the trunk root > > under the folder test). This way the test suite will be separate from > > the framework and things will be clearer. Also, since the tests > > already import Dabo there will be no changes that need to be made to > > the code. > > Ok, that's fine by me. I've seen it done both ways and don't really have > an opinion on the matter. > > > > 4) I don't know if you have played with CruiseControl or AntHill, but > > they are wonderful programs for use in test-driven development. > > Basically, we use them at work for Java and C# projects. They will > > check out the latest code from an SVN repository and run the unit test > > suite at pre-determined time intervals specified by the user. If any > > of the tests fail, it will automatically email the entire development > > team with the test log. I would like to see something similar for > > dabo. I don't think that there is a program like this but it should > > be a simple script to write and it could have a user interface in dabo > > even. Regardless, this Continuous Integration is necessary for a > > test-driven environment which is what we are striving for. > > I don't know that we are striving for test-driven development, but I'm > attempting to get at least some of it going. The unit tests and > continuous integration are just so valuable that we can't not do them. > > > > 5) For the unit test on the UI layer, I suggest we use Python Mock or > > Pmock to create the mock object we need for solid unit tests. I don't > > know of any other way to decouple some of the classes. > > Yes, for testing our code, but we also need to know how wx processes > some of the interactions, so I don't think we can get away from running > the tests against wx, at least where it is possible (no modal user input).
I was thinking more for decoupling the various UI classes from each other rather than decoupling from wx. Since the wx used is not a data layer, I would agree that it is not possible to get away from testing wx. Also, do you have a mock database connection for the Biz Object tests? > > > > 6) A suggestion that I would like to make is that we set up unit tests > > for the various libraries that we subclass. I know that unit tests > > for dabo will be enough work, but look at it this way. Suppose we had > > a set of unit tests for wxPython when it was in 2.6 and then 2.8 came > > out. We would have had a much easier time finding the API > > discrepancies and fixing the code since we could have run the unit > > tests and figured out which ones had failed. By doing this we > > basically create a safety net to ensure that new releases don't > > introduce new bugs in the system through code changes in the > > subclassed libraries. > > Oh boy, let's do it one step at a time. That will have to wait, unless > you have a big itch to scratch on that. > I know small steps. Just saying that it's something that is good to have. > > > Sorry I am chiming in so late on this. I have been busy with > > finishing a thesis and planning a wedding. I am a big advocate of > > test-driven development cycle, especially with a library that is big > > like Dabo. After I free up a lot of time starting the end of March, > > this is one of the top priorities I will have. Paul, I would like to > > work closely with you on this. I have a feeling that we might have to > > refactor some code to define the unit tests properly. But, I do > > believe that once this is implemented, bugs will plummet, code is > > easier to debug, and we can refactor code and add new code with > > certainty that the old code is most likely not broken. > > Nate, please feel free to do whatever you want with reorganizing and > adding to the unit testing. I'll do my part and add tests when bugs > surface and adding new features, but I'm not sure I have time to get > tests written for all the prior work. Also, I'm pretty green at writing > unit tests, so don't worry about stepping on my feet if you think some > tests can be written better. I don't have too much more experience than you, so we'll be doing this together. > > > Note that this does not eliminate the need for acceptance testing, > > user testing, and integration testing. We will still have to do these > > but there should be less defects found as a result of them. > > Absolutely agree. I think Ed and I are just naturally good at "testing" > our changes, which means lots of manual testing of common and edge cases > using the UI, which is why we haven't gotten in way over our heads yet. > But we can't catch everything, and bugs have slipped through on plenty > of occasions where, had the proper unit tests been in place, we would > have caught 'em early on. > > > My $.02, sorry for the long email. > > Don't be sorry, but put some of that energy into writing tests! :) > > -- > pkm ~ http://paulmcnett.com > > > _______________________________________________ > Post Messages to: [email protected] > Subscription Maintenance: http://leafe.com/mailman/listinfo/dabo-dev > _______________________________________________ Post Messages to: [email protected] Subscription Maintenance: http://leafe.com/mailman/listinfo/dabo-dev
