Re: Code correctness, and testing strategies

2008-06-11 Thread David
Thanks again for an informative reply :-) I finished that small app I mentioned last time (before reading the the last reply to this thread). A few points (apologies for the length): I added a few integration tests, to test features which unit tests weren't appropriate for. The main thing they

Re: Code correctness, and testing strategies

2008-06-10 Thread Ben Finney
David [EMAIL PROTECTED] writes: On Sun, Jun 8, 2008 at 12:28 PM, Ben Finney [EMAIL PROTECTED] wrote: I did do it in a mostly top-down way, but didn't stop the BDD process to actually run the app :-) It sounds like what you are suggesting is something like this: 1) Following BDD, get a

Re: Code correctness, and testing strategies

2008-06-08 Thread David
Hi list. Do test-driven development or behaviour-driven development advocate how to do higher-level testing than unit testing? From reading up on the subject, I see that there are formally these types of testing: unit integration system acceptance I'm asking about this, because as suggested by

Re: Code correctness, and testing strategies

2008-06-08 Thread Ben Finney
David [EMAIL PROTECTED] writes: I'm asking about this, because as suggested by various posters, I have written my latest (small) app by following a Behaviour-Driven Development style. Congratulations on taking this step. That went well, and the code ended up much more modular than if I

Re: Code correctness, and testing strategies

2008-06-08 Thread David
Thanks for your informative reply. On Sun, Jun 8, 2008 at 12:28 PM, Ben Finney [EMAIL PROTECTED] wrote: David [EMAIL PROTECTED] writes: [...] My problem is that I haven't run the app once yet during development :-/ That might be an artifact of doing bottom-up implementation exclusively,

Re: Code correctness, and testing strategies

2008-06-03 Thread Paul Rubin
Duncan Booth [EMAIL PROTECTED] writes: I made the mistake at one point when I was trying to sell the concept of TDD telling the people I was trying to persuade that by writing the tests up front it influences the design of the code. I felt the room go cold: they said the customer has to

Re: Code correctness, and testing strategies

2008-06-03 Thread Duncan Booth
[EMAIL PROTECTED] (Jacob Hallen) wrote: The most important aspect of usnit testing is actually that it makes the code testable. This may sound lik an oxymoron but it is actually a really important property. Testable code has to have a level of modularity as well as simplicity and clarity in

Re: Code correctness, and testing strategies

2008-06-03 Thread Ben Finney
Duncan Booth [EMAIL PROTECTED] writes: I made the mistake at one point when I was trying to sell the concept of TDD telling the people I was trying to persuade that by writing the tests up front it influences the design of the code. I felt the room go cold: they said the customer has to sign

Re: Code correctness, and testing strategies

2008-06-03 Thread Nicola Musatti
On Jun 3, 12:35 am, [EMAIL PROTECTED] (Jacob Hallen) wrote: In article [EMAIL PROTECTED], David [EMAIL PROTECTED] wrote: [...] That's why you have human testing QA. Unit tests can help, but they are a poor substitute. If the customer is happy with the first version, you can improve it, fix

Re: Code correctness, and testing strategies

2008-06-02 Thread Jacob Hallen
In article [EMAIL PROTECTED], David [EMAIL PROTECTED] wrote: Clients generally require *working* software. Unfortunately it is all too easy to ship something broken because then you can claim you completed the coding on time and any slippage gets lost in the next 5 years of maintenance.

Re: Code correctness, and testing strategies

2008-05-25 Thread Matthew Woodcraft
Michael L Torrie [EMAIL PROTECTED] wrote: Watch your programmers then. They do have to write and debug the code. And they will spend at least as much or more time debugging as writing the code. It's a fact. I have several programmers working for me on several projects. What you have been

Re: Code correctness, and testing strategies

2008-05-25 Thread David
Hi again. Taking the advice of numerous posters, I've been studying BDD further. I spent a while looking for a Python library which implemented BDD in Python similar to jbehave, as described by Dan North on this page: http://dannorth.net/introducing-bdd. I did find a few, but they either had

Re: Code correctness, and testing strategies

2008-05-25 Thread Scott David Daniels
Ben Finney wrote: David [EMAIL PROTECTED] writes: You need to justify the extra time spent on writing test code. From the perspective of someone who once thought this way, and then dug in and did Behaviour Driven Development, I can tell you the time is entirely justified: by better design,

Re: Code correctness, and testing strategies

2008-05-25 Thread Aahz
In article [EMAIL PROTECTED], David [EMAIL PROTECTED] wrote: Seriously, 10 hours of testing for code developed in 10 hours? What kind of environment do you write code for? This may be practical for large companies with hordes of full-time testing QA staff, but not for small companies with just

Re: Code correctness, and testing strategies

2008-05-25 Thread David
You must have poor project management/tracking. You WILL pay the cost of testing, the only question is when. The when does have an impact on other aspects of the development process. Speaking as someone who started in my current job four years ago as the third developer in a five-person

Re: Code correctness, and testing strategies

2008-05-25 Thread Matthew Woodcraft
David [EMAIL PROTECTED] wrote: Might be a difference in project size/complexity then, rather than company size. Most of my works projects are fairly small (a few thousand lines each), very modular, and each is usually written and maintained by one developer. A lot of the programs will be

Re: Code correctness, and testing strategies

2008-05-25 Thread Paul Rubin
[EMAIL PROTECTED] (Aahz) writes: You must have poor project management/tracking. You WILL pay the cost of testing, the only question is when. The when does have an impact on other aspects of the development process. Well, let's say you used TDD and your program has 5000 tests. One might

Re: Code correctness, and testing strategies

2008-05-25 Thread Scott David Daniels
David wrote: It was an interesting exercise, and I'm encouraged to try it further. Yes, The developed code is missing an overview, but this is an example implementing something we all understand a stack. The layout is a little short on vertical whitespace, but I'm presuming you do that for

Code correctness, and testing strategies

2008-05-24 Thread David
Hi list. What strategies do you use to ensure correctness of new code? Specifically, if you've just written 100 new lines of Python code, then: 1) How do you test the new code? 2) How do you ensure that the code will work correctly in the future? Short version: For (1) I thoroughly (manually)

Re: Code correctness, and testing strategies

2008-05-24 Thread Roy Smith
In article [EMAIL PROTECTED], David [EMAIL PROTECTED] wrote: Hi list. What strategies do you use to ensure correctness of new code? Specifically, if you've just written 100 new lines of Python code, then: 1) How do you test the new code? 2) How do you ensure that the code will work

Re: Code correctness, and testing strategies

2008-05-24 Thread Ben Finney
David [EMAIL PROTECTED] writes: What strategies do you use to ensure correctness of new code? Behaviour Driven Development URL:http://behaviour-driven.org/. * Start with an automated unit test suite that passes. * Describe, as formally or informally as you like, but *not* in code, the new

Re: Code correctness, and testing strategies

2008-05-24 Thread Scott David Daniels
David wrote: Specifically, if you've just written 100 new lines of Python code, then: 1) How do you test the new code? 2) How do you ensure that the code will work correctly in the future? Short version: For (1) I thoroughly (manually) test code as I write it, before checking in to version

Re: Code correctness, and testing strategies

2008-05-24 Thread David
I work in small increments, writing one test at a time, then some code, then another test, then some more code, etc. In fact, I take this to what many people might call an extreme. Thanks for the replies. I've read about the TDD (and similar) approaches. Maybe I need to try it and get used

Re: Code correctness, and testing strategies

2008-05-24 Thread Jacob Hallen
In article [EMAIL PROTECTED], David [EMAIL PROTECTED] wrote: I work in small increments, writing one test at a time, then some code, then another test, then some more code, etc. In fact, I take this to what many people might call an extreme. Thanks for the replies. I've read about the TDD

Re: Code correctness, and testing strategies

2008-05-24 Thread Duncan Booth
David [EMAIL PROTECTED] wrote: Problem 2: Slows down prototyping In order to get a new system working, it's nice to be able to throw together a set of modules quickly, and if that doesn't work, scrap it and try something else. There's a rule (forget where) that your first system will

Re: Code correctness, and testing strategies

2008-05-24 Thread David
Hi again list. As you come up with the corner case, write a test for it and leave the implementation for later. The hard part of coding is always defining your problem. Once it is defined (by a test) the solution is just a matter of tidy work. Is it considered to be cheating if you make a

Re: Code correctness, and testing strategies

2008-05-24 Thread David
In order to get a new system working, it's nice to be able to throw together a set of modules quickly, and if that doesn't work, scrap it and try something else. There's a rule (forget where) that your first system will always be a prototype, regardless of intent. That's fine. It's alright

Re: Code correctness, and testing strategies

2008-05-24 Thread D'Arcy J.M. Cain
On Sat, 24 May 2008 17:51:23 +0200 David [EMAIL PROTECTED] wrote: Basically, with TDD you write the tests first, then the code which passes/fails the tests as appropriate. However, as you're writing the code you will also think of a lot of corner cases you should also handle. The natural way

Re: Code correctness, and testing strategies

2008-05-24 Thread Patrick Mullen
I can see both sides of this argument, and I think TDD is great in some cases and not so great in others. I have used it before, but don't use it often, as the programs I write are very difficult to automatically test. (Games, which many of the bugs have to do with whether an optimization

Re: Code correctness, and testing strategies

2008-05-24 Thread D'Arcy J.M. Cain
On Sat, 24 May 2008 21:14:36 +0200 David [EMAIL PROTECTED] wrote: Is it considered to be cheating if you make a test case which always fails with a TODO: Make a proper test case message? Yes. It's better to have the daily reminder that some code needs to be finished. While it is possible to

Re: Code correctness, and testing strategies

2008-05-24 Thread Duncan Booth
David [EMAIL PROTECTED] wrote: So, at what point do you start writing unit tests? Do you decide: Version 1 I am going to definitely throw away and not put it into production, but version 2 will definitely go into production, so I will start it with TDD?. If you are going to prototype

Re: Code correctness, and testing strategies

2008-05-24 Thread Casey McGinty
Seriously, 10 hours of testing for code developed in 10 hours? What kind of environment do you write code for? This may be practical for large companies with hordes of full-time testing QA staff, but not for small companies with just a handful of developers (and where you need to borrow

Re: Code correctness, and testing strategies

2008-05-24 Thread Roy Smith
David [EMAIL PROTECTED] wrote: Problem 1: You can only code against tests Yup. That's the flavor of Kool-Aide being served at a convenient TDD outlet near you. Basically, with TDD you write the tests first, then the code which passes/fails the tests as appropriate. However, as you're

Re: Code correctness, and testing strategies

2008-05-24 Thread Roy Smith
David [EMAIL PROTECTED] wrote: While it is possible to describe all problems in docs, it can be very hard to write actual test code. For example: sanity tests. Functions can have tests for situations that can never occur, or are very hard to reproduce. How do you unit test for those? In

Re: Code correctness, and testing strategies

2008-05-24 Thread Ben Finney
David [EMAIL PROTECTED] writes: Problem 1: You can only code against tests Basically, with TDD you write the tests first, then the code which passes/fails the tests as appropriate. However, as you're writing the code you will also think of a lot of corner cases you should also handle. The

Re: Code correctness, and testing strategies

2008-05-24 Thread Ben Finney
David [EMAIL PROTECTED] writes: Is it considered to be cheating if you make a test case which always fails with a TODO: Make a proper test case message? I consider it so. What I often do, though, is write a TODO comment in the unit test suite: # TODO:

Re: Code correctness, and testing strategies

2008-05-24 Thread Ben Finney
Roy Smith [EMAIL PROTECTED] writes: But, you are right, there are certainly cases which are difficult or impossible to test for. TDD is a very powerful tool, but it's just that: a tool. It's not a magic wand. My suggestion is to make using TDD a habit, but don't turn it into a religion.

Re: Code correctness, and testing strategies

2008-05-24 Thread Ben Finney
Duncan Booth [EMAIL PROTECTED] writes: You still need human testing and QA, the difference is that with a good set of unit tests you reduce the number of times code comes back from QA before it can be passed and make it more likely that the customer will be happy with the first version.

Re: Code correctness, and testing strategies

2008-05-24 Thread Ben Finney
D'Arcy J.M. Cain [EMAIL PROTECTED] writes: On Sat, 24 May 2008 17:51:23 +0200 David [EMAIL PROTECTED] wrote: If I did start doing some kind of TDD, it would be more of the 'smoke test' variety. Call all of the functions with various parameters, test some common scenarios, all the 'low

Re: Code correctness, and testing strategies

2008-05-24 Thread Terry Reedy
D'Arcy J.M. Cain [EMAIL PROTECTED] wrote in message news:[EMAIL PROTECTED] | But once you track down problems like the above you can write more | unit tests to catch those exact bugs in the future. This is one case | where I do favour unit tests. | | Yes! One of the biggest advantages to

Re: Code correctness, and testing strategies

2008-05-24 Thread Ben Finney
D'Arcy J.M. Cain [EMAIL PROTECTED] writes: Yes! One of the biggest advantages to unit testing is that you never ever deliver the same bug to the client twice. More specifically, this is a benefit of putting all unit tests into an automated test suite, and running that test suite all the time

Re: Code correctness, and testing strategies

2008-05-24 Thread Michael L Torrie
David wrote: Seriously, 10 hours of testing for code developed in 10 hours? What kind of environment do you write code for? This may be practical for large companies with hordes of full-time testing QA staff, but not for small companies with just a handful of developers (and where you need