I met the OP at the Seaspin presentation on Tuesday, and we talked around two things in our conversation that I thought I'd share:
Tests can document one thing that self-documenting code can never do: Distinguish between "It happens to be written this way" and "It must be written this way." Too many tests drowns you in information. If they're easily explored (via IDE, via organization, via some kind of automatic tagging), then you're more likely to be able to deal with it. Taking Justin's point about communication, that could be restated as: Explorable tests continue to communicate, even when there are many. -Kevin Kevin Klinemeier http://www.pugetworks.com/author/kevin On Thu, Mar 3, 2011 at 8:29 AM, David Foley <[email protected]> wrote: > This isn't exactly "tests" as documentation, but it is a great example of > using executable code as documentation. These are the examples for > mongo_mapper, a ruby MongoDB object-document mapper: > > https://github.com/jnunemaker/mongomapper/tree/master/examples > > > > > On Mar 3, 2011, at 12:30 AM, Peter Gfader wrote: > > @Alan > Yes me and my team are writing the tests in C#, and then those get > converted (1way) to HTML docu. > That HTML docu is used as a base for communication. > As Justin highlighted the problem is the communication. It is hard to > explain someone our business rules without samples. For each business rule > (requirement) we have a couple of tests (scenarios) that document the > requirement. We are not using a BDD approach ... yet... > > > @fschwiet > > >> Does anyone remember looking at some tests for an open source > project where they really stuck out as documentation? > > > I throw in some nice tests from NSubstitute > > https://github.com/nsubstitute/NSubstitute/blob/master/Source/NSubstitute.Specs/CallNotReceivedExceptionThrowerSpecs.cs > Nice code! > > .peter.gfader. > http://blog.gfader.com > > > On Thu, Mar 3, 2011 at 2:38 PM, fschwiet <[email protected]> wrote: > >> I think there have been some good points in this discussion. >> >> I learn best from examples though. Thats one nice thing about open >> source, you can find good code and learn from it. >> >> Does anyone remember looking at some tests for an open source project >> where they really stuck out as documentation? >> >> I'm going to plug a little side library of mine. I put together a >> simple script for updating XML from Powershell. Nothing big, just >> something easier than System.Xml.XmlDocument I'm happy with how the >> specs ended up serving as a good reference. The filenames become a >> table of contents, see: >> https://github.com/fschwiet/PSUpdateXml/tree/master/specs >> >> >> On Mar 2, 10:25 am, Justin Bozonier <[email protected]> wrote: >> > Exactly. This ultimately boils down to a problem of communication. Tests >> as >> > documentation can definitely work as long as the communication it >> > facilitates is valued and prioritized by your product team. If it isn't >> your >> > tests will reflect that by being difficult to understand and >> inconsistent. >> > >> > As developers who want to create awesome/valuable software (they're the >> same >> > right?) and keep our costs as low as possible when we deliver software >> we >> > want to communicate our logic and thought process as clearly as possible >> so >> > as to prevent bugs (which are really just the manifestations of >> > misunderstandings) and to ensure we truly are doing the simplest thing >> that >> > our customers will value. >> > >> > If you agree with that statement then we are on the same page and notice >> how >> > TDD/BDD/Documentation/w/e don't appear in that statement, communication >> > does. BDD and company are all just practices to support something I >> highly >> > value which is teamwork. While 0ne of the things we are notorious for is >> not >> > being able to communicate with our business customers, the heart of the >> > matter here is that we find it hard to even communicate with each other >> or >> > our future selves! >> > >> > So we've identified that wherever there are programmers, there is a >> > communications challenge (CAUTION: simplification, not for use as troll >> > bait). To keep to the Aeden's point, if we can identify what a test can >> > communicate then the answer will be whatever situation could use the >> > communication that a test provides and we can brainstorm a litany of >> those >> > I'm sure. >> > >> > *Unit* Tests Can Answer: >> > >> > - How a specific object in the system works >> > - How a specific behaviour of an object ties into business value >> (this is >> > possible but not always done) >> > - How easy an object is to use and design smells from complicated >> tests >> > - How the language you are using behaves (think of providing >> > documentation for yourself as you're learning a new language) >> > - Did my latest change break other expectations within the object? >> > >> > Unit Tests can NOT answer: >> > >> > - Whether or not the given test is still valuable (to programmers, or >> the >> > business) >> > >> > Integration Tests (combining multiple objects or systems) can answer: >> > >> > - What are the use cases? >> > - Can this system talk to that system, where a system is a collection >> of >> > one or more units? >> > - Did my latest change break any use cases (i.e. did it tangibly >> remove >> > value from my product)? >> > >> > I'm sure I'm probably wrong on some and that this is an incomplete list. >> I'd >> > love to see a community wiki of this sort so y'all could update it with >> your >> > two cents. Then we can start focusing on a group on our values and what >> > tools can support our values and in which ways. >> > >> > In lieu of a public wiki, I'd love to see others' lists for how they see >> > utilizing different tests for communication. >> > >> > On Wed, Mar 2, 2011 at 7:02 AM, Shawn Neal <[email protected]> >> wrote: >> > > I think you need both types of documentation, the english and code >> kind. >> > > One thing not mentioned yet, backfilling tests for a legacy system is >> a >> > > great way to fill in the documentation gaps and learn what the system >> really >> > > does... obviously not ideal, but well worth it. >> > >> > > On Wed, Mar 2, 2011 at 6:16 AM, Alan Ridlehoover < >> [email protected]>wrote: >> > >> > >> Actually, tests are useful as documentation to whoever authors them - >> > >> developers, testers, or in the case of acceptance tests, customers. >> (I use >> > >> the word customer in the extreme programming sense.) >> > >> > >> It looks like Peter's developers are writing the acceptance tests for >> > >> their system in C#, which is hampering their customer's understanding >> of the >> > >> tests. Progratically converting the C# into HTML is a novel way to >> help the >> > >> customer read the tests. In fact, I would argue that it is a lot like >> > >> writing FIT tests on behalf of your customer. Ultimately, though, the >> > >> customer will not truly use the tests as documentation unless they >> > >> themselves are authoring the tests. >> > >> > >> Alan >> > >> > >> Alan Ridlehoover >> > >> [email protected] >> > >> 206.412.2526 >> > >> > >> On Mar 1, 2011, at 10:29 PM, Peter Gfader <[email protected]> wrote: >> > >> > >> +1 tests serve as documentation (for developers) >> > >> > >> I think they don't serve as documentation *to others than devs* in >> their >> > >> source code form, because tests and their source is normally not easy >> > >> "accessible" to others. >> > >> But what about if we generate "documentation" in an accessible format >> and >> > >> publish somewhere? (similar like BDD does) >> > >> > >> In my team we generate documentation with a T4 template that clears >> out >> > >> all C# noise and spits out a HTML document. >> > >> > >> 1 test sample that gets converted to documentation >> > >> > >> Feature: OnlyBidFullIncrements >> > >> > >> 1. Lot1 ActualPrice = 50 >> > >> Lot1 BidIncrement = 50 >> > >> 2. MarketOp Change AuctionState: Catalogued >> > >> 3. Anna enters LimitBid: Lot1, 145 >> > >> 4. MarketOp Change AuctionState: Active >> > >> 5. Bob places Bid: Lot1, 100 >> > >> 6. //Note: Anna is getting it for 100, because it is less than 1 >> bid >> > >> increment away >> > >> Expect HoldingBidder AndPrice: Anna, Lot1, 100 >> > >> > >> These tests / HTML documents act as documentation for us (developers) >> and >> > >> when we talk/explain to others (PO and stakeholders)... >> > >> > >> >>What contexts are tests as documenation "good"/ "helpful" etc... >> for? >> > >> > >> #1 >> > >> > >> >>Learning a new system through tests. >> > >> > >> To me, a system is new if I haven't seen it for a while ;-) >> > >> So re-learning how the system works, is another context where they >> are >> > >> helpful.. >> > >> I learn what the edge cases are, what expected input is, and which >> > >> collaborator the SUT depends on. >> > >> > >> #2 >> > >> What about documenting performance goals? >> > >> Example: Create an integration test that hits a certain a webpage and >> > >> excepts a result in less than 2 seconds. >> > >> This test could act as documentation for that performance goal. >> > >> > >> #3 >> > >> Usage of an API can be nicely documented through tests as well >> > >> > >> >> * As for learning a new system goes, tests as documentation don't >> > >> >>offer historical explanation. For example, the roads not taken >> which >> > >> helps paint the >> > >> >>pictue of why the system is the way that it is. >> > >> > >> Nice idea, but I cant really think of a practical usage. By writing >> your >> > >> tests in a TDD manner, there is not really a road not taken... >> > >> > >> my 2c >> > >> .peter.gfader. >> > >> <http://blog.gfader.com>http://blog.gfader.com >> > >> > >> On Wed, Mar 2, 2011 at 4:14 AM, Frank Schwieterman < < >> [email protected]> >> > >> [email protected]> wrote: >> > >> > >>> I think the point about unit testing vs integration testing is >> true, >> > >>> what you test is what you've documented. So unit tests tell a >> > >>> different story then integration tests. >> > >> > >>> I just wanted to chip in that I do use tests as documentation, but >> > >>> not as something I read through. Its a reference I navigate with >> > >>> Reflector or Resharper. When I have a question about a method or >> > >>> class, I will find references to it in the tests to understand its >> > >>> intent and use. >> > >> > >>> On Tue, Mar 1, 2011 at 8:09 AM, Alan Ridlehoover << >> [email protected]> >> > >>> [email protected]> wrote: >> > >>> > Good conversation. One point I think is missing is this: >> > >>> > Unit tests make good documentation for individual methods/classes. >> > >>> But, >> > >>> > automated functional/acceptance tests make better documentation >> for the >> > >>> > system as a whole. And, yes, I've seen environments where the >> Fitnesse >> > >>> > tests were the documentation for the system. I've also seen STiQ >> tests >> > >>> used >> > >>> > that way (with less success), and WatiN-based NUnit tests (with >> good >> > >>> > success). >> > >>> > Alan >> > >>> > On Feb 27, 2011, at 8:08 PM, Michael Ibarra wrote: >> > >> > >>> > Hi Aeden, >> > >>> > I've never seen tests written and organized well enough to serve >> as >> > >>> > documentation (for neither developers nor "the business"). >> However, I >> > >>> do my >> > >>> > best to write and organize my tests to be intention revealing. >> > >>> > But that's not why I write tests. I write tests primarily because: >> 1. I >> > >>> want >> > >>> > to make sure that the code I write works the way it's supposed to, >> 2. I >> > >>> want >> > >>> > to make sure that adding new features doesn't introduce any bugs, >> 3. I >> > >>> don't >> > >>> > write(and therefore maintain) any more code than is necessary to >> > >>> implement >> > >>> > the feature. 4. I find that writing my tests first helps me write >> > >>> cleaner, >> > >>> > more S.O.L.I.D. code. >> > >>> > I want other developers to be able to read my tests and understand >> what >> > >>> my >> > >>> > code does, but that's just a side effect of well written and well >> > >>> organized >> > >>> > tests. >> > >>> > Anyway, that's my 2 cents. Hope that helps. >> > >>> > Mike >> > >> > >>> > On Sun, Feb 27, 2011 at 7:13 PM, Aeden Jameson << >> [email protected]> >> > >>> [email protected]> >> > >>> > wrote: >> > >> > >>> >> I'm curious how much stock people on this list put in the idea >> that >> > >>> >> tests serve as documentation. >> > >> > >>> >> -- >> > >>> >> Cheers, >> > >>> >> Aeden >> > >> > >>> >> Blog : <http://aedenjameson.blogspot.com/> >> > >>>http://aedenjameson.blogspot.com/ >> > >>> >> Linked In: <http://www.linkedin.com/in/aedenjameson> >> > >>>http://www.linkedin.com/in/aedenjameson >> > >>> >> Blah Blah Blah: <http://www.twitter.com/daliful> >> > >>>http://www.twitter.com/daliful >> > >> > >>> >> -- >> > >>> >> You received this message because you are subscribed to the >> Google >> > >>> Groups >> > >>> >> "Seattle area Alt.Net" group. >> > >>> >> To post to this group, send email to < >> [email protected]> >> > >>> [email protected]. >> > >>> >> To unsubscribe from this group, send email to >> > >>> >> <altnetseattle%[email protected]> >> > >>> [email protected]. >> > >>> >> For more options, visit this group at >> > >>> >> <http://groups.google.com/group/altnetseattle?hl=en> >> > >>>http://groups.google.com/group/altnetseattle?hl=en. >> > >> > >>> > -- >> > >>> > ******************************** >> > >>> > Michael Ibarra >> > >>> > <[email protected]>[email protected] >> > >>> > @bm2yogi >> > >>> > <http://dev.bm2yogi.com>http://dev.bm2yogi.com >> > >> > >>> > -- >> > >>> > You received this message because you are >> > >> > ... >> > >> > read more ยป >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Seattle area Alt.Net" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]. >> For more options, visit this group at >> http://groups.google.com/group/altnetseattle?hl=en. >> >> > > > -- > > .peter.gfader. > http://blog.gfader.com/ > http://twitter.com/peitor > > > > -- > You received this message because you are subscribed to the Google Groups > "Seattle area Alt.Net" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/altnetseattle?hl=en. > > > -- > You received this message because you are subscribed to the Google Groups > "Seattle area Alt.Net" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/altnetseattle?hl=en. > -- You received this message because you are subscribed to the Google Groups "Seattle area Alt.Net" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/altnetseattle?hl=en.
