>> [Big Snip] > > There are a few bad assumptions in your colleague's response, so to > set the record straight: > > * test coverage and tests which use the interaction-based test > approach are not mutually exclusive > * you can have crappy tests which take the state-based approach and > crappy tests which use a interaction-based approach > * interaction-based testing is not merely limited to contrived > examples on people's blogs, it is a real practice which adds value on > lots of "real-world" projects > * using factories to generate required objects in tests has several > pros over the use of fixtures, and very very very few cons > > State-based testing and interaction-based testing both have their > place. There are number of reasons why they are both useful, but I'm > going to pick two: object decomposition (and coordinators) and > integration testing. Others have mentioned the value of writing tests > with the interface you want so I'm going to leave that out. > > As an application grows in features and complexity (business logic of > course) good developers will decompose the problem into a number of > simple objects. Some of these objects are responsible for doing the > work and others are responsible for coordinating other objects to do > the work. Objects which are responsible for coordinating are great > candidates for using interaction-based testing, because you are > concerned in the interaction, not the "state". > > If you don't have integration tests then using an interaction-based > testing approach is not worth it because you need something that is > going to test the real objects working with real objects. In Rails you > can write integration tests as Rail's > ActionController::IntegrationTests, Rail's functional tests, RSpec > stories, or RSpec controller tests w/view isolation turned off. > > IMO, one false benefit of only using a state-based approach when > writing a full-fledged application is that every object is essentially > an integration test at some level. You are always testing everything > with everything that it touches. This can lead to having one failure > in one model make several other model tests fail, and it can make > several controller tests failing (as well as any other object which > touches the model that is failing). I see this has a big negative > because it makes it more difficult to pinpoint the issue. People will > end up tracking it down, but it can be time consuming and frustrating. > > Now on the flip side people will complain that they renamed a model > method and re-ran all of their tests and everything passed, but when > running the application a bug exists. Doh, we forgot to update the > controller that relied on calling that model method. It is normal to > say/think, "well that should have failed because the method doesn't > exist on the model". (It sounds like David Chelimsky may have > something in trunk to help with this.) The main problem here though is > that an integration test didn't fail exposing that you weren't done > with your change. > > Thinking back to coordinating objects, my controllers don't contain > business logic in them because they are application layer classes, > they aren't apart of the domain of my software. They are only used by > the application to allow the software to fulfill the requirements of > my customer. Controllers are coordinators, not DOERS. They ask other > objects to fulfill a business requirement for them like moving stocks > from one portfolio to the another. So I used interaction-based testing > here to ensure that my controller is finding a stock, finding a > portfolio and asking a portfolio manager to move the stock to the > designed portfolio. I don't need to have those things written or even > fully implemented to ensure my controller works as I expect. I should > be able to see that my controller does what it should be doing, even > if the pieces it will use to do the work in the application aren't > finished. Now if those aren't implemented I should have an integration > test which fails showing me that the feature for moving stocks from > one portfolio to another is not completed, but that isn't what I'm > testing in my controller. > > Also after my controller works as expected I can go make sure the > PortfolioManager works as expected, and then I can go down and make > sure the Stock model does what I expect. When these objects are > working correctly individual I run my integration tests to ensure they > work well together. > > Another drawback of only using state-based testing is that you always > have to develop bottom up. You have to start with the low level > components and work your way out. I used to write code this way. I > think I have progressed beyond that, and now I write things in a > Acceptance Test Driven Development style. I start by writing an > integration test from the user's perspective proving that the feature > doesn't work, and then I move to the view, and then to the controller, > then to any manager/factory/presenter/service objects that are > required, and then down to any domain level objects (models and > non-models alike). You can't do this approach with state-based testing > only. There is a lot of value that can be gained by developing > software this way. > > In short: Interaction-based testing allows you to ensure that an > object is doing what you expect, without the underlying implementation > having to exist yet at all or in full. It is great for application > layer objects which typically only coordinate domain layer objects > where the correct interaction is what is important. It also helps you > develop interfaces, and it can scream loudly when you have an object > doing way too much. > > * "Blaming "brittleness" of tests upon interaction-based testing is a > red herring. Both interaction-based tests and state-based tests become > brittle if they make assertions upon implementation details and overly > constrain the interfaces between modules." - Nat Pryce > > * http://nat.truemesh.com/archives/000342.html - a wonderful read on > interaction-based vs state-based testing > > -- > Zach Dennis > http://www.continuousthinking.com > _______________________________________________ > rspec-users mailing list > rspec-users@rubyforge.org > http://rubyforge.org/mailman/listinfo/rspec-users
A lot of what you say makes me wish I was more experienced in this department :) I am very new to this! A part of me wishes I had the knowledge to write in the order of story -> view spec -> controller spec -> model spec. However most of the time (I emphasize MOST) I don't have the foresight to do that. The problem I'm trying to solve is almost always too complicated for me to know right away where to really start (my latest client has some crazy ideas). Maybe the problem is that I make things too complicated for myself :) However I have been a developer (just not using RSpec) for a very long time so I know fairly well how to recognize when things need to be complicated and when they don't. This means .should_receive is often out of the question because I have no idea what the model should receive! My primary concern when writing my specs that are to cover complicated features is that I do NOT want false confidence. If I write a spec, and it passes, I want that to mean it works in my app. When the spec goes green, my next step is to go hit Refresh in my browser. If it doesn't work in my browser, then in my opinion, my spec is crap. It's telling me things work when they don't. I hear the concern being voiced that if you break one thing and 15 specs pass then you're not mocking enough. Well since this is BDD, after all, then we should be working closely to the current spec we're trying to make pass. I change code, and 15 specs break, well I have a good idea of what code got broken because it's the top-most file in my editor! I hit save, Autotest screamed at me, I'm going to go hit undo now. Sometimes I make noob decisions and give model A a certain responsibility when it should have been done by model B. I get it to work in the short term, my spec's pass, but later I need to add another feature and realize that old responsibility needs to be moved from A to B. Now I have a red spec and X green specs. I move that responsibility, X specs are green, with still the same 1 red spec. I implement the new feature, X+1 green specs. I refresh in my browser, sure enough, it all works. I didn't have to go change all of my stubs and should_recieve's everywhere that just got moved. There's no need to, because my specs cover the true business logic behavior, and not the model-specific "behavior". While I do certainly believe the ability to spread everything out well enough so that 1 error = 1 broken spec comes from great wisdom and experience, I certainly don't have it, and I don't want to encourage others like me to try to strive for that because I don't know how to teach them from my own example. What I do know, is that I use a lot of real models, and I don't spend any time fixing specs that are broken by working code. I did that on my last project and it, in my opinion, wasn't worth it. I'd change a variable assignment with a .save to a .update_attribute and then I had a broken spec. My fear is that I'll write green specs, pat myself on the back, and then my company loses money because the site has bugs that my specs show green for because I don't have enough integration tests or whatever. But I don't want to have to double my tests for the same amount of coverage. I should have 1 spec for 1 feature and when that feature is working, that spec is green, plain and simple. I admit I may be ignorant to a lot of the power behind RSpec, but I like this level of simplicity and straightforwardness. Glenn Ford http://www.glennfu.com _______________________________________________ rspec-users mailing list rspec-users@rubyforge.org http://rubyforge.org/mailman/listinfo/rspec-users
