So... In the recent five-hundred-and-forty-seventh iteration of the "What's the point of mocks?" discussion on this list, David Chelimsky ended a message with:
> [ . . . ] Really, I think > that's what we should all be striving for. Not so much "should I use > mocks or not," but "when should I choose to use them?" And here... Here you made me really pause and think. David: Now that you're a Pragmatic author -- have you read Andy's _Pragmatic Thinking and Learning?_ Damn good book. The most valuable part of it to me was in the first couple of chapters, which laid out the Dreyfus model of skill acquisition. Having learned about it a few months ago, I'm starting to see it everywhere now. I don't intend to rehash the whole theory, but it basically says that the development of a skill (_any_ skill) follows a predictable pattern: 1. Novice (needs clear checklists or recipes; not ready for theory or problem-solving) 2. Advanced Beginner (can solve typical problems and alter the recipe within parameters) 3. Competent (understands the theory behind the recipes and can act creatively) 4. Proficient (acts on principles rather than recipes; using the skill is nearly automatic) 5. Expert (defines and innovates the field; works on intuition rather than rules) >From reading a fair amount on TDD/BDD -- including the Rspec Book -- I think there's widespread consensus that learning TDD/BDD is a totally separate skill from programming itself. One can be an expert programmer with decades of experience, but still be at zero when it comes to writing and using tests. (Or even an "anti-learner," hostile to the concept itself.) My thesis in this message is that there is a hurdle built into learning the skill of TDD/BDD with RSpec: that there's a steep curve somewhere in the early part of stage 2 -- "Advanced Beginner" -- that makes it more difficult than it ought to be to ascend to competence. The hurdle is that some easy things are easy, but other easy things are very complicated. Not just hard to do, but hard to *learn.* Hard to understand why stuff works. The tasks where the curve begins can be identified by following the high-level trends of conversations on this list. Usually it's on Rails. Models are easy enough; heck, models are fun. But then we try to do controllers, and... *Boom.* Brick wall. You can run the generator, but... What is it doing? How do you change it? Let's say I'm a novice to Rails, too: what the hell *is* that routing spec? That last sentence is the sharpest way I can think to phrase the problem. "What the hell *is* that spec?" Rails is easy to start rolling with for the default use cases, but moderately hard to _understand_ when you're ready to do more complex things. Its nuances are scattered in a lot of places, and some of the connections are invisible. RSpec on Rails, even in the default cases, exposes more of those nuances and *requires* you to understand them. ActiveRecord stands on its own, so models are fine (and views are mostly just tedious) but you can't even begin to understand that generated controller spec until you understand Rails and all of its nuances better. And if you're trying to learn both at once? And you don't want to go deeper into your Rails code until your specs pass? ...Learning deadlock. The failure to understand one keeps you from making progress with the other. In the ideal case the specs would help to teach you Rails, but in practice it doesn't work that way. They specs are too opaque without Rails knowledge. When I set out to consciously learn this stuff, I was determined to really learn it. I didn't want to type any code I couldn't understand. So I ran the RSpec scaffold in a 'dummy' app and used it only as a reference to build my own by hand. The first controller spec I tried was for a nested resource with parent and child objects. (Bad choice perhaps.) It took me almost two days to write that first full spec with all the mocks and stubs, and get it to pass on every action. I was on contract and even to me that felt hard to justify. And then I had to deal with authentication and filters, and then I felt the pain *again.* I'm a stubborn ass, so I plodded through. Eventually I even figured out what I was doing. (I think.) But I believe I have seen people run into this curve and view it as a brick wall. Many give up, sometimes concluding that RSpec is beyond them -- after all, everyone says it's "easy," so if they can't write a controller spec quickly or see the immediate value of this stuff, they must be too dumb for it. It isn't just Rails either. That's the most common entry point, and controllers seem to be the most common wall. But I hit a wall again when I needed to write an adapter for a complex SOAP interface, building some abstractions on top of soap4r. "Where do I start?" was quite hard for me. I fished around for blog posts and examples to give me some precedent. Eventually I figured out a reasonably efficient way to mock out the SOAP responses without building a whole fake server, and I think I'm better for the exercise. But as soon as I got beyond "this is all my code, so I understand it," I hit that curve again. ...This is a real problem. It's not entirely the fault of RSpec (in which I include RSpec-Rails) as a framework. Some of it's built into the ideology: some types of systems really are hard to specify or interact with, and we don't acknowledge that enough. Or we say "If it's painful to specify you're doing it wrong." Not always true, and a dangerous statement for a novice who's only feeling the sorts of pain everyone feels when they first begin exercising. Some of it's built into Rails and "Rails-like" MVC frameworks This is off the topic here, but I can't shake the feeling that if DHH had had RSpec from the very start and applied some of its philosophies faithfully, Rails would be different in some fundamental ways. Instead, some things are coupled in ways that require RSpec-Rails to jump through some weird hoops. It all works, but it isn't all _simple._ And beginners can't grok beyond simple. Some of it's the fault of the community. The problems I see here are subtle and certainly not malignant. This list, for instance, is excellent for its purpose -- it has probably the highest signal-to-noise ratio of any active technical list I follow, and everyone's eager to help at every level. I've never seen anyone sniped at for asking beginner questions. That's great. But the questions and the answers are both coming from such a broad range of skill levels, and sometimes that makes for a disconnect that's tough for learning. That's what I saw in the quote above that triggered this weird polemic from me. In the Dreyfus model, there's no question that many people here would be classed as Expert -- almost by definition in David's or Aslak's case. Me, I see myself on the level of "mid-to-high Competent, struggling toward Proficient." Whether that's accurate or not, I'd be the wrong person to say. Studies show that most incompetent people are very bad at gauging their incompetence, and I might be one of them. Studies also show that Experts are generally not good at teaching Novices, because they no longer _think_ in terms of rules and recipes. Making an Expert follow a recipe is terribly frustrating and a waste of everyone's time. OTOH telling Novice "use your own judgment" is even more frustrating -- and may result in the loss of that Novice. The Novice doesn't _have_ his or her own judgment yet. When we say things like, "I think that's what we should be striving for -- not 'should I use mocks or not,' but 'when should I choose to use them?'" ...That problem is, if we buy into that Dreyfus hierarchy, that is a harmful statement to deliver to an audience that's broadly distributed along the skill curve. The question of "When should I use X?" is an appropriate question for someone at Competent or Proficient levels. But it would be very challenging for most Advanced Beginners to suggest an answer, and a Novice wouldn't be able to answer it at all. (As for Experts: they probably don't even need to think about it. They just know.) We need more awareness of this. It's not a bad thing, and I think there are probably more benefits in having a skill-level-distributed list like this than to set up some sort of "beginner's" or "learner's" list or anything. (For one thing, most programmers who'd benefit would have too much ego to join it.) We just need to recognize -- and I include myself, as *I* spun off the theory tangent in that last thread -- that a Novice question needs a Novice or Advanced Beginner answer, and that saying "It's really more complicated than that" isn't helpful to certain classes of question. Community-wise: I also think we need better learning resources. The RSpec Book is great. Followed faithfully, I have a hunch it can bring an Advanced Beginner up to Competent with remarkable efficiency. I think it could work as well for *some* Novices. But not all. It depends on their thinking patterns, and whether they need or want to be grounded in theory before they get going with the recipes. The Mastermind puzzle is a good starting exercise, but there's too much explanation going on before the rest of the practical reference begins. (Again, for some people. This was _my_ impression. For others it might be just right.) The RSpec.info site is...problematic. It's quirky in its information structure, and I've gotten lost sometimes clicking around to find something I *knew* was there. I don't feel comfortable pointing people to it as a start-from-zero starting point. The explanations are clear and up-to-date, but it's hard to find a handhold. The Peepcode videos worked better for me, though they were a bit too long for my ADD-driven brain. At one point I was going to write a "Getting Started With RSpec" sort of primer. It was going to be informed by my own growth curve, and truly start with "Step by step, okay, just do this." No generators, write the code by hand, but get actual specs working quickly in actual projects. Of course, my "do this" would be somewhat different from David's -- I've been vocal enough about doing a lot of my tests a different way -- so I felt a bit awkward about breaking from orthodoxy when I don't feel like an expert myself. But I felt bold. I started it, got a little distracted and overplanned the demo app (a wiki to host the primer itself, but I wanted some new sorts of features) and it's languishing. This thread makes me wonder if I should put a little more foreground time into it. Not the stupid wiki, but the document. I'm rambling now, and I've spent *way* too much of my workday on this message, so I'll just summarize: * Some easy things are complicated in RSpec; * Incomplete understanding of RSpec and a framework being spec'ed can create learning deadlock; * Novices need different answers than competent practitioners; * The community as a whole should be aware and respectful of the growth curve difficulty. If anyone had the patience to read this far, any thoughts? Am I identifying real issues, or am I just full of hot air? -- Have Fun, Steve Eley (sfe...@gmail.com) ESCAPE POD - The Science Fiction Podcast Magazine http://www.escapepod.org _______________________________________________ rspec-users mailing list rspec-users@rubyforge.org http://rubyforge.org/mailman/listinfo/rspec-users
