Re: [rspec-users] Best practices for sharing state between story steps?
Rick DeNatale wrote:
I've done something similar, but I might have handled the steps implementation a bit differently
in that I'm not sure my approach would be called stateless.
For example say I had (in a very abstract form)
Given 'Rick' has admin privileges
Then 'Rick' can do admin function X
In the step for the given, I'd get or create an admin user named rick, and set an instance
variable say @admin
Then in the step for the then, I'd have an expectation that the name of @admin matched the name
given in the step.
This was a way of catching cases where the story mixed up actors.
Although the short example doesn't really emphasize the problem I was trying to solve, I'm not
sure in retrospect whether or not this was a valuable thing to do.
I like this discussion because it shows the ways different people have solved a
very practical problem.
I don't think what you describe is bad, I do something very similar with my integration tests that
test a RESTFUL API based web service that returns XML to requests.
I use Hpricot to test the responses, but as this is not rails, I have to store the response between
steps.
A contrived Example...
Scenario test get resource list
Given database has 10 resources
When request made to list resources
Then 10 resources are returned
In my case the Given actually clears the resource table and populates it with 10 resources, I access
the database directly using the Sequel Gem, however if the database is remote this is problematic.
The When makes the RESTful request via http. and captures the result as an
Hpricot object.
The When step looks something like this...
When ... do
resp = @req_http.post(path, params, headers)
if resp.code != 200
raise #{resp.code} #{resp.message}
end
@hdoc = Hpricot.XML(resp.body)
end
The Then step looks something like this...
Then $cnt resources are returned do |cnt|
@hdoc.should have_xpath(/resources, :count = cnt)
@hdoc.should have_xpath(/resources/[EMAIL PROTECTED]'1'])
@hdoc.should have_xpath(/resources/[EMAIL PROTECTED]'2'])
...etc...
end
I have a custom hpricot matcher which provides the have_xpath matcher
Yes this is stateful, but the @hdoc variable is central to the tests and I don't see any other way
to carry the response from the When to the Then, unless you combine the step in a then and skip the
when, so you don't have to carry state across the steps, but I prefer the first method as I can run
more tests on the returned document.
e.g.
Then requesting a list of resources returns 10 items
--
Jim Morris, http://blog.wolfman.com Rick DeNatale wrote:
___
rspec-users mailing list
rspec-users@rubyforge.org
http://rubyforge.org/mailman/listinfo/rspec-users
Re: [rspec-users] Best practices for sharing state between story steps?
On Tue, Sep 9, 2008 at 11:31 PM, Jim Morris [EMAIL PROTECTED] wrote: aslak hellesoy wrote: There is persistent state (database) and object state (the object that serves as a context for a scenario). I'm not saying that state in and of itself is bad. However, *coupling* is bad - for maintenance reasons. If the steps share object state (in @variables) they become coupled. So why is coupling bad for maintenance? When your step library grows to several dozens or like in my case, over a hundred, you want to be able to cherry-pick steps to build new scenarios. If you have coupled them with object state, you can't do that. These steps will simply not work unless they're used alongside the steps they are @coupled to. I agree and have run into this problem, but I have not found a better way in the environment I am using. Cucumber starts a transaction when a scenario starts and rolls it back when it ends (when on Rails). Nice if you are using Rails, which I am not :) I was curious if anyone had run into the same problems I do, because I don't use rails, I test directly against the HTTP API, so the database gets all this test data, you can't use transactions, so the database potentially has this state from previous Scenarios. Sorry for not paying attention to this earlier. Right now in my Before_Scenario I generally try to clear the database, but that gets hard when it is relational with a ton of foreign key constraints, and you can't simply delete or truncate a table. As I said one work around is I create randomly named records every run, so as to avoid collision with previous data. I wonder if anyone has a better idea? BTW my Web service is written in Java/Jetty/Spring so I have no Rails/Ruby niceties on that side, so it is nice to be able to use Ruby in the integration tests. I've done that on several projects (using selenium/watir mostly). Some things you could try: 1) HTTP database endpoint A couple of years ago I was using Selenium against a Java web app. We implemented a special HTTP endpoint for testing only that we would access before a test. It would put the database in a pristine state. It's clunky, but it works. 2) Ruby database endpoint Another way to do it is to access the database directly from Ruby before the test runs and clear it there. 3) Turn off constraints Approaches 1 and 2 may be hard to do if you have complex constraints. One strategy is to turn off constraints in your test database and just do brute force deletes along with 1 or 2. 4) Incremental, semi-random data This is the approach you are describing. The problem with this is risk of false positives and intermittent failures. How risky it is depends on your app of course. 5) Nested transactions Using 1 or 2, start a transaction in the beginning of a scenario and roll it back at the end. Only works if your database supports nested transactions. I don't know if there is any good literature or discussion groups about these topics, but there ought to be. Aslak That is quite interesting, however as I pointed out above there is state between steps, in the database and in the response. That's not the coupling kind of state I'm talking about. That's ok. Ok I think I see the difference, by the state being in the Database or the response variable it does not implicitly couple steps like an @variable would. But it still requires the database to have been setup in a certain state by certain preceding Givens or Whens, which is a kind of coupling. But maybe I am stretching too far on that argument? I still feel coupling is coupling and state is state :) In Cucumber a Before block has access to the same scope as the steps. Good that'll make the transition easier I hope. -- Jim Morris, http://blog.wolfman.com ___ rspec-users mailing list rspec-users@rubyforge.org http://rubyforge.org/mailman/listinfo/rspec-users ___ rspec-users mailing list rspec-users@rubyforge.org http://rubyforge.org/mailman/listinfo/rspec-users
Re: [rspec-users] Best practices for sharing state between story steps?
On Tue, Sep 9, 2008 at 8:42 AM, aslak hellesoy [EMAIL PROTECTED]wrote: On Tue, Sep 9, 2008 at 2:21 PM, Dan North [EMAIL PROTECTED] wrote: Hi Jim. I guess I'm not a purist then - that looks fine to me, and it's probably something I would consider doing too. The debate seems to be whether step definitions should be stateful or not. In practice this is achieved by setting one or more @variables in a step and reusing them in a different step - all within a scenario. Both are fine, but beware that as your codebase grows and you have hundred or so step definitions, things can become *really* hard to maintain. It won't bite you when you start on a new codebase, but in my experience stateful steps will bite you later. Having experienced this pain in one project, I decided to try out stateless steps exclusively on the next project. I've found stateless steps to be a tad more verbose than stateful ones, both in the text and in the implementation (because you have to pass identifiers around). Still, the improved maintainability of my features and scenarios using this approach outweighs this slight increase in verbosity. Here is an example of a stateless scenario: (As a convention I'm quoting variables) http://github.com/aslakhellesoy/ba/tree/master/features/submit_proposal.feature You can see the extra verbosity I'm talking about in the repetition of Aslak, Beerfest etc. I'm still pragmatic about this of course, but now you know some of my experience. I've done something similar, but I might have handled the steps implementation a bit differently in that I'm not sure my approach would be called stateless. For example say I had (in a very abstract form) Given 'Rick' has admin privileges Then 'Rick' can do admin function X In the step for the given, I'd get or create an admin user named rick, and set an instance variable say @admin Then in the step for the then, I'd have an expectation that the name of @admin matched the name given in the step. This was a way of catching cases where the story mixed up actors. Although the short example doesn't really emphasize the problem I was trying to solve, I'm not sure in retrospect whether or not this was a valuable thing to do. -- Rick DeNatale My blog on Ruby http://talklikeaduck.denhaven2.com/ ___ rspec-users mailing list rspec-users@rubyforge.org http://rubyforge.org/mailman/listinfo/rspec-users
Re: [rspec-users] Best practices for sharing state between story steps?
Rick DeNatale wrote:
I've done something similar, but I might have handled the steps
implementation a bit differently in that I'm not sure my approach would
be called stateless.
For example say I had (in a very abstract form)
Given 'Rick' has admin privileges
Then 'Rick' can do admin function X
In the step for the given, I'd get or create an admin user named rick,
and set an instance variable say @admin
Then in the step for the then, I'd have an expectation that the name of
@admin matched the name given in the step.
This was a way of catching cases where the story mixed up actors.
Although the short example doesn't really emphasize the problem I was
trying to solve, I'm not sure in retrospect whether or not this was a
valuable thing to do.
I like this discussion because it shows the ways different people have solved a
very practical problem.
I don't think what you describe is bad, I do something very similar with my integration tests that
test a RESTFUL API based web service that returns XML to requests.
I use Hpricot to test the responses, but as this is not rails, I have to store the response between
steps.
A contrived Example...
Scenario test get resource list
Given database has 10 resources
When request made to list resources
Then 10 resources are returned
In my case the Given actually clears the resource table and populates it with 10 resources, I access
the database directly using the Sequel Gem, however if the database is remote this is problematic.
The When makes the RESTful request via http. and captures the result as an
Hpricot object.
The When step looks something like this...
When ... do
resp = @req_http.post(path, params, headers)
if resp.code != 200
raise #{resp.code} #{resp.message}
end
@hdoc = Hpricot.XML(resp.body)
end
The Then step looks something like this...
Then $cnt resources are returned do |cnt|
@hdoc.should have_xpath(/resources, :count = cnt)
@hdoc.should have_xpath(/resources/[EMAIL PROTECTED]'1'])
@hdoc.should have_xpath(/resources/[EMAIL PROTECTED]'2'])
...etc...
end
I have a custom hpricot matcher which provides the have_xpath matcher
Yes this is stateful, but the @hdoc variable is central to the tests and I don't see any other way
to carry the response from the When to the Then, unless you combine the step in a then and skip the
when, so you don't have to carry state across the steps, but I prefer the first method as I can run
more tests on the returned document.
e.g.
Then requesting a list of resources returns 10 items
--
Jim Morris, http://blog.wolfman.com
___
rspec-users mailing list
rspec-users@rubyforge.org
http://rubyforge.org/mailman/listinfo/rspec-users
Re: [rspec-users] Best practices for sharing state between story steps?
of course, there's one 'global' shared between steps that we cant live without: response :) ___ rspec-users mailing list rspec-users@rubyforge.org http://rubyforge.org/mailman/listinfo/rspec-users
Re: [rspec-users] Best practices for sharing state between story steps?
Hi Jim. I guess I'm not a purist then - that looks fine to me, and it's probably something I would consider doing too. The thing to bear in mind is that there is magic going on when you run steps. Each step in a scenario is run in the context of the same object instance (which you don't get to see explicitly) by pixies, which means any @variables you set in one step should be visible to any other steps in the same scenario. I don't think of this as global state - it's simply a sensible way to share state across steps. You can think of the object the steps run in as a world (it might even still be called that - it was at one point) where you can set and verify state. There are bound to be pathological cases though, such as the before_scenario and after_scenario listeners. Perhaps there's a bug there, in that they should be running in the same object instance as the scenario steps themselves. Perhaps not. In the latter case I would definitely go with $globals to communicate state into your scenario, as long as you promise never to use them in your application code. Never, you hear me? Cheers, Dan 2008/9/8 Jim Morris [EMAIL PROTECTED] Sam Stokes wrote: Jonathan Linowes wrote: I make some limited use of global (instance) variables that correspond to english language pronouns. I have things like @current_user (corresponds to I), @current_project (corresponds to the project), etc. I am careful to be consistent. There's only a handful of these, but I find it extremely convenient. Note, these also tend correspond to 'states' in my app, which might be in the session or part of a nested URL. So could I summarise this approach as, use @variables, with good names and consistent principles for use? I'm of the opinion do what works. I know there are purists that say don't do this and don't do that, but when you come down to it you have to use variables between steps, look at the rails examples and it sets a response variable between steps. What I do is horrible :) but it works, I set GLOBAL variables (yup $current_state), because I found that sometimes @variable didn't get setup properly in some cases. Then I use the listeners to clear those variables between scenarios, so every scenario I have a listener (effectively before_scenario), that clears all the global variables I use between steps, this avoids errors where I don't set up something in a step but the scenario passes because it just happens to have the right value in a global set by a previous test. This does require some maintenance, however I could use a global hash for my inter-step-dependencies, and just $hash.clear in the before_scenario listener. Ahh I remember why I had to use $variable and not @variable, for some reason the before_scenario listener does not have access to the same scope as the scenario so I have to use $variables. So most of my scenarios look like this before_scenario do $v1= nil $v2= nil end Scenario xxx do Given something... When I do something # this will set $v1 and $v2 Then check it was done # this will check $v1 and $v2 end I know purists will hate this ;) But it works for me BTW I use stories entirely for integration testing, testing the entire stack by using the same inputs (in my case a WEB based REST API) to the stack as a client would. I also do nasty things like delve into the database directly in a Then to check the database actually got setup the way I expected by a previous When, as well as checking the returned XML from the Rest-API call. As always YMMV -- Jim Morris, http://blog.wolfman.com ___ rspec-users mailing list rspec-users@rubyforge.org http://rubyforge.org/mailman/listinfo/rspec-users ___ rspec-users mailing list rspec-users@rubyforge.org http://rubyforge.org/mailman/listinfo/rspec-users
Re: [rspec-users] Best practices for sharing state between story steps?
On Tue, Sep 9, 2008 at 2:21 PM, Dan North [EMAIL PROTECTED] wrote: Hi Jim. I guess I'm not a purist then - that looks fine to me, and it's probably something I would consider doing too. The thing to bear in mind is that there is magic going on when you run steps. Each step in a scenario is run in the context of the same object instance (which you don't get to see explicitly) by pixies, which means any @variables you set in one step should be visible to any other steps in the same scenario. I don't think of this as global state - it's simply a sensible way to share state across steps. You can think of the object the steps run in as a world (it might even still be called that - it was at one point) where you can set and verify state. The debate seems to be whether step definitions should be stateful or not. In practice this is achieved by setting one or more @variables in a step and reusing them in a different step - all within a scenario. Both are fine, but beware that as your codebase grows and you have hundred or so step definitions, things can become *really* hard to maintain. It won't bite you when you start on a new codebase, but in my experience stateful steps will bite you later. Having experienced this pain in one project, I decided to try out stateless steps exclusively on the next project. I've found stateless steps to be a tad more verbose than stateful ones, both in the text and in the implementation (because you have to pass identifiers around). Still, the improved maintainability of my features and scenarios using this approach outweighs this slight increase in verbosity. Here is an example of a stateless scenario: (As a convention I'm quoting variables) http://github.com/aslakhellesoy/ba/tree/master/features/submit_proposal.feature You can see the extra verbosity I'm talking about in the repetition of Aslak, Beerfest etc. I'm still pragmatic about this of course, but now you know some of my experience. Aslak There are bound to be pathological cases though, such as the before_scenario and after_scenario listeners. Perhaps there's a bug there, in that they should be running in the same object instance as the scenario steps themselves. Perhaps not. In the latter case I would definitely go with $globals to communicate state into your scenario, as long as you promise never to use them in your application code. Never, you hear me? Cheers, Dan 2008/9/8 Jim Morris [EMAIL PROTECTED] Sam Stokes wrote: Jonathan Linowes wrote: I make some limited use of global (instance) variables that correspond to english language pronouns. I have things like @current_user (corresponds to I), @current_project (corresponds to the project), etc. I am careful to be consistent. There's only a handful of these, but I find it extremely convenient. Note, these also tend correspond to 'states' in my app, which might be in the session or part of a nested URL. So could I summarise this approach as, use @variables, with good names and consistent principles for use? I'm of the opinion do what works. I know there are purists that say don't do this and don't do that, but when you come down to it you have to use variables between steps, look at the rails examples and it sets a response variable between steps. What I do is horrible :) but it works, I set GLOBAL variables (yup $current_state), because I found that sometimes @variable didn't get setup properly in some cases. Then I use the listeners to clear those variables between scenarios, so every scenario I have a listener (effectively before_scenario), that clears all the global variables I use between steps, this avoids errors where I don't set up something in a step but the scenario passes because it just happens to have the right value in a global set by a previous test. This does require some maintenance, however I could use a global hash for my inter-step-dependencies, and just $hash.clear in the before_scenario listener. Ahh I remember why I had to use $variable and not @variable, for some reason the before_scenario listener does not have access to the same scope as the scenario so I have to use $variables. So most of my scenarios look like this before_scenario do $v1= nil $v2= nil end Scenario xxx do Given something... When I do something # this will set $v1 and $v2 Then check it was done # this will check $v1 and $v2 end I know purists will hate this ;) But it works for me BTW I use stories entirely for integration testing, testing the entire stack by using the same inputs (in my case a WEB based REST API) to the stack as a client would. I also do nasty things like delve into the database directly in a Then to check the database actually got setup the way I expected by a previous When, as well as checking the returned XML from the Rest-API call. As always YMMV -- Jim Morris, http://blog.wolfman.com ___ rspec-users
Re: [rspec-users] Best practices for sharing state between story steps?
Dan North wrote: Hi Jim. I guess I'm not a purist then - that looks fine to me, and it's probably something I would consider doing too. I'd never call you a purist Dan ;) But I do feel less dirty now, although after reading Aslaks post I wonder what exactly is meant by stateless steps? It seems to me that you have to set some kind of variable between steps to communicate what was done in one and what needs to be checked in another. Having been guilty of writing unit tests that were not stateless (ie one unit test depended on the result of a previous one, which BTW I really avoid doing), I did not consider setting variables between steps to be stateful. scenario steps themselves. Perhaps not. In the latter case I would definitely go with $globals to communicate state into your scenario, as long as you promise never to use them in your application code. Never, you hear me? Hmmm well OK I'll try to never use them ;) -- Jim Morris, http://blog.wolfman.com ___ rspec-users mailing list rspec-users@rubyforge.org http://rubyforge.org/mailman/listinfo/rspec-users
Re: [rspec-users] Best practices for sharing state between story steps?
On Tue, Sep 9, 2008 at 8:28 PM, Jim Morris [EMAIL PROTECTED] wrote: Dan North wrote: Hi Jim. I guess I'm not a purist then - that looks fine to me, and it's probably something I would consider doing too. I'd never call you a purist Dan ;) But I do feel less dirty now, although after reading Aslaks post I wonder what exactly is meant by stateless steps? It seems to me that you What I mean by a stateless step is a step that doesn't assign or use any @variables in its own context. have to set some kind of variable between steps to communicate what was done in one and what needs to be checked in another. Having been guilty of writing unit tests that were not stateless (ie one unit test depended on the result of a previous one, which BTW I really avoid doing), I did not consider setting variables between steps to be stateful. scenario steps themselves. Perhaps not. In the latter case I would definitely go with $globals to communicate state into your scenario, as long as you promise never to use them in your application code. Never, you hear me? Hmmm well OK I'll try to never use them ;) -- Jim Morris, http://blog.wolfman.com ___ rspec-users mailing list rspec-users@rubyforge.org http://rubyforge.org/mailman/listinfo/rspec-users ___ rspec-users mailing list rspec-users@rubyforge.org http://rubyforge.org/mailman/listinfo/rspec-users
Re: [rspec-users] Best practices for sharing state between story steps?
aslak hellesoy wrote: The debate seems to be whether step definitions should be stateful or not. In practice this is achieved by setting one or more @variables in a step and reusing them in a different step - all within a scenario. I think that is the debate, but I'd like to point out that there is always state between steps, it is just a matter of where it is being kept. In your example for instance most of the state is being kept in the database between steps (ie between Given an active site_user names aslak and he following steps. (As a side question how do you clean up the database between tests, so the state from the previous Scenario doesn't affect the next Scenario?) In some cases I use a randomly generated record each time. Here is an example of a stateless scenario: (As a convention I'm quoting variables) That is quite interesting, however as I pointed out above there is state between steps, in the database and in the response. I actually do something very similar, with named resources in each step, however I find I still need to use variables between steps, here is an example... (Not rails) Scenario: testing a pet Given I have a pet named my dog When I stroke my dog Then my dog's health goes up In the Given I create a pet named my dog in the database either directly or through the REST-ful API to my Web app, however in both cases I need the returned id of the created pet to make any further calls on it. In the When clause I need to make an API call that pets my dog, the API is POST stroke/345 So what I do is in the Given I assign the returned id of the new record to $petid, then in the When clause I do something like, $http_req.post(stroke, id = $petid). Now I needed to carry that variable $petid around otherwise how would I be able to make the call in When. Similarly when I want to check my pets health in the Then clause I still need that $petid, whether I check the value directly in the database or make an API call that returns my pets health. I don't see anyway around keeping state in these cases. Now I do clear that state ($petid= nil) in the before_scenario hook, so another scenario which may be written badly and not assign the $petid, won't succeeed due to a previously successful scenario. I'd definitely be interested in better ways to do this though, as I hate passing global variables around (as I said in an earlier post I can't use @variable because the before_scenario does not seem to have access to the same scope as the scenario that is about to run). Although I think someone said each scenario has its own variables? I didn't notice that behavior. -- Jim Morris, http://blog.wolfman.com ___ rspec-users mailing list rspec-users@rubyforge.org http://rubyforge.org/mailman/listinfo/rspec-users
Re: [rspec-users] Best practices for sharing state between story steps?
On 9 Sep 2008, at 19:52, Jim Morris wrote: aslak hellesoy wrote: The debate seems to be whether step definitions should be stateful or not. In practice this is achieved by setting one or more @variables in a step and reusing them in a different step - all within a scenario. I think that is the debate, but I'd like to point out that there is always state between steps, it is just a matter of where it is being kept. In your example for instance most of the state is being kept in the database between steps (ie between Given an active site_user names aslak and he following steps. (As a side question how do you clean up the database between tests, so the state from the previous Scenario doesn't affect the next Scenario?) In some cases I use a randomly generated record each time. I think the scenarios are each wrapped in a transaction, so (as long as you're using the right type of database / tables) the slate should be wiped clean between each one. Here is an example of a stateless scenario: (As a convention I'm quoting variables) That is quite interesting, however as I pointed out above there is state between steps, in the database and in the response. I actually do something very similar, with named resources in each step, however I find I still need to use variables between steps, here is an example... (Not rails) Scenario: testing a pet Given I have a pet named my dog When I stroke my dog Then my dog's health goes up In the Given I create a pet named my dog in the database either directly or through the REST-ful API to my Web app, however in both cases I need the returned id of the created pet to make any further calls on it. In the When clause I need to make an API call that pets my dog, the API is POST stroke/345 So what I do is in the Given I assign the returned id of the new record to $petid, then in the When clause I do something like, $http_req.post(stroke, id = $petid). Now I needed to carry that variable $petid around otherwise how would I be able to make the call in When. Similarly when I want to check my pets health in the Then clause I still need that $petid, whether I check the value directly in the database or make an API call that returns my pets health. I don't see anyway around keeping state in these cases. Now I do clear that state ($petid= nil) in the before_scenario hook, so another scenario which may be written badly and not assign the $petid, won't succeeed due to a previously successful scenario. I'd definitely be interested in better ways to do this though, as I hate passing global variables around (as I said in an earlier post I can't use @variable because the before_scenario does not seem to have access to the same scope as the scenario that is about to run). Although I think someone said each scenario has its own variables? I didn't notice that behavior. I've been thinking about this a bit, after reading Aslak's advice which went contrary to my instincts. It seems like you could probably do something like this instead: Scenario: testing a pet Given I have a pet When I stroke the pet Then the pet's health goes up Because your Given step creates just one Pet instance, you can safely refer to it as the pet in the other steps, and simply call Pet.find (:first) in the step matcher. See? In a more complicated scenario: Scenario: testing a pet Given I have two pets named my dog and my cat When I stroke my dog Then my cat should be pissed off Now your step matcher needs to refer to a particular pet, but can't it just use the name as a key, since that will be unique within the set of two Pet records you created in your Given step? When /I stroke (.*)/ |pet_name| do Pet.find_by_name(pet_name).stroke end Does that make sense to you? cheers, Matt http://blog.mattwynne.net http://songkick.com In case you wondered: The opinions expressed in this email are my own and do not necessarily reflect the views of any former, current or future employers of mine. ___ rspec-users mailing list rspec-users@rubyforge.org http://rubyforge.org/mailman/listinfo/rspec-users
Re: [rspec-users] Best practices for sharing state between story steps?
On Tue, Sep 9, 2008 at 8:52 PM, Jim Morris [EMAIL PROTECTED] wrote: aslak hellesoy wrote: The debate seems to be whether step definitions should be stateful or not. In practice this is achieved by setting one or more @variables in a step and reusing them in a different step - all within a scenario. I think that is the debate, but I'd like to point out that there is always state between steps, it is just a matter of where it is being kept. In your example for instance most of the state is being kept in the database between steps (ie between Given an active site_user names aslak and he following steps. There is persistent state (database) and object state (the object that serves as a context for a scenario). I'm not saying that state in and of itself is bad. However, *coupling* is bad - for maintenance reasons. If the steps share object state (in @variables) they become coupled. So why is coupling bad for maintenance? When your step library grows to several dozens or like in my case, over a hundred, you want to be able to cherry-pick steps to build new scenarios. If you have coupled them with object state, you can't do that. These steps will simply not work unless they're used alongside the steps they are @coupled to. That's when you start to add new step definitions with a similar semantic and wording, but with a different implementation - to work around the other steps that you realised you can't reuse. Now things get confusing. You have Given /there is a contract named (.*)/ and a Given /there is an existing contract named (.*)/ etc. etc. (As a side question how do you clean up the database between tests, so the state from the previous Scenario doesn't affect the next Scenario?) In some cases I use a randomly generated record each time. Cucumber starts a transaction when a scenario starts and rolls it back when it ends (when on Rails). Here is an example of a stateless scenario: (As a convention I'm quoting variables) That is quite interesting, however as I pointed out above there is state between steps, in the database and in the response. That's not the coupling kind of state I'm talking about. That's ok. I actually do something very similar, with named resources in each step, however I find I still need to use variables between steps, here is an example... (Not rails) Scenario: testing a pet Given I have a pet named my dog When I stroke my dog Then my dog's health goes up In the Given I create a pet named my dog in the database either directly or through the REST-ful API to my Web app, however in both cases I need the returned id of the created pet to make any further calls on it. In the When clause I need to make an API call that pets my dog, the API is POST stroke/345 So what I do is in the Given I assign the returned id of the new record to $petid, then in the When clause I do something like, $http_req.post(stroke, id = $petid). Now I needed to carry that variable $petid around otherwise how would I be able to make the call in When. Similarly when I want to check my pets health in the Then clause I still need that $petid, whether I check the value directly in the database or make an API call that returns my pets health. I don't see anyway around keeping state in these cases. Now I do clear that state ($petid= nil) in the before_scenario hook, so another scenario which may be written badly and not assign the $petid, won't succeeed due to a previously successful scenario. I'd definitely be interested in better ways to do this though, as I hate passing global variables around (as I said in an earlier post I can't use @variable because the before_scenario does not seem to have access to the same scope as the scenario that is about to run). Although I think someone said each scenario has its own variables? I didn't notice that behavior. In Cucumber a Before block has access to the same scope as the steps. -- Jim Morris, http://blog.wolfman.com ___ rspec-users mailing list rspec-users@rubyforge.org http://rubyforge.org/mailman/listinfo/rspec-users ___ rspec-users mailing list rspec-users@rubyforge.org http://rubyforge.org/mailman/listinfo/rspec-users
Re: [rspec-users] Best practices for sharing state between story steps?
Thanks for the hints, but I think I did point out I am not using Rails, I write Integration tests, that talk directly to the web application via HTTP running the full stack. So Active record is not available neither are transactions for the database. Sorry for the confusion. If I were using Rails I would undoubtedly do it the way you suggest :) Matt Wynne wrote: On 9 Sep 2008, at 19:52, Jim Morris wrote: aslak hellesoy wrote: The debate seems to be whether step definitions should be stateful or not. In practice this is achieved by setting one or more @variables in a step and reusing them in a different step - all within a scenario. I think that is the debate, but I'd like to point out that there is always state between steps, it is just a matter of where it is being kept. In your example for instance most of the state is being kept in the database between steps (ie between Given an active site_user names aslak and he following steps. (As a side question how do you clean up the database between tests, so the state from the previous Scenario doesn't affect the next Scenario?) In some cases I use a randomly generated record each time. I think the scenarios are each wrapped in a transaction, so (as long as you're using the right type of database / tables) the slate should be wiped clean between each one. Here is an example of a stateless scenario: (As a convention I'm quoting variables) That is quite interesting, however as I pointed out above there is state between steps, in the database and in the response. I actually do something very similar, with named resources in each step, however I find I still need to use variables between steps, here is an example... (Not rails) Scenario: testing a pet Given I have a pet named my dog When I stroke my dog Then my dog's health goes up In the Given I create a pet named my dog in the database either directly or through the REST-ful API to my Web app, however in both cases I need the returned id of the created pet to make any further calls on it. In the When clause I need to make an API call that pets my dog, the API is POST stroke/345 So what I do is in the Given I assign the returned id of the new record to $petid, then in the When clause I do something like, $http_req.post(stroke, id = $petid). Now I needed to carry that variable $petid around otherwise how would I be able to make the call in When. Similarly when I want to check my pets health in the Then clause I still need that $petid, whether I check the value directly in the database or make an API call that returns my pets health. I don't see anyway around keeping state in these cases. Now I do clear that state ($petid= nil) in the before_scenario hook, so another scenario which may be written badly and not assign the $petid, won't succeeed due to a previously successful scenario. I'd definitely be interested in better ways to do this though, as I hate passing global variables around (as I said in an earlier post I can't use @variable because the before_scenario does not seem to have access to the same scope as the scenario that is about to run). Although I think someone said each scenario has its own variables? I didn't notice that behavior. I've been thinking about this a bit, after reading Aslak's advice which went contrary to my instincts. It seems like you could probably do something like this instead: Scenario: testing a pet Given I have a pet When I stroke the pet Then the pet's health goes up Because your Given step creates just one Pet instance, you can safely refer to it as the pet in the other steps, and simply call Pet.find(:first) in the step matcher. See? In a more complicated scenario: Scenario: testing a pet Given I have two pets named my dog and my cat When I stroke my dog Then my cat should be pissed off Now your step matcher needs to refer to a particular pet, but can't it just use the name as a key, since that will be unique within the set of two Pet records you created in your Given step? When /I stroke (.*)/ |pet_name| do Pet.find_by_name(pet_name).stroke end Does that make sense to you? cheers, Matt http://blog.mattwynne.net http://songkick.com In case you wondered: The opinions expressed in this email are my own and do not necessarily reflect the views of any former, current or future employers of mine. -- Jim Morris, http://blog.wolfman.com ___ rspec-users mailing list rspec-users@rubyforge.org http://rubyforge.org/mailman/listinfo/rspec-users
Re: [rspec-users] Best practices for sharing state between story steps?
On Tue, Sep 9, 2008 at 3:52 PM, Matt Wynne [EMAIL PROTECTED] wrote: On 9 Sep 2008, at 19:52, Jim Morris wrote: aslak hellesoy wrote: The debate seems to be whether step definitions should be stateful or not. In practice this is achieved by setting one or more @variables in a step and reusing them in a different step - all within a scenario. I think that is the debate, but I'd like to point out that there is always state between steps, it is just a matter of where it is being kept. In your example for instance most of the state is being kept in the database between steps (ie between Given an active site_user names aslak and he following steps. (As a side question how do you clean up the database between tests, so the state from the previous Scenario doesn't affect the next Scenario?) In some cases I use a randomly generated record each time. I think the scenarios are each wrapped in a transaction, so (as long as you're using the right type of database / tables) the slate should be wiped clean between each one. I don't think this works as you may expect since My SQL and PostgreSQL don't support nested transactions, which is what would happen if your test environment wrapped stories/scenarios in transactions and your application utilized transactions. Granted there has been hopes of getting savepoints fix the problem, but the ticket has not been resolved: http://rails.lighthouseapp.com/projects/8994/tickets/383-activerecord-should-use-savepoints-for-nested-transactions -- Zach Dennis http://www.continuousthinking.com http://www.mutuallyhuman.com ___ rspec-users mailing list rspec-users@rubyforge.org http://rubyforge.org/mailman/listinfo/rspec-users
Re: [rspec-users] Best practices for sharing state between story steps?
Sam Stokes wrote: It seems to me that with Given an existing post 'lol internet' # creates a post with that title Then the page should include 'lol internet' # assumes a post with that title the assumptions are in line with what you'd naturally understand from the English, whereas Given an existing post 'lol internet' # creates and stores in @the_post Then I should see the post title # verifies @the_post matches what's in the db seems to make more assumptions than are obvious from the English. In fact, couldn't one argue that the second example is redundantly storing the post in two places (the database and @the_post)? I think this is what Aslak means by distinguishing stateful and stateless - @variables are *extra* state, on top of the application state which must change either way or you're not testing anything. Thats true in that case. I also wouldn't do what you show in the second case. If I can get the result from the API or the database I would. In my case I store data between states that I can't get anywhere else, or that tells me where to get the results from. -- Jim Morris, http://blog.wolfman.com ___ rspec-users mailing list rspec-users@rubyforge.org http://rubyforge.org/mailman/listinfo/rspec-users
Re: [rspec-users] Best practices for sharing state between story steps?
On 10.9.2008, at 0.29, Zach Dennis wrote: I don't think this works as you may expect since My SQL and PostgreSQL don't support nested transactions, which is what would happen if your test environment wrapped stories/scenarios in transactions and your application utilized transactions. I've been bitten by this a couple of times. A common idiom in the Rails world is (was at least) to use save! to raise an exception when an object couldn't be saved. In a rescue block you could then do the things you wanted to do when saving failed. Raising the exception also rolled back a transaction, so it was a nice way to ensure some transactional logic: MyModel.transaction do MyModel.save! OtherModel.save! end If either line in the transaction failed, it was ensured that neither of them was persisted in the db. This helped in many cases where you wanted either all or none of your updates to get into the db. However, with stories these cases always fail, because of the reason Zach mentions. The question whether a case where someone posts an empty email address is an exceptional state in strict PragProg sense might be a bit questionable, of course, but it's beyond the point here... //jarkko -- Jarkko Laine http://jlaine.net http://dotherightthing.com http://www.railsecommerce.com http://odesign.fi ___ rspec-users mailing list rspec-users@rubyforge.org http://rubyforge.org/mailman/listinfo/rspec-users
Re: [rspec-users] Best practices for sharing state between story steps?
On Sep 7, 2008, at 7:58 PM, Sam Stokes wrote: What approaches do people use to achieve this? Perhaps I'm bucking what others have advised against, so take it for what it's worth. I make some limited use of global (instance) variables that correspond to english language pronouns. I have things like @current_user (corresponds to I), @current_project (corresponds to the project), etc. I am careful to be consistent. There's only a handful of these, but I find it extremely convenient. Note, these also tend correspond to 'states' in my app, which might be in the session or part of a nested URL. With regard to reusable steps, I have some steps that are generic, some are app specific, and some are (group of) story/feature specific: 1. The generic ones are very reusable, for example When I click the $link link do |link| clicks_link link.strip_quotes end Then the browser should show $a_or_an $tag tag with $contents do |_, tag, contents| response.should have_tag(tag, contents.strip_quotes) end 2. The app specific ones are intended to be reusable in any story but only my app, may do some database initialization (like fixtures), logging in, for example: Given a $pname project with $settings Given I am logged in as a $role member Note, $settings is in the format Given a Test project with foo: 1, bar: bar value, and baz: yadda yadda creates a default project overridden with the specified settings (using to_hash_from_story, http://www.mail-archive.com/rspec- [EMAIL PROTECTED]/msg05771.html ) 3. And the feature-set specific steps are not very reusable at all. As it turns out, there's relatively few of these, and they tend to reflect aggregates of more granular steps already tested in a different story or scenario. hope this helps, linoj ___ rspec-users mailing list rspec-users@rubyforge.org http://rubyforge.org/mailman/listinfo/rspec-users
Re: [rspec-users] Best practices for sharing state between story steps?
Jonathan Linowes wrote: I make some limited use of global (instance) variables that correspond to english language pronouns. I have things like @current_user (corresponds to I), @current_project (corresponds to the project), etc. I am careful to be consistent. There's only a handful of these, but I find it extremely convenient. Note, these also tend correspond to 'states' in my app, which might be in the session or part of a nested URL. So could I summarise this approach as, use @variables, with good names and consistent principles for use? One problem I've had is that some steps implicitly have to be preceded by other steps, so that the @variables get set up right (e.g. Given some comments refers to @post, which has to be set beforehand by Given a post). I can make the wording of the steps more explicit, but then I end up with clunky steps like Given some existing comments by the user for the post. Generally I've found with this sort of approach I end up writing fairly imperative-style scenarios (as in http://www.benmabey.com/2008/05/19/imperative-vs-declarative-scenarios-in-user-stories/), and they're quite brittle with respect to small changes to steps. Do you encounter these problems at all? -- Posted via http://www.ruby-forum.com/. ___ rspec-users mailing list rspec-users@rubyforge.org http://rubyforge.org/mailman/listinfo/rspec-users
Re: [rspec-users] Best practices for sharing state between story steps?
Sam Stokes wrote: Jonathan Linowes wrote: I make some limited use of global (instance) variables that correspond to english language pronouns. I have things like @current_user (corresponds to I), @current_project (corresponds to the project), etc. I am careful to be consistent. There's only a handful of these, but I find it extremely convenient. Note, these also tend correspond to 'states' in my app, which might be in the session or part of a nested URL. So could I summarise this approach as, use @variables, with good names and consistent principles for use? I'm of the opinion do what works. I know there are purists that say don't do this and don't do that, but when you come down to it you have to use variables between steps, look at the rails examples and it sets a response variable between steps. What I do is horrible :) but it works, I set GLOBAL variables (yup $current_state), because I found that sometimes @variable didn't get setup properly in some cases. Then I use the listeners to clear those variables between scenarios, so every scenario I have a listener (effectively before_scenario), that clears all the global variables I use between steps, this avoids errors where I don't set up something in a step but the scenario passes because it just happens to have the right value in a global set by a previous test. This does require some maintenance, however I could use a global hash for my inter-step-dependencies, and just $hash.clear in the before_scenario listener. Ahh I remember why I had to use $variable and not @variable, for some reason the before_scenario listener does not have access to the same scope as the scenario so I have to use $variables. So most of my scenarios look like this before_scenario do $v1= nil $v2= nil end Scenario xxx do Given something... When I do something # this will set $v1 and $v2 Then check it was done # this will check $v1 and $v2 end I know purists will hate this ;) But it works for me BTW I use stories entirely for integration testing, testing the entire stack by using the same inputs (in my case a WEB based REST API) to the stack as a client would. I also do nasty things like delve into the database directly in a Then to check the database actually got setup the way I expected by a previous When, as well as checking the returned XML from the Rest-API call. As always YMMV -- Jim Morris, http://blog.wolfman.com ___ rspec-users mailing list rspec-users@rubyforge.org http://rubyforge.org/mailman/listinfo/rspec-users
Re: [rspec-users] Best practices for sharing state between story steps?
Zach Dennis wrote: This thread may help give you some insight... http://www.mail-archive.com/rspec-users@rubyforge.org/msg05382.html Thanks for pointing me there. The first insight it's given me is that I'm actually asking two different (though related) questions: 1) What's the best way to build reusable steps, and is it a good idea? 2) What's the best way to share state between the steps in a single scenario? That thread addresses 1) directly, and sounds like there's some evidence that reusable steps aren't such a good idea. In which case 2) pretty much drops out - if I don't try so hard to reuse my steps then there are fewer disadvantages to the obvious global @variable tactic. If you take the view that reusable steps may be unhelpful, what do you find is a good way to organise them? I can see an extreme case - each plain-text foo.story gets its own steps defined in foo.rb, so there are no cross-file @variable conventions to worry about. In this way foo.rb is basically a clarification in code of foo.story, rather than part of an app-specific language (ASL?) for integration testing. Does anyone do this, and do they find it a useful way of looking at the problem? -- Posted via http://www.ruby-forum.com/. ___ rspec-users mailing list rspec-users@rubyforge.org http://rubyforge.org/mailman/listinfo/rspec-users
[rspec-users] Best practices for sharing state between story steps?
Hi all, I'm just getting into RSpec stories and liking them (especially with webrat), but I'm finding it tricky to write steps that are self-contained and reusable, particularly where features intersect. What approaches do people use to achieve this? (Maybe a better question is, do people bother? One of the things I like about story-runner is the way I'm building a DSL for integration testing my application, but should I just write the scenario I need, write the steps to make it run and forget about reusing steps?) As an example of where I'm having trouble, say I'm writing a blog (since it's the Web 2.0 version of Hello World), so I have posts and comments. I want to write a scenario something like Given a post And some comments for the post When I view the post Then I should see the comments How do I tell the Given some comments step which post to attach the comments to? I can do it by having Given a post set a @post instance variable and having Given some comments use that, but it feels like global variables all over again. My steps (probably in different files) are coupled via the @post instance variable, and other steps can clobber it, and if I forget to clear it I might pollute later steps, and if I want to refer to more than one post (@post1, @post2) I have to rewrite all my steps... I noticed a little note on the Cucumber wiki (http://github.com/aslakhellesoy/cucumber/wikis/home) explicitly advising against using @variables for this, I'm guessing for these reasons. I could avoid the first issue by combining the top two steps into Given a post with some comments but then I still have the problem of which post When I view the post should GET. Maybe When I view a post with comments...? Any advice, criticism or sympathy appreciated :) -- Sam -- Posted via http://www.ruby-forum.com/. ___ rspec-users mailing list rspec-users@rubyforge.org http://rubyforge.org/mailman/listinfo/rspec-users
Re: [rspec-users] Best practices for sharing state between story steps?
This thread may help give you some insight... http://www.mail-archive.com/rspec-users@rubyforge.org/msg05382.html Zach On Sun, Sep 7, 2008 at 7:58 PM, Sam Stokes [EMAIL PROTECTED] wrote: Hi all, I'm just getting into RSpec stories and liking them (especially with webrat), but I'm finding it tricky to write steps that are self-contained and reusable, particularly where features intersect. What approaches do people use to achieve this? (Maybe a better question is, do people bother? One of the things I like about story-runner is the way I'm building a DSL for integration testing my application, but should I just write the scenario I need, write the steps to make it run and forget about reusing steps?) As an example of where I'm having trouble, say I'm writing a blog (since it's the Web 2.0 version of Hello World), so I have posts and comments. I want to write a scenario something like Given a post And some comments for the post When I view the post Then I should see the comments How do I tell the Given some comments step which post to attach the comments to? I can do it by having Given a post set a @post instance variable and having Given some comments use that, but it feels like global variables all over again. My steps (probably in different files) are coupled via the @post instance variable, and other steps can clobber it, and if I forget to clear it I might pollute later steps, and if I want to refer to more than one post (@post1, @post2) I have to rewrite all my steps... I noticed a little note on the Cucumber wiki (http://github.com/aslakhellesoy/cucumber/wikis/home) explicitly advising against using @variables for this, I'm guessing for these reasons. I could avoid the first issue by combining the top two steps into Given a post with some comments but then I still have the problem of which post When I view the post should GET. Maybe When I view a post with comments...? Any advice, criticism or sympathy appreciated :) -- Sam -- Posted via http://www.ruby-forum.com/. ___ rspec-users mailing list rspec-users@rubyforge.org http://rubyforge.org/mailman/listinfo/rspec-users -- Zach Dennis http://www.continuousthinking.com http://www.mutuallyhuman.com ___ rspec-users mailing list rspec-users@rubyforge.org http://rubyforge.org/mailman/listinfo/rspec-users
