Paul thank you very much for the detailed explanation. I have applied a
prefix to all my test functions and everything is running as
expected.
thanks
-george
From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Paul Carvalho
Sent: Thursday, May 04, 2006 11:36 PM
To: [email protected]
Subject: Re: [Wtr-general] Order of execution for ruby tests with test/unit
(BTW, please forgive me for not using correct Test case/suite/whatever terminology here.. I don't have my Ruby/Watir references handy as I type this.)
Let's say that you have a master script that calls 3 other scripts that have the test classes in them.. something like:
---
# Setup
require 'watir'
require 'test/unit'
# Set up common methods, variables, whatever
# My Tests
require Startup.rb
require HomePage.rb
require ShutDown.rb
---
The imaginary contents of ' Startup.rb' could be something like:
---
class Test_Startup < Test::Unit::TestCase
ie = Watir::IE...
puts 'This is the start'
def test_navigation
ie.goto(..)
ie.blah.blah
assert_equal(yadda, yadda)
end
def test_login
ie.text_field().set()
....
end
puts 'This is the end.'
end
---
And so on. You can assume that a similar pattern exists for the other two scripts. Here's where it gets fun.
My initial, uneducated, guess was that these tests would run in the following order:
1) Startup.rb
a) test_navigation
b) test_login
c) ...
2) HomePage.rb
a) ..
b) ...
3) ShutDown.rb
a) ..
b) ...
Ya, well, reality was something a bit different. That is, I found out that these tests are supposed to run in a 'random' order.
I didn't believe for one second that Ruby was really going to run these tests randomly so I tried to figure out what the pattern actually was. Here's what I found out.
First of all, the code is put into two 'pools': (1) Stuff not in a 'test' definition (i.e. *not* a test), and (2) stuff that's in a test definition (i.e. the tests).
-> All the non-test stuff is executed first, in the order in which they appear.
So, all the lines in the Master script are run first, as you would imagine, followed by the lines it finds in the test scripts that aren't in 'test' definitions either. That means that the following lines will be executed *before* any tests are run, even though they're within test classes:
---
ie = Watir::IE...
puts 'This is the start'
puts 'This is the end.'
---
Next, all the Test Classes are put into alphanumeric order. So regardless of how I called the scripts above, they will be run in the following order:
---
class Test_HomePage < Test::Unit::TestCase
class Test_ShutDown < Test::Unit::TestCase
class Test_Startup < Test::Unit::TestCase
---
Lastly, all of the test cases are sorted in an ascending alphanumeric order. So, given the two tests in my 'Startup' script above, they will actually run in this order:
def test_login
def test_navigation
...and so on..
To get around this, I've been using a naming convention for my test classes that include numeric labels. Something like this:
---
class Test_01_Startup < Test::Unit::TestCase
class Test_02_HomePage < Test::Unit::TestCase
class Test_03_ShutDown < Test::Unit::TestCase
---
-> Which assures me that they will be run by Ruby in that order.
Then I also use numeric labels similarly with the test case names too. That way, I know that they run in the correct order. So using my example from above, you would probably see something like this if you browse through my scripts:
---
class Test_01_Startup < Test::Unit::TestCase
def test_0101_navigation
def test_0102_login
class Test_02_HomePage < Test::Unit::TestCase
def test_0201_check_userdata
def test_0202_check_reports
class Test_03_ShutDown < Test::Unit::TestCase
...
---
This way, I don't have to worry about which order the tests are run because I *know* that they will run from top to bottom, in the order that I specify the tests to run.
I was quite happy to discover that pattern. I don't recall where I got the initial tip, but I'm pretty sure that someone or something mentioned that I should consider ascending labels.
Works for me. I would be happy to have someone confirm this pattern for me. As I said, it was a while ago that I figured this out (probably 4-6 weeks ago now), and I am just winging this off the top of my head, but I'm pretty sure I got the details correct.
Hope this helps. Have a great day!
Cheers. Paul.
On 04/05/06, George
Flaherty < [EMAIL PROTECTED]
> wrote:
Sorry to repeat here, but did someone post a way to execute ruby unit tests in a "predetermined order" vs.the default "random" order?
Thanks
-george
_______________________________________________ Wtr-general mailing list [email protected] http://rubyforge.org/mailman/listinfo/wtr-general
