I'm still for running tests as a set for each suite.
If you disagree with me, take a look at some of the current test suite
XML files and explain to me how it makes sense, or is even possible,
to run most of them with 100% independent tests. You can't even load
or assert data if you run each test case independently...
-David
On Mar 7, 2009, at 1:40 PM, Scott Gray wrote:
I haven't worked on it for a few weeks but I do have some code that
can track changes on the GenericDelegator and then reverse them when
requested. At the moment it makes the test independent at the
component level, mostly because the was the easiest place to do it.
I've tested it by exporting the data from a fresh install, running
the tests, exporting again and comparing the differences and at the
moment the only data that gets left behind is anything coming from
async service calls.
I'll try and make some time for getting it to work at the test level
over the next couple of days and then put a patch in jira for
review. Of course the problem with committing it is that a large
percentage of the tests will fail because they depend on the tests
that came before them.
Regards
Scott
HotWax Media
http://www.hotwaxmedia.com
801.657.2909
----- Original Message -----
From: "Tim Ruppert" <[email protected]>
To: [email protected]
Sent: Saturday, March 7, 2009 1:13:26 PM GMT -07:00 US/Canada Mountain
Subject: Re: how to write a test case
I've been a committer on a number of xxxUnit projects in the past
and grew up as one of the people bringing the agile development
processes to many different organizations, so I'd like to think that
I'm pretty savvy on this stuff. That being said, I've never been
happy with the way the testing frameworks work in OFBiz - some
because of my ignorance, but mostly because of the dependencies.
I've built code in a test-driven environment and let me just say
that we had few bugs that weren't caught, so when people added
stuff, we knew just about each and every time when there were side
effects and were able to fix them quickly.
What I'd like to see sometime soon is something that works like this:
1. Each test (note I did not say component or test suite or test
group, I said test) is totally independent.
2. Each test utilizes entity engine XML files to load the
appropriate data necessary for that test.
-- Sometimes this will mean loading the same or similar XML files a
few times.
-- That's ok :)
3. Each test puts the db back in exactly the same state as it was
before the test.
-- I used to use DbUnit to do this in the past.
-- Did this for both WebTest tests (functional) and normal JUnit
tests.
-- Worked like a charm.
-- This should be even easier for us because the Entity Engine can
keep track of all we do and roll it all back.
-- I know that Scott Gray has been working with this for a bit - and
it would be a HUGE win IMHO.
4. Utilizing the Entity Engine for better testing.
-- This is alluded to in #3 above about the roll backs.
-- It would also be cool if it could keep track of all you and BUILD
an entity engine XML file and save it if you like.
-- -- This should be super easy as well.
-- Then you could use these files you're generating in these tests
for future tests.
Anyways, that's my wish list and something that if we start to get
into place, I think we can build TONS of new unit tests around the
existing work. It will make each everyone's lives easier and the
project even more viable long term. Looking forward to feedback
whenever you guys get a chance, but I really feel this is the way we
should go.
Cheers,
Tim
--
Tim Ruppert
HotWax Media
http://www.hotwaxmedia.com
o:801.649.6594
f:801.649.6595
----- "Vikas Mayur" <[email protected]> wrote:
On Mar 7, 2009, at 2:01 AM, Adam Heath wrote:
Vikas Mayur wrote:
How did it work? I reverted back to 660193, the last patch for
OFBIZ-1790, and the accounting tests failed.
If they worked in the past, I'd like to know when. If so, then
that
means something since then has caused them to break, and I will
more
than gladly track that down.
However, if they have never worked(which is what I'm strongly
suspecting), then I stand by my original assessment.
Do not know why it is not working for you and I have no
idea/solution
for this.
If you run the test individually, and follow the instructions in
the
file, it'll probably work.
Yeah, I think so.
However, that's not how things are done.
All tests are run together. Every testdef/*.xml file that is in
any
ofbiz-component.xml is run one after the other, with no chance for
any
manual setup between each test.
In this circumstance, they do not work, and never did work. It is
this circumstance that an *automated* test case must work.
I do not know what is the point here to discuss same thing again and
again. I agree to your point of making test automated and lot of
people have complaint about
this in past but no one really come forward for the contribution.
Its really useless point to discuss on that these things in the trunk
are making you frustrated because they are not written properly so
why
not complain early in the process and not after a YEAR or so. Sorry
man, no time to look back and why not fix them by yourself if you see
issues.
