2006/11/21, Alexei Fedotov <[EMAIL PROTECTED]>:
Hello,
If a developer runs all tests in one VM and they pass, that's great
and he can commit. Can developer make a commit if tests pass in
"perTest" mode and fail in "once" mode?
Let's imagine following scenarios of cooperative development when
pre-commit test suite demonstrates unpredictable behavior (such as
"once" mode").
Scenario A.
1. A developer runs tests one time, and tests failed.
2. He runs them again and tests passes.
3. He commit changes.
Why do we need to exercise developers' patience? Simple tests are
predictable - they either pass or fail.
Scenario B.
1. Developer A pass tests and commit his changes.
2. Developer B checks out a fresh workspace, makes changes and runs
tests - they failed.
3. He digs out the problem to the stage when he runs tests for the
clean workspace - tests fail since he has slightly different
configuration.
Developer B suspects developer A to run tests multiple times, or skip
acceptance test run at all. This destroys mutual trust. This makes me
think that the simpler pre-commit tests are, the better.
So in extreme case, no testing at all is the best! ;)
Seriously, kernel tests are simple enough unit tests - if we cannot
even trust "sameVM" runs here, do we ever need to invent complex
testing scenarios involving big apps?
Sure, we can reverse the default to "perTest" mode, but won't you feel
deeply sorry for time wasted in such testing? Just to remind, tests
for debug version take up to 2h on Windows server.
Thanks!
