On Feb 23, 2007, at 11:53 AM, Doug Cutting wrote:
Glancing at TestNG and JUnit4, the grouping stuff looks nicer in
TestNG. But does grouping help except when changes are in high-
level components that little else depends on? For example, a
change to HDFS should run all the mapred tests that depend on it.
So if groups are tags (as I think they are in TestNG) then we could
tag those mapred tests that also use HDFS as belonging in the HDFS
group too.
Yup, test groups in TestNG are like tags, thus a test can belong to
multiple groups.
But that may not help overall execution time in many cases, e.g., a
change to HDFS would still require that all the slow tests be run.
It might still be worth trying groups...
Another approach to speeding tests might be to reuse static MiniDFS
and MiniMR daemons. We could have a static method like, getMiniDFS
(..params..). If one is already running with those params, it can
be reused, otherwise the running instance is shutdown and a new
instance started. Then. if we have control of the ordering of
tests, we can arrange for tests that use the same minimr and/or
minidfs configurations to run in sequence.
I still don't like this approach. It requires test writers ensure
that their tests don't have side effects on the Mini clusters. It
also would require tests to be run in the same VM (currently each
test is executed in a new VM - which is cleaner). Perhaps we should
rethink the port rolling of the Mini clusters...I think there is a
faster way to achieve the same goal.
Nigel Daley wrote:
On Feb 22, 2007, at 10:55 AM, Doug Cutting (JIRA) wrote:
[snip]
Each 'ant clean test' run took 16.5 minutes. I wonder if we
could decrease that substantially by merging several tests that
start dfs or mapred daemons into a single test, so that daemons
are started and stopped fewer times? We could even run test jobs
in separate threads, so that, e.g., while one test's reduce tasks
are running another tests map tasks can run...
I think merging or overlapping tests complicates the analysis of
tests when they fail (which is already hard enough).
Perhaps another solution is to categorize our tests so that
developers can choose subsets of the tests to run. This could be
achieved through new ant targets (yuk!) or replacing JUnit with a
test harness that supports categorization. TestNG (http://
testng.org/) is such a harness -- and it runs all JUnit tests
without modification. It's licensed under Apache License 2.0.
Another possibility is upgrade our JUnit to version 4 (good walk
through here: http://today.java.net/pub/a/today/2006/12/07/junit-
reloaded.html) which seems to have a rudimentary categorization
scheme using annotated test suites.
Nige
