Konstantin Boudnik commented on ZOOKEEPER-725:

bq. re 1) we should fix the test pkging, also split out units vs func etc... 
JUnit4 has "category" annotations. We should use these to define things like 
long/short running tests, etc... Not sure if we want to separate diff test 
types into diff directories (unit/func/perf/etc...) or use cats, or some 
combination but we need to think about that as well. src/java/test vs say 
src/java/units & src/java/func etc... Konstantin ideas for best practices here?

My recent experience with having a physical taxonomy of tests (unit/func/perf) 
in Hadoop seems to be pretty positive. I.e. it makes build layout simpler. 
Tests categorization is more clear this way, too. JUnit Categories are rather 
primitive in compare with Spring or TestNG ones (or at least it used to be like 
that last time I've checked).

bq. re 4) maybe create a jira for it and link? I'd like to see perf tests run 
as part of some sort of "nightly" build. Not sure if apache hudson can do this 
or not though... Konstantin any suggestions on this? What does MR/HDFS do for 

For a project with a relatively small test base it might be possible to execute 
everything every, say, night. For MR/HDFS we don't run any performance tests on 
Apache infrastructure. However, whatever JUnit based tests we have are all ran 
a couple times a day. 

What is important to mention, is that performance/stress results need to be 
properly trended/analyzed. Otherwise, it doesn't make much sense to bother with 
these types of tests.

Having subtasked JIRAs for stress and performance tests are good idea, because 
these are pretty different and are better be tracked separately.

bq. 5) there's been some interest to move to maven, would this help/hurt us and 
should we do that beore/after doing this? is there a "best" order for doing 
these changes?

It seems to be that client/server separation and build re-org seem like a 
logical first steps. Mockito + unit tests might come next. Stress/performance 
are ok to be added lately, because these might require some additional 
infrastructure/framework efforts.

> Test organization, methodology, and infrastructure improvements 
> ----------------------------------------------------------------
>                 Key: ZOOKEEPER-725
>                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-725
>             Project: Zookeeper
>          Issue Type: Test
>          Components: build, tests
>            Reporter: Konstantin Boudnik
> This is an umbrella feature for a number of improvements needed to happen in 
> ZK test realm

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

Reply via email to