Hello, I will be working for a month on the subject, on behalf of StumbleUpon / Stack. The goal is to reduced the build time for developer to a minimum, and at least half of the time needed now (i.e: from two hours -> 1 hour).
I created a JIRA to ease the follow up: HBASE-4602. I will put all the future sub-JIRA in this one. I already put the existing ones as "related link". As a start, I extracted the time taken on the apache server today, plus some hints on what the test is doing: the type of cluster used (dfs, zookeeper, hbase, mapreduce), the logs, potential "Thread.sleep". I attach the resulting excel sheet in HBASE-4602, you may want to have a look. BTW, The second sheet contains the script I used for this. Strategy will be mainly: - Cutting down on the number of cluster spinups by coalescing related tests rather than have each spin up its own cluster - Make cluster start/stop faster - Rewriting long-running tests so they do not need to be run on a cluster; e.g. by instead mocking expected signals/messages - Move long running tests out of the unit test suite to instead run as part of the recently introduced 'integration test' step Of course, there will be numerous small JIRAs to avoid any big bang effect. Splitting the tests in unit tests vs. long tests seems quite promising when looking to the excel sheet. Jesse, I understood that you're already working on this? Will you do the split as well? For myself, at the beginning, I will concentrate on cleaning up the tests and improving the start time of the cluster, so you will see some JIRA on this. Then I will look at the "long tests" that we would really like to keep as "unit test". Regards, N.
