You're not missing anything Stefan. I think there is general agreement
that 30 mins is too long to wait on tests (Bryan Duxbury, to his credit,
keeps trying to bring up this fact. He filed HBASE-498 last night).
There is an issue using local filesystem on windows that we need to
figure. Will try leaning on our windows user to see if we can get more
info around the prob. Will post to HBASE-498.
St.Ack
Stefan Groschupf wrote:
I'm sorry if I ask stupid questions, I'm very new to HBase.
Waiting 30 min to see if a change breaks something or not is painful.
I understand why you want to use a hadoop minicluster for integration
tests but for normal unit tests.
I would expect that there is an interface for each hbase component
that the server and the clients implements. Similar to the hadoop
jobrunner and localJobrunner. Since both components implement this
interface it is transparent for the test if we use a distributed mini
cluster or a local non tcp ip implementatipon. Also we could have
mocks implementing this interface.
So which puzzle peace I miss that there is the requirement to start up
this minicluster every time?
On Mar 6, 2008, at 8:24 PM, stack wrote:
Stefan Groschupf wrote:
Is there an hudson that build hbase?
Hudson should run the integration test suite with mini cluster and
dfs where ant test for the user should only run local fs and local
map reduce jobrunner.
Yeah. We have our own little zone on hudson:
http://hudson.zones.apache.org/hudson/job/HBase-Patch/. Needs some
work. Currently just runs a build on checkin. Would like to add
the patch testing that happens in hadoop core.
Starting dfs cluster from ant script is a little awkward.
MiniDFSCluster is nice because it chooses unoccupied ports; important
on a populated build box. Would need to then pass the chosen dfs
port then to tests and mini dfs would have to be in current process
if mindfs so can shut it down when done. I tried running in ant
context as distinct task and as wrapper around junit task but ran
into class loading issues.
Could call out to a script that starts minidfs but need to do a
little dev so can send it a shutdown when done. Was not enthusiastic
since replicating the 'real' dfs start/stop scripts. Alternative
would be to start a 'real' dfs using fixed ports. This would involve
ant exec'ing. Would need to deal with *nix and windows paths to make
sure it worked on both. If I'm not mistaken, even 'local' mode
requires ssh'ing to start datanodes even if on localhost.
Maybe the best thing would be to just punt on trying to have the
start/stop of the dfs be run from ant and just do it as part of the
build scripts we run up on the hudson build box?
St.Ack
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
101tec Inc.
Menlo Park, California, USA
http://www.101tec.com
