Is there an hudson that build hbase?
Hudson should run the integration test suite with mini cluster and dfs
where ant test for the user should only run local fs and local map
reduce jobrunner.
On Mar 6, 2008, at 5:23 PM, Bryan Duxbury wrote:
I also vote for the local file system. However, I understand that
there are some issues with local filesystem implementation on
Windows, and there are some bugs that will only manifest when run on
hdfs, mini or otherwise.
I think the best middle ground is expecting an externally configured
dfs, local or otherwise, rather than spinning one up for every test.
This would require an extra step on the part of developers running
the test suite. Do you think that would make it notably more
difficult for new developers to get the test suite going?
Whatever we end up doing regarding filesystem spin up, my testing
shows that save us maybe a minute over the entire suite. It's
certainly still a worthwhile task to pursue, but won't be a magic
bullet.
-Bryan
On Mar 6, 2008, at 5:11 PM, Stefan Groschupf wrote:
jmock2 is pretty useful, I agree.
Also we in pig I submitted a patch that uses the hadoop
localjobrunner (jobtracker=local) - significant faster than the
minicluster.
Also in case hbase uses hadoop configuration why not just using the
local file system?
On Mar 6, 2008, at 4:45 PM, Bryan Duxbury wrote:
I would add that we have an issue open for speeding up the test
suite, HBASE-410. Probably the most promising idea for speeding up
the test suite is converting some of the tests that don't need to
be integration tests into something more like true unit tests, or
specs for that matter. Mock objects used to test hyper-focused
bits of the system would be absolutely ideal, and has the
advantage of being quite fast.
If you wanted to make some early contributions, anything in this
area would be a FANTASTIC place to put them.
-Bryan
On Mar 6, 2008, at 4:28 PM, stack wrote:
Stefan Groschupf wrote:
Hi there,
I checked out latest svn and did run the test.
The test suite failed at TestTable, are there known problems
today?
Supposed to work. Usually does. New hadoop was checked in last
night. That might have been responsible. Breaks everytime? If
so, open JIRA.
Also the test suite took 24 min to run on my little poor box -
is there any way to speed this up?
Yeah. Its a real prob. Test suite starts up little mini dfs's
and mini hbase clusters and sometimes mini mr clusters. I've
spent time on figuring how to do weird things like start the dfs
cluster inside of an ant task but ran into classloading issues
(joy). Needs to be fixed.
(Our 'unit tests' are more like 'integration tests' -- one
approache would be to call them so and move them out of the unit
test suite).
St.Ack
Thanks.
Stefan
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
101tec Inc.
Menlo Park, California, USA
http://www.101tec.com
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
101tec Inc.
Menlo Park, California, USA
http://www.101tec.com
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
101tec Inc.
Menlo Park, California, USA
http://www.101tec.com
