We have done something like this where we want our unit tests to run
against a 1-machine "cluster". As a starting point we took
HadoopTestCase and have written our own modified version of this which
we configure and tell it whether to run in "local fs" or "mini
mapreduce" mode. The former is slightly faster but doesn't catch all
bugs so we have a continuous integration machine setup to perform tests
in both modes. We have also made some changes so that it only starts and
stops the cluster when the class is loaded instead of between tests
which also improves the speed.
Anyway, I would say that having a look at the source code for
HadoopTestCase is probably your best starting point and hopefully you
can take it from there!
Regards,
Adrian
Eugeny N Dzhurinsky wrote:
Hello there, we would like to make some tests with hadoop.
For the tests we would like to have a hadoop filesystem up and configured, so
using stubs and some mocks of core interfaces we can test the overall storage
functionality we're about to develop (which would be a part of map/reduce jobs
later).
As far as we learned from documentation/tutorials, to run hadoop we need to
use it's own startup scripts, which looks like an overhead and a bit
cumbersome to integrate in maven testing.
Could you please advice?
