The approach I've taken is to use JMock and create a unit test for the mapreduce, then debug that within Eclipse on my workstation. For performance debugging, I use YourKit on the cluster.
Matt On Mon, 2008-06-16 at 16:58 -0700, Mori Bellamy wrote: > Hey Richard, > > I'm interested in the same thing myself :D. I was researching it > earlier today, and the best I know to do is to use Eclipse's remote > debugging functionality (although this won't completely work. each map/ > reduce task spawns on its on JVM, making debugging really hard). but > if you want, you can debug up until the mappers/reducers spawn. To do > this, you need to pass certain debug-flags into the JVM. So you'd need > to do export HADOOP_OPTS=myFLagsForRemoteDebug > and then you'd go to eclips ->run->open debug dialog and set up remote > debugging with the correct port. > > if you find out a way to debug the mappers/reducers on eclipse, let me > know :D > > > On Jun 16, 2008, at 3:10 PM, Richard Zhang wrote: > > >> > >> Hello Hadoopers: > >> > >> Is there a way to debug the hadoop code from Eclipse IDE? I am > >> using Eclipse to read the source and build the project now. > >> How to start the hadoop jobs from Eclipse? Say if we can put the > >> server names, could we trace the running process through > >> eclipse, such as setting breakpoints, check the variable values? > >> That should be very helpful for development. > >> If anyone know how to do it, could you please give some info? > >> Thanks. > >> > >> Richard > >> > >> > >> >
