When you are configuring your job in what most people refer to as their "Driver" class you can simply not set a Reducer and only set a Mapper.
// makes the job a map only one job.setNumReduceTasks(0); job.setMapperClass(MyFooMapper.class); Mischa On 3 Jun 2013, at 22:11, samir das mohapatra <[email protected]> wrote: > Dear All, > > Is there any way to copy the intermediate output file of the mapper into > local folder after each map task complete. > > Right now I am using > > FileSystem.copyToLocalFile(hdfsLocation,localLocation); > indiste the cleanup of mapper task , but it is failing . > > Exception file not found. > > But if I am giving same statement after the job complete in driver class ,it > is working fine. that i dont want. > > protected void cleanup(Context context){ > FileSystem.copyToLocalFile(hdfsLocation,localLocation);//failed > } > > Note: I need to copy the inter mediate output of the mapper to local file > system just after each map task complete. I dont want any reducer. > > If this is not the right solution then how to solve this type of scenario. > > Any help............. > > regards, > samir. _______________________________ Mischa Tuffield PhD http://mmt.me.uk/ @mischat
