Without using the distributed cache i'm getting the same error. It's because i start the job from a remote client / programmatically
2011/11/24 Michel Segel <[email protected]>: > Silly question... Why do you need to use the distributed cache for the word > count program? > What are you trying to accomplish? > > I've only had to play with it for one project where we had to push out a > bunch of c++ code to the nodes as part of a job... > > Sent from a remote device. Please excuse any typos... > > Mike Segel > > On Nov 24, 2011, at 7:05 AM, Denis Kreis <[email protected]> wrote: > >> Hi Bejoy >> >> 1. Old API: >> The Map and Reduce classes are the same as in the example, the main >> method is as follows >> >> public static void main(String[] args) throws IOException, >> InterruptedException { >> UserGroupInformation ugi = >> UserGroupInformation.createProxyUser("<remote user name>", >> UserGroupInformation.getLoginUser()); >> ugi.doAs(new PrivilegedExceptionAction<Void>() { >> public Void run() throws Exception { >> >> JobConf conf = new JobConf(WordCount.class); >> conf.setJobName("wordcount"); >> >> conf.setOutputKeyClass(Text.class); >> conf.setOutputValueClass(IntWritable.class); >> >> conf.setMapperClass(Map.class); >> conf.setCombinerClass(Reduce.class); >> conf.setReducerClass(Reduce.class); >> >> conf.setInputFormat(TextInputFormat.class); >> conf.setOutputFormat(TextOutputFormat.class); >> >> FileInputFormat.setInputPaths(conf, new Path("<path to input >> dir>")); >> FileOutputFormat.setOutputPath(conf, new Path("<path to >> output dir>")); >> >> conf.set("mapred.job.tracker", "<ip:8021>"); >> >> FileSystem fs = FileSystem.get(new URI("hdfs://<ip>:8020"), >> new Configuration()); >> fs.mkdirs(new Path("<remote path>")); >> fs.copyFromLocalFile(new Path("<local path>/test.jar"), new >> Path("<remote path>")); >> >> >
