Silly question... Why do you need to use the distributed cache for the word
count program?
What are you trying to accomplish?
I've only had to play with it for one project where we had to push out a bunch
of c++ code to the nodes as part of a job...
Sent from a remote device. Please excuse any typos...
Mike Segel
On Nov 24, 2011, at 7:05 AM, Denis Kreis <de.kr...@gmail.com> wrote:
> Hi Bejoy
>
> 1. Old API:
> The Map and Reduce classes are the same as in the example, the main
> method is as follows
>
> public static void main(String[] args) throws IOException,
> InterruptedException {
> UserGroupInformation ugi =
> UserGroupInformation.createProxyUser("<remote user name>",
> UserGroupInformation.getLoginUser());
> ugi.doAs(new PrivilegedExceptionAction<Void>() {
> public Void run() throws Exception {
>
> JobConf conf = new JobConf(WordCount.class);
> conf.setJobName("wordcount");
>
> conf.setOutputKeyClass(Text.class);
> conf.setOutputValueClass(IntWritable.class);
>
> conf.setMapperClass(Map.class);
> conf.setCombinerClass(Reduce.class);
> conf.setReducerClass(Reduce.class);
>
> conf.setInputFormat(TextInputFormat.class);
> conf.setOutputFormat(TextOutputFormat.class);
>
> FileInputFormat.setInputPaths(conf, new Path("<path to input
> dir>"));
> FileOutputFormat.setOutputPath(conf, new Path("<path to
> output dir>"));
>
> conf.set("mapred.job.tracker", "<ip:8021>");
>
> FileSystem fs = FileSystem.get(new URI("hdfs://<ip>:8020"),
> new Configuration());
> fs.mkdirs(new Path("<remote path>"));
> fs.copyFromLocalFile(new Path("<local path>/test.jar"), new
> Path("<remote path>"));
>
>
