Hi Srilatha, You could do the following:
Map steps can only collect key, value pairs; they can't collect three things. Instead of collecting (word, 1) in the mapper, you could collect (word, filename). Then, in the reduce step, you could output (filename, "word|count"), where "word|count" is Text, and "word" and "count" are both variables. This solution would require changing the value type from IntWritable to Text. As for getting the file name, read more here: < http://www.nabble.com/Processing-multiple-files---need-to-identify-in-map-td15841235.html > Alex On Sun, Oct 5, 2008 at 10:12 AM, Latha <[EMAIL PROTECTED]> wrote: > Greetings! > > Hi, Am trying to modify the WordCount.java mentioned at Example: WordCount > v1.0< > http://hadoop.apache.org/core/docs/current/mapred_tutorial.html#Example%3A+WordCount+v1.0 > >at > http://hadoop.apache.org/core/docs/current/mapred_tutorial.html > Would like to have output the following way, > > FileOne word1 itsCount > FileOne word2 itsCount > ..(and so on) > FileTwo word1 itsCount > FileTwo wordx its Count > .. > FileThree word1 its Count > .. > > Am trying to do following changes to the code of WordCount.java > > 1) private Text filename = new Text(); // Added this to Map class .Not > sure if I would have access to filename here. > 2) (line 18)OutputCollector<Text, Text, IntWritable> output // Changed > the > argument in the map() function to have another Text field. > 3) (line 23) output.collect(filename, word , one); // Trying to change the > output format as 'filename word count' > > Am not sure what other changes are to be affected to achieve the required > output. filename is not available to the map method. > My requirement is to go through all the data available in hdfs and prepare > an index file with < filename word count> format. > Could you please throw light on how I can achieve this. > > Thankyou > Srilatha >
