Well my first suggestion is since the original String is used as a key - repeat the parse steps in the Mapper in the reduce code using the text from the key - then verify that the values you get are the ones in the key - Also if you know the answer - validate that you get the right answer when you get the values in the map phase
On Wed, Jul 28, 2010 at 11:44 AM, Erik Test <[email protected]> wrote: > Hello, > > I've implemented a program using map reduce for a simple distance > calculations between two 2D points. I've set up my input such that all > calculations should be the same but they are not. This makes me think I'm > doing something wrong within a map and/or reduce function. Here is my > output. > > 1,7,13,19 18.973665961010276 > 10,16,22,28 8.48528137423857 > 11,17,23,29 8.48528137423857 > 12,18,24,30 8.48528137423857 > 13,19,25,31 8.48528137423857 > 14,20,26,32 8.48528137423857 > 15,21,27,33 18.973665961010276 > 16,22,28,34 18.973665961010276 > 17,23,29,35 18.973665961010276 > 18,24,30,36 8.48528137423857 > 19,25,31,37 18.973665961010276 > 2,8,14,20 8.48528137423857 > 20,26,32,38 8.48528137423857 > 21,27,33,39 8.48528137423857 > 22,28,34,40 8.48528137423857 > 23,29,35,41 8.48528137423857 > 24,30,36,42 8.48528137423857 > 25,31,37,43 18.973665961010276 > 26,32,38,44 18.973665961010276 > 27,33,39,45 18.973665961010276 > 28,34,40,46 16.97056274847714 > 29,35,41,47 8.48528137423857 > 3,9,15,21 8.48528137423857 > 30,36,42,48 18.973665961010276 > 31,37,43,49 18.973665961010276 > 32,38,44,50 8.48528137423857 > 33,39,45,51 8.48528137423857 > 34,40,46,52 8.48528137423857 > 35,41,47,53 18.973665961010276 > 4,10,16,22 18.973665961010276 > 5,11,17,23 18.973665961010276 > 6,12,18,24 8.48528137423857 > 7,13,19,25 8.48528137423857 > 8,14,20,26 8.48528137423857 > 9,15,21,27 18.973665961010276 > > My key is the entire line entry of the input file and the value is the > distance calculated. Any value that doesn't begin with an 8 is wrong. > > This is what I do in the mapper. > > public static class Map extends MapReduceBase > implements Mapper<LongWritable, Text, > Text, DoubleWritable> > { > private Text word = new Text(); > > public void map(LongWritable key, Text value, > OutputCollector<Text, DoubleWritable> output, > Reporter reporter) throws IOException > { > String line = value.toString(); > String[] tokens = line.split("[,]"); > > for(int i = 0; i < tokens.length; i++) > { > word.set(line); > output.collect(word, new > DoubleWritable(Double.parseDouble(tokens[i]))); > } > }//public void map > }//public static class Map > > And this is what I do in the reducer. > > public static class Reduce extends MapReduceBase > implements Reducer<Text, DoubleWritable, > Text, DoubleWritable> > { > public void reduce(Text key, Iterator<DoubleWritable> values, > OutputCollector<Text, DoubleWritable> output, Reporter reporter) > throws IOException > { > double distance = 0; > double x1 = 0; > double x2 = 0; > double y1 = 0; > double y2 = 0; > > if(values.hasNext()) > { > x1 = values.next().get(); > } > > if(values.hasNext()) > { > x2 = values.next().get(); > } > > if(values.hasNext()) > { > y1 = values.next().get(); > } > > if(values.hasNext()) > { > y2 = values.next().get(); > } > > distance = StrictMath.sqrt(StrictMath.pow(x2 - x1, 2.0) + > StrictMath.pow(y2 - y1, 2.0)); > output.collect(key, new DoubleWritable(distance)); > } > }//public static class Reduce > > Any suggestions would be appreciated. > > Erik > -- Steven M. Lewis PhD Institute for Systems Biology Seattle WA
