Try to keep col2 values as map output key and map output value as the total values " b,a,v " Regards... Ramya.S
________________________________ From: unmesha sreeveni [mailto:[email protected]] Sent: Thu 1/16/2014 9:29 AM To: User Hadoop Subject: Re: Sorting a csv file Thanks Ramya.s I was trying it to do with NULLWRITABLE.. Thanks alot Ramya. And do u have any idea how to sort a given col. Say if user is giving col2 to sort the i want to get as b,a,v a,c,p d,a,z q,z,a r,a,b b,a,v d,a,z r,a,b a,c,p q,z,a How do i approch to that. I my current implementation i am getting result as a,c,p b,a,v d,a,z q,z,a r,a,b using the above code. On Wed, Jan 15, 2014 at 5:09 PM, Ramya S <[email protected]> wrote: All you need is to change the map output value class to TEXT format. Set this accordingly in the main. Eg: public static class Map extends Mapper<LongWritable, Text, Text, Text> { private Text one = new Text(""); private Text word = new Text(); public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException { System.out.println("in mapper"); String line = value.toString(); StringTokenizer tokenizer = new StringTokenizer(line); while (tokenizer.hasMoreTokens()) { word.set(tokenizer.nextToken()); context.write(word, one); System.out.println("sort: "+word); } } } Regards...? Ramya.S ________________________________ From: unmesha sreeveni [mailto:[email protected]] Sent: Wed 1/15/2014 4:11 PM To: User Hadoop Subject: Re: Sorting a csv file I did a map only job for sorting a txt file by editing wordcount program. I only need the key . How to set value to null. public class SortingCsv { public static class Map extends Mapper<LongWritable, Text, Text, IntWritable> { private final static IntWritable one = new IntWritable(1); private Text word = new Text(); public void map(LongWritable key, Text value, Context context) throws IOException, InterruptedException { System.out.println("in mapper"); String line = value.toString(); StringTokenizer tokenizer = new StringTokenizer(line); while (tokenizer.hasMoreTokens()) { word.set(tokenizer.nextToken()); context.write(word, one); System.out.println("sort: "+word); } } } public static void main(String[] args) throws Exception { System.out.println("in main"); Configuration conf = new Configuration(); Job job = new Job(conf, "wordcount"); job.setJarByClass(SortingCsv.class); //Path intermediateInfo = new Path("out"); job.setOutputKeyClass(Text.class); job.setOutputValueClass(IntWritable.class); job.setMapperClass(Map.class); FileSystem fs = FileSystem.get(conf); /* Delete the files if any in the output path */ if (fs.exists(new Path(args[1]))) fs.delete(new Path(args[1]), true); job.setInputFormatClass(TextInputFormat.class); job.setOutputFormatClass(TextOutputFormat.class); FileInputFormat.addInputPath(job, new Path(args[0])); FileOutputFormat.setOutputPath(job, new Path(args[1])); job.waitForCompletion(true); } } On Wed, Jan 15, 2014 at 2:50 PM, unmesha sreeveni <[email protected]> wrote: How to sort a csv file I know , between map and reduce shuffle and sort is taking place. But how do i sort each column in a csv file? -- Thanks & Regards Unmesha Sreeveni U.B Junior Developer http://www.unmeshasreeveni.blogspot.in/ -- Thanks & Regards Unmesha Sreeveni U.B Junior Developer http://www.unmeshasreeveni.blogspot.in/ -- Thanks & Regards Unmesha Sreeveni U.B Junior Developer http://www.unmeshasreeveni.blogspot.in/
<<winmail.dat>>
