OK, i fixed that problem by adding conf.setInputFormat(SequenceFileInputFormat.class);
to the job configuration. Does the BytesWritable file object point to exactly one file inside the Sequence File? On Wed, Jan 20, 2010 at 10:54 AM, Suhail Rehman <suhailreh...@gmail.com>wrote: > This is the console output trying to run this application. I'm sure the > input sequencefile is a (Text,BytesWritable) pair. > > Suhail > > 10/01/20 10:52:34 WARN conf.Configuration: DEPRECATED: hadoop-site.xml > found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use > core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of > core-default.xml, mapred-default.xml and hdfs-default.xml respectively > 10/01/20 10:52:34 WARN mapred.JobClient: Use GenericOptionsParser for > parsing the arguments. Applications should implement Tool for the same. > 10/01/20 10:52:34 INFO mapred.FileInputFormat: Total input paths to process > : 1 > 10/01/20 10:52:34 INFO mapred.JobClient: Running job: job_201001122200_0019 > 10/01/20 10:52:35 INFO mapred.JobClient: map 0% reduce 0% > 10/01/20 10:52:44 INFO mapred.JobClient: Task Id : > attempt_201001122200_0019_m_000000_0, Status : FAILED > java.lang.ClassCastException: org.apache.hadoop.io.LongWritable > incompatible with org.apache.hadoop.io.Text > at BlurMapper.map(BlurMapper.java:1) > at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) > at org.apache.hadoop.mapred.Child.main(Child.java:170) > > > On Wed, Jan 20, 2010 at 5:41 AM, Jeff Zhang <zjf...@gmail.com> wrote: > >> Could you paste your exception message ? >> >> >> >> On Wed, Jan 20, 2010 at 1:15 AM, Suhail Rehman <suhailreh...@gmail.com>wrote: >> >>> I am using hadoop to write some sample code which takes every image and >>> blurs it using a blurring filter. >>> >>> I was able to convert all my input images into a sequence file, and I've >>> written the following hadoop code to perform the blurring operation. (The >>> input sequencefile key-value pairs are Text (filename of the image), >>> BytesWritable (image contents) for each record). >>> >>> 've used the TAR to sequence file creator available here: >>> http://stuartsierra.com/2008/04/24/a-million-little-files >>> >>> But for some reason, I cannot get it working. I've pasted the code here >>> (not including the imports), let me know what I'm doing wrong (I'm new to >>> Hadoop btw). >>> >>> import com.jhlabs.image.BoxBlurFilter; >>> >>> //Mapper Class >>> >>> public class BlurMapper extends MapReduceBase implements >>> Mapper<Text,BytesWritable,Text,BytesWritable> { >>> >>> public void map(Text key, BytesWritable file, >>> >>> >>> >>> >>> OutputCollector<Text, BytesWritable> output, >>> Reporter reporter) throws >>> IOException { >>> >>> >>> //Read Current Image from File. >>> BufferedImage img = ImageIO.read(new ByteArrayInputStream >>> >>> >>> >>> >>> (file.getBytes())); >>> BufferedImage dest = null; >>> >>> //Apply Blur on Filter Operation - External JAR >>> BoxBlurFilter BlurOp = new BoxBlurFilter(10,10,2); >>> BlurOp.filter(img, dest); >>> >>> >>> >>> >>> ByteArrayOutputStream outputbytes = new >>> ByteArrayOutputStream(); >>> ImageIO.write(dest, "jpeg", outputbytes); >>> BytesWritable outfile = new >>> BytesWritable(outputbytes.toByteArray()); >>> >>> >>> >>> >>> output.collect(key, outfile); >>> } >>> >>> } >>> >>> //MAIN CLASS >>> >>> public class BlurVideoHadoop { >>> >>> public static void main(String[] args) { >>> >>> if(args.length!=2) { >>> >>> >>> >>> >>> System.err.println("Usage: blurvideo input output"); >>> System.exit(-1); >>> >>> } >>> JobClient client = new JobClient(); >>> JobConf conf = new JobConf(BlurVideoHadoop.class); >>> >>> >>> >>> >>> conf.setOutputKeyClass(Text.class); >>> conf.setOutputValueClass(BytesWritable.class); >>> >>> SequenceFileInputFormat.addInputPath(conf, new >>> Path(args[0])); >>> SequenceFileOutputFormat.setOutputPath(conf, new >>> Path(args[1])); >>> >>> >>> >>> >>> >>> conf.setMapperClass(BlurMapper.class); >>> >>> >>> >>> conf.setReducerClass(org.apache.hadoop.mapred.lib.IdentityReducer.class); >>> >>> client.setConf(conf); >>> try { >>> >>> >>> >>> >>> JobClient.runJob(conf); >>> } catch (Exception e) { >>> e.printStackTrace(); >>> } >>> } >>> >>> >>> >>> Thanks, >>> >>> Regards, >>> >>> Suhail Rehman >>> MS by Research in Computer Science >>> International Institute of Information Technology - Hyderabad >>> reh...@research.iiit.ac.in >>> --------------------------------------------------------------------- >>> http://research.iiit.ac.in/~rehman<http://research.iiit.ac.in/%7Erehman> >>> >> >> >> >> -- >> Best Regards >> >> Jeff Zhang >> > > > > -- > Regards, > > Suhail Rehman > MS by Research in Computer Science > International Institute of Information Technology - Hyderabad > reh...@research.iiit.ac.in > --------------------------------------------------------------------- > http://research.iiit.ac.in/~rehman <http://research.iiit.ac.in/%7Erehman> > -- Regards, Suhail Rehman MS by Research in Computer Science International Institute of Information Technology - Hyderabad reh...@research.iiit.ac.in --------------------------------------------------------------------- http://research.iiit.ac.in/~rehman <http://research.iiit.ac.in/%7Erehman>
