The problem is that I d'ont see my job on the JobTracker page. It's like the job don't even start ! Is there a way to improve log level to get more information on the error ?
2013/5/12 Josh Wills <[email protected]> > Something probably failed in the MapReduce job itself, which meant that > there weren't any outputs for Crunch to move around. What do the error logs > for the individual tasks look like on the JobTracker status page(s)? > > > On Sat, May 11, 2013 at 5:02 PM, Quentin Ambard > <[email protected]>wrote: > >> Hi, >> I'm running a simple job on hadoop cdh 4.1.2 based on crunch. >> The job is quite simple : it scan a hbase table, extract some data from >> each entry of hbase, group the result by key and combine them using an >> aggreator, then write it back to another hbase table. >> It works fine on my computer, however when I try to launch it on my >> hadoop cluster I get the following : >> >> >>hadoop jar uber-crunch-1.0-SNAPSHOT.jar description >> /home/quentin/default.properties >> 13/05/12 01:57:50 INFO support.ClassPathXmlApplicationContext: Refreshing >> org.springframework.context.support.ClassPathXmlApplicationContext@1f4384c2: >> startup date [Sun May 12 01:57:50 CEST 2013]; root of context hierarchy >> 13/05/12 01:57:50 INFO xml.XmlBeanDefinitionReader: Loading XML bean >> definitions from class path resource [context/job-description-context.xml] >> 13/05/12 01:57:50 INFO xml.XmlBeanDefinitionReader: Loading XML bean >> definitions from class path resource [context/default-batch-context.xml] >> 13/05/12 01:57:51 INFO annotation.ClassPathBeanDefinitionScanner: JSR-330 >> 'javax.inject.Named' annotation found and supported for component scanning >> 13/05/12 01:57:51 INFO config.PropertyPlaceholderConfigurer: Loading >> properties file from URL >> [file:/tmp/hadoop-hdfs/hadoop-unjar7637839123250781784/default.properties] >> 13/05/12 01:57:51 INFO config.PropertyPlaceholderConfigurer: Loading >> properties file from URL >> [jar:file:/home/quentin/uber-crunch-1.0-SNAPSHOT.jar!/default.properties] >> 13/05/12 01:57:51 INFO config.PropertyPlaceholderConfigurer: Loading >> properties file from URL [file:/home/quentin/default.properties] >> 13/05/12 01:57:51 INFO annotation.AutowiredAnnotationBeanPostProcessor: >> JSR-330 'javax.inject.Inject' annotation found and supported for autowiring >> 13/05/12 01:57:51 INFO support.DefaultListableBeanFactory: >> Pre-instantiating singletons in >> org.springframework.beans.factory.support.DefaultListableBeanFactory@5b7b0998: >> defining beans >> [org.springframework.beans.factory.config.PropertyPlaceholderConfigurer#0,applicationContextHolder,descriptionLauncher,descriptionExtractor,emailExtractor,rawTextExtractor,keywordsExtractor,org.springframework.context.annotation.internalConfigurationAnnotationProcessor,org.springframework.context.annotation.internalAutowiredAnnotationProcessor,org.springframework.context.annotation.internalRequiredAnnotationProcessor,org.springframework.context.annotation.internalCommonAnnotationProcessor,org.springframework.context.annotation.ConfigurationClassPostProcessor$ImportAwareBeanPostProcessor#0]; >> root of factory hierarchy >> 13/05/12 01:57:52 INFO hbase.HBaseTarget: HBaseTarget ignores checks for >> existing outputs... >> 13/05/12 01:57:53 INFO collect.PGroupedTableImpl: Setting num reduce >> tasks to 2 >> 13/05/12 01:57:53 ERROR mr.MRPipeline: >> org.apache.crunch.CrunchRuntimeException: java.io.IOException: >> java.lang.RuntimeException: java.io.IOException: No such file or directory >> 13/05/12 01:57:53 WARN mr.MRPipeline: Not running cleanup while output >> targets remain >> >> Any idea of the origin of the problem ? Maybe it's something with >> permissions or a crunch tmp file, but I can't find out where it come from >> >> Thanks for your help >> >> >> Quentin >> > > > > -- > Director of Data Science > Cloudera <http://www.cloudera.com> > Twitter: @josh_wills <http://twitter.com/josh_wills> > -- Quentin Ambard
