You can try MRPipelien.enableDebug() to lower the log level.
On Mon, May 13, 2013 at 12:06 PM, Quentin Ambard <[email protected]>wrote: > The problem is that I d'ont see my job on the JobTracker page. It's like > the job don't even start ! > Is there a way to improve log level to get more information on the error ? > > > 2013/5/12 Josh Wills <[email protected]> > >> Something probably failed in the MapReduce job itself, which meant that >> there weren't any outputs for Crunch to move around. What do the error logs >> for the individual tasks look like on the JobTracker status page(s)? >> >> >> On Sat, May 11, 2013 at 5:02 PM, Quentin Ambard <[email protected] >> > wrote: >> >>> Hi, >>> I'm running a simple job on hadoop cdh 4.1.2 based on crunch. >>> The job is quite simple : it scan a hbase table, extract some data from >>> each entry of hbase, group the result by key and combine them using an >>> aggreator, then write it back to another hbase table. >>> It works fine on my computer, however when I try to launch it on my >>> hadoop cluster I get the following : >>> >>> >>hadoop jar uber-crunch-1.0-SNAPSHOT.jar description >>> /home/quentin/default.properties >>> 13/05/12 01:57:50 INFO support.ClassPathXmlApplicationContext: >>> Refreshing >>> org.springframework.context.support.ClassPathXmlApplicationContext@1f4384c2: >>> startup date [Sun May 12 01:57:50 CEST 2013]; root of context hierarchy >>> 13/05/12 01:57:50 INFO xml.XmlBeanDefinitionReader: Loading XML bean >>> definitions from class path resource [context/job-description-context.xml] >>> 13/05/12 01:57:50 INFO xml.XmlBeanDefinitionReader: Loading XML bean >>> definitions from class path resource [context/default-batch-context.xml] >>> 13/05/12 01:57:51 INFO annotation.ClassPathBeanDefinitionScanner: >>> JSR-330 'javax.inject.Named' annotation found and supported for component >>> scanning >>> 13/05/12 01:57:51 INFO config.PropertyPlaceholderConfigurer: Loading >>> properties file from URL >>> [file:/tmp/hadoop-hdfs/hadoop-unjar7637839123250781784/default.properties] >>> 13/05/12 01:57:51 INFO config.PropertyPlaceholderConfigurer: Loading >>> properties file from URL >>> [jar:file:/home/quentin/uber-crunch-1.0-SNAPSHOT.jar!/default.properties] >>> 13/05/12 01:57:51 INFO config.PropertyPlaceholderConfigurer: Loading >>> properties file from URL [file:/home/quentin/default.properties] >>> 13/05/12 01:57:51 INFO annotation.AutowiredAnnotationBeanPostProcessor: >>> JSR-330 'javax.inject.Inject' annotation found and supported for autowiring >>> 13/05/12 01:57:51 INFO support.DefaultListableBeanFactory: >>> Pre-instantiating singletons in >>> org.springframework.beans.factory.support.DefaultListableBeanFactory@5b7b0998: >>> defining beans >>> [org.springframework.beans.factory.config.PropertyPlaceholderConfigurer#0,applicationContextHolder,descriptionLauncher,descriptionExtractor,emailExtractor,rawTextExtractor,keywordsExtractor,org.springframework.context.annotation.internalConfigurationAnnotationProcessor,org.springframework.context.annotation.internalAutowiredAnnotationProcessor,org.springframework.context.annotation.internalRequiredAnnotationProcessor,org.springframework.context.annotation.internalCommonAnnotationProcessor,org.springframework.context.annotation.ConfigurationClassPostProcessor$ImportAwareBeanPostProcessor#0]; >>> root of factory hierarchy >>> 13/05/12 01:57:52 INFO hbase.HBaseTarget: HBaseTarget ignores checks for >>> existing outputs... >>> 13/05/12 01:57:53 INFO collect.PGroupedTableImpl: Setting num reduce >>> tasks to 2 >>> 13/05/12 01:57:53 ERROR mr.MRPipeline: >>> org.apache.crunch.CrunchRuntimeException: java.io.IOException: >>> java.lang.RuntimeException: java.io.IOException: No such file or directory >>> 13/05/12 01:57:53 WARN mr.MRPipeline: Not running cleanup while output >>> targets remain >>> >>> Any idea of the origin of the problem ? Maybe it's something with >>> permissions or a crunch tmp file, but I can't find out where it come from >>> >>> Thanks for your help >>> >>> >>> Quentin >>> >> >> >> >> -- >> Director of Data Science >> Cloudera <http://www.cloudera.com> >> Twitter: @josh_wills <http://twitter.com/josh_wills> >> > > > > -- > Quentin Ambard >
