It does sound like a permission issue-- you can set the crunch.tmp.dir property on the commandline (assuming you're implementing the Tool interface) by setting -Dcrunch.tmp.dir=... to see if that helps.
On Mon, May 13, 2013 at 5:15 AM, Christian Tzolov <[email protected]> wrote: > You can try MRPipelien.enableDebug() to lower the log level. > > > On Mon, May 13, 2013 at 12:06 PM, Quentin Ambard <[email protected] > > wrote: > >> The problem is that I d'ont see my job on the JobTracker page. It's like >> the job don't even start ! >> Is there a way to improve log level to get more information on the error ? >> >> >> 2013/5/12 Josh Wills <[email protected]> >> >>> Something probably failed in the MapReduce job itself, which meant that >>> there weren't any outputs for Crunch to move around. What do the error logs >>> for the individual tasks look like on the JobTracker status page(s)? >>> >>> >>> On Sat, May 11, 2013 at 5:02 PM, Quentin Ambard < >>> [email protected]> wrote: >>> >>>> Hi, >>>> I'm running a simple job on hadoop cdh 4.1.2 based on crunch. >>>> The job is quite simple : it scan a hbase table, extract some data >>>> from each entry of hbase, group the result by key and combine them using an >>>> aggreator, then write it back to another hbase table. >>>> It works fine on my computer, however when I try to launch it on my >>>> hadoop cluster I get the following : >>>> >>>> >>hadoop jar uber-crunch-1.0-SNAPSHOT.jar description >>>> /home/quentin/default.properties >>>> 13/05/12 01:57:50 INFO support.ClassPathXmlApplicationContext: >>>> Refreshing >>>> org.springframework.context.support.ClassPathXmlApplicationContext@1f4384c2: >>>> startup date [Sun May 12 01:57:50 CEST 2013]; root of context hierarchy >>>> 13/05/12 01:57:50 INFO xml.XmlBeanDefinitionReader: Loading XML bean >>>> definitions from class path resource [context/job-description-context.xml] >>>> 13/05/12 01:57:50 INFO xml.XmlBeanDefinitionReader: Loading XML bean >>>> definitions from class path resource [context/default-batch-context.xml] >>>> 13/05/12 01:57:51 INFO annotation.ClassPathBeanDefinitionScanner: >>>> JSR-330 'javax.inject.Named' annotation found and supported for component >>>> scanning >>>> 13/05/12 01:57:51 INFO config.PropertyPlaceholderConfigurer: Loading >>>> properties file from URL >>>> [file:/tmp/hadoop-hdfs/hadoop-unjar7637839123250781784/default.properties] >>>> 13/05/12 01:57:51 INFO config.PropertyPlaceholderConfigurer: Loading >>>> properties file from URL >>>> [jar:file:/home/quentin/uber-crunch-1.0-SNAPSHOT.jar!/default.properties] >>>> 13/05/12 01:57:51 INFO config.PropertyPlaceholderConfigurer: Loading >>>> properties file from URL [file:/home/quentin/default.properties] >>>> 13/05/12 01:57:51 INFO annotation.AutowiredAnnotationBeanPostProcessor: >>>> JSR-330 'javax.inject.Inject' annotation found and supported for autowiring >>>> 13/05/12 01:57:51 INFO support.DefaultListableBeanFactory: >>>> Pre-instantiating singletons in >>>> org.springframework.beans.factory.support.DefaultListableBeanFactory@5b7b0998: >>>> defining beans >>>> [org.springframework.beans.factory.config.PropertyPlaceholderConfigurer#0,applicationContextHolder,descriptionLauncher,descriptionExtractor,emailExtractor,rawTextExtractor,keywordsExtractor,org.springframework.context.annotation.internalConfigurationAnnotationProcessor,org.springframework.context.annotation.internalAutowiredAnnotationProcessor,org.springframework.context.annotation.internalRequiredAnnotationProcessor,org.springframework.context.annotation.internalCommonAnnotationProcessor,org.springframework.context.annotation.ConfigurationClassPostProcessor$ImportAwareBeanPostProcessor#0]; >>>> root of factory hierarchy >>>> 13/05/12 01:57:52 INFO hbase.HBaseTarget: HBaseTarget ignores checks >>>> for existing outputs... >>>> 13/05/12 01:57:53 INFO collect.PGroupedTableImpl: Setting num reduce >>>> tasks to 2 >>>> 13/05/12 01:57:53 ERROR mr.MRPipeline: >>>> org.apache.crunch.CrunchRuntimeException: java.io.IOException: >>>> java.lang.RuntimeException: java.io.IOException: No such file or directory >>>> 13/05/12 01:57:53 WARN mr.MRPipeline: Not running cleanup while output >>>> targets remain >>>> >>>> Any idea of the origin of the problem ? Maybe it's something with >>>> permissions or a crunch tmp file, but I can't find out where it come from >>>> >>>> Thanks for your help >>>> >>>> >>>> Quentin >>>> >>> >>> >>> >>> -- >>> Director of Data Science >>> Cloudera <http://www.cloudera.com> >>> Twitter: @josh_wills <http://twitter.com/josh_wills> >>> >> >> >> >> -- >> Quentin Ambard >> > > -- Director of Data Science Cloudera <http://www.cloudera.com> Twitter: @josh_wills <http://twitter.com/josh_wills>
