Hi, I'm running a simple job on hadoop cdh 4.1.2 based on crunch. The job is quite simple : it scan a hbase table, extract some data from each entry of hbase, group the result by key and combine them using an aggreator, then write it back to another hbase table. It works fine on my computer, however when I try to launch it on my hadoop cluster I get the following :
>>hadoop jar uber-crunch-1.0-SNAPSHOT.jar description /home/quentin/default.properties 13/05/12 01:57:50 INFO support.ClassPathXmlApplicationContext: Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@1f4384c2: startup date [Sun May 12 01:57:50 CEST 2013]; root of context hierarchy 13/05/12 01:57:50 INFO xml.XmlBeanDefinitionReader: Loading XML bean definitions from class path resource [context/job-description-context.xml] 13/05/12 01:57:50 INFO xml.XmlBeanDefinitionReader: Loading XML bean definitions from class path resource [context/default-batch-context.xml] 13/05/12 01:57:51 INFO annotation.ClassPathBeanDefinitionScanner: JSR-330 'javax.inject.Named' annotation found and supported for component scanning 13/05/12 01:57:51 INFO config.PropertyPlaceholderConfigurer: Loading properties file from URL [file:/tmp/hadoop-hdfs/hadoop-unjar7637839123250781784/default.properties] 13/05/12 01:57:51 INFO config.PropertyPlaceholderConfigurer: Loading properties file from URL [jar:file:/home/quentin/uber-crunch-1.0-SNAPSHOT.jar!/default.properties] 13/05/12 01:57:51 INFO config.PropertyPlaceholderConfigurer: Loading properties file from URL [file:/home/quentin/default.properties] 13/05/12 01:57:51 INFO annotation.AutowiredAnnotationBeanPostProcessor: JSR-330 'javax.inject.Inject' annotation found and supported for autowiring 13/05/12 01:57:51 INFO support.DefaultListableBeanFactory: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@5b7b0998: defining beans [org.springframework.beans.factory.config.PropertyPlaceholderConfigurer#0,applicationContextHolder,descriptionLauncher,descriptionExtractor,emailExtractor,rawTextExtractor,keywordsExtractor,org.springframework.context.annotation.internalConfigurationAnnotationProcessor,org.springframework.context.annotation.internalAutowiredAnnotationProcessor,org.springframework.context.annotation.internalRequiredAnnotationProcessor,org.springframework.context.annotation.internalCommonAnnotationProcessor,org.springframework.context.annotation.ConfigurationClassPostProcessor$ImportAwareBeanPostProcessor#0]; root of factory hierarchy 13/05/12 01:57:52 INFO hbase.HBaseTarget: HBaseTarget ignores checks for existing outputs... 13/05/12 01:57:53 INFO collect.PGroupedTableImpl: Setting num reduce tasks to 2 13/05/12 01:57:53 ERROR mr.MRPipeline: org.apache.crunch.CrunchRuntimeException: java.io.IOException: java.lang.RuntimeException: java.io.IOException: No such file or directory 13/05/12 01:57:53 WARN mr.MRPipeline: Not running cleanup while output targets remain Any idea of the origin of the problem ? Maybe it's something with permissions or a crunch tmp file, but I can't find out where it come from Thanks for your help Quentin
