BTW, in this job, I set the number of tasks to be 240, since I have 60 groomservers and each server has 4 cores. I also changed bsp.tasks.maximum to be 4 and bsp.max.tasks.per.job to be 300 in hama-default.xml. Why there are only 133 tasks, instead of 240?
Thanks! Yingyi On Fri, Sep 27, 2013 at 3:06 AM, Yingyi Bu <[email protected]> wrote: > It fails again: > > 13/09/27 09:49:12 INFO mortbay.log: Logging to > org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via > org.mortbay.log.Slf4jLog > 13/09/27 09:49:12 INFO bsp.FileInputFormat: Total input paths to process : > 2 > 13/09/27 09:49:12 WARN util.NativeCodeLoader: Unable to load native-hadoop > library for your platform... using builtin-java classes where applicable > 13/09/27 09:49:12 WARN snappy.LoadSnappy: Snappy native library not loaded > 13/09/27 09:49:12 INFO bsp.FileInputFormat: Total input paths to process : > 2 > 13/09/27 09:49:13 INFO bsp.BSPJobClient: Running job: job_201309270937_0004 > 13/09/27 09:49:16 INFO bsp.BSPJobClient: Current supersteps number: 0 > 13/09/27 09:53:09 INFO bsp.BSPJobClient: Job failed. > 13/09/27 09:53:09 ERROR bsp.BSPJobClient: Error partitioning the input > path. > > java.io.IOException: Runtime partition failed for the job. > at > org.apache.hama.bsp.BSPJobClient.partition(BSPJobClient.java:465) > at > org.apache.hama.bsp.BSPJobClient.submitJobInternal(BSPJobClient.java:333) > at > org.apache.hama.bsp.BSPJobClient.submitJob(BSPJobClient.java:293) > at org.apache.hama.bsp.BSPJob.submit(BSPJob.java:229) > at org.apache.hama.graph.GraphJob.submit(GraphJob.java:203) > at org.apache.hama.bsp.BSPJob.waitForCompletion(BSPJob.java:236) > at org.apache.hama.examples.PageRank2.main(PageRank2.java:162) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:601) > at > org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68) > at > org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139) > at > org.apache.hama.examples.ExampleDriver.main(ExampleDriver.java:43) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:601) > at org.apache.hama.util.RunJar.main(RunJar.java:146) > > I set the HAMA_LOG_DIR in conf/hama-env.sh to avoid logs going to NFS. > Just write a script to dump all tasklogs for the job. > Attached is the logs. But I cannot find any clue from the logs. > > Any help is appreciated. Thanks!! > > Yingyi > > > > On Fri, Sep 27, 2013 at 2:44 AM, Edward J. Yoon <[email protected]>wrote: > >> Hi, >> >> The task logs are in ${HAMA_HOME}/logs/tasklogs/. >> >> >> >> On Fri, Sep 27, 2013 at 6:12 PM, Yingyi Bu <[email protected]> wrote: >> > All the groomsevers' tasklogs? >> > >> > Yingyi >> > >> > >> > On Fri, Sep 27, 2013 at 2:11 AM, Edward J. Yoon <[email protected] >> >wrote: >> > >> >> Could you please check the tasklogs? >> >> >> >> On Fri, Sep 27, 2013 at 6:03 PM, Yingyi Bu <[email protected]> wrote: >> >> > Hi, >> >> > >> >> > I got a runtime partition exception as follows: >> >> > >> >> > 13/09/27 08:52:41 INFO mortbay.log: Logging to >> >> > org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via >> >> > org.mortbay.log.Slf4jLog >> >> > 13/09/27 08:52:42 INFO bsp.FileInputFormat: Total input paths to >> process >> >> : 2 >> >> > 13/09/27 08:52:42 WARN util.NativeCodeLoader: Unable to load >> >> native-hadoop >> >> > library for your platform... using builtin-java classes where >> applicable >> >> > 13/09/27 08:52:42 WARN snappy.LoadSnappy: Snappy native library not >> >> loaded >> >> > 13/09/27 08:52:42 INFO bsp.FileInputFormat: Total input paths to >> process >> >> : 2 >> >> > 13/09/27 08:52:42 INFO bsp.BSPJobClient: Running job: >> >> job_201309270852_0002 >> >> > 13/09/27 08:52:45 INFO bsp.BSPJobClient: Current supersteps number: 0 >> >> > 13/09/27 08:58:10 INFO bsp.BSPJobClient: Current supersteps number: 1 >> >> > 13/09/27 08:58:16 INFO bsp.BSPJobClient: Current supersteps number: 2 >> >> > 13/09/27 09:00:04 INFO bsp.BSPJobClient: Job failed. >> >> > 13/09/27 09:00:04 ERROR bsp.BSPJobClient: Error partitioning the >> input >> >> path. >> >> > java.io.IOException: Runtime partition failed for the job. >> >> > at >> >> org.apache.hama.bsp.BSPJobClient.partition(BSPJobClient.java:465) >> >> > at >> >> > >> org.apache.hama.bsp.BSPJobClient.submitJobInternal(BSPJobClient.java:333) >> >> > at >> >> org.apache.hama.bsp.BSPJobClient.submitJob(BSPJobClient.java:293) >> >> > at org.apache.hama.bsp.BSPJob.submit(BSPJob.java:229) >> >> > at org.apache.hama.graph.GraphJob.submit(GraphJob.java:203) >> >> > at >> org.apache.hama.bsp.BSPJob.waitForCompletion(BSPJob.java:236) >> >> > at >> org.apache.hama.examples.PageRank2.main(PageRank2.java:164) >> >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native >> Method) >> >> > at >> >> > >> >> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> >> > at >> >> > >> >> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> >> > at java.lang.reflect.Method.invoke(Method.java:601) >> >> > at >> >> > >> >> >> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68) >> >> > at >> >> > org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139) >> >> > at >> >> > org.apache.hama.examples.ExampleDriver.main(ExampleDriver.java:43) >> >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native >> Method) >> >> > at >> >> > >> >> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> >> > at >> >> > >> >> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> >> > at java.lang.reflect.Method.invoke(Method.java:601) >> >> > at org.apache.hama.util.RunJar.main(RunJar.java:146) >> >> > >> >> > Does anyone know what's the possible reason of this exception? >> >> > Thanks! >> >> > >> >> > Yingyi >> >> >> >> >> >> >> >> -- >> >> Best Regards, Edward J. Yoon >> >> @eddieyoon >> >> >> >> >> >> -- >> Best Regards, Edward J. Yoon >> @eddieyoon >> > >
