The "FileNotFoundException" doesn't mean anything in the pi program. If you have some error and the program didn't run successfully, it will always throw this exception. What do you have in the opts?
Regards, *Stanley Shi,* On Mon, May 12, 2014 at 2:09 PM, Tao Xiao <xiaotao.cs....@gmail.com> wrote: > The *FileNotFoundException* was thrown when I tried to submit a job > calculating PI, actually there is no such exception thrown when I submit a > wordcount job, but I can still see "Exception from container-launch... " > and any other jobs would throw such exceptions. > > Every job runs successfully when I commented out properties > *mapreduce.map.java.opts* > and *mapreduce.reduce.java.opts.* > > Indeed sounds odd, but I think maybe it is because that these two > properties conflict with other memory-related properties, so the container > can not be launched. > > > 2014-05-12 3:37 GMT+08:00 Jay Vyas <jayunit...@gmail.com>: > > Sounds odd....So (1) you got a filenotfound exception and (2) you fixed it >> by commenting out memory specific config parameters? >> >> Not sure how that would work... Any other details or am I missing >> something else? >> >> On May 11, 2014, at 4:16 AM, Tao Xiao <xiaotao.cs....@gmail.com> wrote: >> >> I'm sure this problem is caused by the incorrect configuration. I >> commented out all the configurations regarding memory, then jobs can run >> successfully. >> >> >> 2014-05-11 0:01 GMT+08:00 Tao Xiao <xiaotao.cs....@gmail.com>: >> >>> I installed Hadoop-2.2 in a cluster of 4 nodes, following Hadoop YARN >>> Installation: The definitive >>> guide<http://www.alexjf.net/blog/distributed-systems/hadoop-yarn-installation-definitive-guide> >>> . >>> >>> The configurations are as follows: >>> >>> ~/.bashrc <http://pastebin.com/zQgwuQv2> >>> core-site.xml<http://pastebin.com/rBAaqZps> >>> hdfs-site.xml <http://pastebin.com/bxazvp2G> >>> mapred-site.xml >>> <http://pastebin.com/N00SsMbz> slaves<http://pastebin.com/8VjsZ1uu> >>> yarn-site.xml <http://pastebin.com/XwLQZTQb> >>> >>> >>> I started NameNode, DataNodes, ResourceManager and NodeManagers >>> successfully, but no job can run successfully. For example, I run the >>> following job: >>> >>> [root@Single-Hadoop ~]# yarn jar >>> /var/soft/apache/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar >>> pi 2 4 >>> >>> The output is as follows: >>> >>> 14/05/10 23:56:25 INFO mapreduce.Job: Task Id : >>> attempt_1399733823963_0004_m_000000_0, Status : FAILED >>> Exception from container-launch: >>> org.apache.hadoop.util.Shell$ExitCodeException: >>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:464) >>> at org.apache.hadoop.util.Shell.run(Shell.java:379) >>> at >>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589) >>> at >>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195) >>> at >>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:283) >>> at >>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:79) >>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) >>> at java.util.concurrent.FutureTask.run(FutureTask.java:138) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) >>> at java.lang.Thread.run(Thread.java:662) >>> >>> >>> >>> 14/05/10 23:56:25 INFO mapreduce.Job: Task Id : >>> attempt_1399733823963_0004_m_000001_0, Status : FAILED >>> Exception from container-launch: >>> org.apache.hadoop.util.Shell$ExitCodeException: >>> at org.apache.hadoop.util.Shell.runCommand(Shell.java:464) >>> at org.apache.hadoop.util.Shell.run(Shell.java:379) >>> at >>> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589) >>> at >>> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195) >>> at >>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:283) >>> at >>> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:79) >>> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) >>> at java.util.concurrent.FutureTask.run(FutureTask.java:138) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) >>> at java.lang.Thread.run(Thread.java:662) >>> >>> ... ... >>> >>> >>> 14/05/10 23:56:36 INFO mapreduce.Job: map 100% reduce 100% >>> 14/05/10 23:56:37 INFO mapreduce.Job: Job job_1399733823963_0004 failed >>> with state FAILED due to: Task failed task_1399733823963_0004_m_000000 >>> Job failed as tasks failed. failedMaps:1 failedReduces:0 >>> >>> 14/05/10 23:56:37 INFO mapreduce.Job: Counters: 10 >>> Job Counters >>> Failed map tasks=7 >>> Killed map tasks=1 >>> Launched map tasks=8 >>> Other local map tasks=6 >>> Data-local map tasks=2 >>> Total time spent by all maps in occupied slots (ms)=21602 >>> Total time spent by all reduces in occupied slots (ms)=0 >>> Map-Reduce Framework >>> CPU time spent (ms)=0 >>> Physical memory (bytes) snapshot=0 >>> Virtual memory (bytes) snapshot=0 >>> Job Finished in 24.515 seconds >>> java.io.FileNotFoundException: File does not exist: hdfs:// >>> Single-Hadoop.zd.com/user/root/QuasiMonteCarlo_1399737371038_1022927375/out/reduce-out >>> at >>> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1110) >>> at >>> org.apache.hadoop.hdfs.DistributedFileSystem$17.doCall(DistributedFileSystem.java:1102) >>> at >>> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) >>> at >>> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1102) >>> at >>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1749) >>> at >>> org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1773) >>> at >>> org.apache.hadoop.examples.QuasiMonteCarlo.estimatePi(QuasiMonteCarlo.java:314) >>> at >>> org.apache.hadoop.examples.QuasiMonteCarlo.run(QuasiMonteCarlo.java:354) >>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) >>> at >>> org.apache.hadoop.examples.QuasiMonteCarlo.main(QuasiMonteCarlo.java:363) >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >>> at java.lang.reflect.Method.invoke(Method.java:597) >>> at >>> org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72) >>> at org.apache.hadoop.util.ProgramDriver.run(ProgramDriver.java:144) >>> at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:74) >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >>> at java.lang.reflect.Method.invoke(Method.java:597) >>> at org.apache.hadoop.util.RunJar.main(RunJar.java:212) >>> >>> >>> >>> Why would any job fail? Is it because the configurations are not >>> correct? >>> >> >> >
