After submit a mapreduce job, we get job status,But it tells the job failed! we check this application on RM website(xxx:8088/cluster/apps), we find those log :
Application application_1418904565842_3597024 failed 2 times due to AM Container for appattempt_1418904565842_3597024_000002 exited with exitCode: 1 due to: Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException: at org.apache.hadoop.util.Shell.runCommand(Shell.java:464) at org.apache.hadoop.util.Shell.run(Shell.java:379) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589) at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:252) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:283) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:79) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) at java.lang.Thread.run(Thread.java:662) main : command provided 1 we check yarn log with this command : yarn logs -applicationId application_1418904565842_3597024,get those log : Container: container_1418904565842_3597024_01_000001 on hadoop88.photo.163.org_56708 ====================================================================================== LogType: stderr LogLength: 664 Log Contents: Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/mapreduce/v2/app/MRAppMaster Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapreduce.v2.app.MRAppMaster at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) Could not find the main class: org.apache.hadoop.mapreduce.v2.app.MRAppMaster. Program will exit. I thinks this means the task can not find the jar files, So I upload all the jars that kylin dependent to HDFS, and before submit this job*(in AbstractHadoopJob.attachKylinPropsAndMetadata function )* I set "tmpjars" to those files located on HDFS(this way can avoid uploading all files when submit every mapreduce job). This measure works in kylin-0.7.2, I get the same error in kylin-1.0 and I guess this measure will work in kylin-1.0 too, But I do not think this is a good idea, It will be highly appreciated if you have some good idea or some suggestion. Thanks...
