Hi all, I am the one trying to run Apache Beam example on cluster: I used the
following command with my given input in a folder named
"harrypotter":#!/bin/bash
HADOOP_CLASSPATH="$HADOOP_CLASSPATH:/tmp/beam/jars/*" hadoop jar
/tmp/beam/jars/beam-examples-java-2.1.0-SNAPSHOT.jar
org.apache.beam.examples.complete.TfIdf --runner=ApexRunner
--embeddedExecution=false --output=apexrunnertfidf
--input=/tmp/beam/harrypotter/
java -cp
/homes/org.apache.beam.examples.complete.TfIdf--------------------------------------------------------------------------
However, some configuration seems to go wrong:
Exception in thread "main" java.lang.RuntimeException: Failed to launch the
application on YARN. at
org.apache.beam.runners.apex.ApexRunner.run(ApexRunner.java:204) at
org.apache.beam.runners.apex.ApexRunner.run(ApexRunner.java:82) at
org.apache.beam.sdk.Pipeline.run(Pipeline.java:297) at
org.apache.beam.sdk.Pipeline.run(Pipeline.java:283) at
org.apache.beam.examples.complete.TfIdf.main(TfIdf.java:442) at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498) at
org.apache.hadoop.util.RunJar.run(RunJar.java:234) at
org.apache.hadoop.util.RunJar.main(RunJar.java:148)Caused by:
java.io.FileNotFoundException:hadoop/client/dfs.include (No such file or
directory) at java.io.FileInputStream.open0(Native Method) at
java.io.FileInputStream.open(FileInputStream.java:195) at
java.io.FileInputStream.<init>(FileInputStream.java:138) at
org.apache.commons.io.FileUtils.copyFile(FileUtils.java:1112) at
org.apache.beam.runners.apex.ApexYarnLauncher$2.visitFile(ApexYarnLauncher.java:277)
at
org.apache.beam.runners.apex.ApexYarnLauncher$2.visitFile(ApexYarnLauncher.java:253)
at java.nio.file.Files.walkFileTree(Files.java:2670) at
java.nio.file.Files.walkFileTree(Files.java:2742) at
org.apache.beam.runners.apex.ApexYarnLauncher.createJar(ApexYarnLauncher.java:253)
at
org.apache.beam.runners.apex.ApexYarnLauncher.launchApp(ApexYarnLauncher.java:90)
at org.apache.beam.runners.apex.ApexRunner.run(ApexRunner.java:201)
I checked the :hadoop/client/ folder and found that the dfs.include actually
exists.May any of you give solution to this?
Claire
