Can you provide the full stack trace?
Are you getting the exception on the client (while submitting the job) or in the cluster (after the job started to run)? For the client side, the fix would be to add tachyon jars to the client classpath. Looks like you tried some client side classpath fixes. You could run ‘hadoop classpath’ to print the classpath being picked up by the ‘hadoop jar’ command. And scan its output to check if your tachyon jars are being picked up correctly or not. Bikas From: Jiří Šimša [mailto:[email protected]] Sent: Wednesday, November 11, 2015 6:54 PM To: [email protected] Subject: Running Tez with Tachyon Hello, I have followed the Tez installation instructions (https://tez.apache.org/install.html) and was able to successfully run the ordered word count example: $ hadoop jar ./tez-examples/target/tez-examples-0.8.2-SNAPSHOT.jar orderedwordcount /input.txt /output.txt Next, I wanted to see if I can do the same, this time reading from and writing to Tachyon (http://tachyon-project.org/) using: $ hadoop jar ./tez-examples/target/tez-examples-0.8.2-SNAPSHOT.jar orderedwordcount tachyon://localhost:19998/input.txt tachyon://localhost:19998/output.txt Unsurprisingly, this resulted in the "Class tachyon.hadoop.TFS not found" error because Tez needs the Tachyon client jar that defines the tachyon.hadoop.TFS class. To that end, I have tried several options (listed below) to provide this jar to Tez, none of which seems to have worked: 1) Adding the Tachyon client jar to HADOOP_CLASSPATH 2) Specifying the Tachyon client jar with the -libjars flag for the above command. 3) Copying the Tachyon client jar into the $HADOOP_HOME/share/hadoop/common/lib directory of my HADOOP installation. 4) Copying the Tachyon client jar into HDFS and specifying a path to it through the tez.aux.uris property in the tez-site.xml file (in a similar fashion the tez.lib.uris property specifies the path to the Tez tarball). 5) I modified the source code of the ordered word count example, adding a call to TezClient#addAppMasterLocalFiles(...), providing a URI for the Tachyon client jar uploaded to HDFS. Any advice on how to pass the Tachyon client jar to Tez to resolve this issue would be greatly appreciated. Thank you. Best, -- Jiří Šimša
