Hello,

I have followed the Tez installation instructions (
https://tez.apache.org/install.html) and was able to successfully run the
ordered word count example:

$ hadoop jar ./tez-examples/target/tez-examples-0.8.2-SNAPSHOT.jar
orderedwordcount /input.txt /output.txt

Next, I wanted to see if I can do the same, this time reading from and
writing to Tachyon (http://tachyon-project.org/) using:

$ hadoop jar ./tez-examples/target/tez-examples-0.8.2-SNAPSHOT.jar
orderedwordcount tachyon://localhost:19998/input.txt
tachyon://localhost:19998/output.txt

Unsurprisingly, this resulted in the "Class tachyon.hadoop.TFS not found"
error because Tez needs the Tachyon client jar that defines the
tachyon.hadoop.TFS class. To that end, I have tried several options (listed
below) to provide this jar to Tez, none of which seems to have worked:

1) Adding the Tachyon client jar to HADOOP_CLASSPATH
2) Specifying the Tachyon client jar with the -libjars flag for the above
command.
3) Copying the Tachyon client jar into the
$HADOOP_HOME/share/hadoop/common/lib directory of my HADOOP installation.
4) Copying the Tachyon client jar into HDFS and specifying a path to it
through the tez.aux.uris property in the tez-site.xml file (in a similar
fashion the tez.lib.uris property specifies the path to the Tez tarball).
5) I modified the source code of the ordered word count example, adding a
call to TezClient#addAppMasterLocalFiles(...), providing a URI for the
Tachyon client jar uploaded to HDFS.

Any advice on how to pass the Tachyon client jar to Tez to resolve this
issue would be greatly appreciated. Thank you.

Best,

--
Jiří Šimša

Reply via email to