here's the info i have where i'm obviously missing something. a) i've read and re-read the tez install instructions. still no dice. b) we're running yarn. c) we're also running pepperdata. d) trying to get tez 0.8.4 working. e) i think the client side is pretty easy. get the tez jars in the classpath.
however on the server side i've hit a brick wall. from the datanode log: # i think this looks good. 2016-09-17 15:56:37,483 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.LocalizedResource: Resource *hdfs://dwrdevnn1.sv2.trulia.com:8020/apps/tez-0.8.4/apache-tez-0.8.4-bin.tar.gz <http://dwrdevnn1.sv2.trulia.com:8020/apps/tez-0.8.4/apache-tez-0.8.4-bin.tar.gz>*(->/storage4/hadoop/yarn/local/filecache/10/apache-tez-0.8.4-bin.tar.gz) transitioned from DOWNLOADING to LOCALIZED 2016-09-17 15:56:37,484 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Container container_1474152944798_0001_02_000001 transitioned from LOCALIZING to LOCALIZED 2016-09-17 15:56:37,509 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Container container_1474152944798_0001_02_000001 transitioned from LOCALIZED to RUNNING # here's the command/container that gets run Command $JAVA_HOME/bin/java -Xmx819m -Djava.io.tmpdir=$PWD/tmp -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -XX:+PrintGCDetails -verbose:gc -XX:+PrintGCTimeStamps -XX:+UseNUMA -XX:+UseG1GC -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp/ -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=/storage5/hadoop/yarn/logs/application_1474152944798_0001/container_1474152944798_0001_02_000001 -Dtez.root.logger=WARN,CLA -Dsun.nio.ch.bugLevel='' *org.apache.tez.dag.app.DAGAppMaster* --session 1>/storage5/hadoop/yarn/logs/application_1474152944798_0001/container_1474152944798_0001_02_000001/stdout 2>/storage5/hadoop/yarn/logs/application_1474152944798_0001/container_1474152944798_0001_02_000001/stderr doesn't contain classnames [org.apache.hadoop.mapred.YarnChild, org.apache.hadoop.mapreduce.v2.app.MRAppMaster] (Future similar messages will be omitted up to 1000 times) # and boom 2016-09-17 15:56:37,514 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: launchContainer: [bash, /storage4/hadoop/yarn/local/usercache/dwr/appcache/application_1474152944798_0001/container_1474152944798_0001_02_000001/default_container_executor.sh] 2016-09-17 15:56:37,745 WARN org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Exit code from container container_1474152944798_0001_02_000001 is : 1 # and from stderr from the datanode: Error: Could not find or load main class org.apache.tez.dag.app.DAGAppMaster so i ask why can't the container find DAGAppMaster from the tarball in the first step? Thanks, Stephen. PS I'm sure its something dumb but i just can't grok it.
