Hi Hitesh,
This is the content of tez-site.xml]
hduser@rhes564: /home/hduser/hadoop-2.6.0/etc/hadoop> cat tez-site.xml
<configuration>
<property>
<name>tez.version</name>
<value>0.8.3</value>
</property>
<property>
<name>tez.lib.uris</name>
<value>/usr/lib/tez-0.8.3,/usr/lib/tez-0.8.3/lib/</value>
</property>
</configuration>
This is what is in hdfs
drwxr-xr-x - hduser supergroup 0 2016-05-20 19:25
/usr/lib/tez-0.8.3/lib
-rw-r--r-- 2 hduser supergroup 42821282 2016-05-20 09:03
/usr/lib/tez-0.8.3/tez-0.8.3.tar.gz
Tez is installed in
cd $TEZ_HOME
hduser@rhes564: /usr/lib/tez-0.8.3>
tez-site.xml is soft linked in $TEZ_HOME as follows
hduser@rhes564: /usr/lib/tez-0.8.3> ls -l tez-site.xml
lrwxrwxrwx 1 hduser hadoop 49 May 20 19:22 tez-site.xml ->
/home/hduser/hadoop-2.6.0/etc/hadoop/tez-site.xml
This is error
Application application_1463758195355_0004 failed 2 times due to Error
launching appattempt_1463758195355_0004_000002. Got exception:
org.apache.hadoop.ipc.RemoteException(java.lang.NoSuchMethodError):
org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto.hashLong(J)I
at
org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto.hashCode(YarnProtos.java:2616)
at
org.apache.hadoop.yarn.proto.YarnProtos$ApplicationAttemptIdProto.hashCode(YarnProtos.java:3154)
at
org.apache.hadoop.yarn.proto.YarnSecurityTokenProtos$NMTokenIdentifierProto.hashCode(YarnSecurityTokenProtos.java:410)
at
org.apache.hadoop.yarn.security.NMTokenIdentifier.hashCode(NMTokenIdentifier.java:126)
at java.util.HashMap.hash(HashMap.java:338)
at java.util.HashMap.put(HashMap.java:611)
at java.util.HashSet.add(HashSet.java:219)
at javax.security.auth.Subject$ClassSet.populateSet(Subject.java:1409)
at javax.security.auth.Subject$ClassSet.<init>(Subject.java:1369)
at javax.security.auth.Subject.getPublicCredentials(Subject.java:720)
at
org.apache.hadoop.security.UserGroupInformation.getTokenIdentifiers(UserGroupInformation.java:1400)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.selectNMTokenIdentifier(ContainerManagerImpl.java:618)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.startContainers(ContainerManagerImpl.java:699)
at
org.apache.hadoop.yarn.api.impl.pb.service.ContainerManagementProtocolPBServiceImpl.startContainers(ContainerManagementProtocolPBServiceImpl.java:60)
at
org.apache.hadoop.yarn.proto.ContainerManagementProtocol$ContainerManagementProtocolService$2.callBlockingMethod(ContainerManagementProtocol.java:95)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2039)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2035)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2033)
at org.apache.hadoop.ipc.Client.call(Client.java:1468)
at org.apache.hadoop.ipc.Client.call(Client.java:1399)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232)
at com.sun.proxy.$Proxy80.startContainers(Unknown Source)
at
org.apache.hadoop.yarn.api.impl.pb.client.ContainerManagementProtocolPBClientImpl.startContainers(ContainerManagementProtocolPBClientImpl.java:96)
at
org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.launch(AMLauncher.java:119)
at
org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.run(AMLauncher.java:254)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
. Failing the application.
Thanks
Dr Mich Talebzadeh
LinkedIn *
https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
<https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>*
http://talebzadehmich.wordpress.com
On 20 May 2016 at 19:06, Hitesh Shah <[email protected]> wrote:
> Logs from `bin/yarn logs -applicationId application_1463758195355_0002`
> would be more useful to debug your setup issue. The RM logs usually do not
> shed much light on why an application failed.
> Can you confirm that you configured tez.lib.uris correctly to point to the
> tez tarball on HDFS (tez tar should be the one obtained from
> tez-dist/target/tez-0.8.3.tar.gz) ?
>
> — Hitesh
>
> > On May 20, 2016, at 10:24 AM, Mich Talebzadeh <[email protected]>
> wrote:
> >
> > Hi,
> >
> > I have just compiled and installed TEZ, trying to do a test with
> >
> > hadoop jar ./tez-examples-0.8.3.jar orderedwordcount /tmp/input/test.txt
> /tmp/out
> >
> > The job fails as follows. This is from yarn log
> >
> > 2016-05-20 18:19:26,945 INFO
> SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for
> appattempt_1463758195355_0002_000001 (auth:SIMPLE)
> > 2016-05-20 18:19:26,950 WARN org.apache.hadoop.ipc.Server: IPC Server
> handler 0 on 59093, call
> org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers
> from 50.140.197.217:46784 Call#2 Retry#0
> > java.lang.NoSuchMethodError:
> org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto.hashLong(J)I
> > at
> org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto.hashCode(YarnProtos.java:2616)
> > at
> org.apache.hadoop.yarn.proto.YarnProtos$ApplicationAttemptIdProto.hashCode(YarnProtos.java:3154)
> > at
> org.apache.hadoop.yarn.proto.YarnSecurityTokenProtos$NMTokenIdentifierProto.hashCode(YarnSecurityTokenProtos.java:410)
> > at
> org.apache.hadoop.yarn.security.NMTokenIdentifier.hashCode(NMTokenIdentifier.java:126)
> > at java.util.HashMap.hash(HashMap.java:338)
> > at java.util.HashMap.put(HashMap.java:611)
> > at java.util.HashSet.add(HashSet.java:219)
> > at
> javax.security.auth.Subject$ClassSet.populateSet(Subject.java:1409)
> > at javax.security.auth.Subject$ClassSet.<init>(Subject.java:1369)
> > at
> javax.security.auth.Subject.getPublicCredentials(Subject.java:720)
> > at
> org.apache.hadoop.security.UserGroupInformation.getTokenIdentifiers(UserGroupInformation.java:1400)
> > at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.selectNMTokenIdentifier(ContainerManagerImpl.java:618)
> > at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.startContainers(ContainerManagerImpl.java:699)
> > at
> org.apache.hadoop.yarn.api.impl.pb.service.ContainerManagementProtocolPBServiceImpl.startContainers(ContainerManagementProtocolPBServiceImpl.java:60)
> > at
> org.apache.hadoop.yarn.proto.ContainerManagementProtocol$ContainerManagementProtocolService$2.callBlockingMethod(ContainerManagementProtocol.java:95)
> > at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619)
> > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
> > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2039)
> > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2035)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at javax.security.auth.Subject.doAs(Subject.java:422)
> > at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2033)
> > 2016-05-20 18:19:27,929 WARN
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl:
> Event EventType: KILL_CONTAINER sent to absent container
> container_1463758195355_0002_01_000001
> > 2016-05-20 18:19:27,944 INFO
> SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for
> appattempt_1463758195355_0002_000002 (auth:SIMPLE)
> > 2016-05-20 18:19:27,949 WARN org.apache.hadoop.ipc.Server: IPC Server
> handler 0 on 59093, call
> org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers
> from 50.140.197.217:46785 Call#3 Retry#0
> > java.lang.NoSuchMethodError:
> org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto.hashLong(J)I
> > at
> org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto.hashCode(YarnProtos.java:2616)
> > at
> org.apache.hadoop.yarn.proto.YarnProtos$ApplicationAttemptIdProto.hashCode(YarnProtos.java:3154)
> > at
> org.apache.hadoop.yarn.proto.YarnSecurityTokenProtos$NMTokenIdentifierProto.hashCode(YarnSecurityTokenProtos.java:410)
> > at
> org.apache.hadoop.yarn.security.NMTokenIdentifier.hashCode(NMTokenIdentifier.java:126)
> > at java.util.HashMap.hash(HashMap.java:338)
> > at java.util.HashMap.put(HashMap.java:611)
> > at java.util.HashSet.add(HashSet.java:219)
> > at
> javax.security.auth.Subject$ClassSet.populateSet(Subject.java:1409)
> > at javax.security.auth.Subject$ClassSet.<init>(Subject.java:1369)
> > at
> javax.security.auth.Subject.getPublicCredentials(Subject.java:720)
> > at
> org.apache.hadoop.security.UserGroupInformation.getTokenIdentifiers(UserGroupInformation.java:1400)
> > at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.selectNMTokenIdentifier(ContainerManagerImpl.java:618)
> > at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.startContainers(ContainerManagerImpl.java:699)
> > at
> org.apache.hadoop.yarn.api.impl.pb.service.ContainerManagementProtocolPBServiceImpl.startContainers(ContainerManagementProtocolPBServiceImpl.java:60)
> > at
> org.apache.hadoop.yarn.proto.ContainerManagementProtocol$ContainerManagementProtocolService$2.callBlockingMethod(ContainerManagementProtocol.java:95)
> > at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619)
> > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962)
> > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2039)
> > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2035)
> > at java.security.AccessController.doPrivileged(Native Method)
> > at javax.security.auth.Subject.doAs(Subject.java:422)
> > at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
> > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2033)
> > 2016-05-20 18:19:28,931 WARN
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl:
> Event EventType: KILL_CONTAINER sent to absent container
> container_1463758195355_0002_02_000001
> > 2016-05-20 18:19:28,932 WARN
> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl:
> Event EventType: FINISH_APPLICATION sent to absent application
> application_1463758195355_0002
> >
> > Any ideas will be appreciated!
> >
> > Thanks
> >
> > Dr Mich Talebzadeh
> >
> > LinkedIn
> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw
> >
> > http://talebzadehmich.wordpress.com
> >
>
>
