the issue is that I have protobuf-2.6.1 and I believe I modified the pom.xml under apache-tez-0.8.3-src and changed it from 2.5.0 to 2.6.1.
Now I have to install protobuf-2.5.0 and It is not working. Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* http://talebzadehmich.wordpress.com On 20 May 2016 at 21:46, Mich Talebzadeh <[email protected]> wrote: > sounds like this may ve the issue > > java.lang.NoSuchMethodError: org.apache.hadoop.*yarn.proto.YarnProtos$* > ApplicationIdProto.hashLong(J)I > > Dr Mich Talebzadeh > > > > LinkedIn * > https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* > > > > http://talebzadehmich.wordpress.com > > > > On 20 May 2016 at 19:28, Mich Talebzadeh <[email protected]> > wrote: > >> Hi Hitesh, >> >> This is the content of tez-site.xml] >> >> hduser@rhes564: /home/hduser/hadoop-2.6.0/etc/hadoop> cat tez-site.xml >> <configuration> >> <property> >> <name>tez.version</name> >> <value>0.8.3</value> >> </property> >> <property> >> <name>tez.lib.uris</name> >> <value>/usr/lib/tez-0.8.3,/usr/lib/tez-0.8.3/lib/</value> >> </property> >> </configuration> >> >> >> >> This is what is in hdfs >> >> drwxr-xr-x - hduser supergroup 0 2016-05-20 19:25 >> /usr/lib/tez-0.8.3/lib >> -rw-r--r-- 2 hduser supergroup 42821282 2016-05-20 09:03 >> /usr/lib/tez-0.8.3/tez-0.8.3.tar.gz >> >> >> Tez is installed in >> >> cd $TEZ_HOME >> hduser@rhes564: /usr/lib/tez-0.8.3> >> >> tez-site.xml is soft linked in $TEZ_HOME as follows >> >> hduser@rhes564: /usr/lib/tez-0.8.3> ls -l tez-site.xml >> lrwxrwxrwx 1 hduser hadoop 49 May 20 19:22 tez-site.xml -> >> /home/hduser/hadoop-2.6.0/etc/hadoop/tez-site.xml >> >> This is error >> >> Application application_1463758195355_0004 failed 2 times due to Error >> launching appattempt_1463758195355_0004_000002. Got exception: >> org.apache.hadoop.ipc.RemoteException(java.lang.NoSuchMethodError): >> org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto.hashLong(J)I >> at >> org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto.hashCode(YarnProtos.java:2616) >> at >> org.apache.hadoop.yarn.proto.YarnProtos$ApplicationAttemptIdProto.hashCode(YarnProtos.java:3154) >> at >> org.apache.hadoop.yarn.proto.YarnSecurityTokenProtos$NMTokenIdentifierProto.hashCode(YarnSecurityTokenProtos.java:410) >> at >> org.apache.hadoop.yarn.security.NMTokenIdentifier.hashCode(NMTokenIdentifier.java:126) >> at java.util.HashMap.hash(HashMap.java:338) >> at java.util.HashMap.put(HashMap.java:611) >> at java.util.HashSet.add(HashSet.java:219) >> at javax.security.auth.Subject$ClassSet.populateSet(Subject.java:1409) >> at javax.security.auth.Subject$ClassSet.<init>(Subject.java:1369) >> at javax.security.auth.Subject.getPublicCredentials(Subject.java:720) >> at >> org.apache.hadoop.security.UserGroupInformation.getTokenIdentifiers(UserGroupInformation.java:1400) >> at >> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.selectNMTokenIdentifier(ContainerManagerImpl.java:618) >> at >> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.startContainers(ContainerManagerImpl.java:699) >> at >> org.apache.hadoop.yarn.api.impl.pb.service.ContainerManagementProtocolPBServiceImpl.startContainers(ContainerManagementProtocolPBServiceImpl.java:60) >> at >> org.apache.hadoop.yarn.proto.ContainerManagementProtocol$ContainerManagementProtocolService$2.callBlockingMethod(ContainerManagementProtocol.java:95) >> at >> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619) >> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962) >> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2039) >> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2035) >> at java.security.AccessController.doPrivileged(Native Method) >> at javax.security.auth.Subject.doAs(Subject.java:422) >> at >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) >> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2033) >> at org.apache.hadoop.ipc.Client.call(Client.java:1468) >> at org.apache.hadoop.ipc.Client.call(Client.java:1399) >> at >> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) >> at com.sun.proxy.$Proxy80.startContainers(Unknown Source) >> at >> org.apache.hadoop.yarn.api.impl.pb.client.ContainerManagementProtocolPBClientImpl.startContainers(ContainerManagementProtocolPBClientImpl.java:96) >> at >> org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.launch(AMLauncher.java:119) >> at >> org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.run(AMLauncher.java:254) >> at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) >> at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) >> at java.lang.Thread.run(Thread.java:745) >> . Failing the application. >> Thanks >> >> >> >> >> >> Dr Mich Talebzadeh >> >> >> >> LinkedIn * >> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw >> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* >> >> >> >> http://talebzadehmich.wordpress.com >> >> >> >> On 20 May 2016 at 19:06, Hitesh Shah <[email protected]> wrote: >> >>> Logs from `bin/yarn logs -applicationId application_1463758195355_0002` >>> would be more useful to debug your setup issue. The RM logs usually do not >>> shed much light on why an application failed. >>> Can you confirm that you configured tez.lib.uris correctly to point to >>> the tez tarball on HDFS (tez tar should be the one obtained from >>> tez-dist/target/tez-0.8.3.tar.gz) ? >>> >>> — Hitesh >>> >>> > On May 20, 2016, at 10:24 AM, Mich Talebzadeh < >>> [email protected]> wrote: >>> > >>> > Hi, >>> > >>> > I have just compiled and installed TEZ, trying to do a test with >>> > >>> > hadoop jar ./tez-examples-0.8.3.jar orderedwordcount >>> /tmp/input/test.txt /tmp/out >>> > >>> > The job fails as follows. This is from yarn log >>> > >>> > 2016-05-20 18:19:26,945 INFO >>> SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for >>> appattempt_1463758195355_0002_000001 (auth:SIMPLE) >>> > 2016-05-20 18:19:26,950 WARN org.apache.hadoop.ipc.Server: IPC Server >>> handler 0 on 59093, call >>> org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers >>> from 50.140.197.217:46784 Call#2 Retry#0 >>> > java.lang.NoSuchMethodError: >>> org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto.hashLong(J)I >>> > at >>> org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto.hashCode(YarnProtos.java:2616) >>> > at >>> org.apache.hadoop.yarn.proto.YarnProtos$ApplicationAttemptIdProto.hashCode(YarnProtos.java:3154) >>> > at >>> org.apache.hadoop.yarn.proto.YarnSecurityTokenProtos$NMTokenIdentifierProto.hashCode(YarnSecurityTokenProtos.java:410) >>> > at >>> org.apache.hadoop.yarn.security.NMTokenIdentifier.hashCode(NMTokenIdentifier.java:126) >>> > at java.util.HashMap.hash(HashMap.java:338) >>> > at java.util.HashMap.put(HashMap.java:611) >>> > at java.util.HashSet.add(HashSet.java:219) >>> > at >>> javax.security.auth.Subject$ClassSet.populateSet(Subject.java:1409) >>> > at >>> javax.security.auth.Subject$ClassSet.<init>(Subject.java:1369) >>> > at >>> javax.security.auth.Subject.getPublicCredentials(Subject.java:720) >>> > at >>> org.apache.hadoop.security.UserGroupInformation.getTokenIdentifiers(UserGroupInformation.java:1400) >>> > at >>> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.selectNMTokenIdentifier(ContainerManagerImpl.java:618) >>> > at >>> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.startContainers(ContainerManagerImpl.java:699) >>> > at >>> org.apache.hadoop.yarn.api.impl.pb.service.ContainerManagementProtocolPBServiceImpl.startContainers(ContainerManagementProtocolPBServiceImpl.java:60) >>> > at >>> org.apache.hadoop.yarn.proto.ContainerManagementProtocol$ContainerManagementProtocolService$2.callBlockingMethod(ContainerManagementProtocol.java:95) >>> > at >>> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619) >>> > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962) >>> > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2039) >>> > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2035) >>> > at java.security.AccessController.doPrivileged(Native Method) >>> > at javax.security.auth.Subject.doAs(Subject.java:422) >>> > at >>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) >>> > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2033) >>> > 2016-05-20 18:19:27,929 WARN >>> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: >>> Event EventType: KILL_CONTAINER sent to absent container >>> container_1463758195355_0002_01_000001 >>> > 2016-05-20 18:19:27,944 INFO >>> SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for >>> appattempt_1463758195355_0002_000002 (auth:SIMPLE) >>> > 2016-05-20 18:19:27,949 WARN org.apache.hadoop.ipc.Server: IPC Server >>> handler 0 on 59093, call >>> org.apache.hadoop.yarn.api.ContainerManagementProtocolPB.startContainers >>> from 50.140.197.217:46785 Call#3 Retry#0 >>> > java.lang.NoSuchMethodError: >>> org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto.hashLong(J)I >>> > at >>> org.apache.hadoop.yarn.proto.YarnProtos$ApplicationIdProto.hashCode(YarnProtos.java:2616) >>> > at >>> org.apache.hadoop.yarn.proto.YarnProtos$ApplicationAttemptIdProto.hashCode(YarnProtos.java:3154) >>> > at >>> org.apache.hadoop.yarn.proto.YarnSecurityTokenProtos$NMTokenIdentifierProto.hashCode(YarnSecurityTokenProtos.java:410) >>> > at >>> org.apache.hadoop.yarn.security.NMTokenIdentifier.hashCode(NMTokenIdentifier.java:126) >>> > at java.util.HashMap.hash(HashMap.java:338) >>> > at java.util.HashMap.put(HashMap.java:611) >>> > at java.util.HashSet.add(HashSet.java:219) >>> > at >>> javax.security.auth.Subject$ClassSet.populateSet(Subject.java:1409) >>> > at >>> javax.security.auth.Subject$ClassSet.<init>(Subject.java:1369) >>> > at >>> javax.security.auth.Subject.getPublicCredentials(Subject.java:720) >>> > at >>> org.apache.hadoop.security.UserGroupInformation.getTokenIdentifiers(UserGroupInformation.java:1400) >>> > at >>> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.selectNMTokenIdentifier(ContainerManagerImpl.java:618) >>> > at >>> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.startContainers(ContainerManagerImpl.java:699) >>> > at >>> org.apache.hadoop.yarn.api.impl.pb.service.ContainerManagementProtocolPBServiceImpl.startContainers(ContainerManagementProtocolPBServiceImpl.java:60) >>> > at >>> org.apache.hadoop.yarn.proto.ContainerManagementProtocol$ContainerManagementProtocolService$2.callBlockingMethod(ContainerManagementProtocol.java:95) >>> > at >>> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619) >>> > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:962) >>> > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2039) >>> > at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2035) >>> > at java.security.AccessController.doPrivileged(Native Method) >>> > at javax.security.auth.Subject.doAs(Subject.java:422) >>> > at >>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628) >>> > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2033) >>> > 2016-05-20 18:19:28,931 WARN >>> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: >>> Event EventType: KILL_CONTAINER sent to absent container >>> container_1463758195355_0002_02_000001 >>> > 2016-05-20 18:19:28,932 WARN >>> org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: >>> Event EventType: FINISH_APPLICATION sent to absent application >>> application_1463758195355_0002 >>> > >>> > Any ideas will be appreciated! >>> > >>> > Thanks >>> > >>> > Dr Mich Talebzadeh >>> > >>> > LinkedIn >>> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw >>> > >>> > http://talebzadehmich.wordpress.com >>> > >>> >>> >> >
