Hello Sandeep, 2 things to check: - When compiling Tez, is the hadoop.version in the top-level pom ( and addition of mapr’s maven repo ) being used to compile against MapR’s hadoop distribution and not the std. apache release? The Tez AM cannot seem to do a handshake with the YARN RM. If MapR changed anything in that code path, this issue could crop up. - For tez.lib.uris, are you using the minimal tez tarball and not the full one? The full one has hadoop jars in it so it should not be used in your deployment setup as you are using MapR specific jars. You may want to check for jar clashes in terms of the content of the tarball vs the jars from the cluster. The only ones we do bundle in the minimal one are some mapreduce jars I believe.
FWIW, I think MapR does provide a Tez bundle for their clusters. You may wish to check on their user lists too to see if they made some changes. If you do find out anything, please do let us know in case this requires some fixes in Tez for better compatibility across all distros. thanks — Hitesh > On Jul 30, 2016, at 8:55 AM, Sandeep Khurana <skhurana...@gmail.com> wrote: > > Hello > > > On mapR single node cluster, I compiled Tez as per the documentation on > apache tez site. > > The only thing I changed from documentation was in tez-site.xml, where I gave > the value of tez.use.cluster.hadoop-libs as true. I did it because mapR has > its own jars for hadoop components. And without these tez might not work. > > When I run any MR job or hive query I get this below error in the job on RM > link. (DIGEST-MD5: digest response format violation. Mismatched URI: > default/; expecting: null/default) > > I am using mapr 4.1 and tez 0.7.0. mapR 4.1 uses hive 1.2.0. > > Any pointer or suggestion what might be causing this issue or where should I > look? > > 2016-07-30 09:58:38,737 INFO [main] ipc.Server: Stopping server on 45868 > 2016-07-30 09:58:38,737 INFO [IPC Server Responder] ipc.Server: Stopping IPC > Server Responder > 2016-07-30 09:58:38,739 INFO [IPC Server listener on 45868] ipc.Server: > Stopping IPC Server listener on 45868 > 2016-07-30 09:58:38,739 INFO [IPC Server Responder] ipc.Server: Stopping IPC > Server Responder > 2016-07-30 09:58:38,739 ERROR [main] app.DAGAppMaster: Error starting > DAGAppMaster > org.apache.tez.dag.api.TezUncheckedException: > javax.security.sasl.SaslException: DIGEST-MD5: digest response format > violation. Mismatched URI: default/; expecting: null/default [Caused by > org.apache.hadoop.ipc.RemoteException(javax.security.sasl.SaslException): > DIGEST-MD5: digest response format violation. Mismatched URI: default/; > expecting: null/default] > at > org.apache.tez.dag.app.rm.YarnTaskSchedulerService.serviceStart(YarnTaskSchedulerService.java:384) > at > org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > at > org.apache.tez.dag.app.rm.TaskSchedulerEventHandler.serviceStart(TaskSchedulerEventHandler.java:353) > at > org.apache.hadoop.service.AbstractService.start(AbstractService.java:193) > at > org.apache.tez.dag.app.DAGAppMaster$ServiceWithDependency.start(DAGAppMaster.java:1573) > at > org.apache.tez.dag.app.DAGAppMaster$ServiceThread.run(DAGAppMaster.java:1591) > Caused by: javax.security.sasl.SaslException: DIGEST-MD5: digest response > format violation. Mismatched URI: default/; expecting: null/default [Caused > by org.apache.hadoop.ipc.RemoteException(javax.security.sasl.SaslException): > DIGEST-MD5: digest response format violation. Mismatched URI: default/; > expecting: null/default] > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:526) > at > org.apache.hadoop.yarn.ipc.RPCUtil.instantiateException(RPCUtil.java:53) > at > org.apache.hadoop.yarn.ipc.RPCUtil.unwrapAndThrowException(RPCUtil.java:104) > at > org.apache.hadoop.yarn.api.impl.pb.client.ApplicationMasterProtocolPBClientImpl.registerApplicationMaster(ApplicationMasterProtocolPBClientImpl.java:109) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187) > at > org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) > at com.sun.proxy.$Proxy24.registerApplicationMaster(Unknown Source) > at > org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.registerApplicationMaster(AMRMClientImpl.java:222) > at > org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.registerApplicationMaster(AMRMClientImpl.java:214) > at > org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.registerApplicationMaster(AMRMClientAsyncImpl.java:138) > at > org.apache.tez.dag.app.rm.YarnTaskSchedulerService.serviceStart(YarnTaskSchedulerService.java:368) > ... 5 more > Caused by: > org.apache.hadoop.ipc.RemoteException(javax.security.sasl.SaslException): > DIGEST-MD5: digest response format violation. Mismatched URI: default/; > expecting: null/default > at org.apache.hadoop.ipc.Client.call(Client.java:1468) > at org.apache.hadoop.ipc.Client.call(Client.java:1399) > at > org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:232) > at com.sun.proxy.$Proxy23.registerApplicationMaster(Unknown Source) > at > org.apache.hadoop.yarn.api.impl.pb.client.ApplicationMasterProtocolPBClientImpl.registerApplicationMaster(ApplicationMasterProtocolPBClientImpl.java:106) > ... 16 more > 2016-07-30 09:58:38,741 INFO [Thread-2] app.DAGAppMaster: > DAGAppMasterShutdownHook invoked > > > -- > Thanks and regards > Sandeep Khurana
