Hey Hitesh,
so without chaining the hostname of the ec2 instances to their public dns the
log looks like:
2014-08-13 03:53:15,310 INFO [ServiceThread:DAGClientRPCServer]
org.apache.tez.dag.api.client.DAGClientServer: Instantiated DAGClientRPCServer
at domU-12-31-39-0F-30-03/10.193.51.241:31000
2014-08-13 03:53:15,332 INFO [IPC Server Responder]
org.apache.hadoop.ipc.Server: IPC Server Responder: starting
2014-08-13 03:53:15,336 INFO [IPC Server listener on 50192]
org.apache.hadoop.ipc.Server: IPC Server listener on 50192: starting
and the exception on the client is then:
com.google.protobuf.ServiceException: java.net.UnknownHostException: Invalid
host name: local host is: (unknown); destination host is:
"domU-12-31-39-0F-30-03":31000; java.net.UnknownHostException; For more details
see: http://wiki.apache.org/hadoop/UnknownHost
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:216)
at com.sun.proxy.$Proxy37.getAMStatus(Unknown Source)
at
org.apache.tez.client.TezClient.getAppMasterStatus(TezClient.java:503)
at org.apache.tez.client.TezClient.waitTillReady(TezClient.java:576)
HTH
Johannes
On 12 Aug 2014, at 22:05, Hitesh Shah <[email protected]> wrote:
> It seems like the AM is binding to the external/public hostname and not the
> internal IP.
>
> Could you look for this log message in the AM logs: "Instantiated
> DAGClientRPCServer at”. This will provide some information as to what the AM
> is binding to.
>
> thanks
> — Hitesh
>
> On Aug 11, 2014, at 7:43 AM, Johannes Zillmann <[email protected]>
> wrote:
>
>> Hey guys,
>>
>> having a test-infrastructure for Hadoop on ec2. The client sits usually
>> outside of ec2.
>> Using plain map-reduce on YARN everything works fine.
>> Using Tez i run into following exception:
>>
>> INFO [2014-07-29 00:09:06.653] [MrPlanRunnerV2] (TezClient.java:507) -
>> Failed to retrieve AM Status via proxy
>> com.google.protobuf.ServiceException:
>> org.apache.hadoop.net.ConnectTimeoutException: Call From
>> ip-10-73-6-154.ec2.internal/10.73.6.154 to
>> ec2-54-81-245-144.compute-1.amazonaws.com:60914 failed on socket timeout
>> exception: org.apache.hadoop.net.ConnectTimeoutException: connect timed out;
>> For more details see: http:
>> //wiki.apache.org/hadoop/SocketTimeout
>>
>> at
>> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:216)
>> at com.sun.proxy.$Proxy116.getAMStatus(Unknown Source)
>> at
>> org.apache.tez.client.TezClient.getAppMasterStatus(TezClient.java:500)
>> at org.apache.tez.client.TezClient.waitTillReady(TezClient.java:586)
>>
>>
>> I could resolve the problem for Tez changing the hostname of the instances
>> to their public dns’. However, that is causing problems with other
>> components.
>> Do you know of any place in Tez which is related to that ? Any tweak which
>> could make chaining the hostname superfluous ?
>>
>> best
>> Johannes
>
