So created https://issues.apache.org/jira/browse/TEZ-2561 for it. As of the patch if have to see if i find time…
thanks Johannes > On 17 Jun 2015, at 19:05, Hitesh Shah <[email protected]> wrote: > > This may likely be TaskAttemptListenerImpTezDag used for communication > between the AM and the task containers. There are no configuration knobs for > it currently as it is always internal to a cluster. > > If you don’t mind, can you file a jira and contribute a patch to add a > clarifying log message to help other folks down the line? > > — Hitesh > > On Jun 17, 2015, at 9:45 AM, Johannes Zillmann <[email protected]> > wrote: > >> Hey guys, >> >> i’m trying to hunt down some port usage from the tez AM. >> Log looks like >> >> 2015-06-17 18:33:30,104 INFO [main] history.HistoryEventHandler: >> Initializing HistoryEventHandler >> 2015-06-17 18:33:30,110 INFO [main] impl.SimpleHistoryLoggingService: Log >> file location for SimpleHistoryLoggingService not specified, defaulting to >> containerLogDir=/Users/jz/Desktop/hadoop/hadoop-2.4.0/logs/userlogs/application_1434557883115_0003/container_1434557883115_0003_01_000001 >> 2015-06-17 18:33:30,111 INFO [main] impl.SimpleHistoryLoggingService: >> Initializing SimpleHistoryLoggingService, >> logFileLocation=/Users/jz/Desktop/hadoop/hadoop-2.4.0/logs/userlogs/application_1434557883115_0003/container_1434557883115_0003_01_000001/history.txt.appattempt_1434557883115_0003_000001, >> maxErrors=10 >> 2015-06-17 18:33:30,111 INFO [main] recovery.RecoveryService: Initializing >> RecoveryService >> 2015-06-17 18:33:30,115 INFO [main] history.HistoryEventHandler: >> [HISTORY][DAG:N/A][Event:APP_LAUNCHED]: >> applicationId=application_1434557883115_0003, appSubmitTime=1434558806972, >> launchTime=1434558808608 >> 2015-06-17 18:33:30,128 INFO [main] history.HistoryEventHandler: >> [HISTORY][DAG:N/A][Event:AM_LAUNCHED]: >> appAttemptId=appattempt_1434557883115_0003_000001, >> appSubmitTime=1434558806972, launchTime=1434558808608 >> 2015-06-17 18:33:30,132 INFO >> [ServiceThread:org.apache.tez.dag.history.HistoryEventHandler] >> history.HistoryEventHandler: Starting HistoryEventHandler >> 2015-06-17 18:33:30,132 INFO >> [ServiceThread:org.apache.tez.dag.history.HistoryEventHandler] >> impl.SimpleHistoryLoggingService: Starting SimpleHistoryLoggingService >> 2015-06-17 18:33:30,134 INFO >> [ServiceThread:org.apache.tez.dag.app.launcher.ContainerLauncherImpl] >> impl.ContainerManagementProtocolProxy: yarn.client.max-nodemanagers-proxies >> : 500 >> 2015-06-17 18:33:30,140 INFO >> [ServiceThread:org.apache.tez.dag.app.TaskAttemptListenerImpTezDag] >> ipc.CallQueueManager: Using callQueue class >> java.util.concurrent.LinkedBlockingQueue >> 2015-06-17 18:33:30,140 INFO [ServiceThread:DAGClientRPCServer] >> ipc.CallQueueManager: Using callQueue class >> java.util.concurrent.LinkedBlockingQueue >> 2015-06-17 18:33:30,157 INFO >> [ServiceThread:org.apache.tez.dag.history.HistoryEventHandler] >> recovery.RecoveryService: Starting RecoveryService >> 2015-06-17 18:33:30,157 INFO [HistoryEventHandlingThread] >> impl.SimpleHistoryLoggingService: Writing event APP_LAUNCHED to history file >> 2015-06-17 18:33:30,162 INFO [Socket Reader #1 for port 60875] ipc.Server: >> Starting Socket Reader #1 for port 60875 >> 2015-06-17 18:33:30,163 INFO [Socket Reader #1 for port 41111] ipc.Server: >> Starting Socket Reader #1 for port 41111 >> 2015-06-17 18:33:30,189 INFO [IPC Server Responder] ipc.Server: IPC Server >> Responder: starting >> 2015-06-17 18:33:30,189 INFO [IPC Server listener on 41111] ipc.Server: IPC >> Server listener on 41111: starting >> 2015-06-17 18:33:30,193 INFO [IPC Server Responder] ipc.Server: IPC Server >> Responder: starting >> 2015-06-17 18:33:30,193 INFO [IPC Server listener on 60875] ipc.Server: IPC >> Server listener on 60875: starting >> 2015-06-17 18:33:30,195 INFO [ServiceThread:DAGClientRPCServer] >> client.DAGClientServer: Instantiated DAGClientRPCServer at >> jzillmann.lan/192.168.178.28:41111 >> >> Port 41111 is controlled by tez.am.client.am.port-range , you’ve any idea >> what service 60875 could be ? And if so is there a way to configure it ? >> It does seem to vary for each AM, usually about 60xxx. >> >> Any ideas ?\ >> Johannes >
