Johannes Zillmann created TEZ-2561:
--------------------------------------
Summary: Port for TaskAttemptListenerImpTezDag should be
configurable
Key: TEZ-2561
URL: https://issues.apache.org/jira/browse/TEZ-2561
Project: Apache Tez
Issue Type: Improvement
Reporter: Johannes Zillmann
Noticed sporadic DAG failures in our ec2 test environment.
Tasks failing with that:
{noformat}
2015-06-17 11:19:51,064 INFO [main] impl.MetricsSystemImpl: Scheduled snapshot
period at 10 second(s).
2015-06-17 11:19:51,064 INFO [main] impl.MetricsSystemImpl: TezTask metrics
system started
2015-06-17 11:19:51,259 INFO [TezChild] task.ContainerReporter: Attempting to
fetch new task
2015-06-17 11:20:11,311 INFO [TezChild] ipc.Client: Retrying connect to server:
ip-10-149-102-100.ec2.internal/10.149.102.100:60630. Already tried 0 time(s);
maxRetries=5
2015-06-17 11:20:31,312 INFO [TezChild] ipc.Client: Retrying connect to server:
ip-10-149-102-100.ec2.internal/10.149.102.100:60630. Already tried 1 time(s);
maxRetries=5
2015-06-17 11:20:51,313 INFO [TezChild] ipc.Client: Retrying connect to server:
ip-10-149-102-100.ec2.internal/10.149.102.100:60630. Already tried 2 time(s);
maxRetries=5
2015-06-17 11:21:11,314 INFO [TezChild] ipc.Client: Retrying connect to server:
ip-10-149-102-100.ec2.internal/10.149.102.100:60630. Already tried 3 time(s);
maxRetries=5
2015-06-17 11:21:31,315 INFO [TezChild] ipc.Client: Retrying connect to server:
ip-10-149-102-100.ec2.internal/10.149.102.100:60630. Already tried 4 time(s);
maxRetries=5
2015-06-17 11:21:51,317 INFO [main] impl.MetricsSystemImpl: Stopping TezTask
metrics system...
2015-06-17 11:21:51,318 INFO [main] impl.MetricsSystemImpl: TezTask metrics
system stopped.
2015-06-17 11:21:51,318 INFO [main] impl.MetricsSystemImpl: TezTask metrics
system shutdown complete.
{noformat}
>From the AppMaster:
{noformat}
Created DAGAppMaster for application appattempt_1434553606315_0022_000001
2015-06-17 11:19:43,655 INFO [Socket Reader #1 for port 60630] ipc.Server:
Starting Socket Reader #1 for port 60630
2015-06-17 11:19:43,656 INFO [Socket Reader #1 for port 31001] ipc.Server:
Starting Socket Reader #1 for port 31001
2015-06-17 11:19:43,713 WARN
[ServiceThread:org.apache.tez.dag.history.HistoryEventHandler]
conf.Configuration: mapred-site.xml:an attempt to override final parameter:
mapreduce.cluster.local.dir; Ignoring.
{noformat}
[~hitesh] mentioned its likely to be the TaskAttemptListenerImpTezDag which
starts on that port. Would be nice if the port(-range) can be configured!!!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
