Repository: spark
Updated Branches:
refs/heads/master 73f065569 -> 339419145
[SPARK-19508][CORE] Improve error message when binding service fails
## What changes were proposed in this pull request?
Utils provides a helper function to bind service on port. This function can
bind the service to a random free port. However, if the binding fails on a
random free port, the retrying and final exception messages look confusing.
17/02/06 16:25:43 WARN Utils: Service 'sparkDriver' could not bind on port
0. Attempting port 1.
17/02/06 16:25:43 WARN Utils: Service 'sparkDriver' could not bind on port
0. Attempting port 1.
17/02/06 16:25:43 WARN Utils: Service 'sparkDriver' could not bind on port
0. Attempting port 1.
17/02/06 16:25:43 WARN Utils: Service 'sparkDriver' could not bind on port
0. Attempting port 1.
17/02/06 16:25:43 WARN Utils: Service 'sparkDriver' could not bind on port
0. Attempting port 1.
...
17/02/06 16:25:43 ERROR SparkContext: Error initializing SparkContext.
java.net.BindException: Can't assign requested address: Service
'sparkDriver' failed after 16 retries (starting from 0)! Consider explicitly
setting the appropriate port for the service 'sparkDriver' (for example
spark.ui.port for SparkUI) to an available port or increasing
spark.port.maxRetries.
## How was this patch tested?
Jenkins tests.
Please review http://spark.apache.org/contributing.html before opening a pull
request.
Author: Liang-Chi Hsieh <[email protected]>
Closes #16851 from viirya/better-log-message.
Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/33941914
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/33941914
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/33941914
Branch: refs/heads/master
Commit: 33941914548cc5a65e8467821745d65728176368
Parents: 73f0655
Author: Liang-Chi Hsieh <[email protected]>
Authored: Mon Feb 20 21:25:21 2017 -0800
Committer: Sean Owen <[email protected]>
Committed: Mon Feb 20 21:25:21 2017 -0800
----------------------------------------------------------------------
.../scala/org/apache/spark/util/Utils.scala | 27 +++++++++++++++-----
1 file changed, 21 insertions(+), 6 deletions(-)
----------------------------------------------------------------------
http://git-wip-us.apache.org/repos/asf/spark/blob/33941914/core/src/main/scala/org/apache/spark/util/Utils.scala
----------------------------------------------------------------------
diff --git a/core/src/main/scala/org/apache/spark/util/Utils.scala
b/core/src/main/scala/org/apache/spark/util/Utils.scala
index 1e6e9a2..5538289 100644
--- a/core/src/main/scala/org/apache/spark/util/Utils.scala
+++ b/core/src/main/scala/org/apache/spark/util/Utils.scala
@@ -2210,17 +2210,32 @@ private[spark] object Utils extends Logging {
} catch {
case e: Exception if isBindCollision(e) =>
if (offset >= maxRetries) {
- val exceptionMessage = s"${e.getMessage}: Service$serviceString
failed after " +
- s"$maxRetries retries (starting from $startPort)! Consider
explicitly setting " +
- s"the appropriate port for the service$serviceString (for
example spark.ui.port " +
- s"for SparkUI) to an available port or increasing
spark.port.maxRetries."
+ val exceptionMessage = if (startPort == 0) {
+ s"${e.getMessage}: Service$serviceString failed after " +
+ s"$maxRetries retries (on a random free port)! " +
+ s"Consider explicitly setting the appropriate binding address
for " +
+ s"the service$serviceString (for example
spark.driver.bindAddress " +
+ s"for SparkDriver) to the correct binding address."
+ } else {
+ s"${e.getMessage}: Service$serviceString failed after " +
+ s"$maxRetries retries (starting from $startPort)! Consider
explicitly setting " +
+ s"the appropriate port for the service$serviceString (for
example spark.ui.port " +
+ s"for SparkUI) to an available port or increasing
spark.port.maxRetries."
+ }
val exception = new BindException(exceptionMessage)
// restore original stack trace
exception.setStackTrace(e.getStackTrace)
throw exception
}
- logWarning(s"Service$serviceString could not bind on port $tryPort.
" +
- s"Attempting port ${tryPort + 1}.")
+ if (startPort == 0) {
+ // As startPort 0 is for a random free port, it is most possibly
binding address is
+ // not correct.
+ logWarning(s"Service$serviceString could not bind on a random free
port. " +
+ "You may check whether configuring an appropriate binding
address.")
+ } else {
+ logWarning(s"Service$serviceString could not bind on port
$tryPort. " +
+ s"Attempting port ${tryPort + 1}.")
+ }
}
}
// Should never happen
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
