Github user HyukjinKwon commented on a diff in the pull request:

    https://github.com/apache/spark/pull/20424#discussion_r168086187
  
    --- Diff: 
core/src/main/scala/org/apache/spark/api/python/PythonWorkerFactory.scala ---
    @@ -191,7 +192,26 @@ private[spark] class PythonWorkerFactory(pythonExec: 
String, envVars: Map[String
             daemon = pb.start()
     
             val in = new DataInputStream(daemon.getInputStream)
    -        daemonPort = in.readInt()
    +        try {
    +          daemonPort = in.readInt()
    +        } catch {
    +          case _: EOFException =>
    +            throw new IOException(s"No port number in $daemonModule's 
stdout")
    +        }
    +
    +        // test that the returned port number is within a valid range.
    +        // note: this does not cover the case where the port number
    +        // is arbitrary data but is also coincidentally within range
    +        if (daemonPort < 1 || daemonPort > 0xffff) {
    +          val exceptionMessage = f"""
    +               |Bad data in  $daemonModule's standard output.
    +               |Expected valid port number, got 0x$daemonPort%08x.
    +               |PYTHONPATH set to '$pythonPath'
    +               |Python command is '${command.asScala.mkString(" ")}'
    +               |One possibility is a sitecustomize.py module in your 
python installation
    +               |that is printing to stdout"""
    --- End diff --
    
    But I think there's no need to put some additional logics here to deal with 
it tho. Let's just fix the error message to be in the similar format.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

Reply via email to