Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/1643#discussion_r15731478
--- Diff:
core/src/main/scala/org/apache/spark/api/python/PythonWorkerFactory.scala ---
@@ -107,7 +113,9 @@ private[spark] class PythonWorkerFactory(pythonExec:
String, envVars: Map[String
// Wait for it to connect to our socket
serverSocket.setSoTimeout(10000)
try {
- return serverSocket.accept()
+ val socket = serverSocket.accept()
+ simpleWorkers.put(socket, pb)
--- End diff --
Yep, this looks like a compiler bug. This file compiles in 2.10.4 and
gives the expected error in 2.11.2:
```scala
import scala.collection.JavaConversions._
import java.lang.Process
import java.lang.ProcessBuilder
object ImplicitBug {
def main(args: Seq[String]) {
val simpleWorkers = new scala.collection.mutable.WeakHashMap[Int,
Process]()
val pb = new ProcessBuilder()
simpleWorkers.put(1, pb)
}
}
```
For the curious, here's the actual implicit conversions that led to this
bug (run `/scala-2.10.4/bin/scalac -Xprint:typer ImplicitBug.scala` to get this
output):
```scala
[[syntax trees at end of typer]] // ImplicitBug.scala
package <empty> {
import scala.collection.JavaConversions._;
import java.lang.Process;
import java.lang.ProcessBuilder;
object ImplicitBug extends scala.AnyRef {
def <init>(): ImplicitBug.type = {
ImplicitBug.super.<init>();
()
};
def main(args: Seq[String]): Unit = {
val simpleWorkers: scala.collection.mutable.WeakHashMap[Int,Process]
= new scala.collection.mutable.WeakHashMap[Int,Process]();
val pb: ProcessBuilder = new java.lang.ProcessBuilder();
{
// Somehow it became a map with `Object` values:
scala.collection.JavaConversions.mapAsJavaMap[Int,
Object](simpleWorkers).put(1, pb);
()
}
}
}
}
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
