Sebastian Hillig created SPARK-50713:
----------------------------------------
Summary: PythonWorker throws NoSuchElementException
Key: SPARK-50713
URL: https://issues.apache.org/jira/browse/SPARK-50713
Project: Spark
Issue Type: Bug
Components: PySpark
Affects Versions: 3.5.4, 3.5.3, 3.5.2, 3.5.1
Reporter: Sebastian Hillig
The fix of https://issues.apache.org/jira/browse/SPARK-47565 was not complete
because it overlooked the possibility of an element being removed from
`daemonWorkers` before being pulled for reuse.
```
java.util.NoSuchElementException: key not found:
PythonWorker(java.nio.channels.SocketChannel[closed],sun.nio.ch.EPollSelectorImpl@750d157f,sun.nio.ch.SelectionKeyImpl@4929fbb7)
at scala.collection.MapLike.default(MapLike.scala:236) at
scala.collection.MapLike.default$(MapLike.scala:235) at
scala.collection.AbstractMap.default(Map.scala:65) at
scala.collection.MapLike.apply(MapLike.scala:144) at
scala.collection.MapLike.apply$(MapLike.scala:143) at
scala.collection.AbstractMap.apply(Map.scala:65) at
org.apache.spark.api.python.PythonWorkerFactory.create(PythonWorkerFactory.scala:199)
```
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
