[
https://issues.apache.org/jira/browse/SPARK-40122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ihor Bobak updated SPARK-40122:
-------------------------------
Description:
Without any visible reason I am getting this error in my Jupyter notebook (see
stacktrace below) with pyspark kernel. Often it occurs even if no Spark
operations are made, e.g. when I am working with multiprocessing Pool for a
local piece of code that should parallelize on the cores of the driver, with no
spark transformations/actions done in that jupyter cell.
INFO:py4j.clientserver:Error while sending or receiving.
Traceback (most recent call last):
File
"/opt/spark-3.3.0-bin-without-hadoop/python/lib/py4j-0.10.9.5-src.zip/py4j/clientserver.py",
line 503, in send_command
self.socket.sendall(command.encode("utf-8"))
ConnectionResetError: [Errno 104] Connection reset by peer
INFO:py4j.clientserver:Closing down clientserver connection
INFO:root:Exception while sending command.
Traceback (most recent call last):
File
"/opt/spark-3.3.0-bin-without-hadoop/python/lib/py4j-0.10.9.5-src.zip/py4j/clientserver.py",
line 503, in send_command
self.socket.sendall(command.encode("utf-8"))
ConnectionResetError: [Errno 104] Connection reset by peer
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File
"/opt/spark-3.3.0-bin-without-hadoop/python/lib/py4j-0.10.9.5-src.zip/py4j/java_gateway.py",
line 1038, in send_command
response = connection.send_command(command)
File
"/opt/spark-3.3.0-bin-without-hadoop/python/lib/py4j-0.10.9.5-src.zip/py4j/clientserver.py",
line 506, in send_command
raise Py4JNetworkError(
py4j.protocol.Py4JNetworkError: Error while sending
INFO:py4j.clientserver:Closing down clientserver connection
was:
Without any visible reason I am getting this error in my Jupyter notebook (see
stacktrace below) with pyspark kernel. Often it occurs even if no Spark
operations are made, e.g. when I am working with multiprocessing Pool for a
local piece of code that should parallelize on the cores of the driver, with no
spark transformations/actions done in that jupyter cell.
{{INFO:py4j.clientserver:Error while sending or receiving.
Traceback (most recent call last):
File
"/opt/spark-3.3.0-bin-without-hadoop/python/lib/py4j-0.10.9.5-src.zip/py4j/clientserver.py",
line 503, in send_command
self.socket.sendall(command.encode("utf-8"))
ConnectionResetError: [Errno 104] Connection reset by peer
INFO:py4j.clientserver:Closing down clientserver connection
INFO:root:Exception while sending command.
Traceback (most recent call last):
File
"/opt/spark-3.3.0-bin-without-hadoop/python/lib/py4j-0.10.9.5-src.zip/py4j/clientserver.py",
line 503, in send_command
self.socket.sendall(command.encode("utf-8"))
ConnectionResetError: [Errno 104] Connection reset by peer
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File
"/opt/spark-3.3.0-bin-without-hadoop/python/lib/py4j-0.10.9.5-src.zip/py4j/java_gateway.py",
line 1038, in send_command
response = connection.send_command(command)
File
"/opt/spark-3.3.0-bin-without-hadoop/python/lib/py4j-0.10.9.5-src.zip/py4j/clientserver.py",
line 506, in send_command
raise Py4JNetworkError(
py4j.protocol.Py4JNetworkError: Error while sending
INFO:py4j.clientserver:Closing down clientserver connection
}}
> py4j-0.10.9.5 often produces "Connection reset by peer" in Spark 3.3.0
> -----------------------------------------------------------------------
>
> Key: SPARK-40122
> URL: https://issues.apache.org/jira/browse/SPARK-40122
> Project: Spark
> Issue Type: Bug
> Components: PySpark
> Affects Versions: 3.3.0
> Reporter: Ihor Bobak
> Priority: Major
>
> Without any visible reason I am getting this error in my Jupyter notebook
> (see stacktrace below) with pyspark kernel. Often it occurs even if no Spark
> operations are made, e.g. when I am working with multiprocessing Pool for a
> local piece of code that should parallelize on the cores of the driver, with
> no spark transformations/actions done in that jupyter cell.
> INFO:py4j.clientserver:Error while sending or receiving.
> Traceback (most recent call last):
> File
> "/opt/spark-3.3.0-bin-without-hadoop/python/lib/py4j-0.10.9.5-src.zip/py4j/clientserver.py",
> line 503, in send_command
> self.socket.sendall(command.encode("utf-8"))
> ConnectionResetError: [Errno 104] Connection reset by peer
> INFO:py4j.clientserver:Closing down clientserver connection
> INFO:root:Exception while sending command.
> Traceback (most recent call last):
> File
> "/opt/spark-3.3.0-bin-without-hadoop/python/lib/py4j-0.10.9.5-src.zip/py4j/clientserver.py",
> line 503, in send_command
> self.socket.sendall(command.encode("utf-8"))
> ConnectionResetError: [Errno 104] Connection reset by peer
> During handling of the above exception, another exception occurred:
> Traceback (most recent call last):
> File
> "/opt/spark-3.3.0-bin-without-hadoop/python/lib/py4j-0.10.9.5-src.zip/py4j/java_gateway.py",
> line 1038, in send_command
> response = connection.send_command(command)
> File
> "/opt/spark-3.3.0-bin-without-hadoop/python/lib/py4j-0.10.9.5-src.zip/py4j/clientserver.py",
> line 506, in send_command
> raise Py4JNetworkError(
> py4j.protocol.Py4JNetworkError: Error while sending
> INFO:py4j.clientserver:Closing down clientserver connection
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
