juliuszsompolski commented on PR #42818: URL: https://github.com/apache/spark/pull/42818#issuecomment-1708631041
https://github.com/juliuszsompolski/apache-spark/actions/runs/6097172856/job/16544375244 ``` test_other_than_dataframe_iter (pyspark.sql.tests.connect.test_parity_pandas_map.MapInPandasParityTests) ... malloc(): unsorted double linked list corrupted ERROR (612.473s) test_self_join (pyspark.sql.tests.connect.test_parity_pandas_map.MapInPandasParityTests) ... ERROR (611.358s) ERROR:root:Exception while sending command. Traceback (most recent call last): File "/__w/apache-spark/apache-spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py", line 516, in send_command raise Py4JNetworkError("Answer from Java side is empty") py4j.protocol.Py4JNetworkError: Answer from Java side is empty During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/__w/apache-spark/apache-spark/python/lib/py4j-0.10.9.7-src.zip/py4j/java_gateway.py", line 1038, in send_command response = connection.send_command(command) File "/__w/apache-spark/apache-spark/python/lib/py4j-0.10.9.7-src.zip/py4j/clientserver.py", line 539, in send_command raise Py4JNetworkError( py4j.protocol.Py4JNetworkError: Error while sending or receiving /__w/apache-spark/apache-spark/python/pyspark/context.py:657: RuntimeWarning: Unable to cleanly shutdown Spark JVM process. It is possible that the process has crashed, been killed or may also be in a zombie state. warnings.warn( /__w/apache-spark/apache-spark/python/pyspark/sql/connect/client/reattach.py:219: UserWarning: ReleaseExecute failed with exception: Cannot invoke RPC on closed channel!. warnings.warn(f"ReleaseExecute failed with exception: {e}.") ====================================================================== ERROR [612.473s]: test_other_than_dataframe_iter (pyspark.sql.tests.connect.test_parity_pandas_map.MapInPandasParityTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/__w/apache-spark/apache-spark/python/pyspark/sql/tests/connect/test_parity_pandas_map.py", line 26, in test_other_than_dataframe_iter self.check_other_than_dataframe_iter() File "/__w/apache-spark/apache-spark/python/pyspark/sql/tests/pandas/test_pandas_map.py", line 163, in check_other_than_dataframe_iter (self.spark.range(10, numPartitions=3).mapInPandas(bad_iter_elem, "a int").count()) File "/__w/apache-spark/apache-spark/python/pyspark/sql/connect/dataframe.py", line 252, in count pdd = self.agg(_invoke_function("count", lit(1))).toPandas() File "/__w/apache-spark/apache-spark/python/pyspark/sql/connect/dataframe.py", line 1703, in toPandas return self._session.client.to_pandas(query) File "/__w/apache-spark/apache-spark/python/pyspark/sql/connect/client/core.py", line 873, in to_pandas table, schema, metrics, observed_metrics, _ = self._execute_and_fetch( File "/__w/apache-spark/apache-spark/python/pyspark/sql/connect/client/core.py", line 1282, in _execute_and_fetch for response in self._execute_and_fetch_as_iterator(req): File "/__w/apache-spark/apache-spark/python/pyspark/sql/connect/client/core.py", line 1263, in _execute_and_fetch_as_iterator self._handle_error(error) File "/__w/apache-spark/apache-spark/python/pyspark/sql/connect/client/core.py", line 1502, in _handle_error self._handle_rpc_error(error) File "/__w/apache-spark/apache-spark/python/pyspark/sql/connect/client/core.py", line 1542, in _handle_rpc_error raise SparkConnectGrpcException(str(rpc_error)) from None pyspark.errors.exceptions.connect.SparkConnectGrpcException: <_MultiThreadedRendezvous of RPC that terminated with: status = StatusCode.UNAVAILABLE details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:38175: Failed to connect to remote host: Connection refused" debug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:127.0.0.1:38175: Failed to connect to remote host: Connection refused {created_time:"2023-09-06T13:29:14.012172836+00:00", grpc_status:14}" > ``` Unclean shutdown of the client after another failure, leaves the ReleaseExecute in another thread hanging... failure not related to this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
