Hyukjin Kwon created SPARK-48083:
------------------------------------
Summary: session.copyFromLocalToFs failure with 3.5 client <> 4.0
server
Key: SPARK-48083
URL: https://issues.apache.org/jira/browse/SPARK-48083
Project: Spark
Issue Type: Sub-task
Components: Connect, ML, PySpark
Affects Versions: 4.0.0
Reporter: Hyukjin Kwon
{code}
======================================================================
ERROR [1.120s]: test_save_load
(pyspark.ml.tests.connect.test_connect_classification.ClassificationTestsOnConnect.test_save_load)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/home/runner/work/spark/spark-3.5/python/pyspark/ml/tests/connect/test_legacy_mode_classification.py",
line 144, in test_save_load
estimator.save(fs_path)
File
"/home/runner/work/spark/spark-3.5/python/pyspark/ml/connect/io_utils.py", line
248, in save
_copy_dir_from_local_to_fs(tmp_local_dir, path)
File
"/home/runner/work/spark/spark-3.5/python/pyspark/ml/connect/io_utils.py", line
57, in _copy_dir_from_local_to_fs
_copy_file_from_local_to_fs(file_path, dest_file_path)
File
"/home/runner/work/spark/spark-3.5/python/pyspark/ml/connect/io_utils.py", line
39, in _copy_file_from_local_to_fs
session.copyFromLocalToFs(local_path, dest_path)
File
"/home/runner/work/spark/spark-3.5/python/pyspark/sql/connect/session.py", line
756, in copyFromLocalToFs
self._client.copy_from_local_to_fs(local_path, dest_path)
File
"/home/runner/work/spark/spark-3.5/python/pyspark/sql/connect/client/core.py",
line 1549, in copy_from_local_to_fs
self._artifact_manager._add_forward_to_fs_artifacts(local_path, dest_path)
File
"/home/runner/work/spark/spark-3.5/python/pyspark/sql/connect/client/artifact.py",
line 280, in _add_forward_to_fs_artifacts
self._request_add_artifacts(requests)
File
"/home/runner/work/spark/spark-3.5/python/pyspark/sql/connect/client/artifact.py",
line 259, in _request_add_artifacts
response: proto.AddArtifactsResponse = self._retrieve_responses(requests)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/runner/work/spark/spark-3.5/python/pyspark/sql/connect/client/artifact.py",
line 256, in _retrieve_responses
return self._stub.AddArtifacts(requests, metadata=self._metadata)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/grpc/_channel.py",
line 1536, in __call__
return _end_unary_response_blocking(state, call, False, None)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/opt/hostedtoolcache/Python/3.11.9/x64/lib/python3.11/site-packages/grpc/_channel.py",
line 1006, in _end_unary_response_blocking
raise _InactiveRpcError(state) # pytype: disable=not-instantiable
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.INTERNAL
details = "Uploading artifact file to local file system destination
path is not supported."
debug_error_string = "UNKNOWN:Error received from peer
{grpc_message:"Uploading artifact file to local file system destination path is
not supported.", grpc_status:13,
created_time:"2024-05-01T03:01:32.[558](https://github.com/HyukjinKwon/spark/actions/runs/8904629949/job/24454181142#step:9:559)489983+00:00"}"
>
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
