Yi Zhang created SPARK-48741:
--------------------------------
Summary: Wrong FS error when InsertIntoHadoopFSCommand with
HadoopMapreduceCommitterProtocol when table path and partition path are in
different FS
Key: SPARK-48741
URL: https://issues.apache.org/jira/browse/SPARK-48741
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 3.4.3, 3.5.1
Reporter: Yi Zhang
If table path and partition path are in different FS (for example different gcs
bucket), the staging directory is under the table path,
HadoopMapreduceCommitProtocol create fs object using this staging path, so the
operation on the partition paths will fail with 'Wrong FS' error.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
