Ravi Dalal created SPARK-48417:
----------------------------------
Summary: Filesystems do not load with spark.jars.packages
configuration
Key: SPARK-48417
URL: https://issues.apache.org/jira/browse/SPARK-48417
Project: Spark
Issue Type: Bug
Components: Input/Output
Affects Versions: 3.5.1
Reporter: Ravi Dalal
When we use spark.jars.packages configuration parameter in Python SparkSession
Builder (Pyspark), it appears that the filesystems are not loaded when session
starts. Because of this, Spark fails to read file from Google Cloud Storage
(GCS) bucket (with GCS Connector).
I tested this with different packages so it does not appear specific to a
particular package. I will attach the sample code and debug logs.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
