Yuanjian Li created SPARK-34526:
-----------------------------------
Summary: Add a flag to skip checking file sink format and handle
glob path
Key: SPARK-34526
URL: https://issues.apache.org/jira/browse/SPARK-34526
Project: Spark
Issue Type: Bug
Components: Structured Streaming
Affects Versions: 3.1.0
Reporter: Yuanjian Li
This ticket fixes the following issues related to file sink format checking
together:
* Some users may use a very long glob path to read and `isDirectory`{{}} may
fail when the path is too long. We should ignore the error when the path is a
glob path since file streaming sink doesn’t support glob paths.
* Checking whether a directory is outputted by File Streaming Sink may fail
for various issues happening in the storage. We should add a flag to allow
users to disable it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
