Gengliang Wang created SPARK-27136:
--------------------------------------
Summary: Remove data source option check_files_exist
Key: SPARK-27136
URL: https://issues.apache.org/jira/browse/SPARK-27136
Project: Spark
Issue Type: Task
Components: SQL
Affects Versions: 3.0.0
Reporter: Gengliang Wang
The data source option check_files_exist is introduced in In #23383 when the
file source V2 framework is implemented. In the PR, FileIndex was created as a
member of FileTable, so that we could implement partition pruning like 0f9fcab
in the future. For file writes, we needed the option to decide whether to check
file existence.
After https://github.com/apache/spark/pull/23774, the option is not needed
anymore. This PR is to clean the option.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
