parthchandra opened a new issue, #7430: URL: https://github.com/apache/datafusion-comet/issues/7430
### What is the problem the feature request solves? The native CSV scan (`spark.comet.scan.csv.v2.enabled`) covers DataSource V2 `CSVScan` only. Most CSV reads plan V1 `FileSourceScanExec` with `CSVFileFormat`, which Spark reads. ### Describe the potential solution Route V1 CSV file-source scans to the native CSV reader, with the same fallback rules as the V2 path (corrupt-record column, `inferSchema`, multi-character delimiters, non-UTC timestamp parsing). Then decide whether native CSV can become default. ### Additional context Related: #7243, which changes how native CSV scan tasks get their file groups. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
