parthchandra commented on issue #1121: URL: https://github.com/apache/datafusion-comet/issues/1121#issuecomment-2512600034
> Longer term it would be interesting if it is possible to delegate the necessary behavior to custom file formats I would hope that custom formats use Spark's DSV2 apis (like iceberg does). > all the work trying to push down the Parquet scans to datafusion might make that impossible true. CometParquetFileFormat will become unnecessary and Comet scan will become entirely native (or so we think). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
