devinjdangelo commented on code in PR #8667:
URL: https://github.com/apache/arrow-datafusion/pull/8667#discussion_r1438307213
##########
datafusion/core/src/physical_planner.rs:
##########
@@ -605,6 +605,7 @@ impl DefaultPhysicalPlanner {
FileType::JSON => Arc::new(JsonFormat::default()),
FileType::AVRO => Arc::new(AvroFormat {} ),
FileType::ARROW => Arc::new(ArrowFormat {}),
+ FileType::Extension(_ext) => return
not_impl_err!("Extension FileTypes not supported in Copy To statements.")
Review Comment:
I tried this (as well as combining ExtensionFileType trait into FileFormat),
but that would require adding lots of dependencies between the DataFusion
crates. FileType is in common and used in most of DataFusion while FileFormat
is only in core.
This could work but I think the consequence would be DataFusion becomes one
big crate. I think the only way to keep the independent crates is to have at
least two File related abstractions. One simple one for logical planning in
Common and one heavy duty one for Execution plans in core.
Even linking them is a challenge because then common depends on core 🤔
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
