alamb commented on code in PR #15865:
URL: https://github.com/apache/datafusion/pull/15865#discussion_r2065156402
##########
datafusion/core/src/datasource/listing/table.rs:
##########
@@ -1129,7 +1130,17 @@ impl ListingTable {
let (file_group, inexact_stats) =
get_files_with_limit(files, limit,
self.options.collect_stat).await?;
- let file_groups =
file_group.split_files(self.options.target_partitions);
+ let mut file_groups =
file_group.split_files(self.options.target_partitions);
+ let (schema_mapper, _) =
DefaultSchemaAdapterFactory::from_schema(self.schema())
Review Comment:
I think using the default schema mapper makes sense for now / in this PR,
but in general I think it would make sense to allow the user to provide their
own schema mapping rules here (so a default value that is not `NULL` can be
used, for example) via their own mapper.
However, we woudl have to add a schema mapper factory to `ListingOptions`
https://github.com/apache/datafusion/blob/f1bbb1d636650c7f28f52dc507f36e64d71e1aa8/datafusion/core/src/datasource/listing/table.rs#L256-L255
(this is not a change needed for this PR, I just noticed it while reviewing
this PR)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
