gene-bordegaray commented on code in PR #22657:
URL: https://github.com/apache/datafusion/pull/22657#discussion_r3355278332
##########
datafusion/catalog-listing/src/options.rs:
##########
@@ -61,6 +64,17 @@ pub struct ListingOptions {
/// multiple equivalent orderings, the outer `Vec` will have a
/// single element.
pub file_sort_order: Vec<Vec<SortExpr>>,
+ /// Optional declared output partitioning for this table.
+ ///
+ /// Expressions are specified against the full table schema. When set,
+ /// [`ListingTable`](crate::ListingTable) creates one scan file group per
+ /// declared output partition instead of using [`Self::target_partitions`].
+ /// Empty file groups are added when needed to preserve that count.
+ ///
+ /// Files are sorted by path before grouping. DataFusion does not validate
+ /// that rows match the declaration, so callers must ensure file group `i`
+ /// contains only rows for declared output partition `i`.
+ pub output_partitioning: Option<Partitioning>,
Review Comment:
Mmm I see, we will need to introduce `Range` at the logical evel for
partitioning since it only hash `Hash` and `RoundRobin` right now... would that
be too large of a change in this PR
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
