Rich-T-kid commented on code in PR #27500:
URL: https://github.com/apache/datafusion/pull/27500#discussion_r4656804154
##########
datafusion/common/src/config.rs:
##########
@@ -1451,9 +1451,31 @@ config_namespace! {
/// This applies only when DataFusion infers the table schema. Tables
with
/// a user-supplied schema are not promoted because the Parquet footer
is
/// not read at DDL time, so dictionary pages cannot be detected per
column.
+ ///
+ /// Setting `dictionary_read_max_distinct_values` also enables
dictionary
+ /// reads. When both options are set, the cap controls which plain
columns
+ /// are promoted.
/// See <https://github.com/apache/datafusion/issues/24112>
pub enable_rle_to_dictionary: bool, default = false
+ /// (reading) If set, promote top-level plain string and binary
columns to
+ /// dictionaries when every data page is dictionary encoded and the
+ /// per-file sum of row-group cardinality upper bounds is within this
cap.
Review Comment:
updated this!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]