JingsongLi commented on code in PR #340:
URL: https://github.com/apache/paimon-rust/pull/340#discussion_r3341349211
##########
crates/paimon/src/spec/schema.rs:
##########
@@ -293,6 +294,7 @@ impl Schema {
let fields = Self::normalize_fields(&fields, &partition_keys,
&primary_keys)?;
Self::validate_blob_fields(&fields, &partition_keys, &options)?;
PartialUpdateConfig::new(&options).validate_create_mode(!primary_keys.is_empty())?;
+
AggregationConfig::new(&options).validate_create_mode(!primary_keys.is_empty())?;
Review Comment:
This validation only sees the option map plus whether the table has PKs, so
it lets schema-dependent aggregation mistakes through CREATE TABLE. For
example, a typo like `fields.amout.aggregate-function = 'sum'` is accepted and
then the real `amount` column silently falls back to the
default/`last_non_null_value`; an unknown function name or an incompatible
function/type pair is also accepted until the first read/write constructs
`AggregateMergeFunction`. Java Paimon validates field aggregation options
during schema validation (including factory lookup), so can we make this
create-time check schema-aware and reject unknown fields, unknown functions,
and incompatible function/type pairs before persisting the table schema?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
