QuakeWang commented on code in PR #731:
URL: https://github.com/apache/paimon-rust/pull/731#discussion_r3812850561
##########
crates/paimon/src/table/global_index_scanner.rs:
##########
@@ -536,6 +565,27 @@ impl GlobalIndexScanner {
entries: &[GlobalIndexEntry],
predicates: &[(PredicateOperator, &[Datum], &DataType)],
) -> Result<Option<Vec<RowRange>>> {
+ let normalized_predicates = predicates
+ .iter()
+ .map(|(op, literals, data_type)| {
+ let key_type = if is_multivalue_predicate(*op) {
+ let DataType::Array(array) = data_type else {
+ return Err(Error::DataInvalid {
+ message: format!(
+ "Array global-index predicate {op} requires an
ARRAY field type"
+ ),
+ source: None,
+ });
+ };
+ array.element_type()
Review Comment:
This uses the current array element type for all index files, including
files built before a schema change. For example, after changing `ARRAY<INT>` to
`ARRAY<BIGINT>`, an old 4-byte key is passed to the BIGINT comparator and
causes a panic when it reads 8 bytes. Could we keep the compatibility check
here, or filter and rebuild incompatible index entries? A schema-evolution test
would also help cover this case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
