juntaozhang opened a new pull request, #9550:
URL: https://github.com/apache/paimon/pull/9550

   ### Purpose
   Follow [Parquet's backward-compatibility 
rules](https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#backward-compatibility-rules)
 for the LIST logical type when clipping Parquet schemas, so that nested 
variant column pruning and schema conversion correctly distinguish wrapper 
groups from element types across all standard and legacy list encodings:
   - Rule 1: repeated primitive field is the element type.
   - Rule 2: repeated group with multiple fields is the element type.
   - Rule 3: repeated group whose single child is also repeated is the element 
type.
   - Rule 4: repeated group named `array` or `<list>_tuple` is the element type.
   - Rule 5: any other repeated group with exactly one non-repeated child is a 
wrapper; its child is the element type.
   
   Previously the list-structure check did not fully apply these rules, which 
could misidentify legacy two-level encodings as three-level lists and return 
the wrong element type during nested variant pruning.
   
   ### Tests
   
   `mvn -pl paimon-format -DwildcardSuites=none 
-Dtest=ParquetSchemaConverterTest test`
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to