JingsongLi commented on code in PR #9550:
URL: https://github.com/apache/paimon/pull/9550#discussion_r3921449834
##########
paimon-format/src/main/java/org/apache/paimon/format/parquet/ParquetReaderFactory.java:
##########
@@ -336,11 +337,10 @@ private Type clipParquetType(DataType readType, Type
parquetType) {
// There are two representations for array type in parquet.
// See link:
//
https://impala.apache.org/docs/build/html/topics/impala_parquet_array_resolution.html.
- int level = arrayGroup.getType(0) instanceof GroupType ? 3 : 2;
Type elementType =
clipParquetType(elementReadType,
parquetListElementType(arrayGroup));
- if (level == 3) {
+ if (isThreeLevelList(arrayGroup)) {
Review Comment:
[P1] Preserve the original Rule 2 interpretation after clipping
Rule 2 is identified from the original two-field repeated group here, but
after recursive projection the rebuilt requested schema may contain only one
field. ParquetReaderUtil.getArrayElementColumn then reclassifies that clipped
group as a Rule 5 wrapper and unwraps it. For LIST<STRUCT<x,y>> read as
ARRAY<ROW<x>>, reader construction pairs the RowType with a PrimitiveColumnIO
and throws ClassCastException. Please carry the original list interpretation
into field construction, or otherwise keep Rule 2 unambiguous after projection,
and add a one-field projection read test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
