SinghAsDev commented on a change in pull request #3774:
URL: https://github.com/apache/iceberg/pull/3774#discussion_r791305217
##########
File path:
spark/v3.2/spark/src/main/java/org/apache/iceberg/spark/data/SparkParquetReaders.java
##########
@@ -187,13 +187,14 @@ private SparkParquetReaders() {
@Override
public ParquetValueReader<?> list(Types.ListType expectedList, GroupType
array,
ParquetValueReader<?> elementReader) {
- GroupType repeated = array.getFields().get(0).asGroupType();
+ Type repeated = array.getFields().get(0);
+ boolean isOldListElementType =
ParquetSchemaUtil.isOldListElementType(repeated, array.getName());
String[] repeatedPath = currentPath();
int repeatedD = type.getMaxDefinitionLevel(repeatedPath) - 1;
int repeatedR = type.getMaxRepetitionLevel(repeatedPath) - 1;
- Type elementType = repeated.getType(0);
+ Type elementType = isOldListElementType ? repeated :
repeated.asGroupType().getType(0);
Review comment:
Added. As we discussed, I will backport to older versions of Spark and
flink in a separate diff.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
