nssalian commented on code in PR #18121:
URL: https://github.com/apache/iceberg/pull/18121#discussion_r4418525672
##########
arrow/src/main/java/org/apache/iceberg/arrow/vectorized/VectorizedReaderBuilder.java:
##########
@@ -148,11 +154,55 @@ protected VectorizedReader<?>
vectorizedReader(List<VectorizedReader<?>> reorder
@Override
public VectorizedReader<?> struct(
Types.StructType expected, GroupType groupType,
List<VectorizedReader<?>> fieldReaders) {
- if (expected != null) {
- throw new UnsupportedOperationException(
- "Vectorized reads are not supported yet for struct fields");
+ if (expected == null) {
+ return null;
+ }
+
+ // no field ID / no matching Iceberg field: fall back like primitive()
+ if (groupType.getId() == null) {
+ return null;
}
- return null;
+
+ Types.NestedField structField =
icebergSchema.findField(groupType.getId().intValue());
+ if (structField == null) {
+ return null;
+ }
+
+ List<VectorizedReader<?>> reorderedFields =
+ reorderFields(expected.fields(), groupType.getFields(), fieldReaders);
+
+ int structDefinitionLevel =
parquetSchema.getMaxDefinitionLevel(currentPath());
+
+ VectorizedArrowReader presenceReader = null;
+ // must agree with ParquetSchemaUtil.PresenceColumnSelector, which retains
this presence column
+ if (structDefinitionLevel > 0 && !hasFileBackedLeaf(reorderedFields)) {
+ ColumnDescriptor presence =
+ ParquetSchemaUtil.selectPresenceColumn(parquetSchema, currentPath());
+ // list/map are not vectorized, so the presence leaf is always rep level
0 (iterator
Review Comment:
Good catch, thanks. The split PR will address this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
