RussellSpitzer commented on code in PR #15634:
URL: https://github.com/apache/iceberg/pull/15634#discussion_r2996097359
##########
core/src/main/java/org/apache/iceberg/ManifestReader.java:
##########
@@ -281,8 +294,22 @@ private CloseableIterable<ManifestEntry<F>> open(Schema
projection) {
Preconditions.checkArgument(
format != null, "Unable to determine format of manifest: %s",
file.location());
+ boolean unpartitioned = spec.rawPartitionType().fields().isEmpty();
+
+ // V4+ manifests omit the partition field when unpartitioned (Parquet
cannot represent
+ // empty structs, and the field is meaningless regardless of format). Mark
it optional so
+ // the reader returns null for the missing field instead of throwing. The
field must stay
+ // in the projection to preserve positional access for callers like
StructProjection.
+ // For older versions where the empty struct is present, making it
optional is harmless.
List<Types.NestedField> fields = Lists.newArrayList();
- fields.addAll(projection.asStruct().fields());
+ for (Types.NestedField field : projection.asStruct().fields()) {
Review Comment:
As the large comment above says, this is where we are handling the fact that
Parquet doesn't like empty structs. We just inject an "empty - optional"
parquet field so we can just get nulls in that slot.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
