ajantha-bhat commented on code in PR #13425:
URL: https://github.com/apache/iceberg/pull/13425#discussion_r2188920467
##########
core/src/main/java/org/apache/iceberg/PartitionStatsHandler.java:
##########
@@ -89,17 +91,57 @@ private PartitionStatsHandler() {}
NestedField.optional(11, "last_updated_at", LongType.get());
public static final NestedField LAST_UPDATED_SNAPSHOT_ID =
NestedField.optional(12, "last_updated_snapshot_id", LongType.get());
+ // Using default value for v3 field to support v3 reader reading file
written by v2
+ public static final NestedField DV_COUNT =
+ NestedField.required("dv_count")
+ .withId(13)
+ .ofType(Types.IntegerType.get())
+ .withInitialDefault(Literal.of(0))
Review Comment:
Using default 0 for v3. Because when we try to read v2 stats with v3 schema,
we get the field not found error without the default values configuration as dv
is a required field in schema.
callstack:
```
Missing required field: dv_count
java.lang.IllegalArgumentException: Missing required field: dv_count
at
org.apache.iceberg.data.parquet.BaseParquetReaders$ReadBuilder.defaultReader(BaseParquetReaders.java:269)
at
org.apache.iceberg.data.parquet.BaseParquetReaders$ReadBuilder.struct(BaseParquetReaders.java:252)
at
org.apache.iceberg.data.parquet.BaseParquetReaders$ReadBuilder.message(BaseParquetReaders.java:219)
at
org.apache.iceberg.data.parquet.BaseParquetReaders$ReadBuilder.message(BaseParquetReaders.java:207)
at
org.apache.iceberg.parquet.TypeWithSchemaVisitor.visit(TypeWithSchemaVisitor.java:48)
at
org.apache.iceberg.data.parquet.BaseParquetReaders.createReader(BaseParquetReaders.java:67)
at
org.apache.iceberg.data.parquet.BaseParquetReaders.createReader(BaseParquetReaders.java:59)
at
org.apache.iceberg.data.parquet.InternalReader.create(InternalReader.java:40)
at
org.apache.iceberg.parquet.Parquet$ReadBuilder.lambda$build$0(Parquet.java:1368)
at org.apache.iceberg.parquet.ReadConf.<init>(ReadConf.java:121)
at org.apache.iceberg.parquet.ParquetReader.init(ParquetReader.java:74)
at
org.apache.iceberg.parquet.ParquetReader.iterator(ParquetReader.java:94)
at
org.apache.iceberg.io.CloseableIterable$7$1.<init>(CloseableIterable.java:205)
at
org.apache.iceberg.io.CloseableIterable$7.iterator(CloseableIterable.java:204)
at
org.apache.iceberg.io.CloseableIterable$7.iterator(CloseableIterable.java:196)
at
org.apache.iceberg.relocated.com.google.common.collect.Lists.newArrayList(Lists.java:139)
at
org.apache.iceberg.PartitionStatsHandlerTestBase.testV2toV3SchemaEvolution(PartitionStatsHandlerTestBase.java:695)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
