amogh-jahagirdar commented on code in PR #17960:
URL: https://github.com/apache/iceberg/pull/17960#discussion_r4134911536
##########
spark/v4.1/spark/src/test/java/org/apache/iceberg/spark/source/TestWriteMetricsConfig.java:
##########
@@ -282,13 +283,15 @@ public void testCustomMetricCollectionForNestedParquet() {
Map<Integer, Long> nullValueCounts = file.nullValueCounts();
assertThat(nullValueCounts)
- .hasSize(3)
- .containsKeys(longCol.fieldId(), recordId.fieldId(),
recordData.fieldId());
+ .hasSize(4)
Review Comment:
what's interesting/a bit suspect is that the lower/upper bounds assertions
didn't need to change. I think this is a separate issue though, the metrics
config we're building is correct, and clearly the column is materialized in
parquet, but for whatever reason the parquet writer is only handing back the
null/value counts and not the lower/upper bounds.
I'm looking at this separately though, maybe another plumbing issue in the
writer or something i'm missing as to why parquet is handing back metrics which
omit lower/upper bounds for this string column.
I also don't really know if I agree with the utility of this test class, but
while it's here I'll just stick to updating it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
