dramaticlly commented on code in PR #7613:
URL: https://github.com/apache/iceberg/pull/7613#discussion_r1198287876
##########
core/src/main/java/org/apache/iceberg/BaseFilesTable.java:
##########
@@ -192,14 +181,46 @@ private CloseableIterable<? extends ContentFile<?>>
files(Schema fileProjection)
}
}
- private StructLike withReadableMetrics(ContentFile<?> file, int
metricsPosition) {
+ /**
+ * Given content file metadata, append a 'readable_metrics' column that
return the file's
+ * metrics in human-readable form.
+ *
+ * @file content file metadata
+ * @param readableMetricsField projected "readable_metrics" field
+ * @return struct representing content file, with appended
readable_metrics field
+ */
+ private StructLike withReadableMetrics(
+ ContentFile<?> file, Types.NestedField readableMetricsField) {
+ int metricsPosition = projection.columns().indexOf(readableMetricsField);
int columnCount = projection.columns().size();
Review Comment:
maybe we can reuse the `projectionColumnCount` as variable name above ?
##########
core/src/main/java/org/apache/iceberg/BaseFilesTable.java:
##########
@@ -192,14 +181,46 @@ private CloseableIterable<? extends ContentFile<?>>
files(Schema fileProjection)
}
}
- private StructLike withReadableMetrics(ContentFile<?> file, int
metricsPosition) {
+ /**
+ * Given content file metadata, append a 'readable_metrics' column that
return the file's
+ * metrics in human-readable form.
+ *
+ * @file content file metadata
+ * @param readableMetricsField projected "readable_metrics" field
+ * @return struct representing content file, with appended
readable_metrics field
+ */
+ private StructLike withReadableMetrics(
+ ContentFile<?> file, Types.NestedField readableMetricsField) {
+ int metricsPosition = projection.columns().indexOf(readableMetricsField);
int columnCount = projection.columns().size();
- StructType projectedMetricType =
-
projection.findField(MetricsUtil.READABLE_METRICS).type().asStructType();
+
+ StructType projectedMetricType =
readableMetricsField.type().asStructType();
MetricsUtil.ReadableMetricsStruct readableMetrics =
MetricsUtil.readableMetricsStruct(dataTableSchema, file,
projectedMetricType);
- return new ContentFileStructWithMetrics(
- columnCount, metricsPosition, (StructLike) file, readableMetrics);
+ return new MetricsUtil.StructWithReadableMetrics(
+ (StructLike) file, readableMetrics, columnCount, metricsPosition);
+ }
+
+ /**
+ * Create a projection on content files metadata by removing virtual
'readable_column' and
+ * ensuring that the underlying metrics used to create that column are
part of the final
+ * projection.
+ *
+ * @param projectionSchema projection to transform
Review Comment:
nit: `projection to transform` read a bit weird as first time I read it I am
not clear on why do we need to transform, maybe `requested projection` or
`intended projection` which correspond to returning of actual projection ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
