szehon-ho commented on code in PR #7613:
URL: https://github.com/apache/iceberg/pull/7613#discussion_r1201393571
##########
core/src/main/java/org/apache/iceberg/BaseFilesTable.java:
##########
@@ -192,14 +181,46 @@ private CloseableIterable<? extends ContentFile<?>>
files(Schema fileProjection)
}
}
- private StructLike withReadableMetrics(ContentFile<?> file, int
metricsPosition) {
+ /**
+ * Given content file metadata, append a 'readable_metrics' column that
return the file's
+ * metrics in human-readable form.
+ *
+ * @file content file metadata
+ * @param readableMetricsField projected "readable_metrics" field
+ * @return struct representing content file, with appended
readable_metrics field
+ */
+ private StructLike withReadableMetrics(
+ ContentFile<?> file, Types.NestedField readableMetricsField) {
+ int metricsPosition = projection.columns().indexOf(readableMetricsField);
int columnCount = projection.columns().size();
Review Comment:
Yea, projectionColumnCount is I guess the description of what it is, but the
way I think about the method withReadableMetrics is actually more towards the
goal, of making a struct.
So I think it's better to have the variables in the end: struct, structSize,
metricsStruct, metricsPosition
(projectionColumnCount being structSize). Changed the references to match.
What do you think?
##########
core/src/main/java/org/apache/iceberg/BaseFilesTable.java:
##########
@@ -192,14 +181,46 @@ private CloseableIterable<? extends ContentFile<?>>
files(Schema fileProjection)
}
}
- private StructLike withReadableMetrics(ContentFile<?> file, int
metricsPosition) {
+ /**
+ * Given content file metadata, append a 'readable_metrics' column that
return the file's
+ * metrics in human-readable form.
+ *
+ * @file content file metadata
+ * @param readableMetricsField projected "readable_metrics" field
+ * @return struct representing content file, with appended
readable_metrics field
+ */
+ private StructLike withReadableMetrics(
+ ContentFile<?> file, Types.NestedField readableMetricsField) {
+ int metricsPosition = projection.columns().indexOf(readableMetricsField);
int columnCount = projection.columns().size();
- StructType projectedMetricType =
-
projection.findField(MetricsUtil.READABLE_METRICS).type().asStructType();
+
+ StructType projectedMetricType =
readableMetricsField.type().asStructType();
MetricsUtil.ReadableMetricsStruct readableMetrics =
MetricsUtil.readableMetricsStruct(dataTableSchema, file,
projectedMetricType);
- return new ContentFileStructWithMetrics(
- columnCount, metricsPosition, (StructLike) file, readableMetrics);
+ return new MetricsUtil.StructWithReadableMetrics(
+ (StructLike) file, readableMetrics, columnCount, metricsPosition);
+ }
+
+ /**
+ * Create a projection on content files metadata by removing virtual
'readable_column' and
+ * ensuring that the underlying metrics used to create that column are
part of the final
+ * projection.
+ *
+ * @param projectionSchema projection to transform
Review Comment:
I like that, changed to 'requestedProjection'
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
