peter-toth commented on code in PR #54081:
URL: https://github.com/apache/spark/pull/54081#discussion_r2752983291
##########
sql/catalyst/src/main/java/org/apache/spark/sql/connector/read/PartitionReader.java:
##########
@@ -58,4 +58,13 @@ default CustomTaskMetric[] currentMetricsValues() {
CustomTaskMetric[] NO_METRICS = {};
return NO_METRICS;
}
+
+ /**
+ * Sets the initial value of metrics before fetching any data from the
reader. This is called
+ * when multiple {@link PartitionReader}s are grouped into one partition in
case of
+ * {@link
org.apache.spark.sql.connector.read.partitioning.KeyGroupedPartitioning} and
the reader
+ * is initialized with the metrics returned by the previous reader that
belongs to the same
+ * partition.
Review Comment:
Yes, added in
https://github.com/apache/spark/pull/54081/commits/fae7720bc81485ad8605751e3b14be442b1232d4.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
