pengbo commented on a change in pull request #24666: [SPARK-27482][SQL][WEBUI]
Show estimated BroadcastHashJoinExec numOutputRows statistics info on SparkSQL
UI page
URL: https://github.com/apache/spark/pull/24666#discussion_r289641219
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/metric/SQLMetricInfo.scala
##########
@@ -27,4 +27,5 @@ import org.apache.spark.annotation.DeveloperApi
class SQLMetricInfo(
val name: String,
val accumulatorId: Long,
- val metricType: String)
+ val metricType: String,
+ val stats: Long = -1)
Review comment:
@cloud-fan Thanks for your comments.
The idea is that each SQL metric can have a statistic value (-1 means not
available/initialized). I set the statistic type to Long is because SQL
Metric's value is always Long type as well. `class SQLMetric(val metricType:
String, initValue: Long = 0L)`
Put Option[Statistics] in SQLMetricInfo doesn't sound quite right though. It
means that all SQL metrics have an attribute including rowCount, size & column
stats.
Let me know your feedback, thanks in advance.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
