xuanyuanking commented on a change in pull request #30336:
URL: https://github.com/apache/spark/pull/30336#discussion_r526557807
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStore.scala
##########
@@ -178,11 +178,15 @@ object StateStoreMetrics {
trait StateStoreCustomMetric {
def name: String
def desc: String
+ def unit: String
}
-case class StateStoreCustomSumMetric(name: String, desc: String) extends
StateStoreCustomMetric
-case class StateStoreCustomSizeMetric(name: String, desc: String) extends
StateStoreCustomMetric
-case class StateStoreCustomTimingMetric(name: String, desc: String) extends
StateStoreCustomMetric
+case class StateStoreCustomSumMetric(name: String, desc: String, unit: String)
Review comment:
Sorry for the comment at the last minute. I think the PR is ready for
merge. Only one concern for the API change.
This is an API change and will break extended StateStore if they have custom
metrics. Seems now the unit is the UI only requirement, how about we generate
the unit in `StreamingQueryStatisticsPage.scala`? Or we can add the unit info
in the `desc` string. e.g "StateStoreCustomSizeMetric("uncompressedFileSize",
"uncompressed file size - file bytes")" WDYT?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
