yijiacui-db commented on a change in pull request #31944:
URL: https://github.com/apache/spark/pull/31944#discussion_r618975894
##########
File path: sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala
##########
@@ -188,7 +178,8 @@ class SourceProgress protected[sql](
val latestOffset: String,
val numInputRows: Long,
val inputRowsPerSecond: Double,
- val processedRowsPerSecond: Double) extends Serializable {
+ val processedRowsPerSecond: Double,
+ val metrics: ju.Map[String, String] = Map[String, String]().asJava) extends
Serializable {
Review comment:
I tried to add a type for this Map but find it hard to do so. Because we
also use this Map[String, String] in java, if we define this alias in this
file, we can't import it into the java ReportSourceMetrics File.
If we want to create an alias in java, by extending a map type, then we need
to make it public and add another file, then import them in scala files.
The other option is to only use this alias in the two source metrics files,
which it's not that efficient to have a type.
Do you have any suggestion about which way we should go to define this alias?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
