Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21721#discussion_r206374910 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/progress.scala --- @@ -163,7 +163,8 @@ class SourceProgress protected[sql]( val endOffset: String, val numInputRows: Long, val inputRowsPerSecond: Double, - val processedRowsPerSecond: Double) extends Serializable { + val processedRowsPerSecond: Double, + val customMetrics: Option[JValue] = None) extends Serializable { --- End diff -- Wait .. this is an exposed API, right? I guess this is exposed to Java API too (for instance `query.lastProgress().sources()`)? In that case, we should avoid Scala's Option and `org.json4s.*`. If this is supposed to be hidden here, I think we should better find a way to hide this with package level access modifier.
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org