Github user xuanyuanking commented on a diff in the pull request:
https://github.com/apache/spark/pull/21642#discussion_r200160518
--- Diff:
core/src/main/scala/org/apache/spark/status/AppStatusListener.scala ---
@@ -73,6 +74,10 @@ private[spark] class AppStatusListener(
// around liveExecutors.
@volatile private var activeExecutorCount = 0
+ private val inputDataSetId = new AtomicLong(0)
+ private val outputDataSetId = new AtomicLong(0)
+ private val maxRecords = conf.getInt("spark.data.maxRecords", 1000)
--- End diff --
What's this `spark.data.maxRecords` for? Maybe you should follow the config
in core/src/main/scala/org/apache/spark/status/config.scala
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
