[GitHub] spark pull request #22526: [SPARK-25502][CORE][WEBUI]Empty Page when page nu...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/22526


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #22526: [SPARK-25502][CORE][WEBUI]Empty Page when page nu...

[shahidki31]

Github user shahidki31 commented on a diff in the pull request:

https://github.com/apache/spark/pull/22526#discussion_r219897175
  
--- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala ---
@@ -685,7 +685,7 @@ private[ui] class TaskDataSource(
 
   private var _tasksToShow: Seq[TaskData] = null
 
-  override def dataSize: Int = taskCount(stage)
+  override def dataSize: Int = store.taskCount(stage.stageId, 
stage.attemptId).toInt
--- End diff --

Thanks. Done.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #22526: [SPARK-25502][CORE][WEBUI]Empty Page when page nu...

[jiangxb1987]

Github user jiangxb1987 commented on a diff in the pull request:

https://github.com/apache/spark/pull/22526#discussion_r219891354
  
--- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala ---
@@ -685,7 +685,7 @@ private[ui] class TaskDataSource(
 
   private var _tasksToShow: Seq[TaskData] = null
 
-  override def dataSize: Int = taskCount(stage)
+  override def dataSize: Int = store.taskCount(stage.stageId, 
stage.attemptId).toInt
--- End diff --

nit: after this change, the function `taskCount()` is only referenced by 
`totalTasks`, we can inline that.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #22526: [SPARK-25502][CORE][WEBUI]Empty Page when page nu...

[shahidki31]

Github user shahidki31 commented on a diff in the pull request:

https://github.com/apache/spark/pull/22526#discussion_r219876198
  
--- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala ---
@@ -685,7 +685,10 @@ private[ui] class TaskDataSource(
 
   private var _tasksToShow: Seq[TaskData] = null
 
-  override def dataSize: Int = taskCount(stage)
+  override def dataSize: Int = {
+val storedTasks = store.taskCount(stage.stageId, stage.attemptId).toInt
--- End diff --

Done. Thanks


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #22526: [SPARK-25502][CORE][WEBUI]Empty Page when page nu...

[srowen]

Github user srowen commented on a diff in the pull request:

https://github.com/apache/spark/pull/22526#discussion_r219869516
  
--- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala ---
@@ -685,7 +685,10 @@ private[ui] class TaskDataSource(
 
   private var _tasksToShow: Seq[TaskData] = null
 
-  override def dataSize: Int = taskCount(stage)
+  override def dataSize: Int = {
+val storedTasks = store.taskCount(stage.stageId, stage.attemptId).toInt
--- End diff --

You don't need the intermediate variable here.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #22526: [SPARK-25502][CORE][WEBUI]Empty Page when page nu...

[shahidki31]

Github user shahidki31 commented on a diff in the pull request:

https://github.com/apache/spark/pull/22526#discussion_r219844417
  
--- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala ---
@@ -685,7 +685,15 @@ private[ui] class TaskDataSource(
 
   private var _tasksToShow: Seq[TaskData] = null
 
-  override def dataSize: Int = taskCount(stage)
+  override def dataSize: Int = {
+val storedTasks = store.taskCount(stage.stageId, stage.attemptId).toInt
+val totalTasks = taskCount(stage)
+if (totalTasks > storedTasks) {
--- End diff --

Thanks. I have updated the PR  by  returning only storedTasks. Kindly look 
into the updated code.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #22526: [SPARK-25502][CORE][WEBUI]Empty Page when page nu...

[srowen]

Github user srowen commented on a diff in the pull request:

https://github.com/apache/spark/pull/22526#discussion_r219827419
  
--- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala ---
@@ -685,7 +685,15 @@ private[ui] class TaskDataSource(
 
   private var _tasksToShow: Seq[TaskData] = null
 
-  override def dataSize: Int = taskCount(stage)
+  override def dataSize: Int = {
+val storedTasks = store.taskCount(stage.stageId, stage.attemptId).toInt
+val totalTasks = taskCount(stage)
+if (totalTasks > storedTasks) {
--- End diff --

Oh, hm I would have thought it's simpler to return the single value that 
this always takes on.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org